openai / evals
File Size

The distribution of size of files (measured in lines of code).

Intro
Learn more...
File Size Overall
5% | 0% | 28% | 25% | 41%
Legend:
1001+
501-1000
201-500
101-200
1-100


explore: grouped by folders | grouped by size | sunburst | 3D view
File Size per Extension
1001+
501-1000
201-500
101-200
1-100
ipynb100% | 0% | 0% | 0% | 0%
py3% | 0% | 32% | 31% | 32%
yaml0% | 0% | 23% | 11% | 64%
html0% | 0% | 0% | 100% | 0%
jsonl0% | 0% | 0% | 0% | 100%
js0% | 0% | 0% | 0% | 100%
toml0% | 0% | 0% | 0% | 100%
in0% | 0% | 0% | 0% | 100%
File Size per Logical Decomposition
primary
1001+
501-1000
201-500
101-200
1-100
evals5% | 0% | 28% | 24% | 41%
scripts0% | 0% | 0% | 65% | 34%
ROOT0% | 0% | 0% | 0% | 100%
Longest Files (Top 50)
File# lines# units
generate_samples.ipynb
in evals/registry/data/backgammon
 1349 -
actions.py
in evals/elsuite/multistep_web_tasks/webarena/browser_env
 1014 50
tools.py
in evals/elsuite/bugged_tools
 497 37
processors.py
in evals/elsuite/multistep_web_tasks/webarena/browser_env
 495 21
record.py
in evals
 450 54
make_plots.py
in evals/elsuite/error_recovery/scripts
 446 20
session.py
in evals/elsuite/multistep_web_tasks
 416 28
mmlu.yaml
in evals/registry/evals
 399 -
theory_of_mind.yaml
in evals/registry/solvers
 394 -
mmmu.yaml
in evals/registry/evals
 390 -
make_plots.py
in evals/elsuite/identifying_variables/scripts
 325 17
gen_data.py
in evals/elsuite/identifying_variables/scripts
 319 13
eval.py
in evals/elsuite/skill_acquisition
 313 7
plot_experiments.py
in evals/elsuite/hr_ml_agent_bench/scripts
 307 -
low_level_actions.py
in evals/elsuite/hr_ml_agent_bench
 304 13
defaults.yaml
in evals/registry/solvers
 294 -
environment.py
in evals/elsuite/hr_ml_agent_bench
 283 21
constants.py
in evals/elsuite/multistep_web_tasks/webarena/browser_env
 282 -
playwright_api.py
in evals/elsuite/multistep_web_tasks/webarena/core
 279 32
eval_run.py
in evals/elsuite/multistep_web_tasks/webarena
 277 14
evaluators.py
in evals/elsuite/multistep_web_tasks/webarena/evaluation_harness
 273 19
skill_acquisition.yaml
in evals/registry/solvers
 267 -
make_plots.py
in evals/elsuite/already_said_that/scripts
 263 8
oaieval.py
in evals/cli
 253 6
eval.py
in evals/elsuite/incontext_rl
 246 12
eval.py
in evals/elsuite/function_deduction
 244 14
registry.py
in evals
 242 27
dataset_creation.py
in evals/elsuite/cant_do_that_anymore/scripts
 235 8
make_plots.py
in evals/elsuite/track_the_stat/scripts
 235 9
make_plots.py
in evals/elsuite/ballots/scripts
 233 12
plot_experiments.py
in evals/elsuite/incontext_rl/scripts
 233 8
solve.py
in evals/registry/data/solve-for-variable/tools
 231 30
eval.py
in evals/elsuite/identifying_variables
 227 14
raven-matrices.yaml
in evals/registry/evals
 224 -
core.py
in evals/elsuite/make_me_say
 223 18
makemepay.py
in evals/elsuite/make_me_pay
 222 8
diagonal_dataset_creation.py
in evals/elsuite/cant_do_that_anymore/scripts
 216 6
corrset.py
in evals/elsuite/identifying_variables/renderers
 216 11
eval.py
in evals/elsuite/self_prompting
 210 7
eval.py
in evals/elsuite/bugged_tools
 210 9
cards.py
in evals/elsuite/bluff/bluff
 206 37
eval.py
in evals/elsuite/error_recovery
 204 9
pieces.py
in evals/elsuite/cant_do_that_anymore/chess
 203 9
custom_datasets.py
in evals/elsuite/steganography/scripts/dataset
 197 12
make_plots.py
in evals/elsuite/function_deduction/scripts
 195 4
basic_browser_env.py
in evals/elsuite/multistep_web_tasks/webarena/browser_env
 191 11
high_level_actions.py
in evals/elsuite/hr_ml_agent_bench
 191 4
utils.py
in evals/elsuite/multistep_web_tasks/webarena/core
 188 7
openai_assistants_solver.py
in evals/solvers/providers/openai
 186 11
185 1
Files With Most Units (Top 50)
File# lines# units
record.py
in evals
 450 54
actions.py
in evals/elsuite/multistep_web_tasks/webarena/browser_env
 1014 50
cards.py
in evals/elsuite/bluff/bluff
 206 37
tools.py
in evals/elsuite/bugged_tools
 497 37
playwright_api.py
in evals/elsuite/multistep_web_tasks/webarena/core
 279 32
solve.py
in evals/registry/data/solve-for-variable/tools
 231 30
session.py
in evals/elsuite/multistep_web_tasks
 416 28
registry.py
in evals
 242 27
processors.py
in evals/elsuite/multistep_web_tasks/webarena/browser_env
 495 21
environment.py
in evals/elsuite/hr_ml_agent_bench
 283 21
data.py
in evals
 148 21
make_plots.py
in evals/elsuite/error_recovery/scripts
 446 20
evaluators.py
in evals/elsuite/multistep_web_tasks/webarena/evaluation_harness
 273 19
openai_solver.py
in evals/solvers/providers/openai
 181 18
core.py
in evals/elsuite/make_me_say
 223 18
solver.py
in evals/solvers
 125 17
make_plots.py
in evals/elsuite/identifying_variables/scripts
 325 17
wave_function_collapse.py
in evals/registry/data/simple_physics_engine
 157 17
solvers.py
in evals/elsuite/sandbagging
 152 16
utils.py
in evals/elsuite/skill_acquisition
 115 15
corpus.py
in evals/registry/data/word_association/corpus_tools
 58 15
eval.py
in evals
 170 15
eval_run.py
in evals/elsuite/multistep_web_tasks/webarena
 277 14
tabular.py
in evals/elsuite/identifying_variables/renderers
 125 14
eval.py
in evals/elsuite/identifying_variables
 227 14
eval.py
in evals/elsuite/function_deduction
 244 14
basic_bash_env.py
in evals/elsuite/multistep_web_tasks/webarena/bash_env
 163 13
players.py
in evals/elsuite/bluff/bluff
 107 13
board.py
in evals/elsuite/cant_do_that_anymore/chess
 162 13
gen_data.py
in evals/elsuite/identifying_variables/scripts
 319 13
low_level_actions.py
in evals/elsuite/hr_ml_agent_bench
 304 13
solvers.py
in evals/elsuite/track_the_stat
 72 13
utils.py
in evals/elsuite
 150 13
related_words.py
in evals/registry/data/word_association/corpus_tools
 64 13
strong_solver.py
in evals/elsuite/multistep_web_tasks/solvers/strong_solver
 173 12
make_plots.py
in evals/elsuite/ballots/scripts
 233 12
utils.py
in evals/elsuite/cant_do_that_anymore
 178 12
custom_datasets.py
in evals/elsuite/steganography/scripts/dataset
 197 12
eval.py
in evals/elsuite/incontext_rl
 246 12
openai_assistants_solver.py
in evals/solvers/providers/openai
 186 11
basic_browser_env.py
in evals/elsuite/multistep_web_tasks/webarena/browser_env
 191 11
corrset.py
in evals/elsuite/identifying_variables/renderers
 216 11
solver_tools_convo.py
in evals/elsuite
 181 11
validators.py
in evals/registry/data/word_association/corpus_tools
 151 11
together_solver.py
in evals/solvers/providers/together
 68 10
baselines.py
in evals/elsuite/function_deduction
 91 10
openai.py
in evals/completion_fns
 147 10
gemini_solver.py
in evals/solvers/providers/google
 157 9
eval.py
in evals/elsuite/error_recovery
 204 9
eval.py
in evals/elsuite/bluff
 164 9
Files With Long Lines (Top 50)

There are 132 files with lines longer than 120 characters. In total, there are 335 long lines.

File# lines# units# long lines
raven-matrices.yaml
in evals/registry/evals
 224 - 28
tools.py
in evals/elsuite/bugged_tools
 497 37 26
hr-ml-agent-bench.yaml
in evals/registry/evals
 137 - 16
generate_samples.ipynb
in evals/registry/data/backgammon
 1349 - 15
high_level_actions.py
in evals/elsuite/hr_ml_agent_bench
 191 4 12
plot_experiments.py
in evals/elsuite/incontext_rl/scripts
 233 8 11
csv_to_json.py
in evals/registry/data/canto_wu_pronunciation
 55 - 8
theory_of_mind.yaml
in evals/registry/evals
 48 - 8
prompts.py
in evals/elsuite/ballots
 44 - 7
oaieval.py
in evals/cli
 253 6 7
task_description.py
in evals/elsuite/make_me_pay
 57 - 6
utils.py
in evals/elsuite/twenty_questions
 47 4 6
helper_functions.py
in evals/elsuite/multistep_web_tasks/webarena/browser_env
 129 5 5
prompts.py
in evals/elsuite/make_me_pay/solvers
 18 - 5
low_level_actions.py
in evals/elsuite/hr_ml_agent_bench
 304 13 5
defaults.py
in evals/elsuite/make_me_say
 34 5 5
singlestore.yaml
in evals/registry/modelgraded
 24 - 5
fewshot_solver.py
in evals/solvers/nested
 91 5 4
session.py
in evals/elsuite/multistep_web_tasks
 416 28 4
sql.yaml
in evals/registry/modelgraded
 24 - 4
product-ie.yaml
in evals/registry/evals
 28 - 4
defaults.py
in evals/elsuite/error_recovery
 12 - 3
data_generation.py
in evals/elsuite/theory_of_mind/scripts
 66 1 3
eval.py
in evals/elsuite/self_prompting
 210 7 3
environment.py
in evals/elsuite/hr_ml_agent_bench
 283 21 3
utils.py
in evals/elsuite/skill_acquisition
 115 15 3
solvers.py
in evals/elsuite/function_deduction
 140 9 3
self_prompting.yaml
in evals/registry/solvers
 96 - 3
research-question-extraction.yaml
in evals/registry/modelgraded
 19 - 3
self_consistency_solver.py
in evals/solvers/nested
 118 6 2
actions.py
in evals/elsuite/multistep_web_tasks/webarena/browser_env
 1014 50 2
eval.py
in evals/elsuite/make_me_pay
 126 3 2
task_description.py
in evals/elsuite/bugged_tools
 9 - 2
solvers.py
in evals/elsuite/identifying_variables
 27 4 2
classify_utils.py
in evals/elsuite/modelgraded
 145 8 2
eval.py
in evals/elsuite/skill_acquisition
 313 7 2
eval.py
in evals/elsuite/mmmu
 159 4 2
convert.js
in evals/registry/data/medmcqa
 44 1 2
convert.js
in evals/registry/data/unsolvable_questions
 51 1 2
findFailures.js
in evals/registry/data/unsolvable_questions
 43 1 2
fact.yaml
in evals/registry/modelgraded
 19 - 2
keywords.yaml
in evals/registry/modelgraded
 20 - 2
arithmetic-expression.yaml
in evals/registry/modelgraded
 24 - 2
translation.yaml
in evals/registry/modelgraded
 19 - 2
abstract-causal-reasoning.yaml
in evals/registry/evals
 16 - 2
openai.py
in evals/completion_fns
 147 10 2
49 1 1
cot.py
in evals/solvers/prompts
 4 - 1
hhh.py
in evals/solvers/prompts
 99 - 1
openai_solver.py
in evals/solvers/providers/openai
 181 18 1
Correlations

File Size vs. Commits (all time): 1496 points

evals/cli/oaieval.py x: 22 commits (all time) y: 253 lines of code evals/registry/data/imo_exact_answers/samples.jsonl x: 1 commits (all time) y: 3 lines of code evals/registry/evals/imo_exact_answers.yaml x: 1 commits (all time) y: 8 lines of code pyproject.toml x: 32 commits (all time) y: 64 lines of code evals/data.py x: 10 commits (all time) y: 148 lines of code evals/completion_fns/solver_completion_fn.py x: 1 commits (all time) y: 47 lines of code evals/elsuite/bluff/strategy_solver.py x: 3 commits (all time) y: 88 lines of code evals/elsuite/bugged_tools/task_description.py x: 2 commits (all time) y: 9 lines of code evals/elsuite/function_deduction/solvers.py x: 2 commits (all time) y: 140 lines of code evals/elsuite/hr_ml_agent_bench/solvers/baseline.py x: 2 commits (all time) y: 90 lines of code evals/elsuite/make_me_say/eval.py x: 3 commits (all time) y: 48 lines of code evals/eval.py x: 10 commits (all time) y: 170 lines of code evals/registry.py x: 24 commits (all time) y: 242 lines of code evals/registry/solvers/already_said_that.yaml x: 2 commits (all time) y: 75 lines of code evals/registry/solvers/bluff.yaml x: 2 commits (all time) y: 80 lines of code evals/registry/solvers/cant_do_that_anymore.yaml x: 2 commits (all time) y: 16 lines of code evals/registry/solvers/defaults.yaml x: 3 commits (all time) y: 294 lines of code evals/registry/solvers/error_recovery.yaml x: 2 commits (all time) y: 33 lines of code evals/registry/solvers/function_deduction.yaml x: 2 commits (all time) y: 174 lines of code evals/registry/solvers/incontext_rl.yaml x: 2 commits (all time) y: 24 lines of code evals/registry/solvers/make-me-pay.yaml x: 2 commits (all time) y: 101 lines of code evals/registry/solvers/self_prompting.yaml x: 2 commits (all time) y: 96 lines of code evals/registry/solvers/skill_acquisition.yaml x: 2 commits (all time) y: 267 lines of code evals/registry/solvers/theory_of_mind.yaml x: 3 commits (all time) y: 394 lines of code evals/solvers/memory.py x: 1 commits (all time) y: 50 lines of code evals/solvers/nested/cot_solver.py x: 3 commits (all time) y: 61 lines of code evals/solvers/nested/self_consistency_solver.py x: 3 commits (all time) y: 118 lines of code evals/solvers/prompts/cot.py x: 2 commits (all time) y: 4 lines of code evals/solvers/providers/openai/openai_assistants_solver.py x: 1 commits (all time) y: 186 lines of code evals/solvers/providers/openai/openai_solver.py x: 1 commits (all time) y: 181 lines of code evals/solvers/providers/together/together_solver.py x: 1 commits (all time) y: 68 lines of code evals/solvers/solver.py x: 4 commits (all time) y: 125 lines of code evals/solvers/utils.py x: 4 commits (all time) y: 37 lines of code evals/registry/solvers/gemini.yaml x: 1 commits (all time) y: 15 lines of code evals/solvers/providers/google/gemini_solver.py x: 1 commits (all time) y: 157 lines of code evals/completion_fns/openai.py x: 6 commits (all time) y: 147 lines of code evals/utils/api_utils.py x: 11 commits (all time) y: 15 lines of code evals/registry/solvers/anthropic.yaml x: 1 commits (all time) y: 90 lines of code evals/elsuite/hr_ml_agent_bench/actions.py x: 1 commits (all time) y: 37 lines of code evals/elsuite/hr_ml_agent_bench/autoeval.py x: 1 commits (all time) y: 172 lines of code evals/elsuite/hr_ml_agent_bench/benchmarks/ant/baselines/naive.py x: 1 commits (all time) y: 28 lines of code evals/elsuite/hr_ml_agent_bench/benchmarks/bipedal_walker/baselines/human.py x: 1 commits (all time) y: 57 lines of code evals/elsuite/hr_ml_agent_bench/benchmarks/cartpole/scripts/grade.py x: 1 commits (all time) y: 40 lines of code evals/elsuite/hr_ml_agent_bench/benchmarks/cifar10/env/train.py x: 1 commits (all time) y: 105 lines of code evals/elsuite/hr_ml_agent_bench/benchmarks/feedback/scripts/prepare.py x: 1 commits (all time) y: 24 lines of code evals/elsuite/hr_ml_agent_bench/benchmarks/house_price/scripts/prepare.py x: 1 commits (all time) y: 20 lines of code evals/elsuite/hr_ml_agent_bench/benchmarks/ogbn_arxiv/env/train.py x: 1 commits (all time) y: 126 lines of code evals/elsuite/hr_ml_agent_bench/benchmarks/parkinsons_disease/scripts/prepare.py x: 1 commits (all time) y: 98 lines of code evals/elsuite/hr_ml_agent_bench/benchmarks/vectorization/env/train.py x: 1 commits (all time) y: 118 lines of code evals/elsuite/hr_ml_agent_bench/benchmarks/vectorization/scripts/human_baseline.py x: 1 commits (all time) y: 83 lines of code evals/elsuite/hr_ml_agent_bench/environment.py x: 1 commits (all time) y: 283 lines of code evals/elsuite/hr_ml_agent_bench/eval.py x: 1 commits (all time) y: 91 lines of code evals/elsuite/hr_ml_agent_bench/low_level_actions.py x: 1 commits (all time) y: 304 lines of code evals/elsuite/hr_ml_agent_bench/scripts/run_experiments.py x: 1 commits (all time) y: 60 lines of code evals/elsuite/hr_ml_agent_bench/utils.py x: 1 commits (all time) y: 112 lines of code evals/registry/evals/hr-ml-agent-bench.yaml x: 1 commits (all time) y: 137 lines of code evals/elsuite/multistep_web_tasks/constants.py x: 1 commits (all time) y: 70 lines of code evals/elsuite/multistep_web_tasks/docker/flask-playwright/app.py x: 1 commits (all time) y: 165 lines of code evals/elsuite/multistep_web_tasks/docker/homepage/templates/index.html x: 1 commits (all time) y: 108 lines of code evals/elsuite/multistep_web_tasks/session.py x: 1 commits (all time) y: 416 lines of code evals/elsuite/multistep_web_tasks/webarena/bash_env/basic_bash_env.py x: 1 commits (all time) y: 163 lines of code evals/elsuite/multistep_web_tasks/webarena/browser_env/actions.py x: 1 commits (all time) y: 1014 lines of code evals/elsuite/multistep_web_tasks/webarena/browser_env/browser_utils.py x: 1 commits (all time) y: 78 lines of code evals/elsuite/multistep_web_tasks/webarena/browser_env/helper_functions.py x: 1 commits (all time) y: 129 lines of code evals/elsuite/multistep_web_tasks/webarena/browser_env/processors.py x: 1 commits (all time) y: 495 lines of code evals/elsuite/multistep_web_tasks/webarena/core/playwright_api.py x: 1 commits (all time) y: 279 lines of code evals/elsuite/multistep_web_tasks/webarena/evaluation_harness/evaluators.py x: 1 commits (all time) y: 273 lines of code evals/elsuite/incontext_rl/eval.py x: 1 commits (all time) y: 246 lines of code evals/elsuite/incontext_rl/scripts/plot_experiments.py x: 1 commits (all time) y: 233 lines of code evals/elsuite/function_deduction/eval.py x: 1 commits (all time) y: 244 lines of code evals/elsuite/function_deduction/scripts/make_plots.py x: 1 commits (all time) y: 195 lines of code evals/elsuite/identifying_variables/eval.py x: 1 commits (all time) y: 227 lines of code evals/elsuite/identifying_variables/renderers/corrset.py x: 1 commits (all time) y: 216 lines of code evals/elsuite/identifying_variables/scripts/gen_data.py x: 1 commits (all time) y: 319 lines of code evals/elsuite/identifying_variables/scripts/make_plots.py x: 1 commits (all time) y: 325 lines of code evals/elsuite/track_the_stat/scripts/make_plots.py x: 1 commits (all time) y: 235 lines of code evals/elsuite/already_said_that/scripts/make_plots.py x: 1 commits (all time) y: 263 lines of code evals/utils/log_utils.py x: 2 commits (all time) y: 53 lines of code evals/elsuite/skill_acquisition/eval.py x: 1 commits (all time) y: 313 lines of code evals/elsuite/error_recovery/eval.py x: 1 commits (all time) y: 204 lines of code evals/elsuite/error_recovery/scripts/make_plots.py x: 1 commits (all time) y: 446 lines of code evals/elsuite/cant_do_that_anymore/utils.py x: 1 commits (all time) y: 178 lines of code evals/elsuite/bugged_tools/eval.py x: 1 commits (all time) y: 210 lines of code evals/elsuite/basic/match_with_solvers.py x: 2 commits (all time) y: 65 lines of code evals/elsuite/make_me_pay/eval.py x: 3 commits (all time) y: 126 lines of code evals/elsuite/make_me_pay/task_description.py x: 3 commits (all time) y: 57 lines of code evals/elsuite/make_me_say/core.py x: 3 commits (all time) y: 223 lines of code evals/elsuite/steganography/scripts/dataset/custom_datasets.py x: 1 commits (all time) y: 197 lines of code evals/prompt/base.py x: 4 commits (all time) y: 64 lines of code evals/registry/evals/ballots.yaml x: 3 commits (all time) y: 34 lines of code evals/registry/evals/schelling_point.yaml x: 3 commits (all time) y: 66 lines of code evals/solvers/human_cli_solver.py x: 4 commits (all time) y: 29 lines of code evals/solvers/nested/fewshot_solver.py x: 2 commits (all time) y: 91 lines of code evals/solvers/prompts/hhh.py x: 4 commits (all time) y: 99 lines of code evals/elsuite/sandbagging/solvers.py x: 3 commits (all time) y: 152 lines of code evals/elsuite/self_prompting/solvers/baselines.py x: 2 commits (all time) y: 45 lines of code evals/elsuite/self_prompting/solvers/custom_cot_solver.py x: 2 commits (all time) y: 57 lines of code evals/record.py x: 15 commits (all time) y: 450 lines of code evals/cli/oaievalset.py x: 5 commits (all time) y: 111 lines of code evals/elsuite/ballots/prompts.py x: 3 commits (all time) y: 44 lines of code evals/elsuite/sandbagging/defaults.py x: 2 commits (all time) y: 12 lines of code evals/elsuite/schelling_point/prompts.py x: 3 commits (all time) y: 25 lines of code evals/elsuite/schelling_point/utils.py x: 2 commits (all time) y: 61 lines of code evals/registry/data/word_association/corpus_tools/validators.py x: 4 commits (all time) y: 151 lines of code scripts/battle_generator.py x: 7 commits (all time) y: 49 lines of code scripts/modelgraded_generator.py x: 9 commits (all time) y: 185 lines of code scripts/pattern_identification_generator.py x: 6 commits (all time) y: 48 lines of code evals/elsuite/mmmu/eval.py x: 3 commits (all time) y: 159 lines of code evals/completion_fns/langchain_llm.py x: 4 commits (all time) y: 70 lines of code evals/registry/evals/mmmu.yaml x: 1 commits (all time) y: 390 lines of code evals/elsuite/make_me_say/utils.py x: 6 commits (all time) y: 34 lines of code evals/elsuite/ballots/eval.py x: 3 commits (all time) y: 161 lines of code evals/elsuite/schelling_point/eval.py x: 2 commits (all time) y: 73 lines of code evals/elsuite/bluff/eval.py x: 2 commits (all time) y: 164 lines of code evals/completion_fns/langchain_math.py x: 4 commits (all time) y: 21 lines of code evals/elsuite/make_me_say/autoeval.py x: 4 commits (all time) y: 116 lines of code evals/elsuite/make_me_pay/makemepay.py x: 2 commits (all time) y: 222 lines of code evals/elsuite/lambada.py x: 2 commits (all time) y: 39 lines of code evals/elsuite/modelgraded/classify.py x: 27 commits (all time) y: 97 lines of code evals/elsuite/utils.py x: 10 commits (all time) y: 150 lines of code evals/registry/data/mapping_to_matricies/data_generator.py x: 2 commits (all time) y: 30 lines of code evals/registry/data/simple_physics_engine/wave_function_collapse.py x: 2 commits (all time) y: 157 lines of code evals/registry/data/solve-for-variable/tools/solve.py x: 2 commits (all time) y: 231 lines of code evals/registry/data/word_association/corpus_tools/sample_generators.py x: 2 commits (all time) y: 154 lines of code evals/base.py x: 5 commits (all time) y: 51 lines of code evals/elsuite/basic/match.py x: 5 commits (all time) y: 57 lines of code evals/registry/evals/test-modelgraded.yaml x: 7 commits (all time) y: 81 lines of code evals/registry/modelgraded/best.yaml x: 4 commits (all time) y: 10 lines of code evals/elsuite/modelgraded/classify_utils.py x: 5 commits (all time) y: 145 lines of code evals/registry/data/backgammon/generate_samples.ipynb x: 1 commits (all time) y: 1349 lines of code evals/elsuite/basic/json_validator.py x: 4 commits (all time) y: 41 lines of code evals/api.py x: 5 commits (all time) y: 61 lines of code evals/registry/data/balance_chemical_equation/samples.jsonl x: 3 commits (all time) y: 3 lines of code evals/registry/evals/mmlu.yaml x: 1 commits (all time) y: 399 lines of code evals/registry/eval_sets/coqa-ex.yaml x: 3 commits (all time) y: 7 lines of code evals/registry/eval_sets/test-all.yaml x: 8 commits (all time) y: 21 lines of code MANIFEST.in x: 4 commits (all time) y: 4 lines of code evals/registry/eval_sets/test-modelgraded.yaml x: 4 commits (all time) y: 14 lines of code
1349.0
lines of code
  min: 1.0
  average: 28.01
  25th percentile: 3.0
  median: 3.0
  75th percentile: 16.0
  max: 1349.0
0 32.0
commits (all time)
min: 1.0 | average: 1.29 | 25th percentile: 1.0 | median: 1.0 | 75th percentile: 1.0 | max: 32.0

File Size vs. Contributors (all time): 1496 points

evals/cli/oaieval.py x: 18 contributors (all time) y: 253 lines of code evals/registry/data/imo_exact_answers/samples.jsonl x: 1 contributors (all time) y: 3 lines of code evals/registry/evals/imo_exact_answers.yaml x: 1 contributors (all time) y: 8 lines of code pyproject.toml x: 19 contributors (all time) y: 64 lines of code evals/data.py x: 10 contributors (all time) y: 148 lines of code evals/completion_fns/solver_completion_fn.py x: 1 contributors (all time) y: 47 lines of code evals/elsuite/bluff/strategy_solver.py x: 3 contributors (all time) y: 88 lines of code evals/elsuite/function_deduction/prompts.py x: 2 contributors (all time) y: 6 lines of code evals/elsuite/function_deduction/solvers.py x: 2 contributors (all time) y: 140 lines of code evals/elsuite/hr_ml_agent_bench/solvers/baseline.py x: 2 contributors (all time) y: 90 lines of code evals/elsuite/make_me_say/eval.py x: 2 contributors (all time) y: 48 lines of code evals/eval.py x: 9 contributors (all time) y: 170 lines of code evals/registry.py x: 17 contributors (all time) y: 242 lines of code evals/registry/solvers/already_said_that.yaml x: 2 contributors (all time) y: 75 lines of code evals/registry/solvers/bluff.yaml x: 2 contributors (all time) y: 80 lines of code evals/registry/solvers/cant_do_that_anymore.yaml x: 1 contributors (all time) y: 16 lines of code evals/registry/solvers/defaults.yaml x: 2 contributors (all time) y: 294 lines of code evals/registry/solvers/error_recovery.yaml x: 1 contributors (all time) y: 33 lines of code evals/registry/solvers/function_deduction.yaml x: 2 contributors (all time) y: 174 lines of code evals/registry/solvers/hr-ml-agent-bench.yaml x: 2 contributors (all time) y: 37 lines of code evals/registry/solvers/incontext_rl.yaml x: 2 contributors (all time) y: 24 lines of code evals/registry/solvers/make-me-pay.yaml x: 2 contributors (all time) y: 101 lines of code evals/registry/solvers/self_prompting.yaml x: 2 contributors (all time) y: 96 lines of code evals/registry/solvers/skill_acquisition.yaml x: 2 contributors (all time) y: 267 lines of code evals/registry/solvers/theory_of_mind.yaml x: 2 contributors (all time) y: 394 lines of code evals/solvers/memory.py x: 1 contributors (all time) y: 50 lines of code evals/solvers/nested/cot_solver.py x: 2 contributors (all time) y: 61 lines of code evals/solvers/nested/self_consistency_solver.py x: 2 contributors (all time) y: 118 lines of code evals/solvers/prompts/cot.py x: 2 contributors (all time) y: 4 lines of code evals/solvers/providers/openai/openai_assistants_solver.py x: 1 contributors (all time) y: 186 lines of code evals/solvers/providers/openai/openai_solver.py x: 1 contributors (all time) y: 181 lines of code evals/solvers/providers/together/together_solver.py x: 1 contributors (all time) y: 68 lines of code evals/solvers/solver.py x: 2 contributors (all time) y: 125 lines of code evals/registry/solvers/gemini.yaml x: 1 contributors (all time) y: 15 lines of code evals/solvers/providers/google/gemini_solver.py x: 1 contributors (all time) y: 157 lines of code evals/completion_fns/openai.py x: 6 contributors (all time) y: 147 lines of code evals/utils/api_utils.py x: 8 contributors (all time) y: 15 lines of code evals/registry/solvers/anthropic.yaml x: 1 contributors (all time) y: 90 lines of code evals/elsuite/hr_ml_agent_bench/autoeval.py x: 1 contributors (all time) y: 172 lines of code evals/elsuite/hr_ml_agent_bench/benchmarks/ant/baselines/naive.py x: 1 contributors (all time) y: 28 lines of code evals/elsuite/hr_ml_agent_bench/benchmarks/bipedal_walker/baselines/human.py x: 1 contributors (all time) y: 57 lines of code evals/elsuite/hr_ml_agent_bench/benchmarks/cartpole/scripts/grade.py x: 1 contributors (all time) y: 40 lines of code evals/elsuite/hr_ml_agent_bench/benchmarks/cifar10/env/train.py x: 1 contributors (all time) y: 105 lines of code evals/elsuite/hr_ml_agent_bench/benchmarks/feedback/scripts/prepare.py x: 1 contributors (all time) y: 24 lines of code evals/elsuite/hr_ml_agent_bench/benchmarks/ogbn_arxiv/env/train.py x: 1 contributors (all time) y: 126 lines of code evals/elsuite/hr_ml_agent_bench/benchmarks/parkinsons_disease/scripts/prepare.py x: 1 contributors (all time) y: 98 lines of code evals/elsuite/hr_ml_agent_bench/benchmarks/vectorization/env/train.py x: 1 contributors (all time) y: 118 lines of code evals/elsuite/hr_ml_agent_bench/benchmarks/vectorization/scripts/human_baseline.py x: 1 contributors (all time) y: 83 lines of code evals/elsuite/hr_ml_agent_bench/environment.py x: 1 contributors (all time) y: 283 lines of code evals/elsuite/hr_ml_agent_bench/eval.py x: 1 contributors (all time) y: 91 lines of code evals/elsuite/hr_ml_agent_bench/low_level_actions.py x: 1 contributors (all time) y: 304 lines of code evals/elsuite/hr_ml_agent_bench/scripts/run_experiments.py x: 1 contributors (all time) y: 60 lines of code evals/elsuite/hr_ml_agent_bench/utils.py x: 1 contributors (all time) y: 112 lines of code evals/registry/evals/hr-ml-agent-bench.yaml x: 1 contributors (all time) y: 137 lines of code evals/elsuite/multistep_web_tasks/constants.py x: 1 contributors (all time) y: 70 lines of code evals/elsuite/multistep_web_tasks/docker/flask-playwright/app.py x: 1 contributors (all time) y: 165 lines of code evals/elsuite/multistep_web_tasks/docker/homepage/templates/index.html x: 1 contributors (all time) y: 108 lines of code evals/elsuite/multistep_web_tasks/session.py x: 1 contributors (all time) y: 416 lines of code evals/elsuite/multistep_web_tasks/webarena/bash_env/basic_bash_env.py x: 1 contributors (all time) y: 163 lines of code evals/elsuite/multistep_web_tasks/webarena/browser_env/actions.py x: 1 contributors (all time) y: 1014 lines of code evals/elsuite/multistep_web_tasks/webarena/browser_env/browser_utils.py x: 1 contributors (all time) y: 78 lines of code evals/elsuite/multistep_web_tasks/webarena/browser_env/helper_functions.py x: 1 contributors (all time) y: 129 lines of code evals/elsuite/multistep_web_tasks/webarena/browser_env/processors.py x: 1 contributors (all time) y: 495 lines of code evals/elsuite/multistep_web_tasks/webarena/core/playwright_api.py x: 1 contributors (all time) y: 279 lines of code evals/elsuite/multistep_web_tasks/webarena/evaluation_harness/evaluators.py x: 1 contributors (all time) y: 273 lines of code evals/elsuite/incontext_rl/eval.py x: 1 contributors (all time) y: 246 lines of code evals/elsuite/incontext_rl/scripts/plot_experiments.py x: 1 contributors (all time) y: 233 lines of code evals/elsuite/function_deduction/eval.py x: 1 contributors (all time) y: 244 lines of code evals/elsuite/function_deduction/scripts/make_plots.py x: 1 contributors (all time) y: 195 lines of code evals/elsuite/identifying_variables/eval.py x: 1 contributors (all time) y: 227 lines of code evals/elsuite/identifying_variables/renderers/corrset.py x: 1 contributors (all time) y: 216 lines of code evals/elsuite/identifying_variables/scripts/gen_data.py x: 1 contributors (all time) y: 319 lines of code evals/elsuite/identifying_variables/scripts/make_plots.py x: 1 contributors (all time) y: 325 lines of code evals/elsuite/track_the_stat/scripts/make_plots.py x: 1 contributors (all time) y: 235 lines of code evals/elsuite/already_said_that/scripts/make_plots.py x: 1 contributors (all time) y: 263 lines of code evals/elsuite/skill_acquisition/eval.py x: 1 contributors (all time) y: 313 lines of code evals/elsuite/error_recovery/eval.py x: 1 contributors (all time) y: 204 lines of code evals/elsuite/error_recovery/scripts/make_plots.py x: 1 contributors (all time) y: 446 lines of code evals/elsuite/cant_do_that_anymore/utils.py x: 1 contributors (all time) y: 178 lines of code evals/elsuite/bugged_tools/eval.py x: 1 contributors (all time) y: 210 lines of code evals/elsuite/basic/match_with_solvers.py x: 2 contributors (all time) y: 65 lines of code evals/elsuite/make_me_pay/task_description.py x: 2 contributors (all time) y: 57 lines of code evals/elsuite/make_me_say/core.py x: 2 contributors (all time) y: 223 lines of code evals/elsuite/steganography/scripts/dataset/custom_datasets.py x: 1 contributors (all time) y: 197 lines of code evals/prompt/base.py x: 3 contributors (all time) y: 64 lines of code evals/registry/evals/ballots.yaml x: 3 contributors (all time) y: 34 lines of code evals/registry/evals/self_prompting.yaml x: 2 contributors (all time) y: 19 lines of code evals/solvers/human_cli_solver.py x: 3 contributors (all time) y: 29 lines of code evals/solvers/nested/fewshot_solver.py x: 2 contributors (all time) y: 91 lines of code evals/solvers/prompts/hhh.py x: 3 contributors (all time) y: 99 lines of code evals/elsuite/sandbagging/solvers.py x: 3 contributors (all time) y: 152 lines of code evals/record.py x: 9 contributors (all time) y: 450 lines of code evals/cli/oaievalset.py x: 4 contributors (all time) y: 111 lines of code evals/elsuite/ballots/prompts.py x: 2 contributors (all time) y: 44 lines of code evals/elsuite/sandbagging/defaults.py x: 2 contributors (all time) y: 12 lines of code evals/registry/data/word_association/corpus_tools/validators.py x: 4 contributors (all time) y: 151 lines of code scripts/battle_generator.py x: 5 contributors (all time) y: 49 lines of code scripts/modelgraded_generator.py x: 5 contributors (all time) y: 185 lines of code scripts/pattern_identification_generator.py x: 6 contributors (all time) y: 48 lines of code evals/completion_fns/langchain_llm.py x: 4 contributors (all time) y: 70 lines of code evals/registry/completion_fns/langchain_llms.yaml x: 3 contributors (all time) y: 24 lines of code evals/registry/evals/mmmu.yaml x: 1 contributors (all time) y: 390 lines of code evals/elsuite/make_me_say/utils.py x: 5 contributors (all time) y: 34 lines of code evals/elsuite/ballots/eval.py x: 2 contributors (all time) y: 161 lines of code evals/elsuite/schelling_point/eval.py x: 2 contributors (all time) y: 73 lines of code evals/completion_fns/langchain_math.py x: 4 contributors (all time) y: 21 lines of code evals/elsuite/make_me_pay/utils.py x: 3 contributors (all time) y: 47 lines of code evals/elsuite/make_me_say/autoeval.py x: 4 contributors (all time) y: 116 lines of code evals/elsuite/lambada.py x: 2 contributors (all time) y: 39 lines of code evals/elsuite/modelgraded/classify.py x: 7 contributors (all time) y: 97 lines of code evals/elsuite/utils.py x: 6 contributors (all time) y: 150 lines of code evals/metrics.py x: 3 contributors (all time) y: 52 lines of code evals/registry/data/mapping_to_matricies/data_generator.py x: 2 contributors (all time) y: 30 lines of code evals/registry/data/simple_physics_engine/wave_function_collapse.py x: 2 contributors (all time) y: 157 lines of code evals/registry/data/solve-for-variable/tools/solve.py x: 2 contributors (all time) y: 231 lines of code evals/registry/data/word_association/corpus_tools/sample_generators.py x: 2 contributors (all time) y: 154 lines of code evals/base.py x: 4 contributors (all time) y: 51 lines of code evals/elsuite/basic/match.py x: 4 contributors (all time) y: 57 lines of code evals/registry/evals/test-modelgraded.yaml x: 3 contributors (all time) y: 81 lines of code evals/elsuite/modelgraded/classify_utils.py x: 2 contributors (all time) y: 145 lines of code evals/registry/data/backgammon/generate_samples.ipynb x: 1 contributors (all time) y: 1349 lines of code evals/api.py x: 4 contributors (all time) y: 61 lines of code evals/registry/evals/mmlu.yaml x: 1 contributors (all time) y: 399 lines of code evals/registry/eval_sets/test-all.yaml x: 3 contributors (all time) y: 21 lines of code evals/registry/evals/coqa-ex.yaml x: 3 contributors (all time) y: 55 lines of code MANIFEST.in x: 3 contributors (all time) y: 4 lines of code
1349.0
lines of code
  min: 1.0
  average: 28.01
  25th percentile: 3.0
  median: 3.0
  75th percentile: 16.0
  max: 1349.0
0 19.0
contributors (all time)
min: 1.0 | average: 1.19 | 25th percentile: 1.0 | median: 1.0 | 75th percentile: 1.0 | max: 19.0

File Size vs. Commits (30 days): 0 points

No data for "commits (30d)" vs. "lines of code".

File Size vs. Contributors (30 days): 0 points

No data for "contributors (30d)" vs. "lines of code".

File Size vs. Commits (90 days): 0 points

No data for "commits (90d)" vs. "lines of code".

File Size vs. Contributors (90 days): 0 points

No data for "contributors (90d)" vs. "lines of code".