openai / evals
File Size

The distribution of size of files (measured in lines of code).

Intro
  • File size measurements show the distribution of size of files.
  • Files are classified in four categories based on their size (lines of code): 1-100 (very small files), 101-200 (small files), 201-500 (medium size files), 501-1000 (long files), 1001+(very long files).
  • It is a good practice to keep files small. Long files may become "bloaters", code that have increased to such gargantuan proportions that they are hard to work with.
Learn more...
File Size Overall
  • There are 1,492 files with 41,212 lines of code.
    • 2 very long files (2,363 lines of code)
    • 0 long files (0 lines of code)
    • 39 medium size files (11,307 lines of code)
    • 75 small files (10,548 lines of code)
    • 1,376 very small files (16,994 lines of code)
5% | 0% | 27% | 25% | 41%
Legend:
1001+
501-1000
201-500
101-200
1-100


explore: grouped by folders | grouped by size | sunburst | 3D view
File Size per Extension
1001+
501-1000
201-500
101-200
1-100
ipynb100% | 0% | 0% | 0% | 0%
py3% | 0% | 31% | 31% | 32%
yaml0% | 0% | 24% | 11% | 64%
html0% | 0% | 0% | 100% | 0%
jsonl0% | 0% | 0% | 0% | 100%
js0% | 0% | 0% | 0% | 100%
in0% | 0% | 0% | 0% | 100%
File Size per Logical Decomposition
primary
1001+
501-1000
201-500
101-200
1-100
evals5% | 0% | 27% | 25% | 41%
scripts0% | 0% | 0% | 65% | 34%
ROOT0% | 0% | 0% | 0% | 100%
.github0% | 0% | 0% | 0% | 100%
Longest Files (Top 50)
File# lines# units
generate_samples.ipynb
in evals/registry/data/backgammon
1349 -
actions.py
in evals/elsuite/multistep_web_tasks/webarena/browser_env
1014 50
tools.py
in evals/elsuite/bugged_tools
497 37
processors.py
in evals/elsuite/multistep_web_tasks/webarena/browser_env
495 21
record.py
in evals
450 54
make_plots.py
in evals/elsuite/error_recovery/scripts
446 20
session.py
in evals/elsuite/multistep_web_tasks
416 28
mmlu.yaml
in evals/registry/evals
399 -
theory_of_mind.yaml
in evals/registry/solvers
394 -
mmmu.yaml
in evals/registry/evals
390 -
make_plots.py
in evals/elsuite/identifying_variables/scripts
325 17
gen_data.py
in evals/elsuite/identifying_variables/scripts
319 13
eval.py
in evals/elsuite/skill_acquisition
313 7
plot_experiments.py
in evals/elsuite/hr_ml_agent_bench/scripts
307 -
low_level_actions.py
in evals/elsuite/hr_ml_agent_bench
304 13
defaults.yaml
in evals/registry/solvers
294 -
environment.py
in evals/elsuite/hr_ml_agent_bench
283 21
constants.py
in evals/elsuite/multistep_web_tasks/webarena/browser_env
282 -
playwright_api.py
in evals/elsuite/multistep_web_tasks/webarena/core
279 32
eval_run.py
in evals/elsuite/multistep_web_tasks/webarena
277 14
evaluators.py
in evals/elsuite/multistep_web_tasks/webarena/evaluation_harness
273 19
skill_acquisition.yaml
in evals/registry/solvers
267 -
make_plots.py
in evals/elsuite/already_said_that/scripts
263 8
oaieval.py
in evals/cli
253 6
eval.py
in evals/elsuite/function_deduction
244 14
registry.py
in evals
242 27
make_plots.py
in evals/elsuite/track_the_stat/scripts
235 9
dataset_creation.py
in evals/elsuite/cant_do_that_anymore/scripts
235 8
make_plots.py
in evals/elsuite/ballots/scripts
233 12
solve.py
in evals/registry/data/solve-for-variable/tools
231 30
eval.py
in evals/elsuite/identifying_variables
227 14
raven-matrices.yaml
in evals/registry/evals
224 -
core.py
in evals/elsuite/make_me_say
223 18
makemepay.py
in evals/elsuite/make_me_pay
222 8
corrset.py
in evals/elsuite/identifying_variables/renderers
216 11
diagonal_dataset_creation.py
in evals/elsuite/cant_do_that_anymore/scripts
216 6
eval.py
in evals/elsuite/bugged_tools
210 9
eval.py
in evals/elsuite/self_prompting
210 7
cards.py
in evals/elsuite/bluff/bluff
206 37
eval.py
in evals/elsuite/error_recovery
204 9
pieces.py
in evals/elsuite/cant_do_that_anymore/chess
203 9
custom_datasets.py
in evals/elsuite/steganography/scripts/dataset
197 12
make_plots.py
in evals/elsuite/function_deduction/scripts
195 4
basic_browser_env.py
in evals/elsuite/multistep_web_tasks/webarena/browser_env
191 11
high_level_actions.py
in evals/elsuite/hr_ml_agent_bench
191 4
utils.py
in evals/elsuite/multistep_web_tasks/webarena/core
188 7
openai_assistants_solver.py
in evals/solvers/providers/openai
186 11
185 1
solver_tools_convo.py
in evals/elsuite
181 11
openai_solver.py
in evals/solvers/providers/openai
181 18
Files With Most Units (Top 50)
File# lines# units
record.py
in evals
450 54
actions.py
in evals/elsuite/multistep_web_tasks/webarena/browser_env
1014 50
tools.py
in evals/elsuite/bugged_tools
497 37
cards.py
in evals/elsuite/bluff/bluff
206 37
playwright_api.py
in evals/elsuite/multistep_web_tasks/webarena/core
279 32
solve.py
in evals/registry/data/solve-for-variable/tools
231 30
session.py
in evals/elsuite/multistep_web_tasks
416 28
registry.py
in evals
242 27
processors.py
in evals/elsuite/multistep_web_tasks/webarena/browser_env
495 21
environment.py
in evals/elsuite/hr_ml_agent_bench
283 21
data.py
in evals
148 21
make_plots.py
in evals/elsuite/error_recovery/scripts
446 20
evaluators.py
in evals/elsuite/multistep_web_tasks/webarena/evaluation_harness
273 19
core.py
in evals/elsuite/make_me_say
223 18
openai_solver.py
in evals/solvers/providers/openai
181 18
make_plots.py
in evals/elsuite/identifying_variables/scripts
325 17
solver.py
in evals/solvers
125 17
wave_function_collapse.py
in evals/registry/data/simple_physics_engine
157 17
solvers.py
in evals/elsuite/sandbagging
152 16
utils.py
in evals/elsuite/skill_acquisition
115 15
eval.py
in evals
170 15
corpus.py
in evals/registry/data/word_association/corpus_tools
58 15
eval.py
in evals/elsuite/identifying_variables
227 14
tabular.py
in evals/elsuite/identifying_variables/renderers
125 14
eval_run.py
in evals/elsuite/multistep_web_tasks/webarena
277 14
eval.py
in evals/elsuite/function_deduction
244 14
gen_data.py
in evals/elsuite/identifying_variables/scripts
319 13
players.py
in evals/elsuite/bluff/bluff
107 13
solvers.py
in evals/elsuite/track_the_stat
72 13
basic_bash_env.py
in evals/elsuite/multistep_web_tasks/webarena/bash_env
163 13
low_level_actions.py
in evals/elsuite/hr_ml_agent_bench
304 13
utils.py
in evals/elsuite
150 13
board.py
in evals/elsuite/cant_do_that_anymore/chess
162 13
related_words.py
in evals/registry/data/word_association/corpus_tools
64 13
make_plots.py
in evals/elsuite/ballots/scripts
233 12
custom_datasets.py
in evals/elsuite/steganography/scripts/dataset
197 12
strong_solver.py
in evals/elsuite/multistep_web_tasks/solvers/strong_solver
173 12
utils.py
in evals/elsuite/cant_do_that_anymore
178 12
corrset.py
in evals/elsuite/identifying_variables/renderers
216 11
basic_browser_env.py
in evals/elsuite/multistep_web_tasks/webarena/browser_env
191 11
solver_tools_convo.py
in evals/elsuite
181 11
openai_assistants_solver.py
in evals/solvers/providers/openai
186 11
validators.py
in evals/registry/data/word_association/corpus_tools
151 11
baselines.py
in evals/elsuite/function_deduction
91 10
openai.py
in evals/completion_fns
147 10
together_solver.py
in evals/solvers/providers/together
68 10
eval.py
in evals/elsuite/bugged_tools
210 9
eval.py
in evals/elsuite/error_recovery
204 9
eval.py
in evals/elsuite/bluff
164 9
make_plots.py
in evals/elsuite/track_the_stat/scripts
235 9
Files With Long Lines (Top 50)

There are 128 files with lines longer than 120 characters. In total, there are 321 long lines.

File# lines# units# long lines
raven-matrices.yaml
in evals/registry/evals
224 - 28
tools.py
in evals/elsuite/bugged_tools
497 37 26
hr-ml-agent-bench.yaml
in evals/registry/evals
137 - 16
generate_samples.ipynb
in evals/registry/data/backgammon
1349 - 15
high_level_actions.py
in evals/elsuite/hr_ml_agent_bench
191 4 12
theory_of_mind.yaml
in evals/registry/evals
48 - 8
csv_to_json.py
in evals/registry/data/canto_wu_pronunciation
55 - 8
prompts.py
in evals/elsuite/ballots
44 - 7
oaieval.py
in evals/cli
253 6 7
task_description.py
in evals/elsuite/make_me_pay
57 - 6
utils.py
in evals/elsuite/twenty_questions
47 4 6
prompts.py
in evals/elsuite/make_me_pay/solvers
18 - 5
helper_functions.py
in evals/elsuite/multistep_web_tasks/webarena/browser_env
129 5 5
low_level_actions.py
in evals/elsuite/hr_ml_agent_bench
304 13 5
defaults.py
in evals/elsuite/make_me_say
34 5 5
singlestore.yaml
in evals/registry/modelgraded
24 - 5
session.py
in evals/elsuite/multistep_web_tasks
416 28 4
fewshot_solver.py
in evals/solvers/nested
91 5 4
product-ie.yaml
in evals/registry/evals
28 - 4
sql.yaml
in evals/registry/modelgraded
24 - 4
data_generation.py
in evals/elsuite/theory_of_mind/scripts
66 1 3
defaults.py
in evals/elsuite/error_recovery
12 - 3
utils.py
in evals/elsuite/skill_acquisition
115 15 3
eval.py
in evals/elsuite/self_prompting
210 7 3
solvers.py
in evals/elsuite/function_deduction
140 9 3
environment.py
in evals/elsuite/hr_ml_agent_bench
283 21 3
research-question-extraction.yaml
in evals/registry/modelgraded
19 - 3
self_prompting.yaml
in evals/registry/solvers
96 - 3
eval.py
in evals/elsuite/make_me_pay
126 3 2
solvers.py
in evals/elsuite/identifying_variables
27 4 2
task_description.py
in evals/elsuite/bugged_tools
9 - 2
eval.py
in evals/elsuite/skill_acquisition
313 7 2
actions.py
in evals/elsuite/multistep_web_tasks/webarena/browser_env
1014 50 2
classify_utils.py
in evals/elsuite/modelgraded
145 8 2
eval.py
in evals/elsuite/mmmu
159 4 2
openai.py
in evals/completion_fns
147 10 2
self_consistency_solver.py
in evals/solvers/nested
118 6 2
abstract-causal-reasoning.yaml
in evals/registry/evals
16 - 2
fact.yaml
in evals/registry/modelgraded
19 - 2
keywords.yaml
in evals/registry/modelgraded
20 - 2
arithmetic-expression.yaml
in evals/registry/modelgraded
24 - 2
translation.yaml
in evals/registry/modelgraded
19 - 2
convert.js
in evals/registry/data/medmcqa
44 1 2
convert.js
in evals/registry/data/unsolvable_questions
51 1 2
findFailures.js
in evals/registry/data/unsolvable_questions
43 1 2
make_plots.py
in evals/elsuite/error_recovery/scripts
446 20 1
eval.py
in evals/elsuite/bluff
164 9 1
solver_player.py
in evals/elsuite/bluff
82 7 1
prompts.py
in evals/elsuite/bluff
6 - 1
eval.py
in evals/elsuite/steganography
69 3 1
Correlations

File Size vs. Commits (all time): 1489 points

.pre-commit-config.yaml x: 3 commits (all time) y: 32 lines of code evals/cli/oaieval.py x: 22 commits (all time) y: 253 lines of code evals/registry/data/imo_exact_answers/samples.jsonl x: 1 commits (all time) y: 3 lines of code evals/registry/evals/imo_exact_answers.yaml x: 1 commits (all time) y: 8 lines of code evals/data.py x: 10 commits (all time) y: 148 lines of code evals/completion_fns/solver_completion_fn.py x: 1 commits (all time) y: 47 lines of code evals/elsuite/bluff/strategy_solver.py x: 3 commits (all time) y: 88 lines of code evals/elsuite/bugged_tools/task_description.py x: 2 commits (all time) y: 9 lines of code evals/elsuite/function_deduction/solvers.py x: 2 commits (all time) y: 140 lines of code evals/elsuite/hr_ml_agent_bench/solvers/baseline.py x: 2 commits (all time) y: 90 lines of code evals/elsuite/make_me_say/eval.py x: 3 commits (all time) y: 48 lines of code evals/eval.py x: 10 commits (all time) y: 170 lines of code evals/registry.py x: 24 commits (all time) y: 242 lines of code evals/registry/solvers/already_said_that.yaml x: 2 commits (all time) y: 75 lines of code evals/registry/solvers/bluff.yaml x: 2 commits (all time) y: 80 lines of code evals/registry/solvers/cant_do_that_anymore.yaml x: 2 commits (all time) y: 16 lines of code evals/registry/solvers/defaults.yaml x: 3 commits (all time) y: 294 lines of code evals/registry/solvers/error_recovery.yaml x: 2 commits (all time) y: 33 lines of code evals/registry/solvers/function_deduction.yaml x: 2 commits (all time) y: 174 lines of code evals/registry/solvers/make-me-pay.yaml x: 2 commits (all time) y: 101 lines of code evals/registry/solvers/self_prompting.yaml x: 2 commits (all time) y: 96 lines of code evals/registry/solvers/skill_acquisition.yaml x: 2 commits (all time) y: 267 lines of code evals/registry/solvers/theory_of_mind.yaml x: 3 commits (all time) y: 394 lines of code evals/solvers/memory.py x: 1 commits (all time) y: 50 lines of code evals/solvers/nested/cot_solver.py x: 3 commits (all time) y: 61 lines of code evals/solvers/nested/self_consistency_solver.py x: 3 commits (all time) y: 118 lines of code evals/solvers/prompts/cot.py x: 2 commits (all time) y: 4 lines of code evals/solvers/providers/openai/openai_assistants_solver.py x: 1 commits (all time) y: 186 lines of code evals/solvers/providers/openai/openai_solver.py x: 1 commits (all time) y: 181 lines of code evals/solvers/providers/together/together_solver.py x: 1 commits (all time) y: 68 lines of code evals/solvers/solver.py x: 4 commits (all time) y: 125 lines of code evals/solvers/utils.py x: 4 commits (all time) y: 37 lines of code evals/registry/solvers/gemini.yaml x: 1 commits (all time) y: 15 lines of code evals/solvers/providers/google/gemini_solver.py x: 1 commits (all time) y: 157 lines of code evals/completion_fns/openai.py x: 6 commits (all time) y: 147 lines of code evals/utils/api_utils.py x: 11 commits (all time) y: 15 lines of code evals/registry/solvers/anthropic.yaml x: 1 commits (all time) y: 90 lines of code evals/elsuite/hr_ml_agent_bench/actions.py x: 1 commits (all time) y: 37 lines of code evals/elsuite/hr_ml_agent_bench/autoeval.py x: 1 commits (all time) y: 172 lines of code evals/elsuite/hr_ml_agent_bench/benchmarks/ant/baselines/naive.py x: 1 commits (all time) y: 28 lines of code evals/elsuite/hr_ml_agent_bench/benchmarks/bipedal_walker/baselines/human.py x: 1 commits (all time) y: 57 lines of code evals/elsuite/hr_ml_agent_bench/benchmarks/cartpole/scripts/grade.py x: 1 commits (all time) y: 40 lines of code evals/elsuite/hr_ml_agent_bench/benchmarks/cifar10/env/train.py x: 1 commits (all time) y: 105 lines of code evals/elsuite/hr_ml_agent_bench/benchmarks/feedback/scripts/prepare.py x: 1 commits (all time) y: 24 lines of code evals/elsuite/hr_ml_agent_bench/benchmarks/house_price/scripts/prepare.py x: 1 commits (all time) y: 20 lines of code evals/elsuite/hr_ml_agent_bench/benchmarks/ogbn_arxiv/env/train.py x: 1 commits (all time) y: 126 lines of code evals/elsuite/hr_ml_agent_bench/benchmarks/parkinsons_disease/scripts/prepare.py x: 1 commits (all time) y: 98 lines of code evals/elsuite/hr_ml_agent_bench/benchmarks/vectorization/env/train.py x: 1 commits (all time) y: 118 lines of code evals/elsuite/hr_ml_agent_bench/benchmarks/vectorization/scripts/human_baseline.py x: 1 commits (all time) y: 83 lines of code evals/elsuite/hr_ml_agent_bench/environment.py x: 1 commits (all time) y: 283 lines of code evals/elsuite/hr_ml_agent_bench/eval.py x: 1 commits (all time) y: 91 lines of code evals/elsuite/hr_ml_agent_bench/low_level_actions.py x: 1 commits (all time) y: 304 lines of code evals/elsuite/hr_ml_agent_bench/scripts/run_experiments.py x: 1 commits (all time) y: 60 lines of code evals/elsuite/hr_ml_agent_bench/utils.py x: 1 commits (all time) y: 112 lines of code evals/registry/evals/hr-ml-agent-bench.yaml x: 1 commits (all time) y: 137 lines of code evals/elsuite/multistep_web_tasks/constants.py x: 1 commits (all time) y: 70 lines of code evals/elsuite/multistep_web_tasks/docker/flask-playwright/app.py x: 1 commits (all time) y: 165 lines of code evals/elsuite/multistep_web_tasks/docker/homepage/templates/index.html x: 1 commits (all time) y: 108 lines of code evals/elsuite/multistep_web_tasks/session.py x: 1 commits (all time) y: 416 lines of code evals/elsuite/multistep_web_tasks/webarena/bash_env/basic_bash_env.py x: 1 commits (all time) y: 163 lines of code evals/elsuite/multistep_web_tasks/webarena/browser_env/actions.py x: 1 commits (all time) y: 1014 lines of code evals/elsuite/multistep_web_tasks/webarena/browser_env/browser_utils.py x: 1 commits (all time) y: 78 lines of code evals/elsuite/multistep_web_tasks/webarena/browser_env/helper_functions.py x: 1 commits (all time) y: 129 lines of code evals/elsuite/multistep_web_tasks/webarena/browser_env/processors.py x: 1 commits (all time) y: 495 lines of code evals/elsuite/multistep_web_tasks/webarena/core/playwright_api.py x: 1 commits (all time) y: 279 lines of code evals/elsuite/multistep_web_tasks/webarena/evaluation_harness/evaluators.py x: 1 commits (all time) y: 273 lines of code evals/elsuite/function_deduction/eval.py x: 1 commits (all time) y: 244 lines of code evals/elsuite/function_deduction/scripts/make_plots.py x: 1 commits (all time) y: 195 lines of code evals/elsuite/identifying_variables/eval.py x: 1 commits (all time) y: 227 lines of code evals/elsuite/identifying_variables/renderers/corrset.py x: 1 commits (all time) y: 216 lines of code evals/elsuite/identifying_variables/scripts/gen_data.py x: 1 commits (all time) y: 319 lines of code evals/elsuite/identifying_variables/scripts/make_plots.py x: 1 commits (all time) y: 325 lines of code evals/elsuite/track_the_stat/scripts/make_plots.py x: 1 commits (all time) y: 235 lines of code evals/elsuite/already_said_that/scripts/make_plots.py x: 1 commits (all time) y: 263 lines of code evals/utils/log_utils.py x: 2 commits (all time) y: 53 lines of code evals/elsuite/skill_acquisition/eval.py x: 1 commits (all time) y: 313 lines of code evals/elsuite/error_recovery/eval.py x: 1 commits (all time) y: 204 lines of code evals/elsuite/error_recovery/scripts/make_plots.py x: 1 commits (all time) y: 446 lines of code evals/elsuite/cant_do_that_anymore/utils.py x: 1 commits (all time) y: 178 lines of code evals/elsuite/bugged_tools/eval.py x: 1 commits (all time) y: 210 lines of code evals/elsuite/ballots/scripts/make_plots.py x: 1 commits (all time) y: 233 lines of code evals/elsuite/basic/match_with_solvers.py x: 2 commits (all time) y: 65 lines of code evals/elsuite/make_me_pay/eval.py x: 3 commits (all time) y: 126 lines of code evals/elsuite/make_me_pay/task_description.py x: 3 commits (all time) y: 57 lines of code evals/elsuite/make_me_say/core.py x: 3 commits (all time) y: 223 lines of code evals/elsuite/steganography/scripts/dataset/custom_datasets.py x: 1 commits (all time) y: 197 lines of code evals/prompt/base.py x: 4 commits (all time) y: 64 lines of code evals/registry/evals/schelling_point.yaml x: 3 commits (all time) y: 66 lines of code evals/solvers/human_cli_solver.py x: 4 commits (all time) y: 29 lines of code evals/solvers/nested/fewshot_solver.py x: 2 commits (all time) y: 91 lines of code evals/solvers/prompts/hhh.py x: 4 commits (all time) y: 99 lines of code evals/elsuite/sandbagging/solvers.py x: 3 commits (all time) y: 152 lines of code evals/elsuite/self_prompting/solvers/baselines.py x: 2 commits (all time) y: 45 lines of code evals/elsuite/self_prompting/solvers/custom_cot_solver.py x: 2 commits (all time) y: 57 lines of code evals/record.py x: 15 commits (all time) y: 450 lines of code evals/cli/oaievalset.py x: 5 commits (all time) y: 111 lines of code evals/elsuite/ballots/prompts.py x: 3 commits (all time) y: 44 lines of code evals/elsuite/sandbagging/defaults.py x: 2 commits (all time) y: 12 lines of code evals/elsuite/schelling_point/prompts.py x: 3 commits (all time) y: 25 lines of code evals/elsuite/schelling_point/utils.py x: 2 commits (all time) y: 61 lines of code evals/registry/data/word_association/corpus_tools/validators.py x: 4 commits (all time) y: 151 lines of code scripts/battle_generator.py x: 7 commits (all time) y: 49 lines of code scripts/modelgraded_generator.py x: 9 commits (all time) y: 185 lines of code scripts/pattern_identification_generator.py x: 6 commits (all time) y: 48 lines of code evals/elsuite/mmmu/eval.py x: 3 commits (all time) y: 159 lines of code evals/completion_fns/langchain_llm.py x: 4 commits (all time) y: 70 lines of code evals/registry/evals/mmmu.yaml x: 1 commits (all time) y: 390 lines of code evals/elsuite/make_me_say/utils.py x: 6 commits (all time) y: 34 lines of code evals/elsuite/ballots/eval.py x: 3 commits (all time) y: 161 lines of code evals/elsuite/schelling_point/eval.py x: 2 commits (all time) y: 73 lines of code evals/elsuite/bluff/eval.py x: 2 commits (all time) y: 164 lines of code evals/completion_fns/langchain_math.py x: 4 commits (all time) y: 21 lines of code evals/elsuite/make_me_say/autoeval.py x: 4 commits (all time) y: 116 lines of code evals/elsuite/make_me_pay/makemepay.py x: 2 commits (all time) y: 222 lines of code .github/workflows/parse_yaml.py x: 3 commits (all time) y: 10 lines of code evals/elsuite/lambada.py x: 2 commits (all time) y: 39 lines of code evals/elsuite/modelgraded/classify.py x: 27 commits (all time) y: 97 lines of code evals/elsuite/utils.py x: 10 commits (all time) y: 150 lines of code evals/registry/data/mapping_to_matricies/data_generator.py x: 2 commits (all time) y: 30 lines of code evals/registry/data/simple_physics_engine/wave_function_collapse.py x: 2 commits (all time) y: 157 lines of code evals/registry/data/solve-for-variable/tools/solve.py x: 2 commits (all time) y: 231 lines of code evals/registry/data/word_association/corpus_tools/sample_generators.py x: 2 commits (all time) y: 154 lines of code evals/base.py x: 5 commits (all time) y: 51 lines of code evals/elsuite/basic/match.py x: 5 commits (all time) y: 57 lines of code evals/registry/evals/test-modelgraded.yaml x: 7 commits (all time) y: 81 lines of code evals/registry/modelgraded/best.yaml x: 4 commits (all time) y: 10 lines of code evals/elsuite/modelgraded/classify_utils.py x: 5 commits (all time) y: 145 lines of code evals/registry/data/backgammon/generate_samples.ipynb x: 1 commits (all time) y: 1349 lines of code evals/elsuite/basic/json_validator.py x: 4 commits (all time) y: 41 lines of code evals/api.py x: 5 commits (all time) y: 61 lines of code evals/registry/data/balance_chemical_equation/samples.jsonl x: 3 commits (all time) y: 3 lines of code evals/registry/evals/mmlu.yaml x: 1 commits (all time) y: 399 lines of code evals/registry/eval_sets/test-all.yaml x: 8 commits (all time) y: 21 lines of code MANIFEST.in x: 4 commits (all time) y: 4 lines of code evals/registry/eval_sets/test-modelgraded.yaml x: 4 commits (all time) y: 14 lines of code
1349.0
lines of code
  min: 1.0
  average: 27.66
  25th percentile: 3.0
  median: 3.0
  75th percentile: 16.0
  max: 1349.0
0 27.0
commits (all time)
min: 1.0 | average: 1.27 | 25th percentile: 1.0 | median: 1.0 | 75th percentile: 1.0 | max: 27.0

File Size vs. Contributors (all time): 1489 points

.pre-commit-config.yaml x: 3 contributors (all time) y: 32 lines of code evals/cli/oaieval.py x: 18 contributors (all time) y: 253 lines of code evals/registry/data/imo_exact_answers/samples.jsonl x: 1 contributors (all time) y: 3 lines of code evals/registry/evals/imo_exact_answers.yaml x: 1 contributors (all time) y: 8 lines of code evals/data.py x: 10 contributors (all time) y: 148 lines of code evals/completion_fns/solver_completion_fn.py x: 1 contributors (all time) y: 47 lines of code evals/elsuite/bluff/strategy_solver.py x: 3 contributors (all time) y: 88 lines of code evals/elsuite/function_deduction/prompts.py x: 2 contributors (all time) y: 6 lines of code evals/elsuite/function_deduction/solvers.py x: 2 contributors (all time) y: 140 lines of code evals/elsuite/hr_ml_agent_bench/solvers/baseline.py x: 2 contributors (all time) y: 90 lines of code evals/elsuite/make_me_say/eval.py x: 2 contributors (all time) y: 48 lines of code evals/eval.py x: 9 contributors (all time) y: 170 lines of code evals/registry.py x: 17 contributors (all time) y: 242 lines of code evals/registry/solvers/already_said_that.yaml x: 2 contributors (all time) y: 75 lines of code evals/registry/solvers/bluff.yaml x: 2 contributors (all time) y: 80 lines of code evals/registry/solvers/cant_do_that_anymore.yaml x: 1 contributors (all time) y: 16 lines of code evals/registry/solvers/defaults.yaml x: 2 contributors (all time) y: 294 lines of code evals/registry/solvers/error_recovery.yaml x: 1 contributors (all time) y: 33 lines of code evals/registry/solvers/function_deduction.yaml x: 2 contributors (all time) y: 174 lines of code evals/registry/solvers/hr-ml-agent-bench.yaml x: 2 contributors (all time) y: 37 lines of code evals/registry/solvers/make-me-pay.yaml x: 2 contributors (all time) y: 101 lines of code evals/registry/solvers/self_prompting.yaml x: 2 contributors (all time) y: 96 lines of code evals/registry/solvers/skill_acquisition.yaml x: 2 contributors (all time) y: 267 lines of code evals/registry/solvers/theory_of_mind.yaml x: 2 contributors (all time) y: 394 lines of code evals/solvers/memory.py x: 1 contributors (all time) y: 50 lines of code evals/solvers/nested/cot_solver.py x: 2 contributors (all time) y: 61 lines of code evals/solvers/nested/self_consistency_solver.py x: 2 contributors (all time) y: 118 lines of code evals/solvers/prompts/cot.py x: 2 contributors (all time) y: 4 lines of code evals/solvers/providers/openai/openai_assistants_solver.py x: 1 contributors (all time) y: 186 lines of code evals/solvers/providers/openai/openai_solver.py x: 1 contributors (all time) y: 181 lines of code evals/solvers/providers/together/together_solver.py x: 1 contributors (all time) y: 68 lines of code evals/solvers/solver.py x: 2 contributors (all time) y: 125 lines of code evals/registry/solvers/gemini.yaml x: 1 contributors (all time) y: 15 lines of code evals/solvers/providers/google/gemini_solver.py x: 1 contributors (all time) y: 157 lines of code evals/completion_fns/openai.py x: 6 contributors (all time) y: 147 lines of code evals/utils/api_utils.py x: 8 contributors (all time) y: 15 lines of code evals/registry/solvers/anthropic.yaml x: 1 contributors (all time) y: 90 lines of code evals/elsuite/hr_ml_agent_bench/autoeval.py x: 1 contributors (all time) y: 172 lines of code evals/elsuite/hr_ml_agent_bench/benchmarks/ant/baselines/naive.py x: 1 contributors (all time) y: 28 lines of code evals/elsuite/hr_ml_agent_bench/benchmarks/bipedal_walker/baselines/human.py x: 1 contributors (all time) y: 57 lines of code evals/elsuite/hr_ml_agent_bench/benchmarks/cartpole/scripts/grade.py x: 1 contributors (all time) y: 40 lines of code evals/elsuite/hr_ml_agent_bench/benchmarks/cifar10/env/train.py x: 1 contributors (all time) y: 105 lines of code evals/elsuite/hr_ml_agent_bench/benchmarks/feedback/scripts/prepare.py x: 1 contributors (all time) y: 24 lines of code evals/elsuite/hr_ml_agent_bench/benchmarks/ogbn_arxiv/env/train.py x: 1 contributors (all time) y: 126 lines of code evals/elsuite/hr_ml_agent_bench/benchmarks/parkinsons_disease/scripts/prepare.py x: 1 contributors (all time) y: 98 lines of code evals/elsuite/hr_ml_agent_bench/benchmarks/vectorization/env/train.py x: 1 contributors (all time) y: 118 lines of code evals/elsuite/hr_ml_agent_bench/benchmarks/vectorization/scripts/human_baseline.py x: 1 contributors (all time) y: 83 lines of code evals/elsuite/hr_ml_agent_bench/environment.py x: 1 contributors (all time) y: 283 lines of code evals/elsuite/hr_ml_agent_bench/eval.py x: 1 contributors (all time) y: 91 lines of code evals/elsuite/hr_ml_agent_bench/low_level_actions.py x: 1 contributors (all time) y: 304 lines of code evals/elsuite/hr_ml_agent_bench/scripts/run_experiments.py x: 1 contributors (all time) y: 60 lines of code evals/elsuite/hr_ml_agent_bench/utils.py x: 1 contributors (all time) y: 112 lines of code evals/registry/evals/hr-ml-agent-bench.yaml x: 1 contributors (all time) y: 137 lines of code evals/elsuite/multistep_web_tasks/constants.py x: 1 contributors (all time) y: 70 lines of code evals/elsuite/multistep_web_tasks/docker/flask-playwright/app.py x: 1 contributors (all time) y: 165 lines of code evals/elsuite/multistep_web_tasks/docker/homepage/templates/index.html x: 1 contributors (all time) y: 108 lines of code evals/elsuite/multistep_web_tasks/session.py x: 1 contributors (all time) y: 416 lines of code evals/elsuite/multistep_web_tasks/webarena/bash_env/basic_bash_env.py x: 1 contributors (all time) y: 163 lines of code evals/elsuite/multistep_web_tasks/webarena/browser_env/actions.py x: 1 contributors (all time) y: 1014 lines of code evals/elsuite/multistep_web_tasks/webarena/browser_env/browser_utils.py x: 1 contributors (all time) y: 78 lines of code evals/elsuite/multistep_web_tasks/webarena/browser_env/helper_functions.py x: 1 contributors (all time) y: 129 lines of code evals/elsuite/multistep_web_tasks/webarena/browser_env/processors.py x: 1 contributors (all time) y: 495 lines of code evals/elsuite/multistep_web_tasks/webarena/core/playwright_api.py x: 1 contributors (all time) y: 279 lines of code evals/elsuite/multistep_web_tasks/webarena/evaluation_harness/evaluators.py x: 1 contributors (all time) y: 273 lines of code evals/elsuite/function_deduction/eval.py x: 1 contributors (all time) y: 244 lines of code evals/elsuite/function_deduction/scripts/make_plots.py x: 1 contributors (all time) y: 195 lines of code evals/elsuite/identifying_variables/eval.py x: 1 contributors (all time) y: 227 lines of code evals/elsuite/identifying_variables/renderers/corrset.py x: 1 contributors (all time) y: 216 lines of code evals/elsuite/identifying_variables/scripts/gen_data.py x: 1 contributors (all time) y: 319 lines of code evals/elsuite/identifying_variables/scripts/make_plots.py x: 1 contributors (all time) y: 325 lines of code evals/elsuite/track_the_stat/scripts/make_plots.py x: 1 contributors (all time) y: 235 lines of code evals/elsuite/already_said_that/scripts/make_plots.py x: 1 contributors (all time) y: 263 lines of code evals/elsuite/skill_acquisition/eval.py x: 1 contributors (all time) y: 313 lines of code evals/elsuite/error_recovery/eval.py x: 1 contributors (all time) y: 204 lines of code evals/elsuite/error_recovery/scripts/make_plots.py x: 1 contributors (all time) y: 446 lines of code evals/elsuite/cant_do_that_anymore/utils.py x: 1 contributors (all time) y: 178 lines of code evals/elsuite/bugged_tools/eval.py x: 1 contributors (all time) y: 210 lines of code evals/elsuite/ballots/scripts/make_plots.py x: 1 contributors (all time) y: 233 lines of code evals/elsuite/basic/match_with_solvers.py x: 2 contributors (all time) y: 65 lines of code evals/elsuite/make_me_pay/task_description.py x: 2 contributors (all time) y: 57 lines of code evals/elsuite/make_me_say/core.py x: 2 contributors (all time) y: 223 lines of code evals/elsuite/steganography/scripts/dataset/custom_datasets.py x: 1 contributors (all time) y: 197 lines of code evals/prompt/base.py x: 3 contributors (all time) y: 64 lines of code evals/registry/evals/self_prompting.yaml x: 2 contributors (all time) y: 19 lines of code evals/solvers/human_cli_solver.py x: 3 contributors (all time) y: 29 lines of code evals/solvers/nested/fewshot_solver.py x: 2 contributors (all time) y: 91 lines of code evals/solvers/prompts/hhh.py x: 3 contributors (all time) y: 99 lines of code evals/elsuite/sandbagging/solvers.py x: 3 contributors (all time) y: 152 lines of code evals/record.py x: 9 contributors (all time) y: 450 lines of code evals/cli/oaievalset.py x: 4 contributors (all time) y: 111 lines of code evals/elsuite/ballots/prompts.py x: 2 contributors (all time) y: 44 lines of code evals/elsuite/sandbagging/defaults.py x: 2 contributors (all time) y: 12 lines of code evals/elsuite/schelling_point/prompts.py x: 2 contributors (all time) y: 25 lines of code evals/registry/data/word_association/corpus_tools/validators.py x: 4 contributors (all time) y: 151 lines of code scripts/battle_generator.py x: 5 contributors (all time) y: 49 lines of code scripts/modelgraded_generator.py x: 5 contributors (all time) y: 185 lines of code scripts/pattern_identification_generator.py x: 6 contributors (all time) y: 48 lines of code evals/completion_fns/langchain_llm.py x: 4 contributors (all time) y: 70 lines of code evals/registry/completion_fns/langchain_llms.yaml x: 3 contributors (all time) y: 24 lines of code evals/registry/evals/mmmu.yaml x: 1 contributors (all time) y: 390 lines of code evals/elsuite/make_me_say/utils.py x: 5 contributors (all time) y: 34 lines of code evals/elsuite/ballots/eval.py x: 2 contributors (all time) y: 161 lines of code evals/elsuite/schelling_point/eval.py x: 2 contributors (all time) y: 73 lines of code evals/completion_fns/langchain_math.py x: 4 contributors (all time) y: 21 lines of code evals/elsuite/make_me_pay/utils.py x: 3 contributors (all time) y: 47 lines of code evals/elsuite/make_me_say/autoeval.py x: 4 contributors (all time) y: 116 lines of code .github/workflows/parse_yaml.py x: 3 contributors (all time) y: 10 lines of code evals/elsuite/lambada.py x: 2 contributors (all time) y: 39 lines of code evals/elsuite/modelgraded/classify.py x: 7 contributors (all time) y: 97 lines of code evals/elsuite/utils.py x: 6 contributors (all time) y: 150 lines of code evals/metrics.py x: 3 contributors (all time) y: 52 lines of code evals/registry/data/mapping_to_matricies/data_generator.py x: 2 contributors (all time) y: 30 lines of code evals/registry/data/simple_physics_engine/wave_function_collapse.py x: 2 contributors (all time) y: 157 lines of code evals/registry/data/solve-for-variable/tools/solve.py x: 2 contributors (all time) y: 231 lines of code evals/registry/data/word_association/corpus_tools/sample_generators.py x: 2 contributors (all time) y: 154 lines of code evals/base.py x: 4 contributors (all time) y: 51 lines of code evals/elsuite/basic/match.py x: 4 contributors (all time) y: 57 lines of code evals/registry/evals/test-modelgraded.yaml x: 3 contributors (all time) y: 81 lines of code evals/elsuite/modelgraded/classify_utils.py x: 2 contributors (all time) y: 145 lines of code evals/registry/data/backgammon/generate_samples.ipynb x: 1 contributors (all time) y: 1349 lines of code evals/api.py x: 4 contributors (all time) y: 61 lines of code evals/registry/evals/mmlu.yaml x: 1 contributors (all time) y: 399 lines of code evals/registry/eval_sets/test-all.yaml x: 3 contributors (all time) y: 21 lines of code evals/registry/evals/coqa-ex.yaml x: 3 contributors (all time) y: 55 lines of code MANIFEST.in x: 3 contributors (all time) y: 4 lines of code
1349.0
lines of code
  min: 1.0
  average: 27.66
  25th percentile: 3.0
  median: 3.0
  75th percentile: 16.0
  max: 1349.0
0 18.0
contributors (all time)
min: 1.0 | average: 1.18 | 25th percentile: 1.0 | median: 1.0 | 75th percentile: 1.0 | max: 18.0

File Size vs. Commits (30 days): 0 points

No data for "commits (30d)" vs. "lines of code".

File Size vs. Contributors (30 days): 0 points

No data for "contributors (30d)" vs. "lines of code".


File Size vs. Commits (90 days): 1 points

.pre-commit-config.yaml x: 1 commits (90d) y: 32 lines of code
32.0
lines of code
  min: 32.0
  average: 32.0
  25th percentile: 32.0
  median: 32.0
  75th percentile: 32.0
  max: 32.0
0 1.0
commits (90d)
min: 1.0 | average: 1.0 | 25th percentile: 1.0 | median: 1.0 | 75th percentile: 1.0 | max: 1.0

File Size vs. Contributors (90 days): 1 points

.pre-commit-config.yaml x: 1 contributors (90d) y: 32 lines of code
32.0
lines of code
  min: 32.0
  average: 32.0
  25th percentile: 32.0
  median: 32.0
  75th percentile: 32.0
  max: 32.0
0 1.0
contributors (90d)
min: 1.0 | average: 1.0 | 25th percentile: 1.0 | median: 1.0 | 75th percentile: 1.0 | max: 1.0