huggingface / lm-evaluation-harness
File Size

The distribution of size of files (measured in lines of code).

Intro
Learn more...
File Size Overall
10% | 3% | 13% | 11% | 60%
Legend:
1001+
501-1000
201-500
101-200
1-100


explore: grouped by folders | grouped by size | sunburst | 3D view
File Size per Extension
1001+
501-1000
201-500
101-200
1-100
py24% | 7% | 28% | 20% | 19%
yaml0% | 0% | 1% | 4% | 94%
cpp0% | 0% | 0% | 100% | 0%
toml0% | 0% | 0% | 0% | 100%
bib0% | 0% | 0% | 0% | 100%
File Size per Logical Decomposition
primary
1001+
501-1000
201-500
101-200
1-100
lm_eval11% | 3% | 13% | 10% | 61%
scripts0% | 0% | 0% | 53% | 46%
ROOT0% | 0% | 0% | 0% | 100%
templates0% | 0% | 0% | 0% | 100%
Longest Files (Top 50)
File# lines# units
instructions_util.py
in lm_eval/tasks/leaderboard/ifeval
1635 6
instructions_util.py
in lm_eval/tasks/ifeval
1635 6
task.py
in lm_eval/api
1218 79
huggingface.py
in lm_eval/models
1037 35
instructions.py
in lm_eval/tasks/leaderboard/ifeval
838 123
instructions.py
in lm_eval/tasks/ifeval
838 123
neuron_optimum.py
in lm_eval/models
497 23
__init__.py
in lm_eval/tasks
471 34
evaluator.py
in lm_eval
466 4
nemo_lm.py
in lm_eval/models
423 25
vllm_causallms.py
in lm_eval/models
412 18
__main__.py
in lm_eval
401 6
evaluation_tracker.py
in lm_eval/loggers
397 8
384 14
metrics.py
in lm_eval/api
352 44
utils.py
in lm_eval/models
346 27
flan_held_in.yaml
in lm_eval/tasks/benchmarks/flan
345 -
openai_completions.py
in lm_eval/models
342 24
task.py
in lm_eval/tasks/scrolls
304 47
neuralmagic.py
in lm_eval/models
292 16
utils.py
in lm_eval
271 32
wandb_logger.py
in lm_eval/loggers
254 11
anthropic_llms.py
in lm_eval/models
238 23
utils.py
in lm_eval/tasks/minerva_math
229 13
generate_tasks.py
in lm_eval/tasks/bigbench
221 1
utils.py
in lm_eval/tasks/afrixnli
216 3
utils.py
in lm_eval/tasks/agieval
197 9
utils.py
in lm_eval/tasks/afrimgsm
194 3
mmlu_high_school_european_history.yaml
in lm_eval/tasks/mmlu/flan_cot_fewshot
187 -
utils.py
in lm_eval/tasks/mgsm
185 3
model.py
in lm_eval/api
183 26
_generate_configs.py
in lm_eval/tasks/tmmluplus/default
176 1
174 5
bleu.py
in lm_eval/tasks/code_x_glue/code-text
167 10
utils.py
in lm_eval/tasks/hendrycks_math
165 10
utils.py
in lm_eval/tasks/bbh/cot_zeroshot
162 9
utils.py
in lm_eval/tasks/bbh/zeroshot
162 9
regression.py
in scripts
155 6
task.py
in lm_eval/tasks/squadv2
154 16
mmlu_high_school_us_history.yaml
in lm_eval/tasks/mmlu/flan_cot_fewshot
152 -
utils.py
in lm_eval/tasks/drop
148 16
utils.py
in lm_eval/tasks/xnli
144 2
generate_13_grams.py
in scripts/clean_training_data
140 7
registry.py
in lm_eval/api
140 12
_generate_configs.py
in lm_eval/tasks/cmmlu
140 1
archiver.py
in lm_eval/decontamination
134 16
_generate_configs.py
in lm_eval/tasks/mmlu
134 1
_belebele.yaml
in lm_eval/tasks/belebele
133 -
123 4
extraction.py
in lm_eval/filters
122 6
Files With Most Units (Top 50)
File# lines# units
instructions.py
in lm_eval/tasks/leaderboard/ifeval
838 123
instructions.py
in lm_eval/tasks/ifeval
838 123
task.py
in lm_eval/api
1218 79
task.py
in lm_eval/tasks/scrolls
304 47
metrics.py
in lm_eval/api
352 44
huggingface.py
in lm_eval/models
1037 35
__init__.py
in lm_eval/tasks
471 34
utils.py
in lm_eval
271 32
utils.py
in lm_eval/models
346 27
model.py
in lm_eval/api
183 26
nemo_lm.py
in lm_eval/models
423 25
openai_completions.py
in lm_eval/models
342 24
anthropic_llms.py
in lm_eval/models
238 23
neuron_optimum.py
in lm_eval/models
497 23
vllm_causallms.py
in lm_eval/models
412 18
neuralmagic.py
in lm_eval/models
292 16
archiver.py
in lm_eval/decontamination
134 16
task.py
in lm_eval/tasks/squadv2
154 16
utils.py
in lm_eval/tasks/drop
148 16
janitor.py
in lm_eval/decontamination
119 15
task.py
in lm_eval/tasks/unitxt
72 15
textsynth.py
in lm_eval/models
108 14
384 14
group.py
in lm_eval/api
81 13
utils.py
in lm_eval/tasks/minerva_math
229 13
utils.py
in lm_eval/tasks/crows_pairs
31 13
registry.py
in lm_eval/api
140 12
task.py
in lm_eval/tasks/squad_completion
49 12
task.py
in lm_eval/tasks/swde
49 12
task.py
in lm_eval/tasks/fda
49 12
wandb_logger.py
in lm_eval/loggers
254 11
bleu.py
in lm_eval/tasks/code_x_glue/code-text
167 10
utils.py
in lm_eval/tasks/hendrycks_math
165 10
utils.py
in lm_eval/tasks/bbh/cot_zeroshot
162 9
utils.py
in lm_eval/tasks/bbh/zeroshot
162 9
utils.py
in lm_eval/tasks/french_bench
63 9
utils.py
in lm_eval/tasks/agieval
197 9
samplers.py
in lm_eval/api
117 8
evaluation_tracker.py
in lm_eval/loggers
397 8
generate_13_grams.py
in scripts/clean_training_data
140 7
agg_functions.py
in lm_eval/tasks/tinyBenchmarks
37 7
utils.py
in lm_eval/tasks/kobest
33 7
t5_utils.py
in lm_eval/tasks/super_glue/record
95 7
regression.py
in scripts
155 6
72 6
__main__.py
in lm_eval
401 6
gguf.py
in lm_eval/models
115 6
selection.py
in lm_eval/filters
28 6
extraction.py
in lm_eval/filters
122 6
transformation.py
in lm_eval/filters
32 6
Files With Long Lines (Top 50)

There are 176 files with lines longer than 120 characters. In total, there are 284 long lines.

File# lines# units# long lines
flan_held_in.yaml
in lm_eval/tasks/benchmarks/flan
345 - 42
huggingface.py
in lm_eval/models
1037 35 9
evaluation_tracker.py
in lm_eval/loggers
397 8 8
__main__.py
in lm_eval
401 6 7
utils.py
in lm_eval/tasks/minerva_math
229 13 7
utils.py
in lm_eval/tasks/leaderboard/math
95 6 7
task.py
in lm_eval/api
1218 79 5
nemo_lm.py
in lm_eval/models
423 25 5
__init__.py
in lm_eval/tasks
471 34 5
174 5 3
regression.py
in scripts
155 6 3
evaluator.py
in lm_eval
466 4 3
384 14 3
boolean_expressions.yaml
in lm_eval/tasks/bbh/cot_fewshot
21 - 3
registry.py
in lm_eval/api
140 12 2
samplers.py
in lm_eval/api
117 8 2
model.py
in lm_eval/api
183 26 2
tracking_shuffled_objects_five_objects.yaml
in lm_eval/tasks/bbh/cot_zeroshot
19 - 2
tracking_shuffled_objects_seven_objects.yaml
in lm_eval/tasks/bbh/cot_zeroshot
19 - 2
navigate.yaml
in lm_eval/tasks/bbh/cot_zeroshot
17 - 2
snarks.yaml
in lm_eval/tasks/bbh/cot_zeroshot
19 - 2
tracking_shuffled_objects_three_objects.yaml
in lm_eval/tasks/bbh/cot_zeroshot
19 - 2
french_bench_fquadv2_genq.yaml
in lm_eval/tasks/french_bench
26 - 2
metric.py
in lm_eval/tasks/realtoxicityprompts
67 1 2
polemo2_in.yaml
in lm_eval/tasks/polemo2
46 - 2
t5_utils.py
in lm_eval/tasks/super_glue/wsc
76 4 2
72 6 1
write_out.py
in scripts
78 2 1
123 4 1
group.py
in lm_eval/api
81 13 1
openai_completions.py
in lm_eval/models
342 24 1
vllm_causallms.py
in lm_eval/models
412 18 1
neuron_optimum.py
in lm_eval/models
497 23 1
selection.py
in lm_eval/filters
28 6 1
boolean_expressions.yaml
in lm_eval/tasks/bbh/cot_zeroshot
17 - 1
causal_judgement.yaml
in lm_eval/tasks/bbh/cot_zeroshot
17 - 1
logical_deduction_seven_objects.yaml
in lm_eval/tasks/bbh/cot_zeroshot
19 - 1
word_sorting.yaml
in lm_eval/tasks/bbh/cot_zeroshot
15 - 1
penguins_in_a_table.yaml
in lm_eval/tasks/bbh/cot_zeroshot
19 - 1
formal_fallacies.yaml
in lm_eval/tasks/bbh/cot_zeroshot
17 - 1
ruin_names.yaml
in lm_eval/tasks/bbh/cot_zeroshot
19 - 1
temporal_sequences.yaml
in lm_eval/tasks/bbh/cot_zeroshot
19 - 1
web_of_lies.yaml
in lm_eval/tasks/bbh/cot_zeroshot
20 - 1
logical_deduction_five_objects.yaml
in lm_eval/tasks/bbh/cot_zeroshot
19 - 1
dyck_languages.yaml
in lm_eval/tasks/bbh/cot_zeroshot
17 - 1
hyperbaton.yaml
in lm_eval/tasks/bbh/cot_zeroshot
19 - 1
logical_deduction_three_objects.yaml
in lm_eval/tasks/bbh/cot_zeroshot
19 - 1
geometric_shapes.yaml
in lm_eval/tasks/bbh/cot_zeroshot
19 - 1
salient_translation_error_detection.yaml
in lm_eval/tasks/bbh/cot_zeroshot
19 - 1
reasoning_about_colored_objects.yaml
in lm_eval/tasks/bbh/cot_zeroshot
19 - 1