huggingface / lm-evaluation-harness
File Age & Freshness

File age measurements show the distribution of file ages (days since the first commit) and the file freshness (days since the latest commit).

Summary
File Change History Overall
File Age Distribution Overall
Days since first update
  • There are 3,012 files with 51,606 lines of code in files.
    • 2,503 files that are 366+ days old (47,004 lines of code)
    • 509 files that are 181-365 days old (4,602 lines of code)
    • 0 files that are 91-180 days old (0 lines of code)
    • 0 files that are 31-90 days old (0 lines of code)
    • 0 files that are 1-30 days old (0 lines of code)
91% | 8% | 0% | 0% | 0%
Legend:
366+
181-365
91-180
31-90
1-30

explore: grouped by folders | grouped by age
File Freshness Distribution Overall
Days since last update
  • There are 3,012 files with 51,606 lines of code in files.
    • 1,820 files have been last changed 366+ days ago (26,387 lines of code)
    • 1,190 files have been last changed 181-365 days ago (25,026 lines of code)
    • 2 files have been last changed 91-180 days ago (193 lines of code)
    • 0 files have been last changed 31-90 days ago (0 lines of code)
    • 0 files have been last changed 1-30 days ago (0 lines of code)
51% | 48% | <1% | 0% | 0%
Legend:
366+
181-365
91-180
31-90
1-30

explore: grouped by folders | grouped by freshness
File Change History per File Extension
yaml, json, py, md, pkl, txt, sh, ipynb, cpp, ini, gitignore, toml, bib
File Age Distribution per Extension
Days since first update
366+
181-365
91-180
31-90
1-30
yaml86% | 13% | 0% | 0% | 0%
py96% | 3% | 0% | 0% | 0%
cpp100% | 0% | 0% | 0% | 0%
toml100% | 0% | 0% | 0% | 0%
bib100% | 0% | 0% | 0% | 0%
File Freshness Distribution per Extension
Days since last update
366+
181-365
91-180
31-90
1-30
yaml51% | 48% | 0% | 0% | 0%
py50% | 49% | <1% | 0% | 0%
cpp100% | 0% | 0% | 0% | 0%
bib100% | 0% | 0% | 0% | 0%
toml0% | 0% | 100% | 0% | 0%
File Change History per Logical Decomposition
primary
primary (file age distribution)
Days since first update
366+
181-365
91-180
31-90
1-30
lm_eval90% | 9% | 0% | 0% | 0%
scripts100% | 0% | 0% | 0% | 0%
ROOT100% | 0% | 0% | 0% | 0%
templates100% | 0% | 0% | 0% | 0%
primary (file freshness distribution)
Days since last update
366+
181-365
91-180
31-90
1-30
lm_eval49% | 49% | <1% | 0% | 0%
scripts100% | 0% | 0% | 0% | 0%
ROOT10% | 0% | 89% | 0% | 0%
templates100% | 0% | 0% | 0% | 0%
Oldest Files (Top 50)
File# lines# unitscreatedlast modified# changes
(days)		# contributorsfirst
contributor		latest
contributor
__init__.py
in lm_eval/tasks
 471 34 2020-09-07 2024-10-29 205 69 email@jasonphang.com nathan.habib19@gmail.com
utils.py
in lm_eval
 271 32 2020-09-07 2024-10-29 142 47 email@jasonphang.com nathan.habib19@gmail.com
dummy.py
in lm_eval/models
 27 5 2020-09-07 2024-10-31 50 23 email@jasonphang.com 30601243+nathanhb@users.nor...
__init__.py
in lm_eval/models
 20 - 2020-09-07 2024-04-16 60 29 email@jasonphang.com michael@neuralmagic.com
__init__.py
in lm_eval
 1 - 2020-09-07 2023-11-21 12 8 email@jasonphang.com lintang@sutawika.com
setup.py
in root
 2 - 2021-02-01 2023-12-20 108 40 leogao31@gmail.com 92168766+baberabb@users.nor...
evaluator.py
in lm_eval
 466 4 2021-02-04 2024-10-29 185 50 pfarrington@gmail.com nathan.habib19@gmail.com
write_out.py
in scripts
 78 2 2021-02-05 2024-06-27 52 33 jeffhsu3@gmail.com nathan.habib@huggingface.co
72 6 2021-02-14 2024-03-06 26 18 leogao31@gmail.com lsinev@users.noreply.github...
40 1 2021-02-14 2024-06-27 14 10 leogao31@gmail.com nathan.habib@huggingface.co
get_prompts.py
in scripts
 20 - 2021-02-14 2023-12-20 9 10 leogao31@gmail.com 92168766+baberabb@users.nor...
generate_13_grams.py
in scripts/clean_training_data
 140 7 2021-05-12 2024-01-28 10 9 32028883+researcher2@users.... lsinev@users.noreply.github...
janitor_util.cpp
in scripts/clean_training_data
 118 3 2021-05-12 2024-03-03 9 9 32028883+researcher2@users.... 92168766+baberabb@users.nor...
process_sorted_buckets.py
in scripts/clean_training_data
 80 2 2021-05-12 2023-12-20 7 8 32028883+researcher2@users.... 92168766+baberabb@users.nor...
sort_13_gram_buckets.py
in scripts/clean_training_data
 34 2 2021-05-12 2023-12-20 9 8 32028883+researcher2@users.... 92168766+baberabb@users.nor...
__init__.py
in scripts/clean_training_data
 1 - 2021-05-12 2021-09-17 4 4 32028883+researcher2@users.... jonathantow1@gmail.com
__init__.py
in scripts
 1 - 2021-05-12 2021-09-17 4 4 32028883+researcher2@users.... jonathantow1@gmail.com
bib
 10 - 2021-09-15 2024-01-15 4 4 stellabiderman@gmail.com 65563625+haileyschoelkopf@u...
archiver.py
in lm_eval/decontamination
 134 16 2022-03-03 2024-01-28 13 11 32028883+researcher2@users.... lsinev@users.noreply.github...
janitor.py
in lm_eval/decontamination
 119 15 2022-03-03 2023-12-20 18 12 32028883+researcher2@users.... 92168766+baberabb@users.nor...
decontaminate.py
in lm_eval/decontamination
 115 2 2022-03-03 2024-01-28 13 12 32028883+researcher2@users.... lsinev@users.noreply.github...
investigate_pile.py
in scripts/clean_training_data
 71 3 2022-03-03 2024-01-28 6 8 32028883+researcher2@users.... lsinev@users.noreply.github...
__init__.py
in lm_eval/decontamination
 1 - 2022-03-03 2022-06-11 3 4 32028883+researcher2@users.... jonathantow1@gmail.com
compress_and_package.py
in scripts/clean_training_data
 53 2 2022-03-13 2023-12-20 5 7 32028883+researcher2@users.... 92168766+baberabb@users.nor...
textsynth.py
in lm_eval/models
 108 14 2022-11-04 2024-06-27 35 18 41410219+jon-tow@users.nore... nathan.habib@huggingface.co
huggingface.py
in lm_eval/models
 1037 35 2023-01-17 2024-10-31 147 45 hailey.schoelkopf@yale.edu nathan.habib@huggingface.co
task.py
in lm_eval/api
 1218 79 2023-04-19 2024-10-29 154 28 hailey.schoelkopf@yale.edu nathan.habib19@gmail.com
metrics.py
in lm_eval/api
 352 44 2023-04-19 2024-10-29 51 14 hailey.schoelkopf@yale.edu nathan.habib19@gmail.com
model.py
in lm_eval/api
 183 26 2023-04-19 2024-10-29 61 20 hailey.schoelkopf@yale.edu nathan.habib19@gmail.com
extraction.py
in lm_eval/filters
 122 6 2023-04-19 2024-07-15 34 16 hailey.schoelkopf@yale.edu lintang@eleuther.ai
samplers.py
in lm_eval/api
 117 8 2023-04-19 2024-10-29 40 18 hailey.schoelkopf@yale.edu nathan.habib19@gmail.com
__init__.py
in lm_eval/prompts
 89 4 2023-04-19 2024-02-26 30 10 hailey.schoelkopf@yale.edu lsinev@users.noreply.github...
selection.py
in lm_eval/filters
 28 6 2023-04-19 2024-04-26 19 10 hailey.schoelkopf@yale.edu nikitml@gmail.com
instance.py
in lm_eval/api
 26 2 2023-04-19 2024-03-06 23 10 hailey.schoelkopf@yale.edu lsinev@users.noreply.github...
filter.py
in lm_eval/api
 20 3 2023-04-19 2024-01-30 20 9 hailey.schoelkopf@yale.edu 92168766+baberabb@users.nor...
__init__.py
in lm_eval/filters
 15 1 2023-04-19 2024-04-26 33 14 hailey.schoelkopf@yale.edu nikitml@gmail.com
decontamination.py
in lm_eval/filters
 9 2 2023-04-19 2024-06-27 18 9 hailey.schoelkopf@yale.edu nathan.habib@huggingface.co
__init__.py
in lm_eval/api
 1 - 2023-04-19 2023-06-13 13 6 hailey.schoelkopf@yale.edu lintang@sutawika.com
51 1 2023-05-06 2024-06-27 11 7 juletxara@gmail.com nathan.habib@huggingface.co
arc_easy.yaml
in lm_eval/tasks/arc
 23 - 2023-05-13 2024-10-29 30 14 lintang@sutawika.com nathan.habib19@gmail.com
arc_challenge.yaml
in lm_eval/tasks/arc
 3 - 2023-05-13 2023-08-29 22 10 lintang@sutawika.com lintang@sutawika.com
t5-prompt.yaml
in lm_eval/tasks/super_glue/cb
 25 - 2023-05-15 2024-10-29 28 11 lintang@sutawika.com nathan.habib19@gmail.com
t5-prompt.yaml
in lm_eval/tasks/super_glue/copa
 22 - 2023-05-15 2024-10-29 27 11 lintang@sutawika.com nathan.habib19@gmail.com
t5-prompt.yaml
in lm_eval/tasks/super_glue/record
 22 - 2023-05-15 2024-10-29 29 11 lintang@sutawika.com nathan.habib19@gmail.com
t5-prompt.yaml
in lm_eval/tasks/super_glue/wsc
 20 - 2023-05-15 2024-10-29 37 11 lintang@sutawika.com nathan.habib19@gmail.com
preprocess_wsc.py
in lm_eval/tasks/super_glue/wsc
 14 1 2023-05-15 2023-10-09 19 7 lintang@sutawika.com lintang@sutawika.com
pile_enron.yaml
in lm_eval/tasks/pile
 3 - 2023-05-23 2023-06-16 6 5 46356259+fattorib@users.nor... lintang@sutawika.com
regression.py
in scripts
 155 6 2023-06-01 2024-01-28 6 4 gkpyon@gmail.com lsinev@users.noreply.github...
preprocess_wikitext.py
in lm_eval/tasks/wikitext
 37 2 2023-06-02 2023-10-09 11 9 hailey.schoelkopf@yale.edu lintang@sutawika.com
piqa.yaml
in lm_eval/tasks/piqa
 23 - 2023-06-02 2024-06-27 28 12 hailey.schoelkopf@yale.edu nathan.habib@huggingface.co
Files Not Recently Changed (Top 50)
File# lines# unitscreatedlast modified# changes
(days)		# contributorsfirst
contributor		latest
contributor
__init__.py
in scripts
 1 - 2021-05-12 2021-09-17 4 4 32028883+researcher2@users.... jonathantow1@gmail.com
__init__.py
in scripts/clean_training_data
 1 - 2021-05-12 2021-09-17 4 4 32028883+researcher2@users.... jonathantow1@gmail.com
__init__.py
in lm_eval/decontamination
 1 - 2022-03-03 2022-06-11 3 4 32028883+researcher2@users.... jonathantow1@gmail.com
__init__.py
in lm_eval/api
 1 - 2023-04-19 2023-06-13 13 6 hailey.schoelkopf@yale.edu lintang@sutawika.com
blank_yaml.yaml
in templates/new_yaml_task
 1 - 2023-06-12 2023-06-16 3 3 hailey.schoelkopf@yale.edu lintang@sutawika.com
pile_github.yaml
in lm_eval/tasks/pile
 3 - 2023-06-02 2023-06-16 7 5 hailey.schoelkopf@yale.edu lintang@sutawika.com
pile_stackexchange.yaml
in lm_eval/tasks/pile
 3 - 2023-06-02 2023-06-16 8 6 hailey.schoelkopf@yale.edu lintang@sutawika.com
pile_hackernews.yaml
in lm_eval/tasks/pile
 3 - 2023-06-02 2023-06-16 7 5 hailey.schoelkopf@yale.edu lintang@sutawika.com
pile_openwebtext2.yaml
in lm_eval/tasks/pile
 3 - 2023-06-02 2023-06-16 7 5 hailey.schoelkopf@yale.edu lintang@sutawika.com
pile_pubmed-central.yaml
in lm_eval/tasks/pile
 3 - 2023-06-02 2023-06-16 8 6 hailey.schoelkopf@yale.edu lintang@sutawika.com
pile_uspto.yaml
in lm_eval/tasks/pile
 3 - 2023-06-02 2023-06-16 8 6 hailey.schoelkopf@yale.edu lintang@sutawika.com
pile_pile-cc.yaml
in lm_eval/tasks/pile
 3 - 2023-06-02 2023-06-16 7 5 hailey.schoelkopf@yale.edu lintang@sutawika.com
pile_gutenberg.yaml
in lm_eval/tasks/pile
 3 - 2023-06-02 2023-06-16 8 6 hailey.schoelkopf@yale.edu lintang@sutawika.com
pile_youtubesubtitles.yaml
in lm_eval/tasks/pile
 3 - 2023-06-02 2023-06-16 7 5 hailey.schoelkopf@yale.edu lintang@sutawika.com
pile_dm-mathematics.yaml
in lm_eval/tasks/pile
 3 - 2023-06-02 2023-06-16 7 5 hailey.schoelkopf@yale.edu lintang@sutawika.com
pile_books3.yaml
in lm_eval/tasks/pile
 3 - 2023-06-02 2023-06-16 7 5 hailey.schoelkopf@yale.edu lintang@sutawika.com
pile_pubmed-abstracts.yaml
in lm_eval/tasks/pile
 3 - 2023-06-02 2023-06-16 8 6 hailey.schoelkopf@yale.edu lintang@sutawika.com
pile_philpapers.yaml
in lm_eval/tasks/pile
 3 - 2023-06-02 2023-06-16 7 5 hailey.schoelkopf@yale.edu lintang@sutawika.com
pile_bookcorpus2.yaml
in lm_eval/tasks/pile
 3 - 2023-06-02 2023-06-16 7 5 hailey.schoelkopf@yale.edu lintang@sutawika.com
pile_europarl.yaml
in lm_eval/tasks/pile
 3 - 2023-06-02 2023-06-16 8 6 hailey.schoelkopf@yale.edu lintang@sutawika.com
pile_freelaw.yaml
in lm_eval/tasks/pile
 3 - 2023-06-02 2023-06-16 7 5 hailey.schoelkopf@yale.edu lintang@sutawika.com
pile_nih-exporter.yaml
in lm_eval/tasks/pile
 3 - 2023-06-02 2023-06-16 7 5 hailey.schoelkopf@yale.edu lintang@sutawika.com
pile_wikipedia.yaml
in lm_eval/tasks/pile
 3 - 2023-06-02 2023-06-16 8 6 hailey.schoelkopf@yale.edu lintang@sutawika.com
pile_enron.yaml
in lm_eval/tasks/pile
 3 - 2023-05-23 2023-06-16 6 5 46356259+fattorib@users.nor... lintang@sutawika.com
pile_ubuntu-irc.yaml
in lm_eval/tasks/pile
 3 - 2023-06-02 2023-06-16 8 6 hailey.schoelkopf@yale.edu lintang@sutawika.com
pile_opensubtitles.yaml
in lm_eval/tasks/pile
 3 - 2023-06-02 2023-06-16 7 5 hailey.schoelkopf@yale.edu lintang@sutawika.com
preprocess_qa4mre.py
in lm_eval/tasks/qa4mre
 4 2 2023-07-01 2023-07-12 6 4 farzanehnakhaee70@gmail.com 65563625+haileyschoelkopf@u...
headqa_es.yaml
in lm_eval/tasks/headqa
 3 - 2023-07-05 2023-07-13 4 3 hailey@eleuther.ai lintang@sutawika.com
qa4mre_2013.yaml
in lm_eval/tasks/qa4mre
 4 - 2023-07-04 2023-07-22 7 7 farzanehnakhaee70@gmail.com ben.fattori@mail.utoronto.ca
qa4mre_2012.yaml
in lm_eval/tasks/qa4mre
 4 - 2023-07-04 2023-07-22 7 7 farzanehnakhaee70@gmail.com ben.fattori@mail.utoronto.ca
utils.py
in lm_eval/tasks/toxigen
 5 1 2023-07-05 2023-07-22 5 5 hailey@eleuther.ai ben.fattori@mail.utoronto.ca
utils.py
in lm_eval/tasks/mathqa
 7 1 2023-07-07 2023-07-22 5 5 hailey@eleuther.ai ben.fattori@mail.utoronto.ca
preprocess_winogrande.py
in lm_eval/tasks/winogrande
 10 3 2023-07-02 2023-07-22 7 5 ben.fattori@mail.utoronto.ca ben.fattori@mail.utoronto.ca
utils.py
in lm_eval/tasks/super_glue/copa
 13 4 2023-07-03 2023-07-22 5 4 lintang@sutawika.com ben.fattori@mail.utoronto.ca
preprocess_race.py
in lm_eval/tasks/race
 28 6 2023-07-03 2023-07-22 6 5 ben.fattori@mail.utoronto.ca ben.fattori@mail.utoronto.ca
xwinograd_zh.yaml
in lm_eval/tasks/xwinograd
 3 - 2023-07-21 2023-08-01 4 4 eddy.yeo@gmail.com hailey@eleuther.ai
xwinograd_pt.yaml
in lm_eval/tasks/xwinograd
 3 - 2023-07-21 2023-08-01 4 4 eddy.yeo@gmail.com hailey@eleuther.ai
xwinograd_ru.yaml
in lm_eval/tasks/xwinograd
 3 - 2023-07-21 2023-08-01 4 4 eddy.yeo@gmail.com hailey@eleuther.ai
xwinograd_en.yaml
in lm_eval/tasks/xwinograd
 3 - 2023-07-21 2023-08-01 4 4 eddy.yeo@gmail.com hailey@eleuther.ai
xwinograd_jp.yaml
in lm_eval/tasks/xwinograd
 3 - 2023-07-21 2023-08-01 4 4 eddy.yeo@gmail.com hailey@eleuther.ai
xwinograd_fr.yaml
in lm_eval/tasks/xwinograd
 3 - 2023-07-21 2023-08-01 4 4 eddy.yeo@gmail.com hailey@eleuther.ai
utils.py
in lm_eval/tasks/webqs
 13 3 2023-07-18 2023-08-01 5 4 eddy.yeo@gmail.com hailey@eleuther.ai
utils_logiqa2.py
in lm_eval/tasks/logiqa2
 8 1 2023-08-02 2023-08-15 4 4 92168766+baberabb@users.nor... 72096386+aflah02@users.nore...
utils_logiqa.py
in lm_eval/tasks/logiqa
 11 2 2023-08-02 2023-08-15 4 4 92168766+baberabb@users.nor... 72096386+aflah02@users.nore...
crows_pairs_french.yaml
in lm_eval/tasks/crows_pairs
 3 - 2023-08-08 2023-08-16 5 4 hailey@eleuther.ai lintang@sutawika.com
crows_pairs_english_age.yaml
in lm_eval/tasks/crows_pairs
 4 - 2023-08-08 2023-08-16 5 4 hailey@eleuther.ai lintang@sutawika.com
crows_pairs_english_gender.yaml
in lm_eval/tasks/crows_pairs
 4 - 2023-08-08 2023-08-16 5 4 hailey@eleuther.ai lintang@sutawika.com
crows_pairs_french_socioeconomic.yaml
in lm_eval/tasks/crows_pairs
 4 - 2023-08-08 2023-08-16 5 4 hailey@eleuther.ai lintang@sutawika.com
crows_pairs_french_nationality.yaml
in lm_eval/tasks/crows_pairs
 4 - 2023-08-08 2023-08-16 5 4 hailey@eleuther.ai lintang@sutawika.com
4 - 2023-08-08 2023-08-16 5 4 hailey@eleuther.ai lintang@sutawika.com
Most Recently Created Files (Top 50)
File# lines# unitscreatedlast modified# changes
(days)		# contributorsfirst
contributor		latest
contributor
__init__.py
in lm_eval/caching
 1 - 2024-07-21 2024-10-29 3 3 65563625+haileyschoelkopf@u... nathan.habib19@gmail.com
_generate_configs.py
in lm_eval/tasks/med_concepts_qa
 27 2 2024-07-14 2024-10-29 3 3 33639234+ofir408@users.nore... nathan.habib19@gmail.com
_med_concepts_qa.yaml
in lm_eval/tasks/med_concepts_qa
 10 - 2024-07-14 2024-10-29 3 3 33639234+ofir408@users.nore... nathan.habib19@gmail.com
_med_concepts_qa_icd10cm.yaml
in lm_eval/tasks/med_concepts_qa
 6 - 2024-07-14 2024-10-29 3 3 33639234+ofir408@users.nore... nathan.habib19@gmail.com
_med_concepts_qa_icd9cm.yaml
in lm_eval/tasks/med_concepts_qa
 6 - 2024-07-14 2024-10-29 4 4 33639234+ofir408@users.nore... nathan.habib19@gmail.com
_med_concepts_qa_atc.yaml
in lm_eval/tasks/med_concepts_qa
 6 - 2024-07-14 2024-10-29 4 4 33639234+ofir408@users.nore... nathan.habib19@gmail.com
_med_concepts_qa_icd9proc.yaml
in lm_eval/tasks/med_concepts_qa
 6 - 2024-07-14 2024-10-29 4 4 33639234+ofir408@users.nore... nathan.habib19@gmail.com
_med_concepts_qa_icd10proc.yaml
in lm_eval/tasks/med_concepts_qa
 6 - 2024-07-14 2024-10-29 4 4 33639234+ofir408@users.nore... nathan.habib19@gmail.com
med_concepts_qa_icd10cm_medium.yaml
in lm_eval/tasks/med_concepts_qa
 5 - 2024-07-14 2024-10-29 3 3 33639234+ofir408@users.nore... nathan.habib19@gmail.com
med_concepts_qa_icd9cm_medium.yaml
in lm_eval/tasks/med_concepts_qa
 5 - 2024-07-14 2024-10-29 3 3 33639234+ofir408@users.nore... nathan.habib19@gmail.com
med_concepts_qa_icd10proc_hard.yaml
in lm_eval/tasks/med_concepts_qa
 5 - 2024-07-14 2024-10-29 3 3 33639234+ofir408@users.nore... nathan.habib19@gmail.com
med_concepts_qa_icd9proc_medium.yaml
in lm_eval/tasks/med_concepts_qa
 5 - 2024-07-14 2024-10-29 3 3 33639234+ofir408@users.nore... nathan.habib19@gmail.com
med_concepts_qa_atc_hard.yaml
in lm_eval/tasks/med_concepts_qa
 5 - 2024-07-14 2024-10-29 3 3 33639234+ofir408@users.nore... nathan.habib19@gmail.com
med_concepts_qa_icd10proc_easy.yaml
in lm_eval/tasks/med_concepts_qa
 5 - 2024-07-14 2024-10-29 3 3 33639234+ofir408@users.nore... nathan.habib19@gmail.com
med_concepts_qa_icd9proc_easy.yaml
in lm_eval/tasks/med_concepts_qa
 5 - 2024-07-14 2024-10-29 3 3 33639234+ofir408@users.nore... nathan.habib19@gmail.com
med_concepts_qa_icd9cm_easy.yaml
in lm_eval/tasks/med_concepts_qa
 5 - 2024-07-14 2024-10-29 3 3 33639234+ofir408@users.nore... nathan.habib19@gmail.com
med_concepts_qa_icd9proc_hard.yaml
in lm_eval/tasks/med_concepts_qa
 5 - 2024-07-14 2024-10-29 3 3 33639234+ofir408@users.nore... nathan.habib19@gmail.com
med_concepts_qa_atc_medium.yaml
in lm_eval/tasks/med_concepts_qa
 5 - 2024-07-14 2024-10-29 3 3 33639234+ofir408@users.nore... nathan.habib19@gmail.com
med_concepts_qa_icd10proc_medium.yaml
in lm_eval/tasks/med_concepts_qa
 5 - 2024-07-14 2024-10-29 3 3 33639234+ofir408@users.nore... nathan.habib19@gmail.com
med_concepts_qa_icd10cm_easy.yaml
in lm_eval/tasks/med_concepts_qa
 5 - 2024-07-14 2024-10-29 3 3 33639234+ofir408@users.nore... nathan.habib19@gmail.com
med_concepts_qa_icd9cm_hard.yaml
in lm_eval/tasks/med_concepts_qa
 5 - 2024-07-14 2024-10-29 3 3 33639234+ofir408@users.nore... nathan.habib19@gmail.com
med_concepts_qa_atc_easy.yaml
in lm_eval/tasks/med_concepts_qa
 5 - 2024-07-14 2024-10-29 3 3 33639234+ofir408@users.nore... nathan.habib19@gmail.com
med_concepts_qa_icd10cm_hard.yaml
in lm_eval/tasks/med_concepts_qa
 5 - 2024-07-14 2024-10-29 3 3 33639234+ofir408@users.nore... nathan.habib19@gmail.com
utils.py
in lm_eval/tasks/afrixnli
 216 3 2024-07-12 2024-10-29 4 4 jessicaojo19@gmail.com nathan.habib19@gmail.com
utils.py
in lm_eval/tasks/afrimgsm
 194 3 2024-07-12 2024-10-29 4 4 jessicaojo19@gmail.com nathan.habib19@gmail.com
config.py
in lm_eval/tasks/mmlusr
 121 1 2024-07-12 2024-10-29 3 3 37853425+skysupercat@users.... nathan.habib19@gmail.com
_question_only.yaml
in lm_eval/tasks/mmlusr/question_only
 44 - 2024-07-12 2024-10-29 3 3 37853425+skysupercat@users.... nathan.habib19@gmail.com
_question_and_answer.yaml
in lm_eval/tasks/mmlusr/question_and_answer
 44 - 2024-07-12 2024-10-29 3 3 37853425+skysupercat@users.... nathan.habib19@gmail.com
_answer_only.yaml
in lm_eval/tasks/mmlusr/answer_only
 44 - 2024-07-12 2024-10-29 3 3 37853425+skysupercat@users.... nathan.habib19@gmail.com
utils.py
in lm_eval/tasks/afrimmlu/translate
 22 3 2024-07-12 2024-10-29 4 4 jessicaojo19@gmail.com nathan.habib19@gmail.com
utils.py
in lm_eval/tasks/afrimmlu/direct
 22 3 2024-07-12 2024-10-29 4 4 jessicaojo19@gmail.com nathan.habib19@gmail.com
utils.py
in lm_eval/tasks/afrimmlu
 22 3 2024-07-12 2024-10-29 4 4 jessicaojo19@gmail.com nathan.habib19@gmail.com
utils.py
in lm_eval/tasks/afrixnli/lai prompt/translate
 14 3 2024-07-12 2024-10-29 4 4 jessicaojo19@gmail.com nathan.habib19@gmail.com
utils.py
in lm_eval/tasks/afrixnli/lai prompt/direct
 14 3 2024-07-12 2024-10-29 4 4 jessicaojo19@gmail.com nathan.habib19@gmail.com
utils.py
in lm_eval/tasks/mmlusr/question_only
 13 1 2024-07-12 2024-10-29 3 3 37853425+skysupercat@users.... nathan.habib19@gmail.com
utils.py
in lm_eval/tasks/mmlusr/question_and_answer
 13 1 2024-07-12 2024-10-29 3 3 37853425+skysupercat@users.... nathan.habib19@gmail.com
utils.py
in lm_eval/tasks/mmlusr/answer_only
 13 1 2024-07-12 2024-10-29 3 3 37853425+skysupercat@users.... nathan.habib19@gmail.com
afrimgsm_translate_lug.yaml
in lm_eval/tasks/afrimgsm/translate
 11 - 2024-07-12 2024-10-29 3 3 jessicaojo19@gmail.com nathan.habib19@gmail.com
afrimgsm_translate_sna.yaml
in lm_eval/tasks/afrimgsm/translate
 11 - 2024-07-12 2024-10-29 3 3 jessicaojo19@gmail.com nathan.habib19@gmail.com
afrimgsm_translate_swa.yaml
in lm_eval/tasks/afrimgsm/translate
 11 - 2024-07-12 2024-10-29 3 3 jessicaojo19@gmail.com nathan.habib19@gmail.com
afrimgsm_translate_ewe.yaml
in lm_eval/tasks/afrimgsm/translate
 11 - 2024-07-12 2024-10-29 3 3 jessicaojo19@gmail.com nathan.habib19@gmail.com
afrimgsm_translate_wol.yaml
in lm_eval/tasks/afrimgsm/translate
 11 - 2024-07-12 2024-10-29 3 3 jessicaojo19@gmail.com nathan.habib19@gmail.com
afrimgsm_translate_kin.yaml
in lm_eval/tasks/afrimgsm/translate
 11 - 2024-07-12 2024-10-29 3 3 jessicaojo19@gmail.com nathan.habib19@gmail.com
afrimgsm_translate_yor.yaml
in lm_eval/tasks/afrimgsm/translate
 11 - 2024-07-12 2024-10-29 3 3 jessicaojo19@gmail.com nathan.habib19@gmail.com
afrimgsm_translate_ibo.yaml
in lm_eval/tasks/afrimgsm/translate
 11 - 2024-07-12 2024-10-29 3 3 jessicaojo19@gmail.com nathan.habib19@gmail.com
afrimgsm_translate_twi.yaml
in lm_eval/tasks/afrimgsm/translate
 11 - 2024-07-12 2024-10-29 3 3 jessicaojo19@gmail.com nathan.habib19@gmail.com
afrimgsm_translate_hau.yaml
in lm_eval/tasks/afrimgsm/translate
 11 - 2024-07-12 2024-10-29 3 3 jessicaojo19@gmail.com nathan.habib19@gmail.com
afrimgsm_translate_orm.yaml
in lm_eval/tasks/afrimgsm/translate
 11 - 2024-07-12 2024-10-29 3 3 jessicaojo19@gmail.com nathan.habib19@gmail.com
afrimgsm_translate_xho.yaml
in lm_eval/tasks/afrimgsm/translate
 11 - 2024-07-12 2024-10-29 3 3 jessicaojo19@gmail.com nathan.habib19@gmail.com
afrimgsm_translate_lin.yaml
in lm_eval/tasks/afrimgsm/translate
 11 - 2024-07-12 2024-10-29 3 3 jessicaojo19@gmail.com nathan.habib19@gmail.com
Most Recently Changed Files (Top 50)
File# lines# unitscreatedlast modified# changes
(days)		# contributorsfirst
contributor		latest
contributor
98 - 2023-08-25 2025-02-14 52 24 ethan@ethanhs.me kydlicek.hynek@gmail.com
utils.py
in lm_eval/tasks/leaderboard/math
 95 6 2024-06-26 2025-02-14 7 4 nathan.habib@huggingface.co kydlicek.hynek@gmail.com
leaderboard.yaml
in lm_eval/tasks/leaderboard
 32 - 2024-06-26 2024-11-05 5 4 nathan.habib@huggingface.co nathan.habib@huggingface.co
_leaderboard_bbh.yaml
in lm_eval/tasks/leaderboard/bbh_mc
 30 - 2024-07-08 2024-11-01 4 4 lintang@eleuther.ai nathan.habib@huggingface.co
_leaderboard_math.yaml
in lm_eval/tasks/leaderboard/math
 13 - 2024-07-08 2024-11-01 4 4 lintang@eleuther.ai nathan.habib@huggingface.co
_musr.yaml
in lm_eval/tasks/leaderboard/musr
 9 - 2024-07-08 2024-11-01 4 4 lintang@eleuther.ai nathan.habib@huggingface.co
_leaderboard_gpqa.yaml
in lm_eval/tasks/leaderboard/gpqa
 9 - 2024-07-08 2024-11-01 4 4 lintang@eleuther.ai nathan.habib@huggingface.co
huggingface.py
in lm_eval/models
 1037 35 2023-01-17 2024-10-31 147 45 hailey.schoelkopf@yale.edu nathan.habib@huggingface.co
__main__.py
in lm_eval
 401 6 2023-10-04 2024-10-31 54 22 hailey@eleuther.ai 30601243+nathanhb@users.nor...
utils.py
in lm_eval/models
 346 27 2024-02-14 2024-10-31 7 5 92168766+baberabb@users.nor... 30601243+nathanhb@users.nor...
dummy.py
in lm_eval/models
 27 5 2020-09-07 2024-10-31 50 23 email@jasonphang.com 30601243+nathanhb@users.nor...
task.py
in lm_eval/api
 1218 79 2023-04-19 2024-10-29 154 28 hailey.schoelkopf@yale.edu nathan.habib19@gmail.com
__init__.py
in lm_eval/tasks
 471 34 2020-09-07 2024-10-29 205 69 email@jasonphang.com nathan.habib19@gmail.com
evaluator.py
in lm_eval
 466 4 2021-02-04 2024-10-29 185 50 pfarrington@gmail.com nathan.habib19@gmail.com
vllm_causallms.py
in lm_eval/models
 412 18 2023-11-21 2024-10-29 38 14 92168766+baberabb@users.nor... nathan.habib19@gmail.com
evaluation_tracker.py
in lm_eval/loggers
 397 8 2024-05-26 2024-10-29 15 7 65563625+haileyschoelkopf@u... nathan.habib19@gmail.com
384 14 2024-02-27 2024-10-29 10 7 92168766+baberabb@users.nor... nathan.habib19@gmail.com
metrics.py
in lm_eval/api
 352 44 2023-04-19 2024-10-29 51 14 hailey.schoelkopf@yale.edu nathan.habib19@gmail.com
flan_held_in.yaml
in lm_eval/tasks/benchmarks/flan
 345 - 2023-09-26 2024-10-29 9 5 lintang@eleuther.ai nathan.habib19@gmail.com
task.py
in lm_eval/tasks/scrolls
 304 47 2023-11-09 2024-10-29 16 10 lintang@sutawika.com nathan.habib19@gmail.com
utils.py
in lm_eval
 271 32 2020-09-07 2024-10-29 142 47 email@jasonphang.com nathan.habib19@gmail.com
utils.py
in lm_eval/tasks/afrixnli
 216 3 2024-07-12 2024-10-29 4 4 jessicaojo19@gmail.com nathan.habib19@gmail.com
utils.py
in lm_eval/tasks/afrimgsm
 194 3 2024-07-12 2024-10-29 4 4 jessicaojo19@gmail.com nathan.habib19@gmail.com
mmlu_high_school_european_history.yaml
in lm_eval/tasks/mmlu/flan_cot_fewshot
 187 - 2023-09-03 2024-10-29 13 7 lintang@sutawika.com nathan.habib19@gmail.com
model.py
in lm_eval/api
 183 26 2023-04-19 2024-10-29 61 20 hailey.schoelkopf@yale.edu nathan.habib19@gmail.com
task.py
in lm_eval/tasks/squadv2
 154 16 2023-11-15 2024-10-29 17 10 lintang@sutawika.com nathan.habib19@gmail.com
mmlu_high_school_us_history.yaml
in lm_eval/tasks/mmlu/flan_cot_fewshot
 152 - 2023-09-03 2024-10-29 13 7 lintang@sutawika.com nathan.habib19@gmail.com
_generate_configs.py
in lm_eval/tasks/cmmlu
 140 1 2023-09-15 2024-10-29 16 9 hailey@eleuther.ai nathan.habib19@gmail.com
_generate_configs.py
in lm_eval/tasks/mmlu
 134 1 2023-09-03 2024-10-29 22 8 lintang@sutawika.com nathan.habib19@gmail.com
_belebele.yaml
in lm_eval/tasks/belebele
 133 - 2024-07-08 2024-10-29 3 3 lintang@eleuther.ai nathan.habib19@gmail.com
mmlu_college_computer_science.yaml
in lm_eval/tasks/mmlu/flan_cot_fewshot
 121 - 2023-09-03 2024-10-29 13 7 lintang@sutawika.com nathan.habib19@gmail.com
config.py
in lm_eval/tasks/mmlusr
 121 1 2024-07-12 2024-10-29 3 3 37853425+skysupercat@users.... nathan.habib19@gmail.com
_generate_configs.py
in lm_eval/tasks/ceval
 118 1 2023-09-15 2024-10-29 15 8 hailey@eleuther.ai nathan.habib19@gmail.com
samplers.py
in lm_eval/api
 117 8 2023-04-19 2024-10-29 40 18 hailey.schoelkopf@yale.edu nathan.habib19@gmail.com
mmlu_professional_law.yaml
in lm_eval/tasks/mmlu/flan_cot_fewshot
 114 - 2023-09-03 2024-10-29 13 7 lintang@sutawika.com nathan.habib19@gmail.com
utils_truthfulqa.py
in lm_eval/tasks/tinyBenchmarks
 105 6 2024-05-13 2024-10-29 4 4 35227161+lucweber@users.nor... nathan.habib19@gmail.com
utils.py
in lm_eval/tasks/truthfulqa
 105 6 2023-08-15 2024-10-29 9 6 lintang@sutawika.com nathan.habib19@gmail.com
utils.py
in lm_eval/loggers
 97 6 2024-05-26 2024-10-29 9 5 65563625+haileyschoelkopf@u... nathan.habib19@gmail.com
mmlu_security_studies.yaml
in lm_eval/tasks/mmlu/flan_cot_fewshot
 96 - 2023-09-03 2024-10-29 13 7 lintang@sutawika.com nathan.habib19@gmail.com
mmlu_high_school_world_history.yaml
in lm_eval/tasks/mmlu/flan_cot_fewshot
 96 - 2023-09-03 2024-10-29 13 7 lintang@sutawika.com nathan.habib19@gmail.com
_generate_configs.py
in lm_eval/tasks/arabicmmlu
 94 1 2024-06-19 2024-10-29 5 5 61038456+yazeed7@users.nore... nathan.habib19@gmail.com
gsm8k-cot.yaml
in lm_eval/tasks/gsm8k
 82 - 2023-06-08 2024-10-29 28 13 lintang@sutawika.com nathan.habib19@gmail.com
group.py
in lm_eval/api
 81 13 2024-07-08 2024-10-29 3 3 lintang@eleuther.ai nathan.habib19@gmail.com
mmlu_professional_medicine.yaml
in lm_eval/tasks/mmlu/flan_cot_fewshot
 81 - 2023-09-03 2024-10-29 13 7 lintang@sutawika.com nathan.habib19@gmail.com
mmlu_high_school_computer_science.yaml
in lm_eval/tasks/mmlu/flan_cot_fewshot
 81 - 2023-09-03 2024-10-29 13 7 lintang@sutawika.com nathan.habib19@gmail.com
_cmmlu.yaml
in lm_eval/tasks/cmmlu
 78 - 2024-07-08 2024-10-29 3 3 lintang@eleuther.ai nathan.habib19@gmail.com
_blimp.yaml
in lm_eval/tasks/blimp
 75 - 2024-07-08 2024-10-29 3 3 lintang@eleuther.ai nathan.habib19@gmail.com
_generate_configs.py
in lm_eval/tasks/belebele
 75 2 2023-09-25 2024-10-29 17 9 manuel.faysse@illuin.tech nathan.habib19@gmail.com
mmlu_high_school_statistics.yaml
in lm_eval/tasks/mmlu/flan_cot_fewshot
 73 - 2023-09-03 2024-10-29 13 7 lintang@sutawika.com nathan.habib19@gmail.com
task.py
in lm_eval/tasks/unitxt
 72 15 2024-07-08 2024-10-29 3 3 elronbandel@gmail.com nathan.habib19@gmail.com