huggingface / lm-evaluation-harness

Duplication

Places in code with 6 or more lines that are exactly the same.

Longest Duplicates

The list of 50 longest duplicates.

See data for all 4,773 duplicates...

Size	#	Folders	Files	Lines	Code
1630	x 2	lm_eval/tasks/ifeval lm_eval/tasks/leaderboard/ifeval	instructions_util.py instructions_util.py	25:1682 (100%) 25:1682 (100%)	view
829	x 2	lm_eval/tasks/ifeval lm_eval/tasks/leaderboard/ifeval	instructions.py instructions.py	30:1612 (100%) 30:1612 (100%)	view
155	x 2	lm_eval/tasks/bbh/cot_zeroshot lm_eval/tasks/bbh/zeroshot	utils.py utils.py	9:224 (100%) 9:224 (100%)	view
110	x 2	lm_eval/tasks/ifeval lm_eval/tasks/leaderboard/ifeval	instructions_registry.py instructions_registry.py	20:168 (100%) 20:168 (100%)	view
101	x 2	lm_eval/tasks/tinyBenchmarks lm_eval/tasks/truthfulqa	utils_truthfulqa.py utils.py	9:176 (100%) 7:176 (100%)	view
84	x 2	lm_eval/tasks/ifeval lm_eval/tasks/leaderboard/ifeval	utils.py utils.py	8:112 (79%) 7:111 (81%)	view
75	x 2	lm_eval/filters lm_eval/tasks/mmlu/flan_n_shot/generative	extraction.py utils.py	75:184 (64%) 8:112 (100%)	view
75	x 2	lm_eval/filters lm_eval/tasks/mmlu/flan_cot_zeroshot	extraction.py utils.py	75:184 (64%) 8:112 (100%)	view
75	x 2	lm_eval/tasks/mmlu/flan_cot_zeroshot lm_eval/tasks/mmlu/flan_n_shot/generative	utils.py utils.py	8:112 (100%) 8:112 (100%)	view
64	x 2	lm_eval/tasks/bbh/fewshot lm_eval/tasks/leaderboard/bbh_mc	reasoning_about_colored_objects.yaml reasoning_about_colored_objects.yaml	9:118 (91%) 6:115 (92%)	view
60	x 2	lm_eval/tasks/bbh/cot_fewshot lm_eval/tasks/bbh/cot_fewshot	logical_deduction_seven_objects.yaml logical_deduction_three_objects.yaml	2:92 (96%) 2:92 (96%)	view
60	x 2	lm_eval/tasks/bbh/cot_fewshot lm_eval/tasks/bbh/cot_fewshot	logical_deduction_five_objects.yaml logical_deduction_three_objects.yaml	2:92 (96%) 2:92 (96%)	view
60	x 2	lm_eval/tasks/mmlu lm_eval/tasks/mmlusr	_generate_configs.py config.py	13:74 (46%) 13:74 (52%)	view
60	x 2	lm_eval/tasks/bbh/cot_fewshot lm_eval/tasks/bbh/cot_fewshot	logical_deduction_five_objects.yaml logical_deduction_seven_objects.yaml	2:92 (96%) 2:92 (96%)	view
58	x 2	lm_eval/tasks/bbh/cot_fewshot lm_eval/tasks/bbh/cot_fewshot	tracking_shuffled_objects_five_object... tracking_shuffled_objects_three_objec...	2:93 (96%) 2:93 (96%)	view
58	x 2	lm_eval/tasks/bbh/cot_fewshot lm_eval/tasks/bbh/cot_fewshot	tracking_shuffled_objects_seven_objec... tracking_shuffled_objects_three_objec...	2:93 (96%) 2:93 (96%)	view
58	x 2	lm_eval/tasks/bbh/cot_fewshot lm_eval/tasks/bbh/cot_fewshot	tracking_shuffled_objects_five_object... tracking_shuffled_objects_seven_objec...	2:93 (96%) 2:93 (96%)	view
53	x 2	lm_eval/tasks/bbh/fewshot lm_eval/tasks/leaderboard/bbh_mc	temporal_sequences.yaml temporal_sequences.yaml	10:103 (88%) 7:100 (89%)	view
53	x 2	lm_eval/tasks/afrimgsm lm_eval/tasks/mgsm	utils.py utils.py	153:209 (27%) 162:218 (28%)	view
47	x 2	lm_eval/tasks/bbh/fewshot lm_eval/tasks/leaderboard/bbh_mc	geometric_shapes.yaml geometric_shapes.yaml	9:88 (88%) 6:85 (90%)	view
46	x 2	lm_eval/tasks/bbh/fewshot lm_eval/tasks/leaderboard/bbh_mc	causal_judgement.yaml causal_judgement.yaml	9:66 (88%) 6:63 (90%)	view
42	x 2	lm_eval/tasks/bbh/fewshot lm_eval/tasks/leaderboard/bbh_mc	penguins_in_a_table.yaml penguins_in_a_table.yaml	9:70 (87%) 6:67 (89%)	view
42	x 2	lm_eval/tasks/bbh/fewshot lm_eval/tasks/bbh/fewshot	tracking_shuffled_objects_five_object... tracking_shuffled_objects_seven_objec...	2:61 (95%) 2:61 (95%)	view
42	x 2	lm_eval/tasks/bbh/fewshot lm_eval/tasks/bbh/fewshot	tracking_shuffled_objects_seven_objec... tracking_shuffled_objects_three_objec...	2:61 (95%) 2:61 (95%)	view
42	x 2	lm_eval/tasks/bbh/fewshot lm_eval/tasks/bbh/fewshot	tracking_shuffled_objects_five_object... tracking_shuffled_objects_three_objec...	2:61 (95%) 2:61 (95%)	view
41	x 2	lm_eval/tasks/squad_completion lm_eval/tasks/swde	task.py task.py	13:98 (93%) 13:98 (93%)	view
41	x 2	lm_eval/tasks/fda lm_eval/tasks/squad_completion	task.py task.py	13:98 (93%) 13:98 (93%)	view
41	x 2	lm_eval/tasks/fda lm_eval/tasks/swde	task.py task.py	13:98 (93%) 13:98 (93%)	view
39	x 2	lm_eval/tasks/bbh/fewshot lm_eval/tasks/leaderboard/bbh_mc	formal_fallacies.yaml formal_fallacies.yaml	9:59 (86%) 6:56 (88%)	view
38	x 2	lm_eval/tasks/bbh/fewshot lm_eval/tasks/bbh/fewshot	logical_deduction_five_objects.yaml logical_deduction_three_objects.yaml	2:54 (95%) 2:54 (95%)	view
38	x 2	lm_eval/tasks/bbh/fewshot lm_eval/tasks/bbh/fewshot	logical_deduction_five_objects.yaml logical_deduction_seven_objects.yaml	2:54 (95%) 2:54 (95%)	view
38	x 2	lm_eval/tasks/bbh/fewshot lm_eval/tasks/bbh/fewshot	logical_deduction_seven_objects.yaml logical_deduction_three_objects.yaml	2:54 (95%) 2:54 (95%)	view
37	x 2	lm_eval/tasks/bbh/fewshot lm_eval/tasks/leaderboard/bbh_mc	tracking_shuffled_objects_seven_objec... tracking_shuffled_objects_three_objec...	10:61 (84%) 7:58 (86%)	view
37	x 2	lm_eval/tasks/bbh/fewshot lm_eval/tasks/leaderboard/bbh_mc	tracking_shuffled_objects_three_objec... tracking_shuffled_objects_five_object...	10:61 (84%) 7:58 (86%)	view
37	x 2	lm_eval/tasks/bbh/fewshot lm_eval/tasks/leaderboard/bbh_mc	tracking_shuffled_objects_five_object... tracking_shuffled_objects_five_object...	10:61 (84%) 7:58 (86%)	view
37	x 2	lm_eval/tasks/leaderboard/bbh_mc lm_eval/tasks/leaderboard/bbh_mc	tracking_shuffled_objects_seven_objec... tracking_shuffled_objects_three_objec...	7:58 (86%) 7:58 (86%)	view
37	x 2	lm_eval/tasks/bbh/fewshot lm_eval/tasks/leaderboard/bbh_mc	tracking_shuffled_objects_three_objec... tracking_shuffled_objects_three_objec...	10:61 (84%) 7:58 (86%)	view
37	x 2	lm_eval/tasks/bbh/fewshot lm_eval/tasks/leaderboard/bbh_mc	tracking_shuffled_objects_five_object... tracking_shuffled_objects_seven_objec...	10:61 (84%) 7:58 (86%)	view
37	x 2	lm_eval/tasks/bbh/fewshot lm_eval/tasks/leaderboard/bbh_mc	tracking_shuffled_objects_three_objec... tracking_shuffled_objects_seven_objec...	10:61 (84%) 7:58 (86%)	view
37	x 2	lm_eval/tasks/leaderboard/bbh_mc lm_eval/tasks/leaderboard/bbh_mc	tracking_shuffled_objects_five_object... tracking_shuffled_objects_seven_objec...	7:58 (86%) 7:58 (86%)	view
37	x 2	lm_eval/tasks/bbh/fewshot lm_eval/tasks/leaderboard/bbh_mc	tracking_shuffled_objects_seven_objec... tracking_shuffled_objects_seven_objec...	10:61 (84%) 7:58 (86%)	view
37	x 2	lm_eval/tasks/leaderboard/bbh_mc lm_eval/tasks/leaderboard/bbh_mc	tracking_shuffled_objects_five_object... tracking_shuffled_objects_three_objec...	7:58 (86%) 7:58 (86%)	view
37	x 2	lm_eval/tasks/bbh/fewshot lm_eval/tasks/leaderboard/bbh_mc	tracking_shuffled_objects_seven_objec... tracking_shuffled_objects_five_object...	10:61 (84%) 7:58 (86%)	view
37	x 2	lm_eval/tasks/afrimgsm lm_eval/tasks/mgsm	utils.py utils.py	72:118 (19%) 94:140 (20%)	view
37	x 2	lm_eval/tasks/bbh/fewshot lm_eval/tasks/leaderboard/bbh_mc	tracking_shuffled_objects_five_object... tracking_shuffled_objects_three_objec...	10:61 (84%) 7:58 (86%)	view
35	x 2	lm_eval/tasks/bbh/cot_fewshot lm_eval/tasks/bbh/cot_fewshot	_bbh.yaml _bbh_cot_fewshot.yaml	2:36 (97%) 2:36 (97%)	view
33	x 2	lm_eval/tasks/bbh/fewshot lm_eval/tasks/leaderboard/bbh_mc	logical_deduction_five_objects.yaml logical_deduction_three_objects.yaml	10:54 (82%) 7:51 (84%)	view
33	x 2	lm_eval/tasks/bbh/fewshot lm_eval/tasks/leaderboard/bbh_mc	logical_deduction_three_objects.yaml logical_deduction_three_objects.yaml	10:54 (82%) 7:51 (84%)	view
33	x 2	lm_eval/tasks/bbh/fewshot lm_eval/tasks/leaderboard/bbh_mc	logical_deduction_five_objects.yaml logical_deduction_seven_objects.yaml	10:54 (82%) 7:51 (84%)	view
33	x 2	lm_eval/tasks/leaderboard/bbh_mc lm_eval/tasks/leaderboard/bbh_mc	logical_deduction_seven_objects.yaml logical_deduction_three_objects.yaml	7:51 (84%) 7:51 (84%)	view

Duplicated Units

The list of top 12 duplicated units.

See data for all 12 unit duplicates...

Size	#	Folders	Files	Lines	Code
64	x 3	lm_eval/filters lm_eval/tasks/mmlu/flan_n_shot/generative lm_eval/tasks/mmlu/flan_cot_zeroshot	extraction.py utils.py utils.py	0:0  0:0  0:0	view
62	x 2	lm_eval/tasks/tinyBenchmarks lm_eval/tasks/truthfulqa	utils_truthfulqa.py utils.py	0:0  0:0	view
45	x 2	lm_eval/tasks/leaderboard/ifeval lm_eval/tasks/ifeval	instructions_util.py instructions_util.py	0:0  0:0	view
40	x 2	lm_eval/tasks/bbh/cot_zeroshot lm_eval/tasks/bbh/zeroshot	utils.py utils.py	0:0  0:0	view
43	x 2	lm_eval/tasks/leaderboard/ifeval lm_eval/tasks/ifeval	instructions.py instructions.py	0:0  0:0	view
18	x 3	lm_eval/tasks/gpqa/cot_zeroshot lm_eval/tasks/gpqa/cot_n_shot lm_eval/tasks/gpqa/generative	utils.py utils.py utils.py	0:0  0:0  0:0	view
17	x 2	lm_eval/tasks/leaderboard/gpqa lm_eval/tasks/gpqa/zeroshot	utils.py utils.py	0:0  0:0	view
12	x 3	lm_eval/tasks/mmlusr/question_only lm_eval/tasks/mmlusr/question_and_answer lm_eval/tasks/mmlusr/answer_only	utils.py utils.py utils.py	0:0  0:0  0:0	view
18	x 2	lm_eval/tasks/afrimmlu/translate lm_eval/tasks/afrimmlu	utils.py utils.py	0:0  0:0	view
17	x 3	lm_eval/tasks/squad_completion lm_eval/tasks/swde lm_eval/tasks/fda	task.py task.py task.py	0:0  0:0  0:0	view
8	x 3	lm_eval/tasks/hellaswag lm_eval/tasks/tinyBenchmarks lm_eval/tasks/okapi/hellaswag_multilingual	utils.py utils_hellaswag.py utils.py	0:0  0:0  0:0	view
12	x 2	lm_eval/tasks/leaderboard/ifeval lm_eval/tasks/ifeval	instructions_registry.py instructions_registry.py	0:0  0:0	view