huggingface / smollm

Duplication

Places in code with 6 or more lines that are exactly the same.

Longest Duplicates

The list of 50 longest duplicates.

See data for all 17,238 duplicates...

Size	#	Folders	Files	Lines	Code
1046	x 2	vision/m4/models/vllama3 vision/m4/models/vmistral	evaluation_classification_in_context_... evaluation_classification_in_context_...	359:1404 (31%) 361:1406 (33%)	view
1036	x 2	vision/m4/models/vgpt2 vision/m4/models/vmistral	evaluation_classification_in_context_... evaluation_classification_in_context_...	793:1828 (45%) 361:1396 (32%)	view
1036	x 2	vision/m4/models/vgpt2 vision/m4/models/vllama3	evaluation_classification_in_context_... evaluation_classification_in_context_...	793:1828 (45%) 359:1394 (31%)	view
1033	x 2	vision/m4/models/idefics vision/m4/models/vmistral	evaluation_classification_in_context_... evaluation_classification_in_context_...	361:1393 (34%) 361:1393 (32%)	view
1033	x 2	vision/m4/models/idefics vision/m4/models/vllama3	evaluation_classification_in_context_... evaluation_classification_in_context_...	361:1393 (34%) 359:1391 (31%)	view
1033	x 2	vision/m4/models/idefics vision/m4/models/vgpt2	evaluation_classification_in_context_... evaluation_classification_in_context_...	361:1393 (34%) 793:1825 (45%)	view
209	x 2	vision/m4/models/vllama3 vision/m4/models/vmistral	evaluation_classification_in_context_... evaluation_classification_in_context_...	1512:1720 (6%) 1514:1722 (6%)	view
196	x 2	vision/m4/models/vgpt2 vision/m4/models/vllama3	evaluation_classification_in_context_... evaluation_classification_in_context_...	1838:2033 (8%) 1512:1707 (5%)	view
196	x 2	vision/m4/models/vgpt2 vision/m4/models/vmistral	evaluation_classification_in_context_... evaluation_classification_in_context_...	1838:2033 (8%) 1514:1709 (6%)	view
170	x 2	vision/m4/sourcing/data_collection/visualization vision/m4/sourcing/data_...visualization/wikipedia	global_visualization.py global_visualization.py	72:267 (54%) 71:266 (54%)	view
165	x 2	vision/data/datasets_pro...eriments/explore/assets vision/m4/sourcing/data_...on/visualization/assets	DOM_tree_viz.html DOM_tree_viz.html	2:185 (100%) 2:185 (100%)	view
152	x 2	text/data/smoltalk/constraints text/data/smoltalk/constraints/pipeline	filter_ifeval_data.py pipeline.py	70:221 (49%) 14:165 (63%)	view
136	x 2	vision/m4/models/vllama3 vision/m4/models/vmistral	evaluation_classification_in_context_... evaluation_classification_in_context_...	2505:2642 (4%) 2504:2641 (4%)	view
125	x 2	vision/smolvlm2/smolvlm/model vision/smolvlm2/smolvlm/model	modeling_smollmm.py modeling_smolvlm.py	21:244 (64%) 17:240 (77%)	view
121	x 2	vision/m4/evaluation/custom_metrics vision/m4/evaluation/custom_metrics	utils.py utils.py	12:132 (27%) 185:305 (27%)	view
115	x 2	vision/m4/models/vllama3 vision/m4/models/vmistral	evaluation_classification_in_context_... evaluation_classification_in_context_...	2296:2411 (3%) 2295:2410 (3%)	view
112	x 2	vision/data/datasets_pro...eriments/python_scripts vision/m4/sourcing/get_modelling_metadata_dataset	get_modelling_metadata_dataset.py get_modelling_metadata_dataset.py	10:135 (100%) 10:135 (100%)	view
111	x 2	vision/data/datasets_pro..._dataset/python_scripts vision/m4/sourcing/data_collection/callers	05_filtering_web_docs.py filter_web_documents.py	113:228 (51%) 93:208 (59%)	view
108	x 2	vision/m4/models/vllama3 vision/m4/models/vmistral	evaluation_classification_in_context_... evaluation_classification_in_context_...	2707:2814 (3%) 2706:2813 (3%)	view
105	x 2	vision/smolvlm2/smolvlm/datasets vision/smolvlm2/smolvlm/datasets	dataset.py dataset_clip_sampling.py	427:561 (20%) 559:694 (17%)	view
96	x 2	vision/smolvlm2/smolvlm/datasets vision/smolvlm2/smolvlm/datasets	dataset.py dataset_clip_sampling.py	242:425 (19%) 374:557 (16%)	view
90	x 2	vision/m4/sourcing/data_collection/visualization vision/m4/sourcing/data_...visualization/wikipedia	global_visualization.py global_visualization.py	268:367 (28%) 268:367 (28%)	view
80	x 2	text/pretraining/continu...ining/finemath/60B-runs text/pretraining/continu...ining/finemath/60B-runs	finemath-infiwebmath-4plus.yaml openwebmath.yaml	26:105 (76%) 25:104 (76%)	view
80	x 2	text/pretraining/continu...ining/finemath/60B-runs text/pretraining/continu...ining/finemath/60B-runs	finemath-infiwebmath-3plus.yaml infiwebmath-4plus.yaml	26:105 (76%) 25:104 (76%)	view
80	x 2	text/pretraining/continu...ining/finemath/60B-runs text/pretraining/continu...ining/finemath/60B-runs	infiwebmath-4plus.yaml infiwebmath.yaml	25:104 (76%) 25:104 (76%)	view
80	x 2	text/pretraining/continu...ining/finemath/60B-runs text/pretraining/continu...ining/finemath/60B-runs	finemath-infiwebmath-3plus.yaml infiwebmath.yaml	26:105 (76%) 25:104 (76%)	view
80	x 2	text/pretraining/continu...ining/finemath/60B-runs text/pretraining/continu...ining/finemath/60B-runs	finemath-infiwebmath-3plus.yaml openwebmath.yaml	26:105 (76%) 25:104 (76%)	view
80	x 2	text/pretraining/continu...ining/finemath/60B-runs text/pretraining/continu...ining/finemath/60B-runs	finemath-3plus.yaml infiwebmath-3plus.yaml	25:104 (76%) 25:104 (76%)	view
80	x 2	text/pretraining/continu...ining/finemath/60B-runs text/pretraining/continu...ining/finemath/60B-runs	finemath-4plus.yaml openwebmath.yaml	25:104 (76%) 25:104 (76%)	view
80	x 2	text/pretraining/continu...ining/finemath/60B-runs text/pretraining/continu...ining/finemath/60B-runs	finemath-4plus.yaml infiwebmath-4plus.yaml	25:104 (76%) 25:104 (76%)	view
80	x 2	text/pretraining/continu...ining/finemath/60B-runs text/pretraining/continu...ining/finemath/60B-runs	finemath-4plus.yaml finemath-infiwebmath-4plus.yaml	25:104 (76%) 26:105 (76%)	view
80	x 2	text/pretraining/continu...ining/finemath/60B-runs text/pretraining/continu...ining/finemath/60B-runs	finemath-infiwebmath-4plus.yaml infiwebmath-3plus.yaml	26:105 (76%) 25:104 (76%)	view
80	x 2	text/pretraining/continu...ining/finemath/60B-runs text/pretraining/continu...ining/finemath/60B-runs	finemath-3plus.yaml openwebmath.yaml	25:104 (76%) 25:104 (76%)	view
80	x 2	text/pretraining/continu...ining/finemath/60B-runs text/pretraining/continu...ining/finemath/60B-runs	finemath-3plus.yaml finemath-4plus.yaml	25:104 (76%) 25:104 (76%)	view
80	x 2	text/pretraining/continu...ining/finemath/60B-runs text/pretraining/continu...ining/finemath/60B-runs	finemath-3plus.yaml infiwebmath.yaml	25:104 (76%) 25:104 (76%)	view
80	x 2	text/pretraining/continu...ining/finemath/60B-runs text/pretraining/continu...ining/finemath/60B-runs	finemath-infiwebmath-3plus.yaml finemath-infiwebmath-4plus.yaml	26:105 (76%) 26:105 (76%)	view
80	x 2	text/pretraining/continu...ining/finemath/60B-runs text/pretraining/continu...ining/finemath/60B-runs	finemath-3plus.yaml finemath-infiwebmath-4plus.yaml	25:104 (76%) 26:105 (76%)	view
80	x 2	text/pretraining/continu...ining/finemath/60B-runs text/pretraining/continu...ining/finemath/60B-runs	infiwebmath-3plus.yaml infiwebmath-4plus.yaml	25:104 (76%) 25:104 (76%)	view
80	x 2	text/pretraining/continu...ning/finemath/160B-runs text/pretraining/continu...ning/finemath/160B-runs	fwedu-finemath-infiwebmath-3plus.yaml fwedu-finemath-infiwebmath-4plus.yaml	31:110 (72%) 31:110 (72%)	view
80	x 2	text/pretraining/continu...ining/finemath/60B-runs text/pretraining/continu...ining/finemath/60B-runs	finemath-infiwebmath-4plus.yaml infiwebmath.yaml	26:105 (76%) 25:104 (76%)	view
80	x 2	text/pretraining/continu...ining/finemath/60B-runs text/pretraining/continu...ining/finemath/60B-runs	finemath-infiwebmath-4plus.yaml infiwebmath-4plus.yaml	26:105 (76%) 25:104 (76%)	view
80	x 2	text/pretraining/continu...ining/finemath/60B-runs text/pretraining/continu...ining/finemath/60B-runs	infiwebmath-4plus.yaml openwebmath.yaml	25:104 (76%) 25:104 (76%)	view
80	x 2	text/pretraining/continu...ining/finemath/60B-runs text/pretraining/continu...ining/finemath/60B-runs	finemath-3plus.yaml infiwebmath-4plus.yaml	25:104 (76%) 25:104 (76%)	view
80	x 2	text/pretraining/continu...ining/finemath/60B-runs text/pretraining/continu...ining/finemath/60B-runs	finemath-4plus.yaml infiwebmath-3plus.yaml	25:104 (76%) 25:104 (76%)	view
80	x 2	text/pretraining/continu...ining/finemath/60B-runs text/pretraining/continu...ining/finemath/60B-runs	finemath-4plus.yaml finemath-infiwebmath-3plus.yaml	25:104 (76%) 26:105 (76%)	view
80	x 2	text/pretraining/continu...ining/finemath/60B-runs text/pretraining/continu...ining/finemath/60B-runs	finemath-infiwebmath-3plus.yaml infiwebmath-3plus.yaml	26:105 (76%) 25:104 (76%)	view
80	x 2	vision/data/datasets_pro...ight_v02/python_scripts vision/data/datasets_pro...ight_v02/python_scripts	02_01_generate_html_codes_prompt_1.py 02_02_generate_html_codes_prompt_2.py	80:167 (60%) 82:169 (58%)	view
80	x 2	text/pretraining/continu...ining/finemath/60B-runs text/pretraining/continu...ining/finemath/60B-runs	infiwebmath-3plus.yaml openwebmath.yaml	25:104 (76%) 25:104 (76%)	view
80	x 2	text/pretraining/continu...ining/finemath/60B-runs text/pretraining/continu...ining/finemath/60B-runs	infiwebmath-3plus.yaml infiwebmath.yaml	25:104 (76%) 25:104 (76%)	view
80	x 2	text/pretraining/continu...ining/finemath/60B-runs text/pretraining/continu...ining/finemath/60B-runs	infiwebmath.yaml openwebmath.yaml	25:104 (76%) 25:104 (76%)	view

Duplicated Units

The list of top 5 duplicated units.

See data for all 5 unit duplicates...

Size	#	Folders	Files	Lines	Code
47	x 3	vision/experiments/pretr..._smolvlm_025b_1st_stage vision/experiments/pretr...3_smolvlm_05b_1st_stage vision/experiments/pretr...vloom/tr_348_smolvlm_2B	resize_embed_for_eou.py resize_embed_for_eou.py resize_embed_for_eou.py	0:0  0:0  0:0	view
38	x 5	vision/experiments/pretr...vsmollm2_256M_3rd_stage vision/experiments/pretr...vsmollm2_500M_3rd_stage vision/experiments/pretr...50_smolvlm_2B_2nd_stage vision/experiments/pretr...vsmollm2_256M_2nd_stage vision/experiments/pretr..._smolvlm_500M_2nd_stage	resize_embed_for_eou.py resize_embed_for_eou.py resize_embed_for_eou.py resize_embed_for_eou.py resize_embed_for_eou.py	0:0  0:0  0:0  0:0  0:0	view
26	x 2	vision/data/datasets_pro...ight_v02/python_scripts vision/data/datasets_pro...ight_v02/python_scripts	02_02_generate_html_codes_prompt_2.py 02_01_generate_html_codes_prompt_1.py	0:0  0:0	view
12	x 2	vision/smolvlm2/smolvlm/model vision/smolvlm2/smolvlm/model	modeling_smolvlm.py modeling_smollmm.py	0:0  0:0	view
11	x 2	vision/data/datasets_pro...eriments/python_scripts vision/m4/sourcing/get_modelling_metadata_dataset	get_modelling_metadata_dataset.py get_modelling_metadata_dataset.py	0:0  0:0	view