huggingface / search-and-learn Duplication

Intro

For duplication, we look at places in code where there are 6 or more lines of code that are exactly the same.
Before duplication is calculated, the code is cleaned to remove empty lines, comments, and frequently duplicated constructs such as imports.
You should aim at having as little as possible (<5%) of duplicated code as high-level of duplication can lead to maintenance difficulties, poor factoring, and logical contradictions.

Learn more...

Duplication Overall

9% duplication:

2,579 cleaned lines of cleaned code (without empty lines, comments, and frequently duplicated constructs such as imports)
238 duplicated lines

23 duplicates

dependency graphs: 2D graph | 3D graph | 3D graph (with duplicates)...

Duplication per Extension

Duplication per Component (primary)

Longest Duplicates

The list of 23 longest duplicates.

See data for all 23 duplicates...

Size	#	Folders	Files	Lines	Code
15	x 2	src/sal/utils src/sal/utils	qwen_math_parser.py qwen_math_parser.py	442:456 (2%) 500:514 (2%)	view
14	x 2	src/sal/search src/sal/search	beam_search.py diverse_verifier_tree_search.py	98:112 (8%) 82:97 (10%)	view
8	x 2	src/sal/models/skywork_o1_prm src/sal/models/skywork_o1_prm	modeling_base.py modeling_base.py	343:351 (1%) 359:366 (1%)	view
8	x 2	src/sal/models src/sal/models	reward_models.py reward_models.py	294:304 (2%) 342:352 (2%)	view
8	x 2	recipes/AceMath-7B-Instruct recipes/Qwen2.5-1.5B-Instruct	best_of_n.yaml best_of_n.yaml	5:12 (80%) 4:11 (88%)	view
8	x 2	src/sal/models src/sal/models	reward_models.py reward_models.py	184:192 (2%) 229:236 (2%)	view
7	x 2	recipes/AceMath-7B-Instruct recipes/Qwen2.5-1.5B-Instruct	beam_search.yaml beam_search.yaml	5:11 (77%) 4:10 (87%)	view
7	x 2	recipes/AceMath-7B-Instruct recipes/Llama-3.2-3B-Instruct	best_of_n.yaml best_of_n.yaml	6:12 (70%) 4:10 (87%)	view
7	x 2	src/sal/search src/sal/search	beam_search.py diverse_verifier_tree_search.py	45:51 (4%) 48:54 (5%)	view
7	x 2	recipes/Llama-3.2-1B-Instruct recipes/Llama-3.2-3B-Instruct	best_of_n.yaml best_of_n.yaml	3:9 (100%) 4:10 (87%)	view
7	x 2	recipes/Llama-3.2-3B-Instruct recipes/Qwen2.5-1.5B-Instruct	best_of_n.yaml best_of_n.yaml	4:10 (87%) 5:11 (77%)	view
7	x 2	recipes/Llama-3.2-1B-Instruct recipes/Qwen2.5-1.5B-Instruct	best_of_n.yaml best_of_n.yaml	3:9 (100%) 5:11 (77%)	view
7	x 2	src/sal/models src/sal/models	reward_models.py reward_models.py	283:289 (2%) 330:336 (2%)	view
7	x 2	recipes/AceMath-7B-Instruct recipes/Llama-3.2-1B-Instruct	best_of_n.yaml best_of_n.yaml	6:12 (70%) 3:9 (100%)	view
6	x 2	src/sal/search src/sal/search	beam_search.py diverse_verifier_tree_search.py	205:212 (3%) 167:174 (4%)	view
6	x 2	src/sal/utils src/sal/utils	qwen_math_parser.py qwen_math_parser.py	412:419 (<1%) 565:571 (<1%)	view
6	x 2	recipes/AceMath-7B-Instruct recipes/Llama-3.2-3B-Instruct	beam_search.yaml beam_search.yaml	6:11 (66%) 4:9 (85%)	view
6	x 2	src/sal/search src/sal/search	beam_search.py best_of_n.py	88:93 (3%) 50:55 (10%)	view
6	x 2	recipes/AceMath-7B-Instruct recipes/Llama-3.2-1B-Instruct	beam_search.yaml beam_search.yaml	6:11 (66%) 3:8 (100%)	view
6	x 2	recipes/Llama-3.2-1B-Instruct recipes/Qwen2.5-1.5B-Instruct	beam_search.yaml beam_search.yaml	3:8 (100%) 5:10 (75%)	view
6	x 2	recipes/Llama-3.2-3B-Instruct recipes/Qwen2.5-1.5B-Instruct	beam_search.yaml beam_search.yaml	4:9 (85%) 5:10 (75%)	view
6	x 2	recipes/AceMath-7B-Instruct recipes/Qwen2.5-1.5B-Instruct	dvts.yaml dvts.yaml	5:10 (75%) 4:9 (85%)	view
6	x 2	recipes/Llama-3.2-1B-Instruct recipes/Llama-3.2-3B-Instruct	beam_search.yaml beam_search.yaml	3:8 (100%) 4:9 (85%)	view