huggingface / candle Duplication

Intro

For duplication, we look at places in code where there are 6 or more lines of code that are exactly the same.
Before duplication is calculated, the code is cleaned to remove empty lines, comments, and frequently duplicated constructs such as imports.
You should aim at having as little as possible (<5%) of duplicated code as high-level of duplication can lead to maintenance difficulties, poor factoring, and logical contradictions.

Learn more...

Duplication Overall

34% duplication:

113,181 cleaned lines of cleaned code (without empty lines, comments, and frequently duplicated constructs such as imports)
38,851 duplicated lines

6,030 duplicates

dependency graphs: 2D graph | 3D graph | 3D graph (with duplicates)...

Duplication per Extension

Duplication per Component (primary)

Duplication Between Components (50+ lines)

Download: SVG DOT (open online Graphviz editor)

From Component --> To Component	Duplicated Lines	File Pairs	Details
candle-transformers (<1%) --> candle-wasm-examples (5%)	922	16 file pairs	details...
candle-core (<1%) --> candle-nn (4%)	384	31 file pairs	details...
candle-examples (10%) --> candle-wasm-examples (2%)	348	11 file pairs	details...
candle-kernels (1%) --> candle-metal-kernels (<1%)	212	3 file pairs	details...
candle-flash-attn (1%) --> candle-transformers (<1%)	90	14 file pairs	details...
candle-wasm-examples (<1%) --> tensor-tools (9%)	80	9 file pairs	details...
candle-nn (<1%) --> candle-transformers (<1%)	74	9 file pairs	details...
candle-datasets (7%) --> candle-wasm-examples (<1%)	70	9 file pairs	details...
candle-nn (<1%) --> candle-wasm-examples (<1%)	70	9 file pairs	details...
candle-core (<1%) --> candle-wasm-examples (<1%)	70	9 file pairs	details...
candle-book (11%) --> candle-wasm-examples (<1%)	70	9 file pairs	details...
candle-pyo3 (1%) --> candle-wasm-examples (<1%)	70	9 file pairs	details...
candle-core (<1%) --> candle-metal-kernels (<1%)	59	3 file pairs	details...

Open 2D force graph... Open 3D force graph...

Show more details on duplication between components...

Longest Duplicates

The list of 50 longest duplicates.

See data for all 6,030 duplicates...

Size	#	Folders	Files	Lines	Code
442	x 2	candle-transformers/src/models candle-transformers/src/models	codegeex4_9b.rs glm4.rs	88:610 (86%) 85:607 (86%)	view
312	x 2	candle-transformers/src/models candle-transformers/src/models	chatglm.rs glm4.rs	214:588 (63%) 232:607 (61%)	view
312	x 2	candle-transformers/src/models candle-transformers/src/models	chatglm.rs codegeex4_9b.rs	214:588 (63%) 235:610 (61%)	view
271	x 2	candle-transformers/src/models candle-transformers/src/models	blip_text.rs quantized_blip_text.rs	30:392 (75%) 26:388 (77%)	view
164	x 2	candle-transformers/src/models candle-transformers/src/models	quantized_recurrent_gemma.rs recurrent_gemma.rs	117:312 (47%) 350:545 (30%)	view
162	x 2	candle-transformers/src/models candle-transformers/src/models	blip.rs quantized_blip.rs	99:315 (67%) 60:275 (81%)	view
149	x 2	candle-transformers/src/models candle-transformers/src/models	granite.rs llama.rs	128:305 (40%) 170:349 (34%)	view
148	x 2	candle-transformers/src/models candle-transformers/src/models	llama2_c.rs quantized_llama2_c.rs	137:326 (51%) 22:211 (77%)	view
120	x 2	candle-transformers/src/models candle-transformers/src/models	quantized_t5.rs t5.rs	291:419 (19%) 420:548 (16%)	view
114	x 2	candle-transformers/src/models candle-transformers/src/models	helium.rs mistral.rs	179:317 (35%) 228:366 (30%)	view
107	x 2	candle-transformers/src/models/whisper candle-transformers/src/models/whisper	model.rs quantized_model.rs	25:153 (32%) 22:150 (32%)	view
101	x 2	candle-transformers/src/models candle-transformers/src/models	qwen2.rs qwen2_moe.rs	94:217 (32%) 101:225 (27%)	view
100	x 2	candle-transformers/src/models candle-transformers/src/models	quantized_t5.rs t5.rs	431:558 (15%) 561:688 (13%)	view
100	x 2	candle-transformers/src/models candle-transformers/src/models	quantized_rwkv_v6.rs rwkv_v6.rs	111:230 (35%) 89:208 (40%)	view
99	x 2	candle-transformers/src/models candle-transformers/src/models	granite.rs llama.rs	308:429 (26%) 353:474 (23%)	view
93	x 2	candle-transformers/src/models/flux candle-transformers/src/models/flux	model.rs quantized_model.rs	345:458 (18%) 184:297 (25%)	view
82	x 2	candle-examples/src candle-wasm-examples/yolo/src	coco_classes.rs coco_classes.rs	1:82 (100%) 1:82 (100%)	view
79	x 2	candle-transformers/src/models candle-transformers/src/models	quantized_rwkv_v5.rs quantized_rwkv_v6.rs	206:301 (32%) 253:348 (27%)	view
78	x 2	candle-transformers/src/models candle-transformers/src/models	quantized_rwkv_v5.rs rwkv_v5.rs	86:176 (31%) 137:227 (23%)	view
78	x 2	candle-transformers/src/models candle-transformers/src/models	dinov2.rs dinov2reg4.rs	47:153 (27%) 40:146 (36%)	view
75	x 2	candle-transformers/src/models candle-transformers/src/models	quantized_mistral.rs yi.rs	64:155 (28%) 116:207 (26%)	view
74	x 2	candle-transformers/src/models candle-transformers/src/models	gemma.rs gemma2.rs	46:136 (20%) 39:129 (19%)	view
73	x 2	candle-transformers/src/models/whisper candle-wasm-examples/whisper/src	audio.rs audio.rs	13:111 (30%) 7:105 (47%)	view
71	x 2	candle-transformers/src/models candle-transformers/src/models	olmo.rs qwen2.rs	37:122 (25%) 40:125 (22%)	view
70	x 2	candle-transformers/src/models candle-transformers/src/models	olmo.rs olmo2.rs	30:116 (25%) 22:108 (24%)	view
70	x 2	candle-transformers/src/models/flux candle-transformers/src/models/flux	model.rs quantized_model.rs	542:624 (14%) 381:463 (19%)	view
69	x 2	candle-transformers/src/models candle-transformers/src/models	blip_text.rs quantized_blip_text.rs	395:495 (19%) 391:491 (19%)	view
68	x 2	candle-transformers/src/models candle-transformers/src/models	beit.rs dinov2.rs	314:389 (21%) 301:376 (24%)	view
68	x 2	candle-transformers/src/models candle-transformers/src/models	dinov2.rs eva2.rs	301:376 (24%) 342:417 (20%)	view
68	x 2	candle-transformers/src/models candle-transformers/src/models	beit.rs eva2.rs	314:389 (21%) 342:417 (20%)	view
67	x 2	candle-transformers/src/models candle-transformers/src/models	quantized_rwkv_v6.rs rwkv_v6.rs	237:317 (23%) 211:291 (26%)	view
66	x 2	candle-transformers/src/models candle-transformers/src/models	qwen3.rs qwen3_moe.rs	302:387 (22%) 268:353 (24%)	view
66	x 2	candle-wasm-examples/moondream candle-wasm-examples/phi	index.html index.html	124:191 (24%) 486:553 (12%)	view
66	x 2	candle-transformers/src/models candle-transformers/src/models	beit.rs dinov2reg4.rs	155:247 (20%) 90:182 (30%)	view
65	x 2	candle-transformers/src/models candle-transformers/src/models	olmo2.rs qwen2.rs	29:108 (23%) 40:119 (20%)	view
61	x 2	candle-transformers/src/models candle-transformers/src/models	dinov2.rs dinov2reg4.rs	158:235 (21%) 151:228 (28%)	view
60	x 2	candle-nn/src candle-nn/src	rotary_emb.rs rotary_emb.rs	67:132 (12%) 349:414 (12%)	view
59	x 2	candle-transformers/src/models/whisper candle-transformers/src/models/whisper	model.rs quantized_model.rs	198:267 (17%) 194:263 (17%)	view
57	x 2	candle-transformers/src/models candle-transformers/src/models	quantized_t5.rs t5.rs	560:636 (9%) 690:766 (7%)	view
56	x 2	candle-transformers/src/models candle-transformers/src/models	quantized_mistral.rs qwen2.rs	183:253 (21%) 195:265 (17%)	view
55	x 2	candle-transformers/src/models/nvembed_v2 candle-transformers/src/models	embedding.rs quantized_mistral.rs	30:97 (23%) 45:112 (20%)	view
54	x 2	candle-transformers/src/models candle-transformers/src/models	quantized_t5.rs t5.rs	193:260 (8%) 322:389 (7%)	view
53	x 2	candle-transformers/src/models candle-transformers/src/models	quantized_rwkv_v5.rs rwkv_v6.rs	206:270 (21%) 227:291 (21%)	view
53	x 2	candle-metal-kernels/src candle-metal-kernels/src	metal mlx_gemm.metal scaled_dot_product_attention.metal	134:194 (4%) 488:548 (4%)	view
53	x 2	candle-metal-kernels/src candle-metal-kernels/src	metal mlx_gemm.metal scaled_dot_product_attention.metal	324:386 (4%) 624:686 (4%)	view
52	x 2	candle-transformers/src/models candle-transformers/src/models	mpt.rs quantized_mpt.rs	73:128 (21%) 50:105 (31%)	view
51	x 2	candle-transformers/src/models candle-transformers/src/models	helium.rs phi3.rs	277:337 (15%) 313:373 (15%)	view
51	x 2	candle-transformers/src/models candle-transformers/src/models	mixtral.rs qwen2_moe.rs	423:481 (13%) 417:475 (13%)	view
50	x 2	candle-transformers/src/models candle-transformers/src/models	metavoice.rs quantized_metavoice.rs	725:784 (6%) 28:87 (25%)	view
50	x 2	candle-transformers/src/models candle-transformers/src/models	metavoice.rs quantized_metavoice.rs	786:850 (6%) 89:153 (25%)	view

Duplicated Units

The list of top 50 duplicated units.

See data for all 142 unit duplicates...

Size	#	Folders	Files	Lines	Code
87	x 3	candle-transformers/src/models candle-transformers/src/models candle-transformers/src/models	glm4.rs chatglm.rs codegeex4_9b.rs	254:349 236:331 257:352	view
55	x 2	candle-transformers/src/models candle-transformers/src/models	helium.rs mistral.rs	194:258 243:307	view
53	x 2	candle-transformers/src/models candle-transformers/src/models	quantized_recurrent_gemma.rs recurrent_gemma.rs	215:274 448:507	view
51	x 2	candle-transformers/src/models candle-transformers/src/models	glm4.rs codegeex4_9b.rs	148:201 151:204	view
48	x 2	candle-transformers/src/models candle-transformers/src/models	marian.rs trocr.rs	292:341 206:255	view
47	x 2	candle-transformers/src/models candle-transformers/src/models	qwen2.rs qwen2_moe.rs	155:210 162:217	view
42	x 2	candle-transformers/src/models candle-transformers/src/models	quantized_blip_text.rs blip_text.rs	114:158 118:162	view
39	x 2	candle-transformers/src/models candle-transformers/src/models	llama2_c.rs quantized_llama2_c.rs	170:217 55:102	view
39	x 2	candle-transformers/src/models/flux candle-transformers/src/models/flux	quantized_model.rs model.rs	421:464 582:625	view
37	x 3	candle-transformers/src/models candle-transformers/src/models candle-transformers/src/models	glm4.rs chatglm.rs codegeex4_9b.rs	398:436 380:418 401:439	view
36	x 2	candle-transformers/src/models candle-transformers/src/models	quantized_metavoice.rs metavoice.rs	92:134 789:831	view
35	x 2	candle-transformers/src/models/whisper candle-wasm-examples/whisper/src	audio.rs audio.rs	17:60 11:54	view
35	x 2	candle-transformers/src/models/flux candle-transformers/src/models/flux	quantized_model.rs model.rs	236:278 397:439	view
31	x 3	candle-transformers/src/models candle-transformers/src/models candle-transformers/src/models	glm4.rs chatglm.rs codegeex4_9b.rs	477:509 459:491 480:512	view
31	x 2	candle-transformers/src/models/whisper candle-transformers/src/models/whisper	quantized_model.rs model.rs	57:89 60:92	view
30	x 2	candle-transformers/src/models candle-transformers/src/models	glm4.rs codegeex4_9b.rs	217:248 220:251	view
30	x 2	candle-transformers/src/models/whisper candle-transformers/src/models/whisper	quantized_model.rs model.rs	97:128 100:131	view
25	x 2	candle-transformers/src/models candle-transformers/src/models	mixtral.rs qwen2_moe.rs	438:465 432:459	view
25	x 2	candle-transformers/src/models candle-transformers/src/models	t5.rs quantized_t5.rs	185:211 87:113	view
24	x 3	candle-transformers/src/models candle-transformers/src/models candle-transformers/src/models	quantized_qwen3.rs qwen3_moe.rs qwen3.rs	385:410 283:308 317:342	view
24	x 2	candle-transformers/src/models/flux candle-transformers/src/models/flux	quantized_model.rs model.rs	209:234 370:395	view
23	x 3	candle-transformers/src/models candle-transformers/src/models candle-transformers/src/models	glm4.rs chatglm.rs codegeex4_9b.rs	91:115 82:106 94:118	view
23	x 3	candle-transformers/src/models candle-transformers/src/models candle-transformers/src/models	glm4.rs chatglm.rs codegeex4_9b.rs	442:466 424:448 445:469	view
23	x 2	candle-transformers/src/models candle-transformers/src/models	t5.rs quantized_t5.rs	698:728 568:598	view
22	x 2	candle-transformers/src/models candle-transformers/src/models	granite.rs llama.rs	319:342 364:387	view
22	x 2	candle-transformers/src/models candle-transformers/src/models	qwen2.rs qwen2_moe.rs	130:153 137:160	view
22	x 2	candle-transformers/src/models candle-transformers/src/models	quantized_mistral.rs yi.rs	117:140 169:192	view
21	x 2	candle-transformers/src/models candle-transformers/src/models	quantized_blip_text.rs blip_text.rs	75:97 79:101	view
20	x 2	candle-transformers/src/models candle-transformers/src/models	blip.rs quantized_blip.rs	147:168 108:129	view
20	x 3	candle-transformers/src/models candle-transformers/src/models candle-transformers/src/models	beit.rs dinov2.rs eva2.rs	314:335 301:322 342:363	view
20	x 2	candle-transformers/src/models candle-transformers/src/models	quantized_rwkv_v5.rs quantized_rwkv_v6.rs	177:198 224:245	view
20	x 2	candle-transformers/src/models candle-transformers/src/models	quantized_recurrent_gemma.rs recurrent_gemma.rs	187:208 420:441	view
19	x 2	candle-transformers/src/models/whisper candle-wasm-examples/whisper/src	audio.rs audio.rs	63:86 57:80	view
19	x 2	candle-transformers/src/models/flux candle-transformers/src/models/flux	quantized_model.rs model.rs	110:130 271:291	view
19	x 2	candle-transformers/src/models candle-transformers/src/models	t5.rs quantized_t5.rs	352:372 223:243	view
18	x 2	candle-transformers/src/models/whisper candle-transformers/src/models/whisper	quantized_model.rs model.rs	36:55 39:58	view
17	x 2	candle-transformers/src/models candle-transformers/src/models	olmo2.rs olmo.rs	305:324 310:329	view
17	x 2	candle-transformers/src/models candle-transformers/src/models	mixtral.rs yi.rs	82:100 83:101	view
17	x 3	candle-transformers/src/models candle-transformers/src/models candle-transformers/src/models	quantized_rwkv_v5.rs quantized_rwkv_v6.rs rwkv_v6.rs	225:243 272:290 246:264	view
17	x 3	candle-transformers/src/models candle-transformers/src/models candle-transformers/src/models	dinov2.rs dinov2reg4.rs eva2.rs	194:212 187:205 227:245	view
17	x 6	candle-transformers/src/models candle-transformers/src/models candle-transformers/src/models candle-transformers/src/models candle-transformers/src/models candle-transformers/src/models	yi.rs stable_lm.rs deepseek2.rs recurrent_gemma.rs phi3.rs gemma.rs	327:346 395:414 1004:1022 620:638 386:404 375:393	view
17	x 2	candle-transformers/src/models candle-transformers/src/models	quantized_recurrent_gemma.rs quantized_stable_lm.rs	389:407 263:282	view
16	x 2	candle-transformers/src/models candle-transformers/src/models	granite.rs llama.rs	409:426 454:471	view
16	x 2	candle-transformers/src/models candle-transformers/src/models	llama2_c.rs quantized_llama2_c.rs	233:250 118:135	view
16	x 3	candle-transformers/src/models candle-transformers/src/models candle-transformers/src/models	glm4.rs chatglm.rs codegeex4_9b.rs	360:377 342:359 363:380	view
16	x 4	candle-transformers/src/models candle-transformers/src/models candle-transformers/src/models candle-transformers/src/models	olmo2.rs olmo.rs qwen2.rs qwen2_moe.rs	36:53 44:61 47:64 55:72	view
16	x 2	candle-transformers/src/models candle-transformers/src/models	quantized_blip_text.rs blip_text.rs	35:52 39:56	view
16	x 2	candle-transformers/src/models candle-transformers/src/models	gemma2.rs gemma.rs	76:93 83:100	view
16	x 2	candle-transformers/src/models candle-transformers/src/models	quantized_metavoice.rs metavoice.rs	73:90 770:787	view
16	x 2	candle-transformers/src/models candle-transformers/src/models	quantized_phi.rs quantized_llama.rs	174:191 264:281	view