huggingface / optimum-tpu Duplication

Intro

For duplication, we look at places in code where there are 6 or more lines of code that are exactly the same.
Before duplication is calculated, the code is cleaned to remove empty lines, comments, and frequently duplicated constructs such as imports.
You should aim at having as little as possible (<5%) of duplicated code as high-level of duplication can lead to maintenance difficulties, poor factoring, and logical contradictions.

Learn more...

Duplication Overall

51% duplication:

6,033 cleaned lines of cleaned code (without empty lines, comments, and frequently duplicated constructs such as imports)
3,080 duplicated lines

285 duplicates

dependency graphs: 2D graph | 3D graph | 3D graph (with duplicates)...

Duplication per Extension

Duplication per Component (primary)

Duplication Between Components (50+ lines)

Download: SVG DOT (open online Graphviz editor)

From Component --> To Component	Duplicated Lines	File Pairs	Details
optimum (2%) --> text-generation-inference (5%)	196	2 file pairs	details...

Open 2D force graph... Open 3D force graph...

Show more details on duplication between components...

Longest Duplicates

The list of 50 longest duplicates.

See data for all 285 duplicates...

Size	#	Folders	Files	Lines	Code
87	x 2	optimum/tpu optimum/tpu	modeling_gemma.py modeling_llama.py	1223:1335 (10%) 1315:1428 (7%)	view
80	x 2	optimum/tpu optimum/tpu	modeling_gemma.py modeling_llama.py	950:1053 (9%) 1029:1132 (7%)	view
78	x 2	optimum/tpu optimum/tpu	modeling_llama.py modeling_mistral.py	1463:1558 (7%) 1459:1554 (7%)	view
70	x 2	optimum/tpu optimum/tpu	modeling_gemma.py modeling_llama.py	450:552 (8%) 540:642 (6%)	view
68	x 2	optimum/tpu/generation text-generation-inferenc...er/jetstream_pt_support	token_selector.py token_selector.py	62:158 (62%) 68:164 (54%)	view
54	x 2	optimum/tpu optimum/tpu	modeling_llama.py modeling_mistral.py	1023:1091 (4%) 992:1059 (4%)	view
52	x 2	optimum/tpu optimum/tpu	modeling_gemma.py modeling_mistral.py	950:1012 (6%) 997:1059 (4%)	view
46	x 2	optimum/tpu optimum/tpu	modeling_llama.py modeling_mistral.py	1188:1246 (4%) 1183:1241 (4%)	view
42	x 2	optimum/tpu optimum/tpu	modeling_llama.py modeling_mistral.py	667:724 (3%) 645:702 (3%)	view
40	x 2	optimum/tpu optimum/tpu	modeling_gemma.py modeling_mistral.py	1162:1229 (4%) 1244:1313 (3%)	view
34	x 2	optimum/tpu optimum/tpu	modeling_llama.py modeling_mistral.py	1249:1309 (3%) 1244:1305 (3%)	view
34	x 2	optimum/tpu optimum/tpu	modeling_gemma.py modeling_llama.py	1162:1222 (3%) 1249:1309 (3%)	view
32	x 2	optimum/tpu optimum/tpu	modeling_gemma.py modeling_llama.py	666:720 (3%) 754:808 (2%)	view
32	x 2	optimum/tpu optimum/tpu	modeling_llama.py modeling_mistral.py	754:808 (2%) 729:784 (2%)	view
32	x 2	optimum/tpu optimum/tpu	modeling_gemma.py modeling_mistral.py	666:720 (3%) 729:784 (2%)	view
28	x 2	optimum/tpu optimum/tpu	modeling_llama.py modeling_mistral.py	1397:1443 (2%) 1395:1438 (2%)	view
27	x 2	optimum/tpu optimum/tpu	modeling_gemma.py modeling_llama.py	363:401 (3%) 453:491 (2%)	view
27	x 2	optimum/tpu optimum/tpu	modeling_gemma.py modeling_llama.py	906:941 (3%) 992:1026 (2%)	view
26	x 2	optimum/tpu optimum/tpu	modeling_llama.py modeling_mistral.py	393:429 (2%) 289:325 (2%)	view
26	x 2	optimum/tpu optimum/tpu	modeling_gemma.py modeling_mistral.py	1304:1335 (3%) 1395:1423 (2%)	view
26	x 2	optimum/tpu optimum/tpu	modeling_gemma.py modeling_llama.py	406:447 (3%) 496:537 (2%)	view
25	x 2	optimum/tpu optimum/tpu	modeling_gemma.py modeling_llama.py	1112:1138 (2%) 1198:1224 (2%)	view
25	x 2	optimum/tpu optimum/tpu	modeling_gemma.py modeling_mistral.py	1112:1138 (2%) 1193:1219 (2%)	view
25	x 2	optimum/tpu optimum/tpu	modeling_gemma.py modeling_mistral.py	888:914 (2%) 940:968 (2%)	view
25	x 2	optimum/tpu optimum/tpu	modeling_llama.py modeling_mistral.py	280:307 (2%) 217:243 (2%)	view
25	x 2	optimum/tpu optimum/tpu	modeling_gemma.py modeling_llama.py	1055:1083 (2%) 1136:1164 (2%)	view
25	x 2	optimum/tpu optimum/tpu	modeling_gemma.py modeling_mistral.py	603:640 (2%) 670:707 (2%)	view
24	x 2	optimum/tpu optimum/tpu	modeling_gemma.py modeling_llama.py	223:247 (2%) 267:291 (2%)	view
23	x 2	optimum/tpu optimum/tpu	modeling_llama.py modeling_mistral.py	1329:1354 (2%) 1320:1346 (2%)	view
23	x 2	optimum/tpu optimum/tpu	modeling_gemma.py modeling_mistral.py	319:352 (2%) 300:333 (2%)	view
23	x 2	optimum/tpu optimum/tpu	modeling_gemma.py modeling_mistral.py	1236:1261 (2%) 1320:1346 (2%)	view
22	x 2	optimum/tpu optimum/tpu	modeling_gemma.py modeling_llama.py	603:635 (2%) 692:724 (1%)	view
22	x 2	optimum/tpu optimum/tpu	modeling_gemma.py modeling_mistral.py	529:552 (2%) 596:619 (2%)	view
22	x 2	optimum/tpu optimum/tpu	modeling_llama.py modeling_mistral.py	619:642 (1%) 596:619 (2%)	view
22	x 2	text-generation-inferenc.../text_generation_server text-generation-inferenc...er/jetstream_pt_support	generator.py generator.py	362:395 (3%) 298:330 (5%)	view
20	x 2	optimum/tpu optimum/tpu	modeling_gemma.py modeling_mistral.py	121:168 (2%) 124:171 (1%)	view
18	x 2	optimum/tpu optimum/tpu	modeling_gemma.py modeling_llama.py	888:905 (2%) 971:988 (1%)	view
18	x 2	optimum/tpu optimum/tpu	modeling_llama.py modeling_mistral.py	971:988 (1%) 940:957 (1%)	view
18	x 2	optimum/tpu optimum/tpu	modeling_gemma.py modeling_llama.py	319:344 (2%) 404:429 (1%)	view
18	x 2	optimum/tpu optimum/tpu	modeling_gemma.py modeling_llama.py	578:598 (2%) 667:687 (1%)	view
18	x 2	optimum/tpu optimum/tpu	modeling_gemma.py modeling_mistral.py	578:598 (2%) 645:665 (1%)	view
18	x 2	text-generation-inferenc.../text_generation_server text-generation-inferenc...er/jetstream_pt_support	generator.py generator.py	756:789 (2%) 613:646 (4%)	view
16	x 2	optimum/tpu optimum/tpu	modeling_llama.py modeling_mistral.py	515:533 (1%) 436:454 (1%)	view
16	x 2	optimum/tpu optimum/tpu	modeling_gemma.py modeling_mistral.py	425:443 (1%) 436:454 (1%)	view
16	x 2	optimum/tpu optimum/tpu	modeling_llama.py modeling_mistral.py	1371:1393 (1%) 1360:1382 (1%)	view
16	x 2	optimum/tpu optimum/tpu	modeling_gemma.py modeling_mistral.py	1278:1300 (1%) 1360:1382 (1%)	view
15	x 2	optimum/tpu optimum/tpu	modeling_llama.py modeling_mistral.py	1124:1140 (1%) 1112:1129 (1%)	view
14	x 2	optimum/tpu optimum/tpu	modeling_llama.py modeling_mistral.py	1147:1164 (1%) 1141:1159 (1%)	view
14	x 2	optimum/tpu optimum/tpu	modeling_gemma.py modeling_mistral.py	1066:1083 (1%) 1141:1159 (1%)	view
14	x 2	optimum/tpu optimum/tpu	modeling_llama.py modeling_mistral.py	40:61 (1%) 40:60 (1%)	view

Duplicated Units

The list of top 1 duplicated units.

See data for all 1 unit duplicate

Size	#	Folders	Files	Lines	Code
33	x 2	optimum/tpu optimum/tpu	modeling_gemma.py modeling_llama.py	0:0 0:0	view