GoogleCloudPlatform / t5x-on-vertex-ai

Intro

For duplication, we look at places in code where there are 6 or more lines of code that are exactly the same.
Before duplication is calculated, the code is cleaned to remove empty lines, comments, and frequently duplicated constructs such as imports.
You should aim at having as little as possible (<5%) of duplicated code as high-level of duplication can lead to maintenance difficulties, poor factoring, and logical contradictions.

Learn more...

Duplication Overall

73% duplication:

1,970 cleaned lines of cleaned code (without empty lines, comments, and frequently duplicated constructs such as imports)
1,456 duplicated lines

3,681 duplicates

dependency graphs: 2D graph | 3D graph | 3D graph (with duplicates)...

Duplication per Extension

Duplication per Component (primary)

Duplication Between Components (50+ lines)

Download: SVG DOT (open online Graphviz editor)

From Component --> To Component	Duplicated Lines	File Pairs	Details
notebooks (3%) --> scripts (50%)	112	1 file pair	details...

Open 2D force graph... Open 3D force graph...

Show more details on duplication between components...

Longest Duplicates

The list of 50 longest duplicates.

See data for all 3,681 duplicates...

Size	#	Folders	Files	Lines	Code
112	x 2	notebooks notebooks	finetune-t511-large-squad.ipynb finetune-t511-xl-xsum.ipynb	270:381 (22%) 271:382 (23%)	view
85	x 2	notebooks notebooks	finetune-t511-large-squad.ipynb finetune-t511-xl-xsum.ipynb	85:169 (16%) 86:170 (18%)	view
43	x 2	notebooks notebooks	finetune-t511-large-squad.ipynb finetune-t511-xl-xsum.ipynb	400:442 (8%) 393:435 (9%)	view
39	x 2	notebooks notebooks	finetune-t511-large-squad.ipynb finetune-t511-xl-xsum.ipynb	171:209 (7%) 172:210 (8%)	view
35	x 2	notebooks notebooks	eval-t511-xl-xsum.ipynb finetune-t511-xl-xsum.ipynb	475:509 (6%) 436:470 (7%)	view
30	x 2	notebooks notebooks	eval-t511-xl-xsum.ipynb finetune-t511-large-squad.ipynb	203:232 (5%) 180:209 (5%)	view
30	x 2	notebooks notebooks	eval-t511-xl-xsum.ipynb finetune-t511-xl-xsum.ipynb	203:232 (5%) 181:210 (6%)	view
28	x 2	notebooks scripts	utils.py run.py	56:147 (9%) 61:150 (25%)	view
27	x 2	notebooks notebooks	eval-t511-xl-xsum.ipynb finetune-t511-xl-xsum.ipynb	1:27 (5%) 1:27 (5%)	view
27	x 2	notebooks notebooks	finetune-t511-large-squad.ipynb finetune-t511-xl-xsum.ipynb	54:80 (5%) 57:83 (5%)	view
27	x 2	notebooks notebooks	finetune-t511-large-squad.ipynb finetune-t511-xl-xsum.ipynb	1:27 (5%) 1:27 (5%)	view
27	x 2	notebooks notebooks	eval-t511-xl-xsum.ipynb finetune-t511-large-squad.ipynb	1:27 (5%) 1:27 (5%)	view
25	x 2	notebooks notebooks	eval-t511-xl-xsum.ipynb finetune-t511-xl-xsum.ipynb	71:95 (4%) 61:85 (5%)	view
25	x 2	notebooks notebooks	finetune-t511-large-squad.ipynb finetune-t511-xl-xsum.ipynb	473:497 (4%) 435:459 (5%)	view
24	x 2	notebooks notebooks	eval-t511-xl-xsum.ipynb finetune-t511-large-squad.ipynb	475:498 (4%) 474:497 (4%)	view
23	x 2	notebooks notebooks	eval-t511-xl-xsum.ipynb finetune-t511-large-squad.ipynb	168:190 (4%) 147:169 (4%)	view
23	x 2	notebooks notebooks	eval-t511-xl-xsum.ipynb finetune-t511-xl-xsum.ipynb	168:190 (4%) 148:170 (4%)	view
23	x 2	notebooks notebooks	eval-t511-xl-xsum.ipynb finetune-t511-large-squad.ipynb	71:93 (4%) 58:80 (4%)	view
18	x 2	notebooks notebooks	finetune-t511-large-squad.ipynb finetune-t511-xl-xsum.ipynb	251:268 (3%) 259:276 (3%)	view
18	x 2	notebooks notebooks	eval-t511-xl-xsum.ipynb finetune-t511-large-squad.ipynb	412:429 (3%) 358:375 (3%)	view
18	x 2	notebooks notebooks	eval-t511-xl-xsum.ipynb finetune-t511-xl-xsum.ipynb	412:429 (3%) 359:376 (3%)	view
17	x 2	notebooks notebooks	eval-t511-xl-xsum.ipynb finetune-t511-large-squad.ipynb	53:69 (3%) 36:52 (3%)	view
17	x 2	notebooks notebooks	finetune-t511-large-squad.ipynb finetune-t511-xl-xsum.ipynb	36:52 (3%) 39:55 (3%)	view
17	x 2	notebooks scripts	utils.py run.py	182:200 (5%) 178:196 (15%)	view
17	x 2	notebooks notebooks	eval-t511-xl-xsum.ipynb finetune-t511-xl-xsum.ipynb	53:69 (3%) 39:55 (3%)	view
16	x 2	notebooks notebooks	eval-t511-xl-xsum.ipynb finetune-t511-large-squad.ipynb	377:392 (3%) 330:345 (3%)	view
16	x 2	notebooks notebooks	eval-t511-xl-xsum.ipynb finetune-t511-large-squad.ipynb	336:351 (3%) 289:304 (3%)	view
16	x 2	notebooks notebooks	eval-t511-xl-xsum.ipynb finetune-t511-xl-xsum.ipynb	336:351 (3%) 290:305 (3%)	view
16	x 2	notebooks notebooks	eval-t511-xl-xsum.ipynb finetune-t511-xl-xsum.ipynb	377:392 (3%) 331:346 (3%)	view
15	x 2	notebooks notebooks	eval-t511-xl-xsum.ipynb finetune-t511-large-squad.ipynb	140:154 (2%) 211:225 (2%)	view
15	x 2	notebooks notebooks	eval-t511-xl-xsum.ipynb finetune-t511-xl-xsum.ipynb	140:154 (2%) 212:226 (3%)	view
15	x 2	notebooks notebooks	finetune-t511-large-squad.ipynb finetune-t511-xl-xsum.ipynb	211:225 (2%) 212:226 (3%)	view
13	x 2	notebooks notebooks	eval-t511-xl-xsum.ipynb finetune-t511-xl-xsum.ipynb	102:114 (2%) 94:106 (2%)	view
13	x 2	notebooks notebooks	eval-t511-xl-xsum.ipynb finetune-t511-large-squad.ipynb	102:114 (2%) 93:105 (2%)	view
12	x 2	notebooks notebooks	eval-t511-xl-xsum.ipynb finetune-t511-large-squad.ipynb	49:60 (2%) 54:65 (2%)	view
12	x 2	notebooks notebooks	eval-t511-xl-xsum.ipynb finetune-t511-xl-xsum.ipynb	49:60 (2%) 57:68 (2%)	view
11	x 2	notebooks scripts	utils.py run.py	156:169 (3%) 161:174 (9%)	view
10	x 2	notebooks notebooks	eval-t511-xl-xsum.ipynb finetune-t511-large-squad.ipynb	156:165 (1%) 136:145 (1%)	view
10	x 2	notebooks notebooks	eval-t511-xl-xsum.ipynb finetune-t511-large-squad.ipynb	366:375 (1%) 319:328 (1%)	view
10	x 2	notebooks notebooks	finetune-t511-large-squad.ipynb finetune-t511-xl-xsum.ipynb	227:236 (1%) 229:238 (2%)	view
10	x 2	notebooks notebooks	eval-t511-xl-xsum.ipynb finetune-t511-xl-xsum.ipynb	270:279 (1%) 257:266 (2%)	view
10	x 2	notebooks notebooks	eval-t511-xl-xsum.ipynb finetune-t511-xl-xsum.ipynb	156:165 (1%) 137:146 (2%)	view
10	x 2	notebooks notebooks	eval-t511-xl-xsum.ipynb finetune-t511-xl-xsum.ipynb	366:375 (1%) 320:329 (2%)	view
9	x 2	notebooks notebooks	finetune-t511-large-squad.ipynb finetune-t511-xl-xsum.ipynb	384:392 (1%) 385:393 (1%)	view
9	x 2	notebooks notebooks	eval-t511-xl-xsum.ipynb eval-t511-xl-xsum.ipynb	102:110 (1%) 116:124 (1%)	view
9	x 2	notebooks notebooks	eval-t511-xl-xsum.ipynb finetune-t511-xl-xsum.ipynb	116:124 (1%) 94:102 (1%)	view
9	x 2	notebooks notebooks	eval-t511-xl-xsum.ipynb finetune-t511-xl-xsum.ipynb	436:444 (1%) 383:391 (1%)	view
9	x 2	notebooks notebooks	eval-t511-xl-xsum.ipynb finetune-t511-large-squad.ipynb	116:124 (1%) 93:101 (1%)	view
8	x 2	notebooks notebooks	finetune-t511-xl-xsum.ipynb finetune-t511-xl-xsum.ipynb	61:68 (1%) 418:425 (1%)	view
8	x 2	notebooks notebooks	eval-t511-xl-xsum.ipynb finetune-t511-xl-xsum.ipynb	272:279 (1%) 278:285 (1%)	view