G

Intro

For duplication, we look at places in code where there are 6 or more lines of code that are exactly the same.
Before duplication is calculated, the code is cleaned to remove empty lines, comments, and frequently duplicated constructs such as imports.
You should aim at having as little as possible (<5%) of duplicated code as high-level of duplication can lead to maintenance difficulties, poor factoring, and logical contradictions.

Learn more...

Duplication Overall

14% duplication:

18,695 cleaned lines of cleaned code (without empty lines, comments, and frequently duplicated constructs such as imports)
2,644 duplicated lines

228 duplicates

Duplication per Extension

Duplication per Component (primary)

Duplication Between Components (50+ lines)

Download: SVG DOT (open online Graphviz editor)

From Component --> To Component	Duplicated Lines	File Pairs	Details
mesh_tensorflow/bert (1%) --> mesh_tensorflow/transformer (<1%)	64	4 file pairs	details...

Open 3D force graph...

Show more details on duplication between components...

Longest Duplicates

The list of 20 longest duplicates.

See data for all 228 duplicates...

Size	#	Folders	Files	Lines	Code
49	x 2	mesh_tensorflow/transformer mesh_tensorflow/transformer	funnel_transformer.py transformer.py	387:436 (20%) 1650:1699 (3%)	view
43	x 2	mesh_tensorflow/bert mesh_tensorflow/bert	run_classifier.py run_squad.py	744:792 (5%) 698:746 (4%)	view
42	x 2	mesh_tensorflow/transformer mesh_tensorflow/transformer	moe.py moe.py	1063:1112 (3%) 1323:1372 (3%)	view
32	x 2	mesh_tensorflow/transformer mesh_tensorflow/transformer	heterogeneous_moe.py moe.py	349:387 (9%) 466:504 (2%)	view
30	x 2	mesh_tensorflow/transformer mesh_tensorflow/transformer	funnel_transformer.py transformer.py	253:312 (12%) 1525:1580 (2%)	view
26	x 2	mesh_tensorflow/transformer mesh_tensorflow/transformer	heterogeneous_moe.py moe.py	100:130 (7%) 99:129 (2%)	view
24	x 2	mesh_tensorflow/transformer mesh_tensorflow/transformer	heterogeneous_moe.py moe.py	254:284 (7%) 371:401 (2%)	view
24	x 2	mesh_tensorflow/transformer mesh_tensorflow/transformer	funnel_transformer.py transformer.py	341:382 (10%) 1605:1649 (1%)	view
23	x 2	mesh_tensorflow/transformer mesh_tensorflow/transformer	heterogeneous_moe.py moe.py	39:61 (6%) 40:62 (1%)	view
23	x 2	mesh_tensorflow/bert mesh_tensorflow/bert	run_classifier.py run_squad.py	643:673 (3%) 600:630 (2%)	view
22	x 2	mesh_tensorflow/transformer mesh_tensorflow/transformer	moe.py moe.py	938:963 (1%) 1098:1123 (1%)	view
22	x 2	mesh_tensorflow/bert mesh_tensorflow/bert	run_classifier.py run_pretraining.py	115:141 (2%) 99:125 (5%)	view
22	x 2	mesh_tensorflow/bert mesh_tensorflow/bert	run_classifier.py run_squad.py	115:141 (2%) 132:159 (2%)	view
22	x 2	mesh_tensorflow/bert mesh_tensorflow/bert	run_pretraining.py run_squad.py	99:125 (5%) 132:159 (2%)	view
21	x 2	mesh_tensorflow/bert mesh_tensorflow/bert	run_classifier.py run_squad.py	540:573 (2%) 780:813 (2%)	view
21	x 2	mesh_tensorflow/transformer mesh_tensorflow/transformer	heterogeneous_moe.py moe.py	70:90 (6%) 70:90 (1%)	view
21	x 2	mesh_tensorflow/transformer mesh_tensorflow/transformer	transformer.py transformer.py	1146:1167 (1%) 1321:1342 (1%)	view
20	x 2	mesh_tensorflow/bert mesh_tensorflow/transformer	bert.py transformer_layers.py	716:758 (4%) 1014:1056 (1%)	view
20	x 2	mesh_tensorflow/bert mesh_tensorflow/bert	run_classifier.py run_squad.py	920:942 (2%) 1214:1236 (2%)	view
19	x 2	mesh_tensorflow/transformer mesh_tensorflow/transformer	heterogeneous_moe.py moe.py	437:457 (5%) 531:551 (1%)	view

Duplicated Units

The list of top 2 duplicated units.

See data for all 2 unit duplicates...

Size	#	Folders	Files	Lines	Code
39	x 2	mesh_tensorflow/transformer mesh_tensorflow/bert	transformer_layers.py bert.py	0:0 0:0	view
20	x 2	mesh_tensorflow mesh_tensorflow/transformer	optimize.py learning_rate_schedules.py	0:0 0:0	view