G

Intro

For duplication, we look at places in code where there are 6 or more lines of code that are exactly the same.
Before duplication is calculated, the code is cleaned to remove empty lines, comments, and frequently duplicated constructs such as imports.
You should aim at having as little as possible (<5%) of duplicated code as high-level of duplication can lead to maintenance difficulties, poor factoring, and logical contradictions.

Learn more...

Duplication Overall

21% duplication:

12,798 cleaned lines of cleaned code (without empty lines, comments, and frequently duplicated constructs such as imports)
2,814 duplicated lines

91 duplicates

Duplication per Extension

Duplication per Component (primary)

Duplication Between Components (50+ lines)

Download: SVG DOT (open online Graphviz editor)

From Component --> To Component	Duplicated Lines	File Pairs	Details
fairseq_cli (101%) --> ROOT (82%)	1901	11 file pairs	details...

Open 3D force graph...

Show more details on duplication between components...

Longest Duplicates

The list of 20 longest duplicates.

See data for all 91 duplicates...

Size	#	Folders	Files	Lines	Code
213	x 2	fairseq_cli ROOT	train.py train.py	25:306 (100%) 25:306 (100%)	view
191	x 2	fairseq_cli ROOT	preprocess.py preprocess.py	26:254 (100%) 26:254 (100%)	view
157	x 2	ROOT fairseq_cli	eval_lm.py eval_lm.py	22:225 (100%) 22:225 (100%)	view
134	x 2	fairseq_cli ROOT	generate.py generate.py	18:192 (100%) 18:192 (100%)	view
134	x 2	fairseq_cli ROOT	interactive.py interactive.py	21:195 (100%) 21:195 (100%)	view
58	x 2	fairseq_cli ROOT	score.py score.py	20:90 (100%) 20:90 (100%)	view
52	x 2	fairseq_cli ROOT	setup.py setup.py	13:68 (100%) 13:68 (100%)	view
28	x 2	fairseq/models fairseq/models	bert_seq2seq.py transformer.py	85:112 (5%) 76:103 (5%)	view
25	x 2	fairseq/modules fairseq/modules	sparse_transformer_sentence_encoder.py transformer_sentence_encoder.py	19:43 (39%) 69:93 (17%)	view
17	x 2	fairseq/data fairseq/data	language_pair_dataset.py language_pair_self_dataset_mask.py	105:121 (14%) 127:143 (11%)	view
17	x 2	fairseq/optim fairseq/optim	fp16_optimizer.py fp16_optimizer.py	69:87 (7%) 240:258 (7%)	view
13	x 2	fairseq/models fairseq/models	bert_seq2seq.py transformer.py	323:339 (2%) 411:429 (2%)	view
13	x 2	fairseq/modules fairseq/modules	dynamic_convolution.py lightweight_convolution.py	200:216 (7%) 216:232 (6%)	view
12	x 2	fairseq/modules fairseq/modules	sparse_transformer_sentence_encoder_l... transformer_sentence_encoder_layer.py	17:28 (36%) 25:36 (20%)	view
12	x 2	fairseq/optim fairseq/optim	fp16_optimizer.py fp16_optimizer.py	180:194 (5%) 352:367 (5%)	view
12	x 2	fairseq/tasks fairseq/tasks	translation.py translation_self.py	116:127 (9%) 47:58 (9%)	view
10	x 2	fairseq/optim fairseq/optim	nag.py sgd.py	25:42 (18%) 24:41 (55%)	view
10	x 2	fairseq/modules fairseq/modules	sparse_transformer_sentence_encoder.py transformer_sentence_encoder.py	61:70 (15%) 132:141 (7%)	view
10	x 2	fairseq_cli ROOT	generate.py generate_cmlm.py	65:76 (7%) 65:76 (6%)	view
10	x 2	fairseq/modules fairseq/modules	dynamic_convolution.py dynamic_convolution.py	108:118 (5%) 162:171 (5%)	view

Duplicated Units

The list of top 20 duplicated units.

See data for all 21 unit duplicates...

Size	#	Folders	Files	Lines	Code
148	x 2	ROOT fairseq_cli	preprocess.py preprocess.py	0:0 0:0	view
136	x 2	fairseq_cli ROOT	eval_lm.py eval_lm.py	0:0 0:0	view
141	x 2	ROOT fairseq_cli	generate.py generate.py	0:0 0:0	view
107	x 2	fairseq_cli ROOT	interactive.py interactive.py	0:0 0:0	view
57	x 2	fairseq_cli ROOT	train.py train.py	0:0 0:0	view
60	x 2	fairseq_cli ROOT	train.py train.py	0:0 0:0	view
43	x 2	fairseq_cli ROOT	score.py score.py	0:0 0:0	view
43	x 2	fairseq_cli ROOT	train.py train.py	0:0 0:0	view
32	x 2	fairseq_cli ROOT	train.py train.py	0:0 0:0	view
24	x 2	ROOT fairseq_cli	preprocess.py preprocess.py	0:0 0:0	view
22	x 2	fairseq_cli ROOT	train.py train.py	0:0 0:0	view
18	x 2	fairseq_cli ROOT	interactive.py interactive.py	0:0 0:0	view
13	x 2	fairseq_cli ROOT	train.py train.py	0:0 0:0	view
14	x 2	fairseq_cli ROOT	score.py score.py	0:0 0:0	view
9	x 2	fairseq_cli ROOT	interactive.py interactive.py	0:0 0:0	view
6	x 2	ROOT fairseq_cli	preprocess.py preprocess.py	0:0 0:0	view
6	x 2	fairseq_cli ROOT	eval_lm.py eval_lm.py	0:0 0:0	view
10	x 2	fairseq_cli ROOT	eval_lm.py eval_lm.py	0:0 0:0	view
11	x 2	fairseq/optim fairseq/optim	nag.py sgd.py	0:0 0:0	view
28	x 2	fairseq/data fairseq/data	language_pair_self_dataset_mask.py language_pair_dataset.py	0:0 0:0	view