G

Intro

For duplication, we look at places in code where there are 6 or more lines of code that are exactly the same.
Before duplication is calculated, the code is cleaned to remove empty lines, comments, and frequently duplicated constructs such as imports.
You should aim at having as little as possible (<5%) of duplicated code as high-level of duplication can lead to maintenance difficulties, poor factoring, and logical contradictions.

Learn more...

Duplication Overall

95% duplication:

8,110 cleaned lines of cleaned code (without empty lines, comments, and frequently duplicated constructs such as imports)
7,714 duplicated lines

100 duplicates

Duplication per Extension

Duplication per Component (primary)

Duplication Between Components (50+ lines)

Download: SVG DOT (open online Graphviz editor)

From Component --> To Component	Duplicated Lines	File Pairs	Details
src/model (99%) --> xlm/model (99%)	2748	7 file pairs	details...
src (99%) --> xlm (99%)	2118	5 file pairs	details...
src/evaluation (99%) --> xlm/evaluation (99%)	1618	6 file pairs	details...
src/data (99%) --> xlm/data (99%)	1216	3 file pairs	details...

Open 3D force graph...

Show more details on duplication between components...

Longest Duplicates

The list of 20 longest duplicates.

See data for all 100 duplicates...

Size	#	Folders	Files	Lines	Code
597	x 2	src xlm	trainer.py trainer.py	27:935 (100%) 27:935 (100%)	view
435	x 2	src/model/memory xlm/model/memory	memory.py memory.py	15:687 (100%) 15:687 (100%)	view
432	x 2	src/model xlm/model	transformer.py transformer.py	19:748 (100%) 19:748 (100%)	view
340	x 2	src/evaluation xlm/evaluation	evaluator.py evaluator.py	19:565 (100%) 19:565 (100%)	view
233	x 2	src/data xlm/data	dataset.py dataset.py	14:404 (100%) 14:404 (100%)	view
218	x 2	src/data xlm/data	loader.py loader.py	17:351 (100%) 17:351 (100%)	view
194	x 2	src/evaluation xlm/evaluation	glue.py glue.py	29:325 (100%) 29:325 (100%)	view
183	x 2	src/model/memory xlm/model/memory	query.py query.py	7:252 (100%) 7:252 (100%)	view
170	x 2	src xlm	utils.py utils.py	22:294 (100%) 22:294 (100%)	view
168	x 2	src xlm	optim.py optim.py	16:270 (100%) 16:270 (100%)	view
157	x 2	src/data xlm/data	dictionary.py dictionary.py	14:228 (100%) 14:228 (100%)	view
138	x 2	src/evaluation xlm/evaluation	xnli.py xnli.py	25:254 (100%) 25:254 (100%)	view
137	x 2	src/evaluation xlm/evaluation	multi-bleu.perl multi-bleu.perl	10:177 (100%) 10:177 (100%)	view
111	x 2	src/model xlm/model	__init__.py __init__.py	17:185 (100%) 17:185 (100%)	view
92	x 2	src/model/memory xlm/model/memory	utils.py utils.py	8:163 (100%) 8:163 (100%)	view
89	x 2	src xlm	slurm.py slurm.py	17:170 (100%) 17:170 (100%)	view
69	x 2	src/model xlm/model	embedder.py embedder.py	16:137 (100%) 16:137 (100%)	view
52	x 2	src/model xlm/model	pretrain.py pretrain.py	14:95 (100%) 14:95 (100%)	view
35	x 2	src xlm	logger.py logger.py	13:66 (100%) 13:66 (100%)	view
15	x 2	src/evaluation xlm/evaluation	evaluator.py evaluator.py	272:291 (4%) 347:366 (4%)	view

Duplicated Units

The list of top 20 duplicated units.

See data for all 163 unit duplicates...

Size	#	Folders	Files	Lines	Code
139	x 2	xlm/model src/model	transformer.py transformer.py	0:0 0:0	view
96	x 2	xlm/model/memory src/model/memory	memory.py memory.py	0:0 0:0	view
102	x 2	xlm src	slurm.py slurm.py	0:0 0:0	view
81	x 2	xlm/model/memory src/model/memory	memory.py memory.py	0:0 0:0	view
87	x 2	xlm src	trainer.py trainer.py	0:0 0:0	view
86	x 2	xlm/evaluation src/evaluation	evaluator.py evaluator.py	0:0 0:0	view
82	x 2	xlm/data src/data	loader.py loader.py	0:0 0:0	view
67	x 2	xlm/model src/model	transformer.py transformer.py	0:0 0:0	view
81	x 2	xlm/model src/model	transformer.py transformer.py	0:0 0:0	view
68	x 2	xlm/model/memory src/model/memory	memory.py memory.py	0:0 0:0	view
61	x 2	xlm/evaluation src/evaluation	glue.py glue.py	0:0 0:0	view
58	x 2	xlm/data src/data	dictionary.py dictionary.py	0:0 0:0	view
56	x 2	xlm src	optim.py optim.py	0:0 0:0	view
63	x 2	xlm/model/memory src/model/memory	memory.py memory.py	0:0 0:0	view
59	x 2	xlm/model src/model	__init__.py __init__.py	0:0 0:0	view
66	x 2	xlm/model src/model	__init__.py __init__.py	0:0 0:0	view
58	x 2	xlm/evaluation src/evaluation	glue.py glue.py	0:0 0:0	view
58	x 2	xlm/evaluation src/evaluation	evaluator.py evaluator.py	0:0 0:0	view
71	x 2	xlm/model src/model	transformer.py transformer.py	0:0 0:0	view
54	x 2	xlm/evaluation src/evaluation	xnli.py xnli.py	0:0 0:0	view