mozilla / translations

Intro

For duplication, we look at places in code where there are 6 or more lines of code that are exactly the same.
Before duplication is calculated, the code is cleaned to remove empty lines, comments, and frequently duplicated constructs such as imports.
You should aim at having as little as possible (<5%) of duplicated code as high-level of duplication can lead to maintenance difficulties, poor factoring, and logical contradictions.

Learn more...

Duplication Overall

5% duplication:

16,541 cleaned lines of cleaned code (without empty lines, comments, and frequently duplicated constructs such as imports)
978 duplicated lines

68 duplicates

dependency graphs: 2D graph | 3D graph | 3D graph (with duplicates)...

Duplication per Extension

Duplication per Component (primary)

Duplication Between Components (50+ lines)

Download: SVG DOT (open online Graphviz editor)

From Component --> To Component	Duplicated Lines	File Pairs	Details
pipeline (<1%) --> taskcluster (1%)	50	1 file pair	details...

Open 2D force graph... Open 3D force graph...

Show more details on duplication between components...

Longest Duplicates

The list of 50 longest duplicates.

See data for all 68 duplicates...

Size	#	Folders	Files	Lines	Code
32	x 2	snakemake/profiles/slurm-csd3 snakemake/profiles/slurm-moz	submit.py submit.py	12:59 (62%) 12:59 (68%)	view
25	x 2	pipeline/common taskcluster/translations_taskgraph/util	datasets.py dataset_helpers.py	51:86 (8%) 15:50 (92%)	view
22	x 2	snakemake/profiles/slurm-csd3 snakemake/profiles/slurm-moz	status.py status.py	72:93 (30%) 43:64 (44%)	view
18	x 2	taskcluster/translations_taskgraph/actions taskcluster/translations_taskgraph/actions	train.py train.py	271:288 (4%) 290:307 (4%)	view
16	x 2	site/model-registry site/model-registry	mjs model-registry.mjs model-registry.mjs	688:703 (1%) 742:757 (1%)	view
15	x 2	snakemake/profiles/slurm-csd3 snakemake/profiles/slurm-moz	submit.py submit.py	65:83 (29%) 60:78 (31%)	view
14	x 2	utils utils	find_corpus.py find_corpus.py	249:263 (3%) 296:310 (3%)	view
14	x 2	utils utils	find_corpus.py find_corpus.py	202:216 (3%) 249:263 (3%)	view
14	x 2	utils utils	find_corpus.py find_corpus.py	202:216 (3%) 296:310 (3%)	view
12	x 2	taskcluster/translations_taskgraph/transforms taskcluster/translations_taskgraph/transforms	from_datasets.py from_datasets.py	105:118 (10%) 173:186 (10%)	view
12	x 2	taskcluster/translations_taskgraph/actions taskcluster/translations_taskgraph/actions	train.py train.py	206:217 (2%) 226:237 (2%)	view
12	x 2	taskcluster/translations_taskgraph/actions taskcluster/translations_taskgraph/actions	train.py train.py	226:237 (2%) 244:255 (2%)	view
12	x 2	utils utils	model_registry.py model_registry.py	176:190 (1%) 231:245 (1%)	view
12	x 2	taskcluster/translations_taskgraph/actions taskcluster/translations_taskgraph/actions	train.py train.py	206:217 (2%) 244:255 (2%)	view
12	x 2	snakemake/profiles/slurm-csd3 snakemake/profiles/slurm-moz	config.yaml config.yaml	5:16 (40%) 5:16 (40%)	view
10	x 2	utils utils	tb_log_parser.py tb_log_parser.py	39:48 (5%) 137:146 (5%)	view
10	x 2	site/model-registry site/model-registry	mjs model-registry.mjs model-registry.mjs	470:480 (<1%) 607:617 (<1%)	view
10	x 2	tracking/translations_parser tracking/translations_parser	utils.py utils.py	32:41 (4%) 51:60 (4%)	view
10	x 2	inference/scripts inference/scripts	build-wasm.py test-wasm.py	41:50 (5%) 39:48 (13%)	view
9	x 2	tracking/translations_parser/cli tracking/translations_parser/cli	taskcluster.py taskcluster_group.py	60:68 (7%) 51:59 (3%)	view
9	x 2	snakemake/profiles/local-container snakemake/profiles/local	config.yaml config.yaml	9:18 (52%) 7:20 (60%)	view
9	x 2	snakemake/profiles/slurm-csd3 snakemake/profiles/slurm-moz	status.py status.py	61:69 (12%) 31:39 (18%)	view
9	x 2	snakemake/profiles/slurm-csd3 snakemake/profiles/slurm-moz	status.py status.py	40:49 (12%) 19:28 (18%)	view
8	x 2	pipeline/alignments pipeline/data	align.py parallel_importer.py	368:377 (2%) 229:238 (5%)	view
8	x 2	taskcluster/translations_taskgraph/transforms taskcluster/translations_taskgraph/transforms	find_upstreams.py find_upstreams.py	112:123 (5%) 160:171 (5%)	view
7	x 2	pipeline/train pipeline/translate	train.py translate.py	398:408 (1%) 83:93 (3%)	view
7	x 2	snakemake/profiles/slurm-csd3 snakemake/profiles/slurm-moz	status.py status.py	9:18 (9%) 9:18 (14%)	view
7	x 2	utils utils	model_registry.py model_registry.py	903:910 (<1%) 986:993 (<1%)	view
7	x 2	pipeline/clean pipeline/clean	merge-mono.py merge-parallel.py	201:207 (3%) 243:251 (3%)	view
7	x 2	taskcluster/translations_taskgraph/transforms taskcluster/translations_taskgraph/transforms	cast_to.py marian_args.py	19:28 (33%) 22:31 (31%)	view
7	x 2	site/model-registry site/model-registry	mjs model-registry.mjs model-registry.mjs	716:722 (<1%) 751:757 (<1%)	view
7	x 2	site/model-registry site/model-registry	mjs model-registry.mjs model-registry.mjs	708:714 (<1%) 777:783 (<1%)	view
7	x 2	site/model-registry site/model-registry	mjs model-registry.mjs model-registry.mjs	725:731 (<1%) 760:766 (<1%)	view
7	x 2	site/model-registry site/model-registry	mjs model-registry.mjs model-registry.mjs	697:703 (<1%) 785:791 (<1%)	view
7	x 2	taskcluster/translations_taskgraph/actions taskcluster/translations_taskgraph/actions	train.py train.py	75:85 (1%) 393:403 (1%)	view
7	x 2	taskcluster/translations_taskgraph/actions taskcluster/translations_taskgraph/actions	train.py train.py	75:85 (1%) 405:414 (1%)	view
7	x 2	taskcluster/translations_taskgraph/transforms taskcluster/translations_taskgraph/transforms	cached_tasks.py cast_to.py	36:45 (7%) 19:28 (33%)	view
7	x 2	taskcluster/translations_taskgraph/transforms taskcluster/translations_taskgraph/transforms	cached_tasks.py marian_args.py	36:45 (7%) 22:31 (31%)	view
7	x 2	pipeline/train pipeline/translate	train.py translate.py	429:435 (1%) 112:118 (3%)	view
7	x 2	pipeline/train pipeline/translate	train.py translate.py	440:446 (1%) 117:123 (3%)	view
7	x 2	site/model-registry site/model-registry	mjs model-registry.mjs model-registry.mjs	697:703 (<1%) 716:722 (<1%)	view
7	x 2	taskcluster/translations_taskgraph/actions taskcluster/translations_taskgraph/actions	train.py train.py	393:403 (1%) 405:414 (1%)	view
7	x 2	site/model-registry site/model-registry	mjs model-registry.mjs model-registry.mjs	461:467 (<1%) 598:604 (<1%)	view
7	x 2	site/model-registry site/model-registry	mjs model-registry.mjs model-registry.mjs	716:722 (<1%) 785:791 (<1%)	view
7	x 2	site/model-registry site/model-registry	mjs model-registry.mjs model-registry.mjs	751:757 (<1%) 785:791 (<1%)	view
7	x 2	site/model-registry site/model-registry	mjs model-registry.mjs model-registry.mjs	733:739 (<1%) 768:774 (<1%)	view
6	x 2	snakemake/profiles/local-container snakemake/profiles/local	config.yaml config.yaml	1:6 (35%) 1:6 (40%)	view
6	x 2	utils utils	find_corpus.py find_corpus.py	192:198 (1%) 285:292 (1%)	view
6	x 2	tracking/translations_parser/cli tracking/translations_parser/cli	experiments.py taskcluster_group.py	86:91 (4%) 89:94 (2%)	view
6	x 2	pipeline/train pipeline/translate	train.py translate.py	474:479 (1%) 135:140 (2%)	view

Duplicated Units

The list of top 1 duplicated units.

See data for all 1 unit duplicate

Size	#	Folders	Files	Lines	Code
31	x 2	pipeline/common taskcluster/translations_taskgraph/util	datasets.py dataset_helpers.py	0:0 0:0	view