Intro

For duplication, we look at places in code where there are 6 or more lines of code that are exactly the same.
Before duplication is calculated, the code is cleaned to remove empty lines, comments, and frequently duplicated constructs such as imports.
You should aim at having as little as possible (<5%) of duplicated code as high-level of duplication can lead to maintenance difficulties, poor factoring, and logical contradictions.

Learn more...

Duplication Overall

26% duplication:

2,926 cleaned lines of cleaned code (without empty lines, comments, and frequently duplicated constructs such as imports)
779 duplicated lines

78 duplicates

Duplication per Extension

Duplication per Component (primary)

Longest Duplicates

The list of 20 longest duplicates.

See data for all 78 duplicates...

Size	#	Folders	Files	Lines	Code
41	x 2	dpr_scale dpr_scale	run_retrieval_fb.py run_retrieval_multiset.py	62:111 (35%) 60:109 (33%)	view
30	x 2	dpr_scale dpr_scale	run_retrieval.py run_retrieval_multiset.py	17:50 (37%) 64:97 (24%)	view
30	x 2	dpr_scale dpr_scale	run_retrieval.py run_retrieval_fb.py	17:50 (37%) 66:99 (26%)	view
25	x 2	dpr_scale/utils dpr_scale/utils	prep_wiki.py prep_wiki_exp.py	101:134 (23%) 172:203 (14%)	view
16	x 2	dpr_scale dpr_scale	run_retrieval_fb.py run_retrieval_multiset.py	14:37 (13%) 14:37 (13%)	view
16	x 2	dpr_scale/transforms dpr_scale/transforms	dpr_transform.py dpr_transform.py	105:122 (7%) 233:250 (7%)	view
14	x 2	dpr_scale/transforms dpr_scale/transforms	dpr_transform.py dpr_transform.py	69:87 (6%) 196:214 (6%)	view
13	x 2	dpr_scale dpr_scale	generate_embeddings.py generate_query_embeddings.py	14:30 (76%) 16:32 (68%)	view
13	x 2	dpr_scale/transforms dpr_scale/transforms	hf_bert.py hf_transform.py	23:36 (54%) 22:35 (54%)	view
12	x 2	dpr_scale/transforms dpr_scale/transforms	dpr_transform.py dpr_transform.py	14:25 (5%) 153:164 (5%)	view
12	x 2	dpr_scale/conf dpr_scale/conf	dstc7.yaml ubuntuv2.yaml	31:44 (30%) 29:42 (32%)	view
11	x 2	dpr_scale/conf dpr_scale/conf	convai2.yaml dstc7.yaml	29:41 (30%) 31:43 (28%)	view
11	x 2	dpr_scale/transforms dpr_scale/transforms	dpr_transform.py dpr_transform.py	89:102 (5%) 215:228 (5%)	view
11	x 2	dpr_scale/conf dpr_scale/conf	convai2.yaml ubuntuv2.yaml	29:41 (30%) 29:41 (29%)	view
11	x 2	dpr_scale/utils dpr_scale/utils	ccnews_stats.py prep_ccnews.py	82:98 (14%) 194:210 (6%)	view
10	x 2	dpr_scale/conf dpr_scale/conf	convai2.yaml ubuntuv2.yaml	1:12 (27%) 1:12 (27%)	view
10	x 2	dpr_scale/utils dpr_scale/utils	prep_ccnews.py prep_wiki.py	120:129 (6%) 102:111 (9%)	view
10	x 2	dpr_scale/conf dpr_scale/conf	ccnews_ict.yaml wiki_ict.yaml	1:12 (33%) 1:12 (31%)	view
10	x 2	dpr_scale/utils dpr_scale/utils	prep_wiki.py prep_wiki_exp.py	15:26 (9%) 19:30 (5%)	view
10	x 2	dpr_scale/utils dpr_scale/utils	prep_ccnews.py prep_wiki_exp.py	120:129 (6%) 173:182 (5%)	view

Duplicated Units

The list of top 2 duplicated units.

See data for all 2 unit duplicates...

Size	#	Folders	Files	Lines	Code
9	x 2	dpr_scale dpr_scale	run_retrieval_multiset.py run_retrieval_fb.py	0:0 0:0	view
8	x 2	dpr_scale/transforms dpr_scale/transforms	hf_transform.py hf_bert.py	0:0 0:0	view