G

Intro

For duplication, we look at places in code where there are 6 or more lines of code that are exactly the same.
Before duplication is calculated, the code is cleaned to remove empty lines, comments, and frequently duplicated constructs such as imports.
You should aim at having as little as possible (<5%) of duplicated code as high-level of duplication can lead to maintenance difficulties, poor factoring, and logical contradictions.

Learn more...

Duplication Overall

52% duplication:

7,766 cleaned lines of cleaned code (without empty lines, comments, and frequently duplicated constructs such as imports)
4,092 duplicated lines

174 duplicates

Duplication per Extension

Duplication per Component (primary)

Duplication Between Components (50+ lines)

Download: SVG DOT (open online Graphviz editor)

From Component --> To Component	Duplicated Lines	File Pairs	Details
E2E_TOD (30%) --> data/multiwoz (68%)	1732	5 file pairs	details...
DST (59%) --> E2E_TOD (24%)	1376	9 file pairs	details...
DST (13%) --> IC (35%)	315	5 file pairs	details...
E2E_TOD (5%) --> IC (34%)	305	5 file pairs	details...
IC (30%) --> Pretraining (33%)	272	3 file pairs	details...
DST (10%) --> Pretraining (30%)	255	3 file pairs	details...
E2E_TOD (4%) --> Pretraining (30%)	251	3 file pairs	details...
DST (9%) --> data/multiwoz (8%)	221	2 file pairs	details...
DST/modelling (100%) --> E2E_TOD/modelling (100%)	168	1 file pair	details...

Open 3D force graph...

Show more details on duplication between components...

Longest Duplicates

The list of 20 longest duplicates.

See data for all 174 duplicates...

Size	#	Folders	Files	Lines	Code
203	x 2	E2E_TOD data/multiwoz/utlis	db_ops.py db_ops.py	4:264 (100%) 4:264 (87%)	view
185	x 2	E2E_TOD data/multiwoz/utlis	utils.py utils.py	7:226 (95%) 7:243 (95%)	view
173	x 2	E2E_TOD data/multiwoz/utlis	reader.py reader.py	16:238 (49%) 18:240 (47%)	view
168	x 2	E2E_TOD data/multiwoz/utlis	reader.py reader.py	248:447 (48%) 247:441 (46%)	view
99	x 2	E2E_TOD data/multiwoz/utlis	ontology.py ontology.py	1:178 (100%) 1:178 (100%)	view
99	x 2	DST data/multiwoz/utlis	ontology.py ontology.py	1:178 (100%) 1:178 (100%)	view
99	x 2	DST E2E_TOD	ontology.py ontology.py	1:178 (100%) 1:178 (100%)	view
84	x 2	DST/modelling E2E_TOD/modelling	T5Model.py T5Model.py	7:101 (100%) 7:101 (100%)	view
54	x 2	DST E2E_TOD	dataclass.py dataclass.py	325:386 (13%) 368:429 (10%)	view
51	x 2	data/multiwoz/utlis data/multiwoz/utlis	postprocessing_dataset.py processing_funcs.py	5:61 (26%) 4:60 (34%)	view
50	x 2	DST E2E_TOD	dataclass.py dataclass.py	131:184 (12%) 143:196 (9%)	view
44	x 2	DST E2E_TOD	dataclass.py dataclass.py	230:275 (10%) 247:292 (8%)	view
39	x 2	E2E_TOD Pretraining	dataclass.py dataclass.py	371:414 (7%) 248:291 (16%)	view
39	x 2	DST Pretraining	dataclass.py dataclass.py	328:371 (9%) 248:291 (16%)	view
37	x 2	data/multiwoz/utlis data/multiwoz/utlis	postprocessing_dataset.py processing_funcs.py	63:102 (19%) 62:101 (25%)	view
33	x 2	DST E2E_TOD	dataclass.py dataclass.py	186:221 (8%) 198:232 (6%)	view
33	x 2	DST E2E_TOD	learn.py learn.py	197:232 (12%) 139:174 (15%)	view
33	x 2	IC IC	inference.py learn.py	82:114 (32%) 166:198 (16%)	view
30	x 2	IC Pretraining	learn.py pretrain.py	122:153 (14%) 110:140 (16%)	view
29	x 2	DST Pretraining	learn.py pretrain.py	202:232 (10%) 111:140 (16%)	view

Duplicated Units

The list of top 20 duplicated units.

See data for all 39 unit duplicates...

Size	#	Folders	Files	Lines	Code
81	x 2	E2E_TOD data/multiwoz/utlis	db_ops.py db_ops.py	0:0 0:0	view
46	x 2	E2E_TOD data/multiwoz/utlis	reader.py reader.py	0:0 0:0	view
40	x 2	E2E_TOD data/multiwoz/utlis	reader.py reader.py	0:0 0:0	view
37	x 2	E2E_TOD data/multiwoz/utlis	db_ops.py db_ops.py	0:0 0:0	view
35	x 2	E2E_TOD/modelling DST/modelling	T5Model.py T5Model.py	0:0 0:0	view
31	x 2	E2E_TOD data/multiwoz/utlis	reader.py reader.py	0:0 0:0	view
34	x 2	E2E_TOD data/multiwoz/utlis	reader.py reader.py	0:0 0:0	view
28	x 2	E2E_TOD data/multiwoz/utlis	db_ops.py db_ops.py	0:0 0:0	view
36	x 2	E2E_TOD data/multiwoz/utlis	reader.py reader.py	0:0 0:0	view
24	x 2	data/multiwoz/utlis data/multiwoz/utlis	postprocessing_dataset.py processing_funcs.py	0:0 0:0	view
23	x 2	E2E_TOD data/multiwoz/utlis	db_ops.py db_ops.py	0:0 0:0	view
19	x 4	E2E_TOD E2E_TOD/modelling DST DST/modelling	dataclass.py T5Model.py dataclass.py T5Model.py	0:0 0:0 0:0 0:0	view
19	x 2	E2E_TOD data/multiwoz/utlis	utils.py utils.py	0:0 0:0	view
19	x 2	data/multiwoz/utlis data/multiwoz/utlis	postprocessing_dataset.py processing_funcs.py	0:0 0:0	view
21	x 2	E2E_TOD data/multiwoz/utlis	reader.py reader.py	0:0 0:0	view
17	x 2	E2E_TOD/modelling DST/modelling	T5Model.py T5Model.py	0:0 0:0	view
19	x 2	E2E_TOD data/multiwoz/utlis	reader.py reader.py	0:0 0:0	view
16	x 2	E2E_TOD DST	dataclass.py dataclass.py	0:0 0:0	view
15	x 2	E2E_TOD DST	dataclass.py dataclass.py	0:0 0:0	view
15	x 2	E2E_TOD DST	dataclass.py dataclass.py	0:0 0:0	view