G

Intro

For duplication, we look at places in code where there are 6 or more lines of code that are exactly the same.
Before duplication is calculated, the code is cleaned to remove empty lines, comments, and frequently duplicated constructs such as imports.
You should aim at having as little as possible (<5%) of duplicated code as high-level of duplication can lead to maintenance difficulties, poor factoring, and logical contradictions.

Learn more...

Duplication Overall

9% duplication:

49,426 cleaned lines of cleaned code (without empty lines, comments, and frequently duplicated constructs such as imports)
4,910 duplicated lines

1,433 duplicates

Duplication per Extension

Duplication per Component (primary)

Duplication Between Components (50+ lines)

Download: SVG DOT (open online Graphviz editor)

Open 3D force graph...

Show more details on duplication between components...

From Component --> To Component	Duplicated Lines	File Pairs	Details
tensorflow_datasets/structured (4%) --> tensorflow_datasets/text (2%)	319	101 file pairs	details...
tensorflow_datasets/image (6%) --> tensorflow_datasets/image_classification (1%)	271	69 file pairs	details...
tensorflow_datasets/image (5%) --> tensorflow_datasets/text (1%)	264	84 file pairs	details...
tensorflow_datasets/image_classification (1%) --> tensorflow_datasets/text (1%)	237	48 file pairs	details...
tensorflow_datasets/image (4%) --> tensorflow_datasets/structured (2%)	212	35 file pairs	details...
tensorflow_datasets/question_answering (8%) --> tensorflow_datasets/text (1%)	211	23 file pairs	details...
tensorflow_datasets/summarization (6%) --> tensorflow_datasets/text (1%)	194	58 file pairs	details...
tensorflow_datasets/image_classification (1%) --> tensorflow_datasets/structured (2%)	168	24 file pairs	details...
tensorflow_datasets/image_classification (<1%) --> tensorflow_datasets/summarization (3%)	120	20 file pairs	details...
tensorflow_datasets/image (2%) --> tensorflow_datasets/summarization (3%)	108	26 file pairs	details...
tensorflow_datasets/audio (2%) --> tensorflow_datasets/text (<1%)	102	30 file pairs	details...
tensorflow_datasets/image_classification (<1%) --> tensorflow_datasets/question_answering (3%)	95	13 file pairs	details...
tensorflow_datasets/image (1%) --> tensorflow_datasets/question_answering (3%)	84	11 file pairs	details...
tensorflow_datasets/image (1%) --> tensorflow_datasets/testing (1%)	76	1 file pair	details...
tensorflow_datasets/structured (<1%) --> tensorflow_datasets/summarization (2%)	72	12 file pairs	details...
tensorflow_datasets/question_answering (2%) --> tensorflow_datasets/structured (<1%)	66	11 file pairs	details...
tensorflow_datasets/audio (1%) --> tensorflow_datasets/image (1%)	60	14 file pairs	details...
tensorflow_datasets/audio (1%) --> tensorflow_datasets/question_answering (2%)	60	11 file pairs	details...
tensorflow_datasets/audio (1%) --> tensorflow_datasets/structured (<1%)	54	10 file pairs	details...
tensorflow_datasets/audio (1%) --> tensorflow_datasets/summarization (1%)	54	13 file pairs	details...

Longest Duplicates

The list of 20 longest duplicates.

See data for all 1,433 duplicates...

Size	#	Folders	Files	Lines	Code
56	x 2	tensorflow_datasets/text tensorflow_datasets/text	math_dataset.py math_dataset.py	70:125 (28%) 128:183 (28%)	view
37	x 2	tensorflow_datasets/text/gem tensorflow_datasets/text/gem	gem.py gem.py	840:877 (2%) 909:945 (2%)	view
31	x 2	tensorflow_datasets/audio tensorflow_datasets/audio	crema_d.py savee.py	56:121 (31%) 60:124 (32%)	view
30	x 2	tensorflow_datasets/text/gem tensorflow_datasets/text/gem	gem.py gem.py	840:870 (1%) 1189:1218 (1%)	view
30	x 2	tensorflow_datasets/text/gem tensorflow_datasets/text/gem	gem.py gem.py	909:938 (1%) 1189:1218 (1%)	view
26	x 2	tensorflow_datasets/audio tensorflow_datasets/audio	crema_d.py dementiabank.py	56:114 (26%) 46:103 (29%)	view
26	x 2	tensorflow_datasets/audio tensorflow_datasets/audio	dementiabank.py savee.py	46:103 (29%) 60:117 (27%)	view
24	x 2	tensorflow_datasets/text tensorflow_datasets/text	eraser_multi_rc.py movie_rationales.py	74:101 (38%) 69:93 (38%)	view
24	x 2	tensorflow_datasets/image_classification tensorflow_datasets/image_classification	plant_leaves.py plantae_k.py	101:132 (30%) 92:123 (32%)	view
24	x 2	tensorflow_datasets/rlds...te_panda_pick_place_can tensorflow_datasets/rlds...te_panda_pick_place_can	robosuite_panda_pick_place_can.py robosuite_panda_pick_place_can.py	94:117 (12%) 162:185 (12%)	view
23	x 2	tensorflow_datasets/image_classification tensorflow_datasets/image_classification	imagenette.py imagewang.py	111:135 (34%) 118:142 (32%)	view
22	x 2	tensorflow_datasets/imag...cation/i_naturalist2018 tensorflow_datasets/image_classification	i_naturalist2018.py inaturalist.py	130:155 (21%) 135:160 (22%)	view
22	x 2	tensorflow_datasets/text/gem tensorflow_datasets/text/gem	gem.py gem.py	879:900 (1%) 923:944 (1%)	view
22	x 2	tensorflow_datasets/text/gem tensorflow_datasets/text/gem	gem.py gem.py	855:876 (1%) 879:900 (1%)	view
21	x 2	tensorflow_datasets/question_answering tensorflow_datasets/question_answering	ai2_arc.py ai2_arc_with_ir.py	74:98 (22%) 95:119 (21%)	view
20	x 2	tensorflow_datasets/text tensorflow_datasets/text/xtreme_xnli	xnli.py xtreme_xnli.py	60:83 (40%) 51:74 (27%)	view
19	x 2	tensorflow_datasets/text/gem tensorflow_datasets/text/gem	gem.py gem.py	786:806 (1%) 1061:1080 (1%)	view
19	x 2	tensorflow_datasets/structured/web_nlg tensorflow_datasets/structured	web_nlg.py wiki_bio.py	61:84 (17%) 78:98 (15%)	view
19	x 2	tensorflow_datasets/text/gem tensorflow_datasets/text/gem	gem.py gem.py	955:973 (1%) 1061:1080 (1%)	view
19	x 2	tensorflow_datasets/text/gem tensorflow_datasets/text/gem	gem.py gem.py	786:806 (1%) 955:973 (1%)	view

Duplicated Units

The list of top 15 duplicated units.

See data for all 15 unit duplicates...

Size	#	Folders	Files	Lines	Code
24	x 2	tensorflow_datasets/image_classification tensorflow_datasets/image_classification	plant_leaves.py plantae_k.py	0:0 0:0	view
19	x 2	tensorflow_datasets/text/xtreme_xnli tensorflow_datasets/text	xtreme_xnli.py xnli.py	0:0 0:0	view
27	x 4	tensorflow_datasets/audio tensorflow_datasets/audio tensorflow_datasets/audio tensorflow_datasets/audio	dementiabank.py voxforge.py savee.py crema_d.py	0:0 0:0 0:0 0:0	view
25	x 2	tensorflow_datasets/audio tensorflow_datasets/audio	voxforge.py crema_d.py	0:0 0:0	view
12	x 2	tensorflow_datasets/audio/gtzan_music_speech tensorflow_datasets/audio/gtzan	gtzan_music_speech.py gtzan.py	0:0 0:0	view
12	x 2	tensorflow_datasets/core/logging tensorflow_datasets/core/logging	logging_logger.py base_logger.py	0:0 0:0	view
14	x 2	tensorflow_datasets/translate tensorflow_datasets/translate	ted_hrlr.py flores.py	0:0 0:0	view
12	x 2	tensorflow_datasets/text/paws_x_wiki tensorflow_datasets/text	paws_x_wiki.py paws_wiki.py	0:0 0:0	view
8	x 2	tensorflow_datasets/testing tensorflow_datasets/testing	feature_test_case.py feature_test_case.py	0:0 0:0	view
9	x 2	tensorflow_datasets/image_classification tensorflow_datasets/image_classification	imagewang.py imagenette.py	0:0 0:0	view
7	x 3	tensorflow_datasets/question_answering tensorflow_datasets/question_answering tensorflow_datasets/question_answering	mctaco.py cosmos_qa.py web_questions.py	0:0 0:0 0:0	view
6	x 2	tensorflow_datasets/testing/fake_data_generation tensorflow_datasets/image/bccd	voc2007.py dummy_data_generation.py	0:0 0:0	view
6	x 3	tensorflow_datasets/testing/fake_data_generation tensorflow_datasets/testing/fake_data_generation tensorflow_datasets/testing/fake_data_generation	cassava.py beans.py caltech.py	0:0 0:0 0:0	view
6	x 2	tensorflow_datasets/image tensorflow_datasets/image	shapes3d.py dsprites.py	0:0 0:0	view
6	x 2	tensorflow_datasets/image_classification tensorflow_datasets/image_classification	citrus.py plant_village.py	0:0 0:0	view