G

Intro

For duplication, we look at places in code where there are six or more lines of code that are exactly the same.
Before duplication is calculated, the code is cleaned to remove empty lines, comments, and frequently duplicated constructs such as imports.
You should aim at having as little as possible (<5%) of duplicated code as high-level of duplication can lead to maintenance difficulties, poor factoring, and logical contradictions.

Learn more...

Duplication Overall

15% duplication:

23,433 cleaned lines of cleaned code (without empty lines, comments, and frequently duplicated constructs such as imports)
3,710 duplicated lines

164 duplicates

Duplication per Extension

Duplication per Component (primary)

Duplication Between Components (50+ lines)

Download: SVG DOT (open online Graphviz editor)

From Component --> To Component	Duplicated Lines	File Pairs	Details
examples/elastic --> examples	515	21 file pairs	details...
horovod/keras --> horovod/tensorflow	102	2 file pairs	details...
horovod/mxnet --> horovod/torch	100	4 file pairs	details...
horovod/torch --> horovod/tensorflow	64	2 file pairs	details...

Show more details on duplication between components...

Longest Duplicates

The list of 20 longest duplicates.

Size	#	Folders	Files	Lines	Code
145	x 2	examples examples	keras_spark3_rossmann.py keras_spark_rossmann_estimator.py	83:292 (40%) 78:287 (55%)	view
136	x 2	examples examples	keras_spark_rossmann_run.py keras_spark3_rossmann.py	182:409 (40%) 192:420 (38%)	view
69	x 2	examples examples	keras_spark_rossmann_run.py keras_spark_rossmann_estimator.py	182:282 (20%) 187:287 (26%)	view
63	x 2	examples examples	keras_spark_rossmann_run.py keras_spark3_rossmann.py	66:155 (18%) 75:164 (17%)	view
57	x 2	examples examples	keras_spark_rossmann_run.py keras_spark_rossmann_estimator.py	74:155 (17%) 78:159 (21%)	view
57	x 2	examples examples	keras_mnist.py tensorflow_keras_mnist.py	12:94 (100%) 14:96 (100%)	view
45	x 2	examples examples	keras_spark_rossmann_run.py keras_spark3_rossmann.py	411:491 (13%) 422:502 (12%)	view
38	x 2	horovod/common/ops horovod/common/ops	nccl_operations.cc adasum_gpu_operations.cc	224:297 (10%) 89:162 (19%)	view
30	x 2	horovod/keras horovod/tensorflow/keras	callbacks.py callbacks.py	22:169 (100%) 22:169 (100%)	view
29	x 2	horovod/torch horovod/torch	optimizer.py optimizer.py	36:69 (10%) 213:248 (10%)	view
28	x 3	examples examples examples	tensorflow_keras_mnist.py keras_mnist_advanced.py keras_mnist.py	34:65 (49%) 50:81 (37%) 32:63 (49%)	view
27	x 2	examples examples	tensorflow2_keras_mnist.py tensorflow2_mnist_ray.py	29:73 (69%) 20:64 (54%)	view
25	x 2	horovod/common/ops horovod/common/ops	mpi_operations.cc mpi_operations.cc	97:136 (9%) 191:230 (9%)	view
24	x 2	horovod/common/ops horovod/common/ops	mpi_gpu_operations.cc mpi_operations.cc	47:82 (17%) 39:74 (8%)	view
24	x 2	examples/elastic examples	tensorflow2_mnist_elastic.py tensorflow2_mnist.py	20:48 (42%) 20:48 (53%)	view
24	x 2	horovod/common/ops horovod/common/ops	mpi_operations.cc ccl_operations.cc	97:134 (8%) 148:185 (14%)	view
23	x 3	examples/elastic examples examples	tensorflow2_mnist_elastic.py tensorflow2_keras_mnist.py tensorflow2_mnist.py	20:47 (41%) 20:47 (58%) 20:47 (51%)	view
22	x 2	examples examples	keras_spark_rossmann_run.py keras_spark3_rossmann.py	35:57 (6%) 35:57 (6%)	view
21	x 2	horovod/keras horovod/tensorflow/keras	__init__.py __init__.py	34:159 (91%) 47:172 (77%)	view
21	x 2	horovod/common/ops horovod/common/ops	mpi_gpu_operations.cc mpi_operations.cc	143:167 (15%) 145:169 (7%)	view

Most Frequent Duplicates

The list of 20 most frequently found duplicates.

Size	#	Folders	Files	Lines	Code
6	x 6	examples/elastic examples examples examples examples/elastic examples	tensorflow_keras_mnist_elastic.py tensorflow_mnist_eager.py tensorflow2_mnist.py tensorflow2_mnist_ray.py tensorflow2_mnist_elastic.py tensorflow2_keras_mnist.py	24:30 (11%) 39:45 (14%) 29:35 (13%) 20:26 (12%) 29:35 (10%) 29:35 (15%)	view
9	x 5	examples/elastic examples examples examples examples/elastic	pytorch_synthetic_benchmark_elastic.py tensorflow_synthetic_benchmark.py tensorflow2_synthetic_benchmark.py pytorch_synthetic_benchmark.py tensorflow2_synthetic_benchmark_elast...	18:27 (10%) 17:26 (11%) 31:40 (13%) 18:27 (12%) 31:40 (10%)	view
7	x 5	horovod/common/ops horovod/common/ops horovod/common/ops horovod/common/ops horovod/common/ops	nccl_operations.cc mpi_gpu_operations.cc gloo_operations.cc mpi_operations.cc ccl_operations.cc	550:558 (1%) 174:182 (5%) 262:270 (3%) 173:181 (2%) 223:231 (4%)	view
6	x 5	examples examples examples examples/elastic examples	keras_spark_mnist.py keras_mnist_advanced.py tensorflow_keras_mnist.py tensorflow_keras_mnist_elastic.py keras_mnist.py	90:95 (8%) 75:80 (8%) 59:64 (10%) 37:42 (11%) 57:62 (10%)	view
8	x 5	examples/elastic examples examples examples examples/elastic	pytorch_synthetic_benchmark_elastic.py tensorflow_synthetic_benchmark.py tensorflow2_synthetic_benchmark.py pytorch_synthetic_benchmark.py tensorflow2_synthetic_benchmark_elast...	96:105 (8%) 82:91 (10%) 97:106 (11%) 89:98 (11%) 98:107 (9%)	view
6	x 5	horovod/common/ops horovod/common/ops horovod/common/ops horovod/common/ops horovod/common/ops	mpi_gpu_operations.cc ccl_operations.cc gloo_operations.cc mpi_operations.cc adasum_mpi_operations.cc	72:82 (4%) 103:113 (3%) 151:161 (2%) 64:74 (2%) 83:93 (11%)	view
8	x 4	examples examples examples examples/elastic	tensorflow_synthetic_benchmark.py tensorflow2_keras_synthetic_benchmark.py tensorflow2_synthetic_benchmark.py tensorflow2_synthetic_benchmark_elast...	11:19 (10%) 25:33 (11%) 25:33 (11%) 25:33 (9%)	view
6	x 4	horovod/tensorflow horovod/tensorflow horovod/tensorflow horovod/tensorflow	mpi_ops.cc mpi_ops.cc mpi_ops.cc mpi_ops.cc	638:643 (1%) 661:666 (1%) 686:691 (1%) 709:714 (1%)	view
7	x 4	examples/elastic examples examples examples	tensorflow_keras_mnist_elastic.py keras_mnist.py tensorflow_keras_mnist.py keras_mnist_advanced.py	37:43 (13%) 57:63 (12%) 59:65 (12%) 75:81 (9%)	view
7	x 4	horovod/common/ops horovod/common/ops horovod/common/ops horovod/common/ops	mpi_operations.h mpi_operations.h mpi_operations.h mpi_operations.h	39:47 (13%) 53:61 (13%) 81:89 (13%) 95:103 (13%)	view
8	x 4	horovod/common/ops horovod/common/ops horovod/common/ops horovod/common/ops	gloo_operations.cc mpi_operations.cc adasum_mpi_operations.cc ccl_operations.cc	151:165 (3%) 64:78 (2%) 83:97 (15%) 103:117 (4%)	view
8	x 4	horovod/common/ops horovod/common/ops horovod/common/ops horovod/common/ops	gloo_operations.cc ccl_operations.cc mpi_gpu_operations.cc mpi_operations.cc	262:272 (3%) 223:233 (4%) 174:184 (5%) 173:183 (2%)	view
17	x 4	examples examples examples/elastic examples	tensorflow2_mnist_ray.py tensorflow2_mnist.py tensorflow2_mnist_elastic.py tensorflow2_keras_mnist.py	20:38 (34%) 29:47 (37%) 29:47 (30%) 29:47 (43%)	view
8	x 4	horovod/common/ops horovod/common/ops horovod/common/ops horovod/common/ops	gloo_operations.cc mpi_operations.cc ccl_operations.cc mpi_gpu_operations.cc	195:206 (3%) 106:117 (2%) 157:168 (4%) 110:121 (5%)	view
7	x 3	examples examples examples	keras_spark3_rossmann.py keras_spark_rossmann_estimator.py keras_spark_rossmann_run.py	300:307 (1%) 298:305 (2%) 290:297 (2%)	view
16	x 3	examples examples examples	keras_spark_rossmann_run.py keras_spark3_rossmann.py keras_spark_rossmann_estimator.py	159:180 (4%) 169:190 (4%) 164:185 (6%)	view
6	x 3	horovod/tensorflow horovod/tensorflow horovod/tensorflow	mpi_ops.cc mpi_ops.cc mpi_ops.cc	463:469 (1%) 588:595 (1%) 743:749 (1%)	view
11	x 3	examples examples examples	keras_spark3_rossmann.py keras_spark_rossmann_estimator.py keras_spark_rossmann_run.py	41:51 (3%) 44:54 (4%) 41:51 (3%)	view
12	x 3	horovod/common/ops horovod/common/ops horovod/common/ops	mpi_gpu_operations.cc ccl_operations.cc mpi_operations.cc	104:123 (8%) 151:170 (7%) 100:119 (4%)	view
9	x 3	horovod/common horovod/common horovod/common	operations.cc operations.cc operations.cc	924:935 (1%) 963:974 (1%) 1056:1067 (1%)	view

HorovodDuplication

Duplication Between Components (50+ lines)

Horovod
Duplication