awslabs / deeplearning-benchmark
Duplication

Places in code with 6 or more lines that are exactly the same.

Intro
  • For duplication, we look at places in code where there are 6 or more lines of code that are exactly the same.
  • Before duplication is calculated, the code is cleaned to remove empty lines, comments, and frequently duplicated constructs such as imports.
  • You should aim at having as little as possible (<5%) of duplicated code as high-level of duplication can lead to maintenance difficulties, poor factoring, and logical contradictions.
Learn more...
Duplication Overall
  • 17% duplication:
    • 11,171 cleaned lines of cleaned code (without empty lines, comments, and frequently duplicated constructs such as imports)
    • 2,008 duplicated lines
  • 116 duplicates
system17% (2,008 lines)
Duplication per Extension
py16% (1,723 lines)
scala24% (139 lines)
yaml66% (124 lines)
java11% (22 lines)
Duplication per Component (primary)
image_classification27% (623 lines)
tensorflow19% (525 lines)
word_language_model49% (284 lines)
tensorflow_benchmark5% (184 lines)
scala-mxnet26% (161 lines)
reporting26% (124 lines)
ssd36% (44 lines)
benchmark16% (32 lines)
ROOT18% (16 lines)
djl7% (15 lines)
end_to_end_model_benchmark0% (0 lines)
onnx_benchmark0% (0 lines)
utils0% (0 lines)
dawnbench0% (0 lines)
dependency_update0% (0 lines)

Duplication Between Components (50+ lines)

G tensorflow tensorflow tensorflow_benchmark tensorflow_benchmark tensorflow--tensorflow_benchmark 56

Download: SVG DOT (open online Graphviz editor)

Open 3D force graph...

Show more details on duplication between components...
Longest Duplicates
The list of 20 longest duplicates.
See data for all 116 duplicates...
Size#FoldersFilesLinesCode
57 x 2 word_language_model
word_language_model
word_language_model.py
word_language_model_train.py
 26:93 (37%)
26:93 (38%)
 view
53 x 2 image_classification/symbols
image_classification/symbols
resnet-v1.py
resnet.py
 140:200 (42%)
138:196 (41%)
 view
45 x 2 image_classification
image_classification
infer_imagenet.py
infer_imagenet_gpu.py
 27:71 (26%)
30:74 (24%)
 view
39 x 2 word_language_model
word_language_model
word_language_model.py
word_language_model_train.py
 95:152 (25%)
95:152 (26%)
 view
36 x 2 image_classification/symbols
image_classification/symbols
resnet.py
resnext.py
 152:187 (28%)
165:200 (27%)
 view
36 x 2 image_classification/symbols
image_classification/symbols
resnet-v1.py
resnext.py
 156:191 (28%)
165:200 (27%)
 view
24 x 2 image_classification
image_classification
infer_imagenet.py
infer_imagenet_gpu.py
 90:115 (14%)
77:106 (12%)
 view
22 x 2 tensorflow_benchmark/tf_cnn_benchmarks
tensorflow_benchmark/tf_cnn_benchmarks
variable_mgr.py
variable_mgr.py
 667:694 (3%)
767:794 (3%)
 view
21 x 2 word_language_model
word_language_model
word_language_model.py
word_language_model_train.py
 158:188 (13%)
160:190 (14%)
 view
21 x 2 image_classification/symbols
image_classification/symbols
resnet.py
resnext.py
 107:128 (16%)
122:143 (16%)
 view
21 x 2 tensorflow/inception/inception
tensorflow_benchmark/tf_cnn_benchmarks
image_processing.py
preprocessing.py
 337:395 (15%)
31:89 (4%)
 view
18 x 2 tensorflow/inception/inception/data
tensorflow/inception/inception/data
build_image_data.py
build_imagenet_data.py
 240:261 (10%)
363:384 (6%)
 view
16 x 2 tensorflow/inception/inception/slim
tensorflow/inception/inception/slim
inception_model.py
inception_model.py
 193:208 (6%)
212:227 (6%)
 view
15 x 2 tensorflow/inception/inception/slim
tensorflow/inception/inception/slim
inception_model.py
inception_model.py
 283:297 (6%)
301:315 (6%)
 view
15 x 2 image_classification
image_classification
infer_imagenet.py
infer_imagenet_gpu.py
 11:25 (8%)
14:28 (8%)
 view
15 x 2 tensorflow/inception/inception/data
tensorflow/inception/inception/data
build_image_data.py
build_imagenet_data.py
 268:284 (8%)
394:410 (5%)
 view
15 x 2 ssd
ssd
ssd_inference.py
ssd_inference.py
 80:100 (12%)
110:130 (12%)
 view
14 x 2 scala-mxnet/scala-bm/src/main/scala/mxnet
scala-mxnet/scala-bm/src/main/scala/mxnet
SSDObjectDetection.scala
SSDObjectDetection.scala
 64:87 (16%)
96:120 (16%)
 view
14 x 2 word_language_model
word_language_model
word_language_model.py
word_language_model_train.py
 203:222 (9%)
206:225 (9%)
 view
13 x 2 scala-mxnet/scala-bm/src/main/scala/mxnet
scala-mxnet/scala-bm/src/main/scala/mxnet
ImageClassification.scala
ImageClassification.scala
 77:100 (14%)
114:137 (14%)
 view
Duplicated Units
The list of top 3 duplicated units.
See data for all 3 unit duplicates...
Size#FoldersFilesLinesCode
10 x 2 word_language_model
word_language_model
lstm_bucketing.py
lstm_bucketing_infer.py
 0:0 
0:0 
 view
7 x 2 tensorflow/resnet
tensorflow/resnet
resnet_main.py
resnet_dist.py
 0:0 
0:0 
 view
7 x 2 tensorflow_benchmark/tf_cnn_benchmarks
tensorflow_benchmark/tf_cnn_benchmarks
variable_mgr.py
variable_mgr.py
 0:0 
0:0 
 view