aws-samples / aws-ml-jp
Duplication

Places in code with 6 or more lines that are exactly the same.

Intro
  • For duplication, we look at places in code where there are 6 or more lines of code that are exactly the same.
  • Before duplication is calculated, the code is cleaned to remove empty lines, comments, and frequently duplicated constructs such as imports.
  • You should aim at having as little as possible (<5%) of duplicated code as high-level of duplication can lead to maintenance difficulties, poor factoring, and logical contradictions.
Learn more...
Duplication Overall
  • 40% duplication:
    • 4,312 cleaned lines of cleaned code (without empty lines, comments, and frequently duplicated constructs such as imports)
    • 1,746 duplicated lines
  • 118 duplicates
system40% (1,746 lines)
Duplication per Extension
py40% (1,590 lines)
yaml69% (156 lines)
Duplication per Component (primary)
sagemaker/sagemaker-inference87% (762 lines)
sagemaker/distributed-training84% (504 lines)
sagemaker/sagemaker-experiments100% (162 lines)
sagemaker/hpo-pytorch-mnist72% (140 lines)
sagemaker/workshop67% (102 lines)
inference/edge15% (60 lines)
sagemaker/sagemaker-processing19% (16 lines)
mlops/edge-deploy0% (0 lines)
mlops/sagemaker-pipelines0% (0 lines)
nlp/nlp_amazon_review0% (0 lines)
ai-services/forecast0% (0 lines)
vision/amazon-sagemaker-pytorch-detectron20% (0 lines)
autogluon/tabular-prediction0% (0 lines)
sagemaker/tensorflow2-training-and-serving0% (0 lines)

Duplication Between Components (50+ lines)

G sagemaker/sagemaker-experiments sagemaker/sagemaker-experiments sagemaker/sagemaker-inference sagemaker/sagemaker-inference sagemaker/sagemaker-experiments--sagemaker/sagemaker-inference 815 sagemaker/hpo-pytorch-mnist sagemaker/hpo-pytorch-mnist sagemaker/hpo-pytorch-mnist--sagemaker/sagemaker-experiments 302 sagemaker/hpo-pytorch-mnist--sagemaker/sagemaker-inference 793

Download: SVG DOT (open online Graphviz editor)

Open 3D force graph...

Show more details on duplication between components...
Longest Duplicates
The list of 20 longest duplicates.
See data for all 118 duplicates...
Size#FoldersFilesLinesCode
140 x 2 sagemaker/hpo-pytorch-mnist
sagemaker/sagemaker-experiments/pytorch_mnist/src
mnist.py
mnist_train.py
 15:194 (100%)
15:193 (100%)
 view
116 x 2 sagemaker/sagemaker-inference/pytorch/code_async
sagemaker/sagemaker-inference/pytorch/code_byoc
train.py
train.py
 63:211 (81%)
55:203 (84%)
 view
92 x 2 sagemaker/sagemaker-inference/pytorch/code_async
sagemaker/sagemaker-inference/pytorch/code_mme
train.py
train.py
 63:180 (64%)
104:221 (56%)
 view
92 x 2 sagemaker/sagemaker-inference/pytorch/code_byoc
sagemaker/sagemaker-inference/pytorch/code_mme
train.py
train.py
 55:172 (67%)
104:221 (56%)
 view
91 x 2 sagemaker/sagemaker-inference/pytorch/code_byoc
sagemaker/sagemaker-inference/pytorch/code_lambda
train.py
train.py
 55:171 (66%)
25:141 (77%)
 view
91 x 2 sagemaker/sagemaker-inference/pytorch/code_async
sagemaker/sagemaker-inference/pytorch/code_lambda
train.py
train.py
 63:179 (63%)
25:141 (77%)
 view
91 x 2 sagemaker/sagemaker-inference/pytorch/code_lambda
sagemaker/sagemaker-inference/pytorch/code_mme
train.py
train.py
 25:141 (77%)
104:220 (55%)
 view
87 x 2 sagemaker/hpo-pytorch-mnist
sagemaker/sagemaker-inference/pytorch/code_byoc
mnist.py
train.py
 39:148 (62%)
55:164 (63%)
 view
87 x 2 sagemaker/hpo-pytorch-mnist
sagemaker/sagemaker-inference/pytorch/code_async
mnist.py
train.py
 39:148 (62%)
63:172 (60%)
 view
87 x 2 sagemaker/sagemaker-experiments/pytorch_mnist/src
sagemaker/sagemaker-inference/pytorch/code_async
mnist_train.py
train.py
 39:147 (62%)
63:172 (60%)
 view
87 x 2 sagemaker/sagemaker-experiments/pytorch_mnist/src
sagemaker/sagemaker-inference/pytorch/code_byoc
mnist_train.py
train.py
 39:147 (62%)
55:164 (63%)
 view
87 x 2 sagemaker/hpo-pytorch-mnist
sagemaker/sagemaker-inference/pytorch/code_mme
mnist.py
train.py
 39:148 (62%)
104:213 (53%)
 view
87 x 2 sagemaker/hpo-pytorch-mnist
sagemaker/sagemaker-inference/pytorch/code_lambda
mnist.py
train.py
 39:148 (62%)
25:134 (73%)
 view
87 x 2 sagemaker/sagemaker-experiments/pytorch_mnist/src
sagemaker/sagemaker-inference/pytorch/code_mme
mnist_train.py
train.py
 39:147 (62%)
104:213 (53%)
 view
87 x 2 sagemaker/sagemaker-experiments/pytorch_mnist/src
sagemaker/sagemaker-inference/pytorch/code_lambda
mnist_train.py
train.py
 39:147 (62%)
25:134 (73%)
 view
75 x 2 sagemaker/distributed-training
sagemaker/distributed-training
train_pytorch_single_maskrcnn.py
train_pytorch_smdataparallel_maskrcnn.py
 127:211 (35%)
144:228 (36%)
 view
71 x 2 sagemaker/sagemaker-inference/pytorch/code_inf1
sagemaker/sagemaker-inference/pytorch/code_lambda
train.py
train.py
 59:149 (49%)
25:115 (60%)
 view
71 x 2 sagemaker/sagemaker-experiments/pytorch_mnist/src
sagemaker/sagemaker-inference/pytorch/code_inf1
mnist_train.py
train.py
 39:128 (50%)
59:149 (49%)
 view
71 x 2 sagemaker/sagemaker-inference/pytorch/code_async
sagemaker/sagemaker-inference/pytorch/code_inf1
train.py
train.py
 63:153 (49%)
59:149 (49%)
 view
71 x 2 sagemaker/sagemaker-inference/pytorch/code_byoc
sagemaker/sagemaker-inference/pytorch/code_inf1
train.py
train.py
 55:145 (51%)
59:149 (49%)
 view
Duplicated Units
The list of top 11 duplicated units.
See data for all 11 unit duplicates...
Size#FoldersFilesLinesCode
56 x 6 sagemaker/hpo-pytorch-mnist
sagemaker/sagemaker-inference/pytorch/code_byoc
sagemaker/sagemaker-inference/pytorch/code_async
sagemaker/sagemaker-inference/pytorch/code_lambda
sagemaker/sagemaker-inference/pytorch/code_mme
sagemaker/sagemaker-experiments/pytorch_mnist/src
mnist.py
train.py
train.py
train.py
train.py
mnist_train.py
 0:0 
0:0 
0:0 
0:0 
0:0 
0:0 
 view
77 x 2 sagemaker/workshop/lab_b...ur-own-model/tensorflow
sagemaker/workshop/lab_b...ur-own-model/tensorflow
cnn_mnist_after.py
cnn_mnist_before.py
 0:0 
0:0 
 view
27 x 2 sagemaker/distributed-training
sagemaker/distributed-training
train_pytorch_single_maskrcnn.py
train_pytorch_smdataparallel_maskrcnn.py
 0:0 
0:0 
 view
28 x 4 sagemaker/sagemaker-inference/pytorch/code_inf1
sagemaker/sagemaker-inference/pytorch/code_byoc
sagemaker/sagemaker-inference/pytorch/code_async
sagemaker/sagemaker-inference/pytorch/code_mme
train.py
train.py
train.py
train.py
 0:0 
0:0 
0:0 
0:0 
 view
14 x 7 sagemaker/hpo-pytorch-mnist
sagemaker/sagemaker-inference/pytorch/code_inf1
sagemaker/sagemaker-inference/pytorch/code_byoc
sagemaker/sagemaker-inference/pytorch/code_async
sagemaker/sagemaker-inference/pytorch/code_lambda
sagemaker/sagemaker-inference/pytorch/code_mme
sagemaker/sagemaker-experiments/pytorch_mnist/src
mnist.py
train.py
train.py
train.py
train.py
train.py
mnist_train.py
 0:0 
0:0 
0:0 
0:0 
0:0 
0:0 
0:0 
 view
13 x 2 sagemaker/distributed-training
sagemaker/distributed-training
train_pytorch_single_maskrcnn.py
train_pytorch_smdataparallel_maskrcnn.py
 0:0 
0:0 
 view
7 x 8 sagemaker/hpo-pytorch-mnist
sagemaker/sagemaker-inference/pytorch/code_inf1
sagemaker/sagemaker-inference/pytorch/code_byoc
sagemaker/sagemaker-inference/pytorch/code_async
sagemaker/sagemaker-inference/pytorch/code_lambda
sagemaker/sagemaker-inference/pytorch/code_mme
sagemaker/sagemaker-experiments/pytorch_mnist/src
sagemaker/sagemaker-experiments/pytorch_mnist/src
mnist.py
model.py
model.py
model.py
model.py
model.py
mnist_train.py
mnist_deploy.py
 0:0 
0:0 
0:0 
0:0 
0:0 
0:0 
0:0 
0:0 
 view
7 x 7 sagemaker/hpo-pytorch-mnist
sagemaker/sagemaker-inference/pytorch/code_inf1
sagemaker/sagemaker-inference/pytorch/code_byoc
sagemaker/sagemaker-inference/pytorch/code_async
sagemaker/sagemaker-inference/pytorch/code_lambda
sagemaker/sagemaker-inference/pytorch/code_mme
sagemaker/sagemaker-experiments/pytorch_mnist/src
mnist.py
train.py
train.py
train.py
train.py
train.py
mnist_train.py
 0:0 
0:0 
0:0 
0:0 
0:0 
0:0 
0:0 
 view
7 x 2 sagemaker/sagemaker-proc...ntainer/preprocess_code
sagemaker/sagemaker-proc...inner/preprocess_script
preprocess.py
preprocess.py
 0:0 
0:0 
 view
6 x 8 sagemaker/hpo-pytorch-mnist
sagemaker/sagemaker-inference/pytorch/code_inf1
sagemaker/sagemaker-inference/pytorch/code_byoc
sagemaker/sagemaker-inference/pytorch/code_async
sagemaker/sagemaker-inference/pytorch/code_lambda
sagemaker/sagemaker-inference/pytorch/code_mme
sagemaker/sagemaker-experiments/pytorch_mnist/src
sagemaker/sagemaker-experiments/pytorch_mnist/src
mnist.py
model.py
model.py
model.py
model.py
model.py
mnist_train.py
mnist_deploy.py
 0:0 
0:0 
0:0 
0:0 
0:0 
0:0 
0:0 
0:0 
 view
6 x 7 sagemaker/hpo-pytorch-mnist
sagemaker/sagemaker-inference/pytorch/code_inf1
sagemaker/sagemaker-inference/pytorch/code_byoc
sagemaker/sagemaker-inference/pytorch/code_async
sagemaker/sagemaker-inference/pytorch/code_lambda
sagemaker/sagemaker-inference/pytorch/code_mme
sagemaker/sagemaker-experiments/pytorch_mnist/src
mnist.py
train.py
train.py
train.py
train.py
train.py
mnist_train.py
 0:0 
0:0 
0:0 
0:0 
0:0 
0:0 
0:0 
 view