aws / deep-learning-containers
Duplication

Places in code with 6 or more lines that are exactly the same.

Intro
  • For duplication, we look at places in code where there are 6 or more lines of code that are exactly the same.
  • Before duplication is calculated, the code is cleaned to remove empty lines, comments, and frequently duplicated constructs such as imports.
  • You should aim at having as little as possible (<5%) of duplicated code as high-level of duplication can lead to maintenance difficulties, poor factoring, and logical contradictions.
Learn more...
Duplication Overall
  • 54% duplication:
    • 8,953 cleaned lines of cleaned code (without empty lines, comments, and frequently duplicated constructs such as imports)
    • 4,874 duplicated lines
  • 927 duplicates
system54% (4,874 lines)
Duplication per Extension
yml73% (3,249 lines)
py32% (1,252 lines)
js98% (274 lines)
yaml27% (99 lines)
Duplication per Component (primary)
tensorflow76% (2,573 lines)
pytorch77% (1,282 lines)
mxnet72% (435 lines)
huggingface66% (278 lines)
eks_infrastructure28% (106 lines)
habana60% (79 lines)
ROOT22% (71 lines)
autogluon45% (36 lines)
src<1% (14 lines)
release0% (0 lines)
scheduler0% (0 lines)

Duplication Between Components (50+ lines)

G mxnet mxnet pytorch pytorch mxnet--pytorch 662 tensorflow tensorflow mxnet--tensorflow 313 pytorch--tensorflow 642 autogluon autogluon autogluon--mxnet 119 autogluon--pytorch 156 autogluon--tensorflow 55 habana habana habana--pytorch 124 habana--tensorflow 50

Download: SVG DOT (open online Graphviz editor)

Open 3D force graph...

Show more details on duplication between components...
Longest Duplicates
The list of 20 longest duplicates.
See data for all 927 duplicates...
Size#FoldersFilesLinesCode
101 x 2 tensorflow/inference/doc...ild_artifacts/sagemaker
tensorflow/inference/doc...ifacts/sagemaker_neuron
python_service.py
python_service.py
294:411 (32%)
295:412 (31%)
view
90 x 2 tensorflow/inference/doc...ild_artifacts/sagemaker
tensorflow/inference/doc...ifacts/sagemaker_neuron
tensorflow-serving.js
tensorflow-serving.js
83:233 (64%)
82:231 (64%)
view
89 x 2 pytorch
pytorch
buildspec-1-5-1.yml
buildspec-1-5.yml
5:96 (95%)
5:96 (95%)
view
80 x 2 tensorflow
tensorflow
buildspec-2-3.yml
buildspec-2-4.yml
7:88 (56%)
7:87 (72%)
view
56 x 2 tensorflow/inference/doc...ild_artifacts/sagemaker
tensorflow/inference/doc...ifacts/sagemaker_neuron
serve.py
serve.py
124:194 (16%)
112:182 (19%)
view
54 x 2 pytorch
pytorch
buildspec-1-4.yml
buildspec-1-5.yml
5:61 (36%)
5:61 (58%)
view
54 x 2 pytorch
pytorch
buildspec-1-4.yml
buildspec-1-5-1.yml
5:61 (36%)
5:61 (58%)
view
51 x 2 tensorflow/inference/doc...ild_artifacts/sagemaker
tensorflow/inference/doc...ifacts/sagemaker_neuron
python_service.py
python_service.py
108:168 (16%)
95:155 (15%)
view
47 x 2 tensorflow/inference/doc...ild_artifacts/sagemaker
tensorflow/inference/doc...ifacts/sagemaker_neuron
tensorflow-serving.js
tensorflow-serving.js
1:77 (33%)
1:77 (33%)
view
46 x 2 tensorflow/inference/doc...ild_artifacts/sagemaker
tensorflow/inference/doc...ifacts/sagemaker_neuron
python_service.py
python_service.py
173:221 (14%)
158:206 (14%)
view
42 x 2 tensorflow/inference/doc...ild_artifacts/sagemaker
tensorflow/inference/doc...ifacts/sagemaker_neuron
tfs_utils.py
tfs_utils.py
52:106 (23%)
45:99 (31%)
view
39 x 2 pytorch
pytorch
buildspec-1-7-1.yml
buildspec-1-8-0.yml
7:47 (38%)
6:46 (39%)
view
39 x 2 pytorch
pytorch
buildspec-1-9-1-neuron.yml
buildspec-neuron.yml
1:42 (100%)
1:42 (100%)
view
39 x 2 tensorflow/inference/doc...ild_artifacts/sagemaker
tensorflow/inference/doc...ifacts/sagemaker_neuron
tfs_utils.py
tfs_utils.py
175:220 (21%)
140:185 (29%)
view
39 x 2 pytorch
pytorch
buildspec-1-8-1.yml
buildspec-1-9-1.yml
7:47 (36%)
7:47 (36%)
view
37 x 2 pytorch
pytorch
buildspec-1-6.yml
buildspec-1-7-1.yml
8:46 (39%)
7:45 (36%)
view
37 x 2 pytorch
pytorch
buildspec-1-6.yml
buildspec-1-8-0.yml
8:46 (39%)
6:44 (37%)
view
37 x 2 pytorch
pytorch
buildspec-1-9-1.yml
buildspec-1-9-diy-sm.yml
5:44 (34%)
5:44 (22%)
view
37 x 2 tensorflow
tensorflow
buildspec-2-5-neuron.yml
buildspec-neuron.yml
1:40 (88%)
1:40 (88%)
view
36 x 2 pytorch
pytorch
buildspec-1-8-1.yml
buildspec-1-9-diy-sm.yml
7:44 (33%)
7:44 (22%)
view
Duplicated Units
The list of top 20 duplicated units.
See data for all 32 unit duplicates...
Size#FoldersFilesLinesCode
36 x 2 tensorflow/inference/doc...ifacts/sagemaker_neuron
tensorflow/inference/doc...ild_artifacts/sagemaker
python_service.py
python_service.py
0:0 
0:0 
view
32 x 2 tensorflow/inference/doc...ifacts/sagemaker_neuron
tensorflow/inference/doc...ild_artifacts/sagemaker
tensorflow-serving.js
tensorflow-serving.js
189:232 
191:234 
view
27 x 2 tensorflow/inference/doc...ifacts/sagemaker_neuron
tensorflow/inference/doc...ild_artifacts/sagemaker
python_service.py
python_service.py
0:0 
0:0 
view
28 x 2 tensorflow/inference/doc...ifacts/sagemaker_neuron
tensorflow/inference/doc...ild_artifacts/sagemaker
serve.py
serve.py
0:0 
0:0 
view
26 x 2 tensorflow/inference/doc...ifacts/sagemaker_neuron
tensorflow/inference/doc...ild_artifacts/sagemaker
tensorflow-serving.js
tensorflow-serving.js
107:136 
109:138 
view
17 x 2 tensorflow/inference/doc...ifacts/sagemaker_neuron
tensorflow/inference/doc...ild_artifacts/sagemaker
tensorflow-serving.js
tensorflow-serving.js
168:187 
170:189 
view
16 x 2 tensorflow/inference/doc...ifacts/sagemaker_neuron
tensorflow/inference/doc...ild_artifacts/sagemaker
serve.py
serve.py
0:0 
0:0 
view
14 x 2 tensorflow/inference/doc...ifacts/sagemaker_neuron
tensorflow/inference/doc...ild_artifacts/sagemaker
tfs_utils.py
tfs_utils.py
0:0 
0:0 
view
13 x 2 tensorflow/inference/doc...ifacts/sagemaker_neuron
tensorflow/inference/doc...ild_artifacts/sagemaker
python_service.py
python_service.py
0:0 
0:0 
view
10 x 2 tensorflow/inference/doc...ifacts/sagemaker_neuron
tensorflow/inference/doc...ild_artifacts/sagemaker
python_service.py
python_service.py
0:0 
0:0 
view
10 x 2 tensorflow/inference/doc...ifacts/sagemaker_neuron
tensorflow/inference/doc...ild_artifacts/sagemaker
tensorflow-serving.js
tensorflow-serving.js
92:105 
94:107 
view
9 x 2 tensorflow/inference/doc...ifacts/sagemaker_neuron
tensorflow/inference/doc...ild_artifacts/sagemaker
tensorflow-serving.js
tensorflow-serving.js
4:14 
4:14 
view
9 x 2 tensorflow/inference/doc...ifacts/sagemaker_neuron
tensorflow/inference/doc...ild_artifacts/sagemaker
tensorflow-serving.js
tensorflow-serving.js
138:148 
140:150 
view
9 x 2 tensorflow/inference/doc...ifacts/sagemaker_neuron
tensorflow/inference/doc...ild_artifacts/sagemaker
tfs_utils.py
tfs_utils.py
0:0 
0:0 
view
9 x 2 tensorflow/inference/doc...ifacts/sagemaker_neuron
tensorflow/inference/doc...ild_artifacts/sagemaker
tfs_utils.py
tfs_utils.py
0:0 
0:0 
view
8 x 2 tensorflow/inference/doc...ifacts/sagemaker_neuron
tensorflow/inference/doc...ild_artifacts/sagemaker
multi_model_utils.py
multi_model_utils.py
0:0 
0:0 
view
8 x 2 tensorflow/inference/doc...ifacts/sagemaker_neuron
tensorflow/inference/doc...ild_artifacts/sagemaker
multi_model_utils.py
multi_model_utils.py
0:0 
0:0 
view
9 x 2 tensorflow/inference/doc...ifacts/sagemaker_neuron
tensorflow/inference/doc...ild_artifacts/sagemaker
python_service.py
python_service.py
0:0 
0:0 
view
8 x 2 tensorflow/inference/doc...ifacts/sagemaker_neuron
tensorflow/inference/doc...ild_artifacts/sagemaker
tensorflow-serving.js
tensorflow-serving.js
44:52 
44:52 
view
8 x 2 tensorflow/inference/doc...ifacts/sagemaker_neuron
tensorflow/inference/doc...ild_artifacts/sagemaker
serve.py
serve.py
0:0 
0:0 
view