aws-samples / deep-learning-models
Duplication

Places in code with 6 or more lines that are exactly the same.

Intro
  • For duplication, we look at places in code where there are 6 or more lines of code that are exactly the same.
  • Before duplication is calculated, the code is cleaned to remove empty lines, comments, and frequently duplicated constructs such as imports.
  • You should aim at having as little as possible (<5%) of duplicated code as high-level of duplication can lead to maintenance difficulties, poor factoring, and logical contradictions.
Learn more...
Duplication Overall
  • 47% duplication:
    • 23,415 cleaned lines of cleaned code (without empty lines, comments, and frequently duplicated constructs such as imports)
    • 11,078 duplicated lines
  • 5,972 duplicates
system47% (11,078 lines)
Duplication per Extension
py48% (10,864 lines)
yml34% (214 lines)
Duplication per Component (primary)
models/vision/detection56% (8,461 lines)
models/vision/classification76% (1,691 lines)
ci/frcnn26% (256 lines)
legacy/hpc-cluster34% (214 lines)
ci/albert51% (150 lines)
models/nlp/albert11% (125 lines)
legacy/models/resnet3% (50 lines)
models/nlp/electra11% (48 lines)
models/nlp/t534% (44 lines)
models/nlp/common5% (39 lines)
models/nlp/cloudformation0% (0 lines)
legacy/hpc-cluster/monitor0% (0 lines)
legacy/utils/tensorflow0% (0 lines)
ROOT0% (0 lines)

Duplication Between Components (50+ lines)

G models/vision/classification models/vision/classification models/vision/detection models/vision/detection models/vision/classification--models/vision/detection 2644 ci/frcnn ci/frcnn ci/frcnn--models/vision/detection 766 ci/albert ci/albert ci/albert--ci/frcnn 243 models/nlp/common models/nlp/common ci/albert--models/nlp/common 78 models/nlp/albert models/nlp/albert models/nlp/electra models/nlp/electra models/nlp/albert--models/nlp/electra 84

Download: SVG DOT (open online Graphviz editor)

Open 3D force graph...

Show more details on duplication between components...
Longest Duplicates
The list of 20 longest duplicates.
See data for all 5,972 duplicates...
Size#FoldersFilesLinesCode
424 x 2 models/vision/classification/models
models/vision/detection/awsdet/models/backbones
hrnet.py
hrnet.py
 21:499 (58%)
72:549 (54%)
 view
238 x 2 models/vision/classification/models
models/vision/detection/awsdet/models/backbones
hrnet.py
hrnet.py
 576:824 (33%)
618:866 (30%)
 view
142 x 2 models/vision/detection/configs/mask_rcnn/SM/1
models/vision/detection/configs/mask_rcnn/SM/2
mask_rcnn_r50v1_d_fpn_1x_coco.py
mask_rcnn_r50v1_d_fpn_1x_coco.py
 2:152 (100%)
2:152 (100%)
 view
136 x 2 models/vision/detection/configs/cascade_rcnn/SM/1
models/vision/detection/configs/cascade_rcnn/SM/2
cascade_rcnn_r50v1_d_fpn_1x_coco.py
cascade_rcnn_r50v1_d_fpn_1x_coco.py
 1:148 (100%)
1:148 (100%)
 view
122 x 2 models/vision/detection/configs/mask_rcnn/SM/4
models/vision/detection/configs/mask_rcnn/SM/8
mask_rcnn_r50v1_d_fpn_1x_coco.py
mask_rcnn_r50v1_d_fpn_1x_coco.py
 2:131 (75%)
2:131 (75%)
 view
119 x 2 models/vision/detection/configs/mask_rcnn/SM/1
models/vision/detection/configs/mask_rcnn/SM/8
mask_rcnn_r50v1_d_fpn_1x_coco.py
mask_rcnn_r50v1_d_fpn_1x_coco.py
 2:126 (83%)
2:126 (73%)
 view
119 x 2 models/vision/detection/configs/mask_rcnn/SM/2
models/vision/detection/configs/mask_rcnn/SM/8
mask_rcnn_r50v1_d_fpn_1x_coco.py
mask_rcnn_r50v1_d_fpn_1x_coco.py
 2:126 (83%)
2:126 (73%)
 view
119 x 2 models/vision/detection/configs/mask_rcnn/SM/1
models/vision/detection/configs/mask_rcnn/SM/4
mask_rcnn_r50v1_d_fpn_1x_coco.py
mask_rcnn_r50v1_d_fpn_1x_coco.py
 2:126 (83%)
2:126 (73%)
 view
119 x 2 models/vision/detection/configs/mask_rcnn/SM/2
models/vision/detection/configs/mask_rcnn/SM/4
mask_rcnn_r50v1_d_fpn_1x_coco.py
mask_rcnn_r50v1_d_fpn_1x_coco.py
 2:126 (83%)
2:126 (73%)
 view
111 x 2 models/vision/detection/configs/cascade_rcnn/SM/1
models/vision/detection/configs/cascade_rcnn/SM/4
cascade_rcnn_r50v1_d_fpn_1x_coco.py
cascade_rcnn_r50v1_d_fpn_1x_coco.py
 1:120 (81%)
1:119 (76%)
 view
111 x 2 models/vision/detection/configs/cascade_rcnn/SM/2
models/vision/detection/configs/cascade_rcnn/SM/4
cascade_rcnn_r50v1_d_fpn_1x_coco.py
cascade_rcnn_r50v1_d_fpn_1x_coco.py
 1:120 (81%)
1:119 (76%)
 view
106 x 2 models/vision/detection/configs/cascade_rcnn/SM/1
models/vision/detection/configs/cascade_rcnn/SM/8
cascade_rcnn_r50v1_d_fpn_1x_coco.py
cascade_rcnn_r50v1_d_fpn_1x_coco.py
 1:113 (77%)
1:113 (72%)
 view
106 x 2 models/vision/detection/configs/cascade_rcnn/SM/2
models/vision/detection/configs/cascade_rcnn/SM/8
cascade_rcnn_r50v1_d_fpn_1x_coco.py
cascade_rcnn_r50v1_d_fpn_1x_coco.py
 1:113 (77%)
1:113 (72%)
 view
106 x 2 models/vision/detection/configs/cascade_rcnn/SM/4
models/vision/detection/configs/cascade_rcnn/SM/8
cascade_rcnn_r50v1_d_fpn_1x_coco.py
cascade_rcnn_r50v1_d_fpn_1x_coco.py
 1:112 (72%)
1:113 (72%)
 view
104 x 2 models/vision/detection/configs/mask_rcnn/EC2/1
models/vision/detection/configs/mask_rcnn/SM/8
mask_rcnn_r50v1_d_fpn_1x_coco.py
mask_rcnn_r50v1_d_fpn_1x_coco.py
 12:118 (91%)
12:118 (64%)
 view
104 x 2 models/vision/detection/configs/mask_rcnn/EC2/1
models/vision/detection/configs/mask_rcnn/SM/4
mask_rcnn_r50v1_d_fpn_1x_coco.py
mask_rcnn_r50v1_d_fpn_1x_coco.py
 12:118 (91%)
12:118 (64%)
 view
104 x 2 models/vision/detection/configs/mask_rcnn/EC2/1
models/vision/detection/configs/mask_rcnn/SM/1
mask_rcnn_r50v1_d_fpn_1x_coco.py
mask_rcnn_r50v1_d_fpn_1x_coco.py
 12:118 (91%)
12:118 (73%)
 view
104 x 2 models/vision/detection/configs/mask_rcnn/EC2/1
models/vision/detection/configs/mask_rcnn/SM/2
mask_rcnn_r50v1_d_fpn_1x_coco.py
mask_rcnn_r50v1_d_fpn_1x_coco.py
 12:118 (91%)
12:118 (73%)
 view
95 x 2 models/vision/detection/configs/retinanet/SM/2
models/vision/detection/configs/retinanet/SM/8
retinanet_r50_fpn_1x_coco.py
retinanet_r50_fpn_1x_coco.py
 2:111 (100%)
2:111 (100%)
 view
95 x 2 models/vision/detection/configs/retinanet/SM/1
models/vision/detection/configs/retinanet/SM/8
retinanet_r50_fpn_1x_coco.py
retinanet_r50_fpn_1x_coco.py
 2:111 (100%)
2:111 (100%)
 view
Duplicated Units
The list of top 20 duplicated units.
See data for all 45 unit duplicates...
Size#FoldersFilesLinesCode
51 x 2 models/vision/detection/awsdet/models/backbones
models/vision/classification/models
hrnet.py
hrnet.py
 0:0 
0:0 
 view
45 x 2 models/vision/detection/awsdet/models/backbones
models/vision/classification/models
hrnet.py
hrnet.py
 0:0 
0:0 
 view
42 x 2 models/vision/detection/awsdet/models/backbones
models/vision/classification/models
hrnet.py
hrnet.py
 0:0 
0:0 
 view
37 x 2 models/vision/detection/awsdet/models/backbones
models/vision/classification/models
hrnet.py
hrnet.py
 0:0 
0:0 
 view
36 x 2 models/vision/detection/awsdet/models/backbones
models/vision/classification/models/layers
hrnet.py
conv_module.py
 0:0 
0:0 
 view
36 x 2 models/vision/detection/awsdet/models/backbones
models/vision/classification/models
hrnet.py
hrnet.py
 0:0 
0:0 
 view
49 x 2 models/vision/classification/models
models/vision/classification/models
resnet.py
resnet_evo.py
 0:0 
0:0 
 view
36 x 2 models/vision/detection/awsdet/models/backbones
models/vision/classification/models
hrnet.py
hrnet.py
 0:0 
0:0 
 view
28 x 2 models/vision/detection/awsdet/models/backbones
models/vision/classification/models
hrnet.py
hrnet.py
 0:0 
0:0 
 view
25 x 2 models/vision/detection/awsdet/models/backbones
models/vision/classification/models
hrnet.py
hrnet.py
 0:0 
0:0 
 view
17 x 2 models/vision/detection/awsdet/models/backbones
models/vision/classification/models
resnet_common.py
resnet_evo.py
 0:0 
0:0 
 view
17 x 2 models/vision/detection/awsdet/models/backbones
models/vision/classification/models
resnet_common.py
resnet_evo.py
 0:0 
0:0 
 view
17 x 2 models/vision/detection/awsdet/models/backbones
models/vision/classification/models
resnet_common.py
resnet_evo.py
 0:0 
0:0 
 view
17 x 2 models/vision/detection/awsdet/models/backbones
models/vision/classification/models
resnet_common.py
resnet_evo.py
 0:0 
0:0 
 view
17 x 2 models/vision/detection/awsdet/models/backbones
models/vision/classification/models
resnet_common.py
resnet_evo.py
 0:0 
0:0 
 view
16 x 2 models/vision/detection/awsdet/utils/keras
models/vision/classification
optimizers.py
optimizers.py
 0:0 
0:0 
 view
16 x 2 models/vision/detection/awsdet/models/backbones
models/vision/classification/models
hrnet.py
hrnet.py
 0:0 
0:0 
 view
34 x 2 models/vision/classification/preprocessing
models/vision/classification/preprocessing
darknet_preprocessing.py
resnet_preprocessing.py
 0:0 
0:0 
 view
15 x 2 models/vision/detection/awsdet/utils/keras
models/vision/classification
optimizers.py
optimizers.py
 0:0 
0:0 
 view
14 x 2 models/vision/detection/awsdet/models/backbones
models/vision/classification/models
hrnet.py
hrnet.py
 0:0 
0:0 
 view