aws-samples / amazon-mwaa-examples
Duplication

Places in code with 6 or more lines that are exactly the same.

Intro
  • For duplication, we look at places in code where there are 6 or more lines of code that are exactly the same.
  • Before duplication is calculated, the code is cleaned to remove empty lines, comments, and frequently duplicated constructs such as imports.
  • You should aim at having as little as possible (<5%) of duplicated code as high-level of duplication can lead to maintenance difficulties, poor factoring, and logical contradictions.
Learn more...
Duplication Overall
  • 50% duplication:
    • 1,828 cleaned lines of cleaned code (without empty lines, comments, and frequently duplicated constructs such as imports)
    • 927 duplicated lines
  • 19 duplicates
system50% (927 lines)
Duplication per Extension
py66% (927 lines)
Duplication per Component (primary)
usecases/image-processing/dags100% (488 lines)
dags/xgboost-ml-pipeline/1.1099% (128 lines)
dags/xgboost-ml-pipeline/2.099% (128 lines)
dags/emr_job/1.10100% (77 lines)
dags/emr_job/2.0100% (77 lines)
usecases/mwaa-with-codeartifact/infra7% (23 lines)
dags/metadb_to_secrets_manager/1.1010% (6 lines)
dags/xgboost-ml-pipeline/glue0% (0 lines)
dags/bash_operator_script/1.100% (0 lines)
usecases/image-processing0% (0 lines)
usecases/image-processing/lambda0% (0 lines)
usecases/mwaa-with-codeartifact0% (0 lines)
usecases/mwaa-with-codeartifact/mwaa-ca-bucket-content0% (0 lines)
usecases/mwaa-with-codeartifact/lambda0% (0 lines)

Duplication Between Components (50+ lines)

G dags/xgboost-ml-pipeline/1.10 dags/xgboost-ml-pipeline/1.10 dags/xgboost-ml-pipeline/2.0 dags/xgboost-ml-pipeline/2.0 dags/xgboost-ml-pipeline/1.10--dags/xgboost-ml-pipeline/2.0 256 dags/emr_job/1.10 dags/emr_job/1.10 dags/emr_job/2.0 dags/emr_job/2.0 dags/emr_job/1.10--dags/emr_job/2.0 154

Download: SVG DOT (open online Graphviz editor)

Open 3D force graph...

Show more details on duplication between components...
Longest Duplicates
The list of 19 longest duplicates.
See data for all 19 duplicates...
Size#FoldersFilesLinesCode
164 x 2 usecases/image-processing/dags/1.10
usecases/image-processing/dags
image_processing.py
image_processing.py
 34:225 (100%)
34:225 (100%)
 view
109 x 2 usecases/image-processing/dags/2.0
usecases/image-processing/dags
image_processing.py
image_processing.py
 34:163 (68%)
34:163 (66%)
 view
109 x 2 usecases/image-processing/dags/1.10
usecases/image-processing/dags/2.0
image_processing.py
image_processing.py
 34:163 (66%)
34:163 (68%)
 view
77 x 2 dags/emr_job/1.10
dags/emr_job/2.0
emr_job.py
emr_job.py
 30:114 (100%)
28:112 (100%)
 view
74 x 2 dags/xgboost-ml-pipeline/1.10
dags/xgboost-ml-pipeline/2.0
mwaa-customer-churn-dag.py
mwaa-customer-churn-dag.py
 99:196 (65%)
99:196 (65%)
 view
38 x 2 dags/xgboost-ml-pipeline/1.10
dags/xgboost-ml-pipeline/2.0
mwaa-customer-churn-dag.py
mwaa-customer-churn-dag.py
 37:91 (33%)
37:91 (33%)
 view
20 x 2 usecases/image-processing/dags/2.0
usecases/image-processing/dags
image_processing.py
image_processing.py
 177:197 (12%)
179:199 (12%)
 view
20 x 2 usecases/image-processing/dags/1.10
usecases/image-processing/dags/2.0
image_processing.py
image_processing.py
 179:199 (12%)
177:197 (12%)
 view
16 x 2 dags/xgboost-ml-pipeline/1.10
dags/xgboost-ml-pipeline/2.0
config.py
config.py
 3:24 (100%)
3:24 (100%)
 view
14 x 2 usecases/image-processing/dags/1.10
usecases/image-processing/dags/2.0
image_processing.py
image_processing.py
 208:225 (8%)
204:221 (8%)
 view
14 x 2 usecases/image-processing/dags/2.0
usecases/image-processing/dags
image_processing.py
image_processing.py
 204:221 (8%)
208:225 (8%)
 view
11 x 2 usecases/image-processing/dags/1.10
usecases/image-processing/dags/2.0
image_processing.py
image_processing.py
 165:177 (6%)
164:176 (6%)
 view
11 x 2 usecases/image-processing/dags/2.0
usecases/image-processing/dags
image_processing.py
image_processing.py
 164:176 (6%)
165:177 (6%)
 view
8 x 2 usecases/mwaa-with-codeartifact/infra
usecases/mwaa-with-codeartifact/infra
mwaa_stack.py
mwaa_stack.py
 33:40 (6%)
43:50 (6%)
 view
6 x 2 dags/emr_job/2.0
dags/metadb_to_secrets_manager/1.10
emr_job.py
metadb_to_secrets_manager.py
 30:35 (7%)
28:33 (10%)
 view
6 x 2 usecases/mwaa-with-codeartifact/infra
usecases/mwaa-with-codeartifact/infra
mwaa_stack.py
mwaa_stack.py
 32:37 (4%)
52:57 (4%)
 view
6 x 2 usecases/image-processing/dags/1.10
usecases/image-processing/dags/2.0
image_processing.py
image_processing.py
 201:206 (3%)
198:203 (3%)
 view
6 x 2 usecases/image-processing/dags/2.0
usecases/image-processing/dags
image_processing.py
image_processing.py
 198:203 (3%)
201:206 (3%)
 view
6 x 2 dags/emr_job/1.10
dags/metadb_to_secrets_manager/1.10
emr_job.py
metadb_to_secrets_manager.py
 32:37 (7%)
28:33 (10%)
 view
Duplicated Units
The list of top 6 duplicated units.
See data for all 6 unit duplicates...
Size#FoldersFilesLinesCode
35 x 2 dags/xgboost-ml-pipeline/1.10
dags/xgboost-ml-pipeline/2.0
mwaa-customer-churn-dag.py
mwaa-customer-churn-dag.py
 0:0 
0:0 
 view
24 x 3 usecases/image-processing/dags/1.10
usecases/image-processing/dags/2.0
usecases/image-processing/dags
image_processing.py
image_processing.py
image_processing.py
 0:0 
0:0 
0:0 
 view
22 x 3 usecases/image-processing/dags/1.10
usecases/image-processing/dags/2.0
usecases/image-processing/dags
image_processing.py
image_processing.py
image_processing.py
 0:0 
0:0 
0:0 
 view
15 x 3 usecases/image-processing/dags/1.10
usecases/image-processing/dags/2.0
usecases/image-processing/dags
image_processing.py
image_processing.py
image_processing.py
 0:0 
0:0 
0:0 
 view
14 x 3 usecases/image-processing/dags/1.10
usecases/image-processing/dags/2.0
usecases/image-processing/dags
image_processing.py
image_processing.py
image_processing.py
 0:0 
0:0 
0:0 
 view
7 x 3 usecases/image-processing/dags/1.10
usecases/image-processing/dags/2.0
usecases/image-processing/dags
image_processing.py
image_processing.py
image_processing.py
 0:0 
0:0 
0:0 
 view