Places in code with 6 or more lines that are exactly the same.
Intro
- For duplication, we look at places in code where there are 6 or more lines of code that are exactly the same.
- Before duplication is calculated, the code is cleaned to remove empty lines, comments, and frequently duplicated constructs such as imports.
- You should aim at having as little as possible (<5%) of duplicated code as high-level of duplication can lead to maintenance difficulties, poor factoring, and logical contradictions.
- To learn more about duplications and techniques for eliminating duplication, Sokrates recommends the following resources:
- Avoid Repetition, MartinFlower.com (IEEE Software article)
- Refactoring duplicated code, sourcemaking.com
- Beck Design Rules, MartinFowler.com
- DRY (Don't Repeat Yourself) Principle, Wikipedia
- The Cut-and-Paste Programming Software Development Anti-Pattern, sourcemaking.com
- Code Smells / Duplicated Code, Jeff Atwood, codinghorror.com:
- "Duplicated code is the bane of software development. Stamp out duplication whenever possible. You should always be on the lookout for more subtle cases of near-duplication, too. Don't Repeat Yourself!"
- Reducing Coupling, MartinFlower.com (IEEE Software article):
- "There are several ways to describe coupling, but it boils down to this: If changing one module in a program requires changing another module, then coupling exists. ... Duplication always implies coupling, because changing one piece of duplicate code implies changing the other."
Duplication Overall
- 20% duplication:
- 27,803 cleaned lines of cleaned code (without empty lines, comments, and frequently duplicated constructs such as imports)
- 5,645 duplicated lines
- 978 duplicates
Duplication per Extension
Duplication per Component (primary)
Duplication Between Components (50+ lines)
Open 3D force graph...
Show more details on duplication between components...
From Component
--> To Component
Duplicated
Lines
File Pairs
Details
src/sagemaker/mxnet (65%)
--> src/sagemaker/pytorch (70%)
380
5 file pairs
details...
src/sagemaker (1%)
--> src/sagemaker/tensorflow (35%)
348
10 file pairs
details...
src/sagemaker/huggingface (36%)
--> src/sagemaker/pytorch (52%)
284
3 file pairs
details...
src/sagemaker/huggingface (36%)
--> src/sagemaker/mxnet (48%)
280
3 file pairs
details...
src/sagemaker/model_monitor (4%)
--> src/sagemaker/workflow (4%)
258
3 file pairs
details...
src/sagemaker (1%)
--> src/sagemaker/workflow (4%)
254
12 file pairs
details...
src/sagemaker/pytorch (36%)
--> src/sagemaker/tensorflow (20%)
200
3 file pairs
details...
src/sagemaker/mxnet (34%)
--> src/sagemaker/tensorflow (20%)
200
3 file pairs
details...
src/sagemaker/model_monitor (3%)
--> src/sagemaker (<1%)
195
2 file pairs
details...
src/sagemaker (<1%)
--> src/sagemaker/spark (13%)
181
1 file pair
details...
src/sagemaker/chainer (46%)
--> src/sagemaker/pytorch (31%)
172
4 file pairs
details...
src/sagemaker/huggingface (21%)
--> src/sagemaker/tensorflow (16%)
164
3 file pairs
details...
src/sagemaker/huggingface (20%)
--> src/sagemaker (<1%)
159
4 file pairs
details...
src/sagemaker/chainer (36%)
--> src/sagemaker/mxnet (23%)
136
4 file pairs
details...
src/sagemaker (<1%)
--> src/sagemaker/pytorch (24%)
135
4 file pairs
details...
src/sagemaker/mxnet (23%)
--> src/sagemaker (<1%)
135
4 file pairs
details...
src/sagemaker/model_monitor (2%)
--> src/sagemaker/spark (9%)
130
2 file pairs
details...
src/sagemaker/pytorch (23%)
--> src/sagemaker/sklearn (27%)
126
3 file pairs
details...
src/sagemaker/pytorch (21%)
--> src/sagemaker/xgboost (25%)
114
2 file pairs
details...
src/sagemaker/mxnet (18%)
--> src/sagemaker/xgboost (24%)
110
2 file pairs
details...
src/sagemaker/chainer (29%)
--> src/sagemaker/sklearn (23%)
108
2 file pairs
details...
src/sagemaker/mxnet (18%)
--> src/sagemaker/sklearn (23%)
106
3 file pairs
details...
src/sagemaker/tensorflow (9%)
--> src/sagemaker/xgboost (21%)
94
2 file pairs
details...
src/sagemaker/sklearn (20%)
--> src/sagemaker/xgboost (20%)
92
3 file pairs
details...
src/sagemaker (<1%)
--> src/sagemaker/xgboost (18%)
82
2 file pairs
details...
src/sagemaker/chainer (21%)
--> src/sagemaker/huggingface (10%)
80
2 file pairs
details...
src/sagemaker/model_monitor (1%)
--> src/sagemaker/sklearn (17%)
80
2 file pairs
details...
src/sagemaker/huggingface (9%)
--> src/sagemaker/xgboost (17%)
76
2 file pairs
details...
src/sagemaker/huggingface (8%)
--> src/sagemaker/sklearn (14%)
68
3 file pairs
details...
src/sagemaker/sklearn (14%)
--> src/sagemaker/spark (5%)
66
1 file pair
details...
src/sagemaker/tensorflow (5%)
--> src/sagemaker/workflow (<1%)
50
3 file pairs
details...
| From Component --> To Component |
Duplicated Lines |
File Pairs | Details |
|---|---|---|---|
| src/sagemaker/mxnet (65%) --> src/sagemaker/pytorch (70%) |
380 | 5 file pairs | details... |
| src/sagemaker (1%) --> src/sagemaker/tensorflow (35%) |
348 | 10 file pairs | details... |
| src/sagemaker/huggingface (36%) --> src/sagemaker/pytorch (52%) |
284 | 3 file pairs | details... |
| src/sagemaker/huggingface (36%) --> src/sagemaker/mxnet (48%) |
280 | 3 file pairs | details... |
| src/sagemaker/model_monitor (4%) --> src/sagemaker/workflow (4%) |
258 | 3 file pairs | details... |
| src/sagemaker (1%) --> src/sagemaker/workflow (4%) |
254 | 12 file pairs | details... |
| src/sagemaker/pytorch (36%) --> src/sagemaker/tensorflow (20%) |
200 | 3 file pairs | details... |
| src/sagemaker/mxnet (34%) --> src/sagemaker/tensorflow (20%) |
200 | 3 file pairs | details... |
| src/sagemaker/model_monitor (3%) --> src/sagemaker (<1%) |
195 | 2 file pairs | details... |
| src/sagemaker (<1%) --> src/sagemaker/spark (13%) |
181 | 1 file pair | details... |
| src/sagemaker/chainer (46%) --> src/sagemaker/pytorch (31%) |
172 | 4 file pairs | details... |
| src/sagemaker/huggingface (21%) --> src/sagemaker/tensorflow (16%) |
164 | 3 file pairs | details... |
| src/sagemaker/huggingface (20%) --> src/sagemaker (<1%) |
159 | 4 file pairs | details... |
| src/sagemaker/chainer (36%) --> src/sagemaker/mxnet (23%) |
136 | 4 file pairs | details... |
| src/sagemaker (<1%) --> src/sagemaker/pytorch (24%) |
135 | 4 file pairs | details... |
| src/sagemaker/mxnet (23%) --> src/sagemaker (<1%) |
135 | 4 file pairs | details... |
| src/sagemaker/model_monitor (2%) --> src/sagemaker/spark (9%) |
130 | 2 file pairs | details... |
| src/sagemaker/pytorch (23%) --> src/sagemaker/sklearn (27%) |
126 | 3 file pairs | details... |
| src/sagemaker/pytorch (21%) --> src/sagemaker/xgboost (25%) |
114 | 2 file pairs | details... |
| src/sagemaker/mxnet (18%) --> src/sagemaker/xgboost (24%) |
110 | 2 file pairs | details... |
| src/sagemaker/chainer (29%) --> src/sagemaker/sklearn (23%) |
108 | 2 file pairs | details... |
| src/sagemaker/mxnet (18%) --> src/sagemaker/sklearn (23%) |
106 | 3 file pairs | details... |
| src/sagemaker/tensorflow (9%) --> src/sagemaker/xgboost (21%) |
94 | 2 file pairs | details... |
| src/sagemaker/sklearn (20%) --> src/sagemaker/xgboost (20%) |
92 | 3 file pairs | details... |
| src/sagemaker (<1%) --> src/sagemaker/xgboost (18%) |
82 | 2 file pairs | details... |
| src/sagemaker/chainer (21%) --> src/sagemaker/huggingface (10%) |
80 | 2 file pairs | details... |
| src/sagemaker/model_monitor (1%) --> src/sagemaker/sklearn (17%) |
80 | 2 file pairs | details... |
| src/sagemaker/huggingface (9%) --> src/sagemaker/xgboost (17%) |
76 | 2 file pairs | details... |
| src/sagemaker/huggingface (8%) --> src/sagemaker/sklearn (14%) |
68 | 3 file pairs | details... |
| src/sagemaker/sklearn (14%) --> src/sagemaker/spark (5%) |
66 | 1 file pair | details... |
| src/sagemaker/tensorflow (5%) --> src/sagemaker/workflow (<1%) |
50 | 3 file pairs | details... |
Longest Duplicates
The list of 20 longest duplicates.
| Size | # | Folders | Files | Lines | Code |
|---|---|---|---|---|---|
| 40 | x 2 | src/sagemaker/pytorchsrc/sagemaker/xgboost |  processing.pyprocessing.py |
29:81 (95%)29:81 (95%) | view |
| 40 | x 2 | src/sagemaker/mxnetsrc/sagemaker/xgboost | processing.pyprocessing.py |
29:81 (95%)29:81 (95%) | view |
| 40 | x 2 | src/sagemaker/pytorchsrc/sagemaker/tensorflow | processing.pyprocessing.py |
29:81 (95%)29:81 (95%) | view |
| 40 | x 2 | src/sagemaker/tensorflowsrc/sagemaker/xgboost | processing.pyprocessing.py |
29:81 (95%)29:81 (95%) | view |
| 40 | x 2 | src/sagemaker/mxnetsrc/sagemaker/pytorch | processing.pyprocessing.py |
29:81 (95%)29:81 (95%) | view |
| 40 | x 2 | src/sagemaker/mxnetsrc/sagemaker/tensorflow | processing.pyprocessing.py |
29:81 (95%)29:81 (95%) | view |
| 38 | x 2 | src/sagemaker/model_monitorsrc/sagemaker/model_monitor | model_monitoring.pymodel_monitoring.py |
2271:2318 (1%)2974:3021 (1%) | view |
| 36 | x 2 | src/sagemaker/model_monitorsrc/sagemaker/model_monitor | clarify_model_monitoring.pymodel_monitoring.py |
217:298 (4%)2842:2920 (1%) | view |
| 35 | x 2 | src/sagemaker/model_monitorsrc/sagemaker/model_monitor | clarify_model_monitoring.pymodel_monitoring.py |
218:298 (4%)2142:2216 (1%) | view |
| 35 | x 2 | src/sagemaker/model_monitorsrc/sagemaker/model_monitor | model_monitoring.pymodel_monitoring.py |
2142:2216 (1%)2843:2920 (1%) | view |
| 31 | x 2 | src/sagemaker/model_monitorsrc/sagemaker/model_monitor | clarify_model_monitoring.pymodel_monitoring.py |
373:411 (3%)2983:3021 (1%) | view |
| 31 | x 2 | src/sagemaker/model_monitorsrc/sagemaker/model_monitor | clarify_model_monitoring.pymodel_monitoring.py |
373:411 (3%)2280:2318 (1%) | view |
| 29 | x 2 | src/sagemakersrc/sagemaker | session.pysession.py |
441:469 (1%)593:621 (1%) | view |
| 28 | x 2 | src/sagemaker/huggingfacesrc/sagemaker/pytorch | model.pymodel.py |
170:230 (20%)140:198 (23%) | view |
| 28 | x 2 | src/sagemaker/huggingfacesrc/sagemaker/mxnet | model.pymodel.py |
170:230 (20%)142:199 (23%) | view |
| 28 | x 2 | src/sagemaker/mxnetsrc/sagemaker/pytorch | model.pymodel.py |
142:199 (23%)140:198 (23%) | view |
| 26 | x 2 | src/sagemaker/huggingfacesrc/sagemaker/pytorch | model.pymodel.py |
232:273 (18%)200:241 (21%) | view |
| 26 | x 2 | src/sagemaker/huggingfacesrc/sagemaker/mxnet | model.pymodel.py |
232:273 (18%)201:244 (21%) | view |
| 26 | x 2 | src/sagemaker/mxnetsrc/sagemaker/pytorch | model.pymodel.py |
201:244 (21%)200:241 (21%) | view |
| 26 | x 2 | src/sagemaker/model_monitorsrc/sagemaker/model_monitor | clarify_model_monitoring.pyclarify_model_monitoring.py |
506:555 (3%)900:948 (3%) | view |
Duplicated Units
The list of top 20 duplicated units.
| Size | # | Folders | Files | Lines | Code |
|---|---|---|---|---|---|
| 28 | x 2 | src/sagemakersrc/sagemaker | session.pysession.py |
0:0 0:0 | view |
| 18 | x 4 | src/sagemaker/pytorchsrc/sagemaker/xgboostsrc/sagemaker/mxnetsrc/sagemaker/tensorflow | processing.pyprocessing.pyprocessing.pyprocessing.py |
0:0 0:0 0:0 0:0 | view |
| 17 | x 2 | src/sagemaker/sparksrc/sagemaker/spark | processing.pyprocessing.py |
0:0 0:0 | view |
| 14 | x 5 | src/sagemaker/pytorchsrc/sagemaker/mxnetsrc/sagemakersrc/sagemaker/huggingfacesrc/sagemaker/tensorflow | model.pymodel.pymodel.pymodel.pymodel.py |
0:0 0:0 0:0 0:0 0:0 | view |
| 14 | x 2 | src/sagemakersrc/sagemaker | session.pysession.py |
0:0 0:0 | view |
| 13 | x 4 | src/sagemaker/model_monitorsrc/sagemaker/model_monitorsrc/sagemaker/model_monitorsrc/sagemaker/workflow | model_monitoring.pymodel_monitoring.pyclarify_model_monitoring.pycheck_job_config.py |
0:0 0:0 0:0 0:0 | view |
| 13 | x 2 | src/sagemakersrc/sagemaker | model.pypipeline.py |
0:0 0:0 | view |
| 12 | x 2 | src/sagemakersrc/sagemaker | session.pysession.py |
0:0 0:0 | view |
| 12 | x 2 | src/sagemakersrc/sagemaker | transformer.pytransformer.py |
0:0 0:0 | view |
| 9 | x 2 | src/sagemaker/pytorchsrc/sagemaker/mxnet | estimator.pyestimator.py |
0:0 0:0 | view |
| 9 | x 2 | src/sagemakersrc/sagemaker | predictor.pypredictor_async.py |
0:0 0:0 | view |
| 8 | x 4 | src/sagemaker/pytorchsrc/sagemaker/sklearnsrc/sagemaker/xgboostsrc/sagemaker/chainer | estimator.pyestimator.pyestimator.pyestimator.py |
0:0 0:0 0:0 0:0 | view |
| 8 | x 2 | src/sagemakersrc/sagemaker | session.pysession.py |
0:0 0:0 | view |
| 7 | x 2 | src/sagemakersrc/sagemaker | session.pysession.py |
0:0 0:0 | view |
| 7 | x 2 | src/sagemaker/rlsrc/sagemaker/tensorflow | estimator.pyestimator.py |
0:0 0:0 | view |
| 7 | x 2 | src/sagemaker/wranglersrc/sagemaker | processing.pyprocessing.py |
0:0 0:0 | view |
| 6 | x 2 | src/sagemakersrc/sagemaker | predictor.pypredictor.py |
0:0 0:0 | view |
| 6 | x 2 | src/sagemaker/debuggersrc/sagemaker/debugger | metrics_config.pymetrics_config.py |
0:0 0:0 | view |
| 7 | x 2 | src/sagemaker/model_monitorsrc/sagemaker/model_monitor | model_monitoring.pymodel_monitoring.py |
0:0 0:0 | view |
| 6 | x 2 | src/sagemaker/model_monitorsrc/sagemaker/model_monitor | clarify_model_monitoring.pyclarify_model_monitoring.py |
0:0 0:0 | view |