Places in code with 6 or more lines that are exactly the same.
Intro
- For duplication, we look at places in code where there are 6 or more lines of code that are exactly the same.
- Before duplication is calculated, the code is cleaned to remove empty lines, comments, and frequently duplicated constructs such as imports.
- You should aim at having as little as possible (<5%) of duplicated code as high-level of duplication can lead to maintenance difficulties, poor factoring, and logical contradictions.
- To learn more about duplications and techniques for eliminating duplication, Sokrates recommends the following resources:
- Avoid Repetition, MartinFlower.com (IEEE Software article)
- Refactoring duplicated code, sourcemaking.com
- Beck Design Rules, MartinFowler.com
- DRY (Don't Repeat Yourself) Principle, Wikipedia
- The Cut-and-Paste Programming Software Development Anti-Pattern, sourcemaking.com
- Code Smells / Duplicated Code, Jeff Atwood, codinghorror.com:
- "Duplicated code is the bane of software development. Stamp out duplication whenever possible. You should always be on the lookout for more subtle cases of near-duplication, too. Don't Repeat Yourself!"
- Reducing Coupling, MartinFlower.com (IEEE Software article):
- "There are several ways to describe coupling, but it boils down to this: If changing one module in a program requires changing another module, then coupling exists. ... Duplication always implies coupling, because changing one piece of duplicate code implies changing the other."
Duplication Overall
- 77% duplication:
- 26,775 cleaned lines of cleaned code (without empty lines, comments, and frequently duplicated constructs such as imports)
- 20,721 duplicated lines
- 917 duplicates
Duplication per Extension
Duplication per Component (primary)
Duplication Between Components (50+ lines)
Open 3D force graph...
Show more details on duplication between components...
From Component
--> To Component
Duplicated
Lines
File Pairs
Details
Advanced workshops/AI Driving Olympics 2019/challenge_train_DQN (106%)
--> Advanced workshops/AI Driving Olympics 2019/challenge_train_w_PPO (92%)
13219
80 file pairs
details...
Advanced workshops/AI Driving Olympics 2019/challenge_train_w_PPO (25%)
--> Advanced workshops/reInvent2019-400/src (72%)
3598
25 file pairs
details...
Advanced workshops/AI Driving Olympics 2019/challenge_train_DQN (24%)
--> Advanced workshops/reInvent2019-400/src (60%)
3004
25 file pairs
details...
Advanced workshops/reInvent2019-400/customize (71%)
--> Advanced workshops/reInvent2019-400/src (52%)
2631
13 file pairs
details...
Advanced workshops/AI Driving Olympics 2019/challenge_train_w_PPO (10%)
--> Advanced workshops/reInvent2019-400/customize (42%)
1576
18 file pairs
details...
Advanced workshops/AI Driving Olympics 2019/challenge_train_DQN (7%)
--> Advanced workshops/reInvent2019-400/customize (26%)
984
18 file pairs
details...
Advanced workshops/AI Driving Olympics 2019/challenge_eval_submit (56%)
--> Advanced workshops/reInvent2019-400 (68%)
617
4 file pairs
details...
Advanced workshops/AI Driving Olympics 2019/challenge_eval_submit (51%)
--> Advanced workshops/AI Driving Olympics 2019/challenge_train_w_PPO (3%)
556
5 file pairs
details...
Advanced workshops/AI Driving Olympics 2019/challenge_eval_submit (44%)
--> Advanced workshops/reInvent2019-400/common (100%)
480
3 file pairs
details...
Advanced workshops/AI Driving Olympics 2019/challenge_train_DQN (3%)
--> Advanced workshops/reInvent2019-400/common (94%)
455
3 file pairs
details...
Advanced workshops/AI Driving Olympics 2019/challenge_eval_submit (41%)
--> Advanced workshops/AI Driving Olympics 2019/challenge_train_DQN (3%)
455
3 file pairs
details...
Advanced workshops/AI Driving Olympics 2019/challenge_train_w_PPO (3%)
--> Advanced workshops/reInvent2019-400/common (94%)
455
3 file pairs
details...
Advanced workshops/AI Driving Olympics 2019/challenge_eval_submit (40%)
--> log-analysis (88%)
444
2 file pairs
details...
Advanced workshops/reInvent2019-400 (49%)
--> log-analysis (88%)
442
2 file pairs
details...
Advanced workshops/400 Level DeepRacer Workshop/static (6%)
--> Advanced workshops/400 Level DeepRacer Workshop/themes (5%)
415
6 file pairs
details...
Advanced workshops/400 Level DeepRacer Workshop/layouts (61%)
--> Advanced workshops/400 Level DeepRacer Workshop/themes (4%)
368
4 file pairs
details...
Advanced workshops/AI Driving Olympics 2019/challenge_train_w_PPO (2%)
--> Advanced workshops/reInvent2019-400 (32%)
292
4 file pairs
details...
| From Component --> To Component |
Duplicated Lines |
File Pairs | Details |
|---|---|---|---|
| Advanced workshops/AI Driving Olympics 2019/challenge_train_DQN (106%) --> Advanced workshops/AI Driving Olympics 2019/challenge_train_w_PPO (92%) |
13219 | 80 file pairs | details... |
| Advanced workshops/AI Driving Olympics 2019/challenge_train_w_PPO (25%) --> Advanced workshops/reInvent2019-400/src (72%) |
3598 | 25 file pairs | details... |
| Advanced workshops/AI Driving Olympics 2019/challenge_train_DQN (24%) --> Advanced workshops/reInvent2019-400/src (60%) |
3004 | 25 file pairs | details... |
| Advanced workshops/reInvent2019-400/customize (71%) --> Advanced workshops/reInvent2019-400/src (52%) |
2631 | 13 file pairs | details... |
| Advanced workshops/AI Driving Olympics 2019/challenge_train_w_PPO (10%) --> Advanced workshops/reInvent2019-400/customize (42%) |
1576 | 18 file pairs | details... |
| Advanced workshops/AI Driving Olympics 2019/challenge_train_DQN (7%) --> Advanced workshops/reInvent2019-400/customize (26%) |
984 | 18 file pairs | details... |
| Advanced workshops/AI Driving Olympics 2019/challenge_eval_submit (56%) --> Advanced workshops/reInvent2019-400 (68%) |
617 | 4 file pairs | details... |
| Advanced workshops/AI Driving Olympics 2019/challenge_eval_submit (51%) --> Advanced workshops/AI Driving Olympics 2019/challenge_train_w_PPO (3%) |
556 | 5 file pairs | details... |
| Advanced workshops/AI Driving Olympics 2019/challenge_eval_submit (44%) --> Advanced workshops/reInvent2019-400/common (100%) |
480 | 3 file pairs | details... |
| Advanced workshops/AI Driving Olympics 2019/challenge_train_DQN (3%) --> Advanced workshops/reInvent2019-400/common (94%) |
455 | 3 file pairs | details... |
| Advanced workshops/AI Driving Olympics 2019/challenge_eval_submit (41%) --> Advanced workshops/AI Driving Olympics 2019/challenge_train_DQN (3%) |
455 | 3 file pairs | details... |
| Advanced workshops/AI Driving Olympics 2019/challenge_train_w_PPO (3%) --> Advanced workshops/reInvent2019-400/common (94%) |
455 | 3 file pairs | details... |
| Advanced workshops/AI Driving Olympics 2019/challenge_eval_submit (40%) --> log-analysis (88%) |
444 | 2 file pairs | details... |
| Advanced workshops/reInvent2019-400 (49%) --> log-analysis (88%) |
442 | 2 file pairs | details... |
| Advanced workshops/400 Level DeepRacer Workshop/static (6%) --> Advanced workshops/400 Level DeepRacer Workshop/themes (5%) |
415 | 6 file pairs | details... |
| Advanced workshops/400 Level DeepRacer Workshop/layouts (61%) --> Advanced workshops/400 Level DeepRacer Workshop/themes (4%) |
368 | 4 file pairs | details... |
| Advanced workshops/AI Driving Olympics 2019/challenge_train_w_PPO (2%) --> Advanced workshops/reInvent2019-400 (32%) |
292 | 4 file pairs | details... |
Longest Duplicates
The list of 20 longest duplicates.
| Size | # | Folders | Files | Lines | Code |
|---|---|---|---|---|---|
| 1186 | x 2 | Advanced workshops/...trator/workflow/managerAdvanced workshops/...trator/workflow/manager |  experiment_manager.pyexperiment_manager.py |
14:1849 (100%)14:1849 (100%) | view |
| 755 | x 2 | Advanced workshops/reInvent2019-400/customizeAdvanced workshops/...src/markov/environments | deepracer_racetrack_env_lidar_5cars.pydeepracer_racetrack_env.py |
24:1242 (100%)24:1121 (100%) | view |
| 463 | x 2 | Advanced workshops/reInvent2019-400/customizeAdvanced workshops/...src/markov/environments | deepracer_racetrack_env_lidar_3cars.pydeepracer_racetrack_env.py |
610:1240 (61%)491:1121 (61%) | view |
| 463 | x 2 | Advanced workshops/reInvent2019-400/customizeAdvanced workshops/reInvent2019-400/customize | deepracer_racetrack_env_lidar_3cars.pydeepracer_racetrack_env_lidar_5cars.py |
610:1240 (61%)612:1242 (61%) | view |
| 410 | x 2 | Advanced workshops/...src/markov/environmentsAdvanced workshops/...src/markov/environments | deepracer_racetrack_env.pydeepracer_racetrack_env_cv2.py |
22:579 (100%)22:579 (100%) | view |
| 400 | x 2 | Advanced workshops/...src/markov/environmentsAdvanced workshops/...src/markov/environments | deepracer_racetrack_env.pydeepracer_racetrack_env_original.py |
21:559 (100%)21:559 (100%) | view |
| 360 | x 2 | Advanced workshops/...trator/workflow/managerAdvanced workshops/...trator/workflow/manager | join_manager.pyjoin_manager.py |
14:694 (100%)14:694 (100%) | view |
| 357 | x 2 | Advanced workshops/...gemaker_rl/orchestratorAdvanced workshops/...gemaker_rl/orchestrator | resource_manager.pyresource_manager.py |
19:599 (100%)19:599 (100%) | view |
| 327 | x 2 | Advanced workshops/...trator/workflow/managerAdvanced workshops/...trator/workflow/manager | model_manager.pymodel_manager.py |
24:542 (100%)24:542 (100%) | view |
| 272 | x 2 | Advanced workshops/...r_rl/orchestrator/utilsAdvanced workshops/...r_rl/orchestrator/utils | cloudwatch_logger.pycloudwatch_logger.py |
5:285 (100%)5:285 (100%) | view |
| 260 | x 2 | Advanced workshops/...src/markov/environmentsAdvanced workshops/...src/markov/environments | deepracer_racetrack_env.pydeepracer_racetrack_env.py |
236:559 (65%)256:579 (63%) | view |
| 260 | x 2 | Advanced workshops/...src/markov/environmentsAdvanced workshops/...src/markov/environments | deepracer_racetrack_env.pydeepracer_racetrack_env_original.py |
256:579 (63%)236:559 (65%) | view |
| 260 | x 2 | Advanced workshops/...src/markov/environmentsAdvanced workshops/...src/markov/environments | deepracer_racetrack_env_cv2.pydeepracer_racetrack_env_original.py |
256:579 (63%)236:559 (65%) | view |
| 260 | x 2 | Advanced workshops/...src/markov/environmentsAdvanced workshops/...src/markov/environments | deepracer_racetrack_env.pydeepracer_racetrack_env_cv2.py |
236:559 (65%)256:579 (63%) | view |
| 197 | x 2 | Advanced workshops/...ge_train_DQN/src/markovAdvanced workshops/..._train_w_PPO/src/markov | rollout_worker.pyrollout_worker.py |
39:310 (100%)39:310 (100%) | view |
| 185 | x 2 | Advanced workshops/reInvent2019-400/customizeAdvanced workshops/reInvent2019-400/customize | deepracer_racetrack_env_lidar_3cars.pydeepracer_racetrack_env_lidar_5cars.py |
262:523 (24%)261:523 (24%) | view |
| 185 | x 2 | Advanced workshops/reInvent2019-400/customizeAdvanced workshops/...src/markov/environments | deepracer_racetrack_env_lidar_3cars.pydeepracer_racetrack_env.py |
262:523 (24%)143:402 (24%) | view |
| 184 | x 2 | Advanced workshops/...challenge_train_DQN/srcAdvanced workshops/...allenge_train_w_PPO/src | training_worker.pytraining_worker.py |
28:265 (100%)28:265 (100%) | view |
| 184 | x 2 | Advanced workshops/...ge_train_DQN/src/markovAdvanced workshops/..._train_w_PPO/src/markov | s3_boto_data_store.pys3_boto_data_store.py |
16:257 (100%)16:257 (100%) | view |
| 181 | x 2 | Advanced workshops/...DQN/common/sagemaker_rlAdvanced workshops/...PPO/common/sagemaker_rl | coach_launcher.pycoach_launcher.py |
23:316 (100%)23:316 (100%) | view |
Duplicated Units
The list of top 20 duplicated units.
| Size | # | Folders | Files | Lines | Code |
|---|---|---|---|---|---|
| 275 | x 3 | Advanced workshops/...src/markov/environmentsAdvanced workshops/reInvent2019-400/customizeAdvanced workshops/reInvent2019-400/customize | deepracer_racetrack_env.pydeepracer_racetrack_env_lidar_5cars.pydeepracer_racetrack_env_lidar_3cars.py |
0:0 0:0 0:0 | view |
| 157 | x 2 | Advanced workshops/...r_rl/orchestrator/utilsAdvanced workshops/...r_rl/orchestrator/utils | cloudwatch_logger.pycloudwatch_logger.py |
0:0 0:0 | view |
| 129 | x 4 | Advanced workshops/...src/markov/environmentsAdvanced workshops/...src/markov/environmentsAdvanced workshops/...src/markov/environmentsAdvanced workshops/...src/markov/environments | deepracer_racetrack_env.pydeepracer_racetrack_env_cv2.pydeepracer_racetrack_env.pydeepracer_racetrack_env_original.py |
0:0 0:0 0:0 0:0 | view |
| 107 | x 3 | Advanced workshops/reInvent2019-400/srcAdvanced workshops/...challenge_train_DQN/srcAdvanced workshops/...allenge_train_w_PPO/src | training_worker.pytraining_worker.pytraining_worker.py |
0:0 0:0 0:0 | view |
| 95 | x 2 | Advanced workshops/...ge_train_DQN/src/markovAdvanced workshops/..._train_w_PPO/src/markov | rollout_worker.pyrollout_worker.py |
0:0 0:0 | view |
| 98 | x 4 | Advanced workshops/...src/markov/environmentsAdvanced workshops/...src/markov/environmentsAdvanced workshops/...src/markov/environmentsAdvanced workshops/...src/markov/environments | deepracer_racetrack_env.pydeepracer_racetrack_env_cv2.pydeepracer_racetrack_env.pydeepracer_racetrack_env_original.py |
0:0 0:0 0:0 0:0 | view |
| 99 | x 2 | Advanced workshops/...trator/workflow/managerAdvanced workshops/...trator/workflow/manager | experiment_manager.pyexperiment_manager.py |
0:0 0:0 | view |
| 95 | x 2 | Advanced workshops/...trator/workflow/managerAdvanced workshops/...trator/workflow/manager | experiment_manager.pyexperiment_manager.py |
0:0 0:0 | view |
| 95 | x 2 | Advanced workshops/...trator/workflow/managerAdvanced workshops/...trator/workflow/manager | experiment_manager.pyexperiment_manager.py |
0:0 0:0 | view |
| 95 | x 2 | Advanced workshops/...trator/workflow/managerAdvanced workshops/...trator/workflow/manager | experiment_manager.pyexperiment_manager.py |
0:0 0:0 | view |
| 109 | x 2 | Advanced workshops/...trator/workflow/managerAdvanced workshops/...trator/workflow/manager | model_manager.pymodel_manager.py |
0:0 0:0 | view |
| 81 | x 3 | Advanced workshops/...vent2019-400/src/markovAdvanced workshops/...ge_train_DQN/src/markovAdvanced workshops/..._train_w_PPO/src/markov | sagemaker_graph_manager.pysagemaker_graph_manager.pysagemaker_graph_manager.py |
0:0 0:0 0:0 | view |
| 84 | x 2 | Advanced workshops/...trator/workflow/managerAdvanced workshops/...trator/workflow/manager | experiment_manager.pyexperiment_manager.py |
0:0 0:0 | view |
| 80 | x 2 | Advanced workshops/...trator/workflow/managerAdvanced workshops/...trator/workflow/manager | experiment_manager.pyexperiment_manager.py |
0:0 0:0 | view |
| 67 | x 2 | Advanced workshops/...challenge_train_DQN/srcAdvanced workshops/...allenge_train_w_PPO/src | training_worker.pytraining_worker.py |
0:0 0:0 | view |
| 60 | x 2 | Advanced workshops/...ge_train_DQN/src/markovAdvanced workshops/..._train_w_PPO/src/markov | evaluation_worker.pyevaluation_worker.py |
0:0 0:0 | view |
| 72 | x 2 | Advanced workshops/...trator/workflow/managerAdvanced workshops/...trator/workflow/manager | experiment_manager.pyexperiment_manager.py |
0:0 0:0 | view |
| 67 | x 2 | Advanced workshops/...ge_train_DQN/src/markovAdvanced workshops/..._train_w_PPO/src/markov | s3_boto_data_store.pys3_boto_data_store.py |
0:0 0:0 | view |
| 88 | x 2 | Advanced workshops/...trator/workflow/managerAdvanced workshops/...trator/workflow/manager | join_manager.pyjoin_manager.py |
0:0 0:0 | view |
| 69 | x 2 | Advanced workshops/...trator/workflow/managerAdvanced workshops/...trator/workflow/manager | model_manager.pymodel_manager.py |
0:0 0:0 | view |