Places in code with 6 or more lines that are exactly the same.
Intro
- For duplication, we look at places in code where there are 6 or more lines of code that are exactly the same.
- Before duplication is calculated, the code is cleaned to remove empty lines, comments, and frequently duplicated constructs such as imports.
- You should aim at having as little as possible (<5%) of duplicated code as high-level of duplication can lead to maintenance difficulties, poor factoring, and logical contradictions.
- To learn more about duplications and techniques for eliminating duplication, Sokrates recommends the following resources:
- Avoid Repetition, MartinFlower.com (IEEE Software article)
- Refactoring duplicated code, sourcemaking.com
- Beck Design Rules, MartinFowler.com
- DRY (Don't Repeat Yourself) Principle, Wikipedia
- The Cut-and-Paste Programming Software Development Anti-Pattern, sourcemaking.com
- Code Smells / Duplicated Code, Jeff Atwood, codinghorror.com:
- "Duplicated code is the bane of software development. Stamp out duplication whenever possible. You should always be on the lookout for more subtle cases of near-duplication, too. Don't Repeat Yourself!"
- Reducing Coupling, MartinFlower.com (IEEE Software article):
- "There are several ways to describe coupling, but it boils down to this: If changing one module in a program requires changing another module, then coupling exists. ... Duplication always implies coupling, because changing one piece of duplicate code implies changing the other."
Duplication Overall
- 16% duplication:
- 38,373 cleaned lines of cleaned code (without empty lines, comments, and frequently duplicated constructs such as imports)
- 6,299 duplicated lines
- 1,697 duplicates
Duplication per Extension
Duplication per Component (primary)
Duplication Between Components (50+ lines)
Open 3D force graph...
Show more details on duplication between components...
From Component
--> To Component
Duplicated
Lines
File Pairs
Details
Services/DataX.Flow (9%)
--> Services/DataX.Metrics (74%)
943
6 file pairs
details...
Services/DataX.Flow (6%)
--> Services/DataX.SimulatedData (42%)
663
6 file pairs
details...
Services/DataX.Metrics (41%)
--> Services/DataX.SimulatedData (34%)
528
3 file pairs
details...
Services/DataX.Flow (4%)
--> Services/DataX.Gateway (38%)
474
3 file pairs
details...
Services/DataX.Gateway (33%)
--> Services/DataX.SimulatedData (26%)
418
2 file pairs
details...
Services/DataX.Gateway (33%)
--> Services/DataX.Metrics (32%)
418
2 file pairs
details...
Services/DataX.Flow (3%)
--> Services/DataX.ServiceHost (41%)
305
4 file pairs
details...
Website/Packages (<1%)
--> Website/Website (6%)
228
70 file pairs
details...
Services/DataX.Metrics (9%)
--> Services/DataX.ServiceHost (16%)
122
1 file pair
details...
DeploymentCloud/Deployment.Common (2%)
--> DeploymentCloud/Deployment.DataX (9%)
77
3 file pairs
details...
| From Component --> To Component |
Duplicated Lines |
File Pairs | Details |
|---|---|---|---|
| Services/DataX.Flow (9%) --> Services/DataX.Metrics (74%) |
943 | 6 file pairs | details... |
| Services/DataX.Flow (6%) --> Services/DataX.SimulatedData (42%) |
663 | 6 file pairs | details... |
| Services/DataX.Metrics (41%) --> Services/DataX.SimulatedData (34%) |
528 | 3 file pairs | details... |
| Services/DataX.Flow (4%) --> Services/DataX.Gateway (38%) |
474 | 3 file pairs | details... |
| Services/DataX.Gateway (33%) --> Services/DataX.SimulatedData (26%) |
418 | 2 file pairs | details... |
| Services/DataX.Gateway (33%) --> Services/DataX.Metrics (32%) |
418 | 2 file pairs | details... |
| Services/DataX.Flow (3%) --> Services/DataX.ServiceHost (41%) |
305 | 4 file pairs | details... |
| Website/Packages (<1%) --> Website/Website (6%) |
228 | 70 file pairs | details... |
| Services/DataX.Metrics (9%) --> Services/DataX.ServiceHost (16%) |
122 | 1 file pair | details... |
| DeploymentCloud/Deployment.Common (2%) --> DeploymentCloud/Deployment.DataX (9%) |
77 | 3 file pairs | details... |
Longest Duplicates
The list of 20 longest duplicates.
| Size | # | Folders | Files | Lines | Code |
|---|---|---|---|---|---|
| 199 | x 2 | Services/DataX.Flow/DataX.Flow/ScriptsServices/DataX.Simulated...X.SimulatedData/Scripts |  Deploy-FabricApplication.ps1Deploy-FabricApplication.ps1 |
1:258 (100%)1:258 (100%) | view |
| 199 | x 2 | Services/DataX.Flow/DataX.Flow/ScriptsServices/DataX.Gateway/DataX.Gateway/Scripts | Deploy-FabricApplication.ps1Deploy-FabricApplication.ps1 |
1:258 (100%)1:258 (100%) | view |
| 199 | x 2 | Services/DataX.Gateway/DataX.Gateway/ScriptsServices/DataX.Simulated...X.SimulatedData/Scripts | Deploy-FabricApplication.ps1Deploy-FabricApplication.ps1 |
1:258 (100%)1:258 (100%) | view |
| 97 | x 2 | Services/DataX.Flow/Flow.LiveDataServiceServices/DataX.Metrics/DataX.Metrics.Ingestor |  ServiceEventSource.csServiceEventSource.cs |
17:190 (97%)18:191 (97%) | view |
| 97 | x 2 | Services/DataX.Flow/Flow.InteractiveQueryServiceServices/DataX.Metrics/DataX.Metrics.Ingestor | ServiceEventSource.csServiceEventSource.cs |
17:190 (97%)18:191 (97%) | view |
| 97 | x 2 | Services/DataX.Flow/Flow.InteractiveQueryServiceServices/DataX.Flow/Flow.SchemaInferenceService | ServiceEventSource.csServiceEventSource.cs |
17:190 (97%)13:186 (97%) | view |
| 97 | x 2 | Services/DataX.Flow/Flow.LiveDataServiceServices/DataX.Flow/Flow.SchemaInferenceService | ServiceEventSource.csServiceEventSource.cs |
17:190 (97%)13:186 (97%) | view |
| 97 | x 2 | Services/DataX.Flow/Flow.ManagementServiceServices/DataX.Metrics/DataX.Metrics.Ingestor | ServiceEventSource.csServiceEventSource.cs |
17:190 (97%)18:191 (97%) | view |
| 97 | x 2 | Services/DataX.Flow/Flow.InteractiveQueryServiceServices/DataX.Flow/Flow.ManagementService | ServiceEventSource.csServiceEventSource.cs |
17:190 (97%)17:190 (97%) | view |
| 97 | x 2 | Services/DataX.Flow/Flow.SchemaInferenceServiceServices/DataX.Metrics/DataX.Metrics.Ingestor | ServiceEventSource.csServiceEventSource.cs |
13:186 (97%)18:191 (97%) | view |
| 97 | x 2 | Services/DataX.Flow/Flow.InteractiveQueryServiceServices/DataX.Flow/Flow.LiveDataService | ServiceEventSource.csServiceEventSource.cs |
17:190 (97%)17:190 (97%) | view |
| 97 | x 2 | Services/DataX.Flow/Flow.LiveDataServiceServices/DataX.Flow/Flow.ManagementService | ServiceEventSource.csServiceEventSource.cs |
17:190 (97%)17:190 (97%) | view |
| 97 | x 2 | Services/DataX.Flow/Flow.ManagementServiceServices/DataX.Flow/Flow.SchemaInferenceService | ServiceEventSource.csServiceEventSource.cs |
17:190 (97%)13:186 (97%) | view |
| 70 | x 2 | Services/DataX.Gateway/DataX.Gateway/ScriptsServices/DataX.Metrics/DataX.Metrics/Scripts | Deploy-FabricApplication.ps1Deploy-FabricApplication.ps1 |
1:100 (35%)1:100 (33%) | view |
| 70 | x 2 | Services/DataX.Flow/DataX.Flow/ScriptsServices/DataX.Metrics/DataX.Metrics/Scripts | Deploy-FabricApplication.ps1Deploy-FabricApplication.ps1 |
1:100 (35%)1:100 (33%) | view |
| 70 | x 2 | Services/DataX.Metrics/DataX.Metrics/ScriptsServices/DataX.Simulated...X.SimulatedData/Scripts | Deploy-FabricApplication.ps1Deploy-FabricApplication.ps1 |
1:100 (33%)1:100 (35%) | view |
| 67 | x 2 | Website/Packages/datax-p...ion/components/scheduleWebsite/Packages/datax-p...ion/components/schedule |  oneTimeScheduleSettings.jsxrecurringScheduleSettings.jsx |
23:103 (38%)24:104 (36%) | view |
| 58 | x 2 | Services/DataX.Gateway/DataX.Gateway/ScriptsServices/DataX.Metrics/DataX.Metrics/Scripts | Deploy-FabricApplication.ps1Deploy-FabricApplication.ps1 |
144:215 (29%)148:219 (27%) | view |
| 58 | x 2 | Services/DataX.Flow/DataX.Flow/ScriptsServices/DataX.Metrics/DataX.Metrics/Scripts | Deploy-FabricApplication.ps1Deploy-FabricApplication.ps1 |
144:215 (29%)148:219 (27%) | view |
| 58 | x 2 | Services/DataX.Metrics/DataX.Metrics/ScriptsServices/DataX.Simulated...X.SimulatedData/Scripts | Deploy-FabricApplication.ps1Deploy-FabricApplication.ps1 |
148:219 (27%)144:215 (29%) | view |
Duplicated Units
The list of top 20 duplicated units.
| Size | # | Folders | Files | Lines | Code |
|---|---|---|---|---|---|
| 28 | x 2 | Services/DataX.Flow/DataX.Flow.CodegenRulesServices/DataX.Flow/DataX.Flow.CodegenRules | Engine.csEngine.cs |
200:230 268:298 | view |
| 28 | x 5 | Services/DataX.Flow/Flow.InteractiveQueryServiceServices/DataX.Flow/Flow.LiveDataServiceServices/DataX.Flow/Flow.ManagementServiceServices/DataX.Flow/Flow.SchemaInferenceServiceServices/DataX.Metrics/DataX.Metrics.Ingestor | ServiceEventSource.csServiceEventSource.csServiceEventSource.csServiceEventSource.csServiceEventSource.cs |
99:128 99:128 99:128 95:124 100:129 | view |
| 17 | x 3 | Services/DataX.Flow/Flow.InteractiveQueryServiceServices/DataX.Flow/Flow.LiveDataServiceServices/DataX.Flow/Flow.SchemaInferenceService | InteractiveQueryService.csLiveDataService.csSchemaInferenceService.cs |
41:58 41:59 41:59 | view |
| 15 | x 6 | Services/DataX.Flow/Flow.InteractiveQueryServiceServices/DataX.Flow/Flow.LiveDataServiceServices/DataX.Flow/Flow.ManagementServiceServices/DataX.Flow/Flow.SchemaInferenceServiceServices/DataX.Metrics/DataX.Metrics.IngestorServices/DataX.ServiceHo...rviceHost.ServiceFabric | ServiceEventSource.csServiceEventSource.csServiceEventSource.csServiceEventSource.csServiceEventSource.csServiceEventSource.cs |
72:88 72:88 72:88 68:84 73:89 72:88 | view |
| 13 | x 2 | Services/DataX.Config/Da...Config.DatabricksClientServices/DataX.Config/DataX.Config.LivyClient | DatabricksHttpClient.csLivyHttpClient.cs |
42:55 42:55 | view |
| 12 | x 3 | Services/DataX.Config/DataX.Config/UtilityServices/DataX.Config/DataX.Config/UtilityServices/DataX.Flow/DataX.Flow.Common | ConfigHelper.csConfigHelper.csHelper.cs |
19:32 39:52 220:233 | view |
| 12 | x 2 | Services/DataX.Simulated...Data.DataGenServiceTestServices/DataX.Simulated...Data.DataGenServiceTest | DataGenUnitTests.csDataGenUnitTests.cs |
19:36 39:56 | view |
| 11 | x 2 | Services/DataX.Config/DataX.Utility.EventHubServices/DataX.Utilities/DataX.Utilities.EventHub | EventHubManagementRestClient.csEventHubUtil.cs |
41:54 228:241 | view |
| 11 | x 6 | Services/DataX.Flow/Flow.InteractiveQueryServiceServices/DataX.Flow/Flow.LiveDataServiceServices/DataX.Flow/Flow.ManagementServiceServices/DataX.Flow/Flow.SchemaInferenceServiceServices/DataX.Metrics/DataX.Metrics.IngestorServices/DataX.ServiceHo...rviceHost.ServiceFabric | ServiceEventSource.csServiceEventSource.csServiceEventSource.csServiceEventSource.csServiceEventSource.csServiceEventSource.cs |
163:176 163:176 163:176 159:172 164:177 146:159 | view |
| 10 | x 2 | Services/DataX.Config/Da...Config.DatabricksClientServices/DataX.Config/DataX.Config.LivyClient | DatabricksClient.csLivyClient.cs |
72:82 194:204 | view |
| 10 | x 2 | Services/DataX.Config/DataX.Config/UtilityServices/DataX.Config/DataX.Utility.EventHub | DictionaryExtension.csConnectionStringParser.cs |
27:37 53:63 | view |
| 10 | x 6 | Services/DataX.Flow/Flow.InteractiveQueryServiceServices/DataX.Flow/Flow.LiveDataServiceServices/DataX.Flow/Flow.ManagementServiceServices/DataX.Flow/Flow.SchemaInferenceServiceServices/DataX.Metrics/DataX.Metrics.IngestorServices/DataX.Simulated...atedData.DataGenService | ServiceEventSource.csServiceEventSource.csServiceEventSource.csServiceEventSource.csServiceEventSource.csServiceEventSource.cs |
178:188 178:188 178:188 174:184 179:189 156:166 | view |
| 9 | x 2 | Services/DataX.Flow/DataX.Flow.CommonServices/DataX.Flow/DataX.Flow.InteractiveQuery | Helper.csKernelService.cs |
307:317 756:766 | view |
| 7 | x 3 | DataProcessing/datax-uti...ain/scala/datax/utilityDataProcessing/datax-uti...ain/scala/datax/utilityDataProcessing/datax-uti...ain/scala/datax/utility |  DataMerger.scalaDataMerger.scalaMapManipulation.scala |
16:23 25:32 17:24 | view |
| 7 | x 2 | Services/DataX.Config/Da...figGeneration/ProcessorServices/DataX.Config/Da...figGeneration/Processor | S600_GenerateJobConfig.csS600_GenerateJobConfigBatch.cs |
92:100 283:291 | view |
| 7 | x 2 | Services/DataX.Config/DataX.Config/UtilityServices/DataX.Config/DataX.Config/Utility | ConfigHelper.csConfigHelper.cs |
133:141 161:169 | view |
| 7 | x 6 | Services/DataX.Flow/Flow.InteractiveQueryServiceServices/DataX.Flow/Flow.LiveDataServiceServices/DataX.Flow/Flow.ManagementServiceServices/DataX.Flow/Flow.SchemaInferenceServiceServices/DataX.Metrics/DataX.Metrics.IngestorServices/DataX.ServiceHo...rviceHost.ServiceFabric | ServiceEventSource.csServiceEventSource.csServiceEventSource.csServiceEventSource.csServiceEventSource.csServiceEventSource.cs |
52:59 52:59 52:59 48:55 53:60 52:59 | view |
| 6 | x 2 | Services/DataX.Flow/DataX.Flow.SchemaInferenceServices/DataX.Flow/DataX.Flow.SchemaInference | Schema.csSchema.cs |
38:44 46:52 | view |
| 6 | x 6 | Services/DataX.Flow/Flow.InteractiveQueryServiceServices/DataX.Flow/Flow.LiveDataServiceServices/DataX.Flow/Flow.ManagementServiceServices/DataX.Flow/Flow.SchemaInferenceServiceServices/DataX.Metrics/DataX.Metrics.IngestorServices/DataX.ServiceHo...rviceHost.ServiceFabric | ServiceEventSource.csServiceEventSource.csServiceEventSource.csServiceEventSource.csServiceEventSource.csServiceEventSource.cs |
63:69 63:69 63:69 59:65 64:70 63:69 | view |
| 6 | x 2 | Services/DataX.Utilities/DataX.Utilities.KeyVaultServices/DataX.Utilities/DataX.Utility.KeyVault | KeyVaultManager.csKeyVaultUtility.cs |
144:150 112:118 | view |