microsoft / SynapseML
Duplication

Places in code with 6 or more lines that are exactly the same.

Intro
  • For duplication, we look at places in code where there are 6 or more lines of code that are exactly the same.
  • Before duplication is calculated, the code is cleaned to remove empty lines, comments, and frequently duplicated constructs such as imports.
  • You should aim at having as little as possible (<5%) of duplicated code as high-level of duplication can lead to maintenance difficulties, poor factoring, and logical contradictions.
Learn more...
Duplication Overall
  • 6% duplication:
    • 27,648 cleaned lines of cleaned code (without empty lines, comments, and frequently duplicated constructs such as imports)
    • 1,778 duplicated lines
  • 142 duplicates
system6% (1,778 lines)
Duplication per Extension
scala5% (1,287 lines)
yaml41% (285 lines)
py5% (158 lines)
js3% (38 lines)
sbt7% (10 lines)
Duplication per Component (primary)
core5% (746 lines)
cognitive8% (428 lines)
ROOT36% (295 lines)
lightgbm5% (145 lines)
vw5% (66 lines)
website2% (38 lines)
project7% (32 lines)
deep-learning1% (28 lines)
opencv0% (0 lines)
templates0% (0 lines)
tools0% (0 lines)
Longest Duplicates
The list of 20 longest duplicates.
See data for all 142 duplicates...
Size#FoldersFilesLinesCode
24 x 2 cognitive/src/main/scala...re/synapse/ml/cognitive
cognitive/src/main/scala...re/synapse/ml/cognitive
TextTranslator.scala
TextTranslator.scala
 86:116 (7%)
221:251 (7%)
 view
23 x 2 cognitive/src/main/scala...re/synapse/ml/cognitive
cognitive/src/main/scala...re/synapse/ml/cognitive
SpeechToText.scala
SpeechToTextSDK.scala
 62:89 (29%)
168:195 (5%)
 view
22 x 2 ROOT
ROOT
pipeline.yaml
pipeline.yaml
 409:430 (3%)
639:660 (3%)
 view
22 x 2 ROOT
ROOT
pipeline.yaml
pipeline.yaml
 409:430 (3%)
459:480 (3%)
 view
22 x 2 ROOT
ROOT
pipeline.yaml
pipeline.yaml
 459:480 (3%)
639:660 (3%)
 view
22 x 2 ROOT
ROOT
pipeline.yaml
pipeline.yaml
 358:379 (3%)
409:430 (3%)
 view
22 x 2 ROOT
ROOT
pipeline.yaml
pipeline.yaml
 358:379 (3%)
639:660 (3%)
 view
22 x 2 ROOT
ROOT
pipeline.yaml
pipeline.yaml
 358:379 (3%)
459:480 (3%)
 view
18 x 2 vw/src/main/scala/com/mi...oft/azure/synapse/ml/vw
vw/src/main/scala/com/mi...oft/azure/synapse/ml/vw
VowpalWabbitBase.scala
VowpalWabbitBase.scala
 305:322 (5%)
381:398 (5%)
 view
18 x 2 lightgbm/src/main/scala/...apse/ml/lightgbm/params
lightgbm/src/main/scala/...apse/ml/lightgbm/params
TrainParams.scala
TrainParams.scala
 85:102 (8%)
181:198 (8%)
 view
17 x 2 core/src/main/scala/com/...azure/synapse/ml/stages
core/src/main/scala/com/...azure/synapse/ml/stages
Batchers.scala
Batchers.scala
 26:57 (20%)
80:111 (20%)
 view
17 x 2 cognitive/src/main/scala...re/synapse/ml/cognitive
cognitive/src/main/scala...re/synapse/ml/cognitive
TextAnalytics.scala
TextAnalytics.scala
 70:89 (4%)
508:527 (4%)
 view
17 x 2 lightgbm/src/main/scala/...apse/ml/lightgbm/params
lightgbm/src/main/scala/...apse/ml/lightgbm/params
TrainParams.scala
TrainParams.scala
 107:123 (8%)
156:172 (8%)
 view
17 x 2 lightgbm/src/main/scala/...ure/synapse/ml/lightgbm
lightgbm/src/main/scala/...ure/synapse/ml/lightgbm
LightGBMRanker.scala
LightGBMRegressor.scala
 132:159 (16%)
104:131 (20%)
 view
17 x 2 ROOT
ROOT
pipeline.yaml
pipeline.yaml
 44:61 (2%)
88:105 (2%)
 view
16 x 2 ROOT
ROOT
pipeline.yaml
pipeline.yaml
 88:104 (2%)
130:146 (2%)
 view
16 x 2 cognitive/src/main/scala...re/synapse/ml/cognitive
cognitive/src/main/scala...re/synapse/ml/cognitive
TextTranslator.scala
TextTranslator.scala
 228:247 (4%)
509:528 (4%)
 view
16 x 2 ROOT
ROOT
pipeline.yaml
pipeline.yaml
 44:60 (2%)
130:146 (2%)
 view
16 x 2 lightgbm/src/main/scala/...apse/ml/lightgbm/params
lightgbm/src/main/scala/...apse/ml/lightgbm/params
TrainParams.scala
TrainParams.scala
 140:155 (7%)
185:200 (7%)
 view
16 x 2 cognitive/src/main/scala...re/synapse/ml/cognitive
cognitive/src/main/scala...re/synapse/ml/cognitive
TextTranslator.scala
TextTranslator.scala
 93:112 (4%)
509:528 (4%)
 view
Duplicated Units
The list of top 12 duplicated units.
See data for all 12 unit duplicates...
Size#FoldersFilesLinesCode
16 x 2 cognitive/src/main/scala...re/synapse/ml/cognitive
cognitive/src/main/scala...re/synapse/ml/cognitive
FormRecognizer.scala
FormRecognizer.scala
 289:308 
330:349 
 view
16 x 2 deep-learning/src/main/s...t/azure/synapse/ml/cntk
deep-learning/src/main/s...t/azure/synapse/ml/cntk
ConversionUtils.scala
ConversionUtils.scala
 68:84 
86:102 
 view
16 x 2 lightgbm/src/main/scala/...ure/synapse/ml/lightgbm
lightgbm/src/main/scala/...ure/synapse/ml/lightgbm
LightGBMRanker.scala
LightGBMRegressor.scala
 130:154 
102:126 
 view
13 x 2 core/src/main/scala/com/...e/synapse/ml/explainers
core/src/main/scala/com/...e/synapse/ml/explainers
VectorLIME.scala
VectorSHAP.scala
 72:88 
72:88 
 view
9 x 2 cognitive/src/main/scala...re/synapse/ml/cognitive
cognitive/src/main/scala...re/synapse/ml/cognitive
SpeechToText.scala
SpeechToTextSDK.scala
 71:82 
177:188 
 view
9 x 2 core/src/main/scala/com/...zure/synapse/ml/codegen
core/src/main/scala/com/...zure/synapse/ml/codegen
Wrappable.scala
Wrappable.scala
 148:157 
411:420 
 view
9 x 2 core/src/main/scala/org/apache/spark/ml/param
core/src/main/scala/org/apache/spark/ml/param
PythonWrappableParam.scala
RWrappableParam.scala
 10:19 
16:25 
 view
8 x 2 core/src/main/scala/com/...azure/synapse/ml/stages
core/src/main/scala/com/...azure/synapse/ml/stages
DropColumns.scala
SelectColumns.scala
 52:63 
54:65 
 view
7 x 2 core/src/main/scala/com/...e/synapse/ml/explainers
core/src/main/scala/com/...e/synapse/ml/explainers
ImageLIME.scala
ImageSHAP.scala
 115:123 
113:121 
 view
6 x 2 cognitive/src/main/scala...re/synapse/ml/cognitive
cognitive/src/main/scala...re/synapse/ml/cognitive
AzureSearchAPI.scala
AzureSearchAPI.scala
 126:132 
134:140 
 view
6 x 2 core/src/main/scala/com/...ure/synapse/ml/core/env
project
FileUtilities.scala
build.scala
 29:35 
107:114 
 view
6 x 2 core/src/main/scala/com/...e/synapse/ml/explainers
core/src/main/scala/com/...e/synapse/ml/explainers
TextLIME.scala
TextSHAP.scala
 71:78 
70:77 
 view