awslabs / deequ
Duplication

Places in code with 6 or more lines that are exactly the same.

Intro
  • For duplication, we look at places in code where there are 6 or more lines of code that are exactly the same.
  • Before duplication is calculated, the code is cleaned to remove empty lines, comments, and frequently duplicated constructs such as imports.
  • You should aim at having as little as possible (<5%) of duplicated code as high-level of duplication can lead to maintenance difficulties, poor factoring, and logical contradictions.
Learn more...
Duplication Overall
  • 6% duplication:
    • 6,387 cleaned lines of cleaned code (without empty lines, comments, and frequently duplicated constructs such as imports)
    • 417 duplicated lines
  • 28 duplicates
system6% (417 lines)
Duplication per Extension
scala6% (417 lines)
Duplication per Component (primary)
deequ/suggestions18% (124 lines)
deequ/profiles16% (117 lines)
deequ/analyzers2% (59 lines)
deequ/repository5% (36 lines)
deequ9% (33 lines)
deequ/checks5% (24 lines)
deequ/constraints6% (24 lines)
deequ/anomalydetection0% (0 lines)
deequ/metrics0% (0 lines)
deequ/io0% (0 lines)
deequ/schema0% (0 lines)

Duplication Between Components (50+ lines)

G deequ/profiles deequ/profiles deequ/suggestions deequ/suggestions deequ/profiles--deequ/suggestions 167

Download: SVG DOT (open online Graphviz editor)

Open 3D force graph...

Show more details on duplication between components...
Longest Duplicates
The list of 20 longest duplicates.
See data for all 28 duplicates...
Size#FoldersFilesLinesCode
18 x 2 src/main/scala/com/amazon/deequ/repository/fs
src/main/scala/com/amazon/deequ/repository/memory
FileSystemMetricsRepository.scala
InMemoryMetricsRepository.scala
79:127 (19%)
71:119 (39%)
view
14 x 2 src/main/scala/com/amazon/deequ/profiles
src/main/scala/com/amazon/deequ/suggestions
ColumnProfilerRunBuilder.scala
ConstraintSuggestionRunBuilder.scala
51:66 (12%)
58:73 (9%)
view
13 x 2 src/main/scala/com/amazon/deequ/profiles
src/main/scala/com/amazon/deequ/suggestions
ColumnProfilerRunBuilder.scala
ConstraintSuggestionRunBuilder.scala
27:41 (11%)
32:46 (8%)
view
12 x 2 src/main/scala/com/amazon/deequ/suggestions/rules
src/main/scala/com/amazon/deequ/suggestions/rules
CategoricalRangeRule.scala
FractionalCategoricalRangeRule.scala
29:44 (30%)
31:46 (17%)
view
11 x 2 src/main/scala/com/amazon/deequ/profiles
src/main/scala/com/amazon/deequ/suggestions
ColumnProfilerRunBuilder.scala
ConstraintSuggestionRunBuilder.scala
195:221 (9%)
254:280 (7%)
view
11 x 2 src/main/scala/com/amazon/deequ
src/main/scala/com/amazon/deequ/profiles
VerificationRunBuilder.scala
ColumnProfilerRunBuilder.scala
189:216 (8%)
195:221 (9%)
view
11 x 2 src/main/scala/com/amazon/deequ
src/main/scala/com/amazon/deequ/suggestions
VerificationRunBuilder.scala
ConstraintSuggestionRunBuilder.scala
189:216 (8%)
254:280 (7%)
view
11 x 2 src/main/scala/com/amazon/deequ/analyzers/runners
src/main/scala/com/amazon/deequ/profiles
AnalysisRunBuilder.scala
ColumnProfilerRunBuilder.scala
124:151 (14%)
195:221 (9%)
view
11 x 2 src/main/scala/com/amazon/deequ/analyzers/runners
src/main/scala/com/amazon/deequ/suggestions
AnalysisRunBuilder.scala
ConstraintSuggestionRunBuilder.scala
124:151 (14%)
254:280 (7%)
view
11 x 2 src/main/scala/com/amazon/deequ/profiles
src/main/scala/com/amazon/deequ/suggestions
ColumnProfilerRunner.scala
ConstraintSuggestionRunner.scala
90:103 (16%)
240:253 (5%)
view
11 x 2 src/main/scala/com/amazon/deequ
src/main/scala/com/amazon/deequ/analyzers/runners
VerificationRunBuilder.scala
AnalysisRunBuilder.scala
189:216 (8%)
124:151 (14%)
view
10 x 2 src/main/scala/com/amazon/deequ/analyzers/runners
src/main/scala/com/amazon/deequ/profiles
AnalysisRunner.scala
ColumnProfiler.scala
216:231 (3%)
342:357 (2%)
view
9 x 2 src/main/scala/com/amazon/deequ/profiles
src/main/scala/com/amazon/deequ/suggestions
ColumnProfilerRunner.scala
ConstraintSuggestionRunner.scala
71:81 (13%)
101:111 (4%)
view
8 x 2 src/main/scala/com/amazon/deequ/analyzers/catalyst
src/main/scala/com/amazon/deequ/analyzers/catalyst
StatefulHyperloglogPlus.scala
StatefulHyperloglogPlus.scala
127:134 (5%)
195:202 (5%)
view
8 x 2 src/main/scala/com/amazon/deequ/suggestions/rules
src/main/scala/com/amazon/deequ/suggestions/rules
CategoricalRangeRule.scala
FractionalCategoricalRangeRule.scala
55:65 (20%)
65:75 (11%)
view
8 x 2 src/main/scala/com/amazon/deequ
src/main/scala/com/amazon/deequ/analyzers/runners
VerificationRunBuilder.scala
AnalysisRunBuilder.scala
271:288 (5%)
167:184 (10%)
view
7 x 2 src/main/scala/com/amazon/deequ/profiles
src/main/scala/com/amazon/deequ/suggestions
ColumnProfilerRunBuilder.scala
ConstraintSuggestionRunBuilder.scala
89:110 (6%)
131:153 (4%)
view
7 x 2 src/main/scala/com/amazon/deequ/analyzers
src/main/scala/com/amazon/deequ/analyzers
ApproxQuantile.scala
ApproxQuantiles.scala
59:69 (15%)
44:55 (18%)
view
7 x 2 src/main/scala/com/amazon/deequ/profiles
src/main/scala/com/amazon/deequ/suggestions
ColumnProfiler.scala
ConstraintSuggestionRunner.scala
212:219 (1%)
228:235 (3%)
view
7 x 2 src/main/scala/com/amazon/deequ
src/main/scala/com/amazon/deequ
VerificationSuite.scala
VerificationSuite.scala
208:214 (4%)
302:308 (4%)
view
Duplicated Units
The list of top 5 duplicated units.
See data for all 5 unit duplicates...
Size#FoldersFilesLinesCode
12 x 2 src/main/scala/com/amazon/deequ/analyzers/runners
src/main/scala/com/amazon/deequ/profiles
AnalysisRunner.scala
ColumnProfiler.scala
216:234 
342:360 
view
8 x 2 src/main/scala/com/amazon/deequ/checks
src/main/scala/com/amazon/deequ/checks
Check.scala
Check.scala
850:859 
890:899 
view
8 x 2 src/main/scala/com/amazon/deequ/checks
src/main/scala/com/amazon/deequ/checks
Check.scala
Check.scala
870:879 
911:920 
view
7 x 4 src/main/scala/com/amazon/deequ/analyzers/runners
src/main/scala/com/amazon/deequ/profiles
src/main/scala/com/amazon/deequ/suggestions
src/main/scala/com/amazon/deequ
AnalysisRunBuilder.scala
ColumnProfilerRunBuilder.scala
ConstraintSuggestionRunBuilder.scala
VerificationRunBuilder.scala
133:141 
203:211 
262:270 
198:206 
view
7 x 2 src/main/scala/com/amazon/deequ/profiles
src/main/scala/com/amazon/deequ/suggestions
ColumnProfiler.scala
ConstraintSuggestionRunner.scala
212:220 
228:236 
view