awslabs / emr-dynamodb-connector
Duplication

Places in code with 6 or more lines that are exactly the same.

Intro
  • For duplication, we look at places in code where there are 6 or more lines of code that are exactly the same.
  • Before duplication is calculated, the code is cleaned to remove empty lines, comments, and frequently duplicated constructs such as imports.
  • You should aim at having as little as possible (<5%) of duplicated code as high-level of duplication can lead to maintenance difficulties, poor factoring, and logical contradictions.
Learn more...
Duplication Overall
  • 4% duplication:
    • 4,243 cleaned lines of cleaned code (without empty lines, comments, and frequently duplicated constructs such as imports)
    • 186 duplicated lines
  • 21 duplicates
system4% (186 lines)
Duplication per Extension
java4% (186 lines)
Duplication per Component (primary)
emr-dynamodb-hive/src/main/java/org/apache/hadoop/hive/dynamodb4% (67 lines)
emr-dynamodb-tools/src/main/java/org/apache/hadoop/dynamodb/tools24% (50 lines)
shims/hive1-shims/src/main/java/org/apache/hadoop/hive51% (15 lines)
emr-dynamodb-hadoop/src/main/java/org/apache/hadoop/dynamodb2% (14 lines)
emr-dynamodb-hadoop/src/main/java/org/apache/hadoop/dynamodb/split14% (13 lines)
emr-dynamodb-tools/src/main/java/org/apache/hadoop/dynamodb/exportformat6% (12 lines)
shims/hive2-shims/src/main/java/org/apache/hadoop/hive32% (9 lines)
shims/hive1.2-shims/src/main/java/org/apache/hadoop/hive35% (6 lines)
shims/loader/src/main/java/org/apache/hadoop/hive0% (0 lines)
shims/common/src/main/java/org/apache/hadoop/hive0% (0 lines)
emr-dynamodb-hadoop/src/main/java/org/apache/hadoop/dynamodb/preader0% (0 lines)
emr-dynamodb-hadoop/src/main/java/org/apache/hadoop/dynamodb/filter0% (0 lines)
emr-dynamodb-hadoop/src/main/java/org/apache/hadoop/dynamodb/util0% (0 lines)
emr-dynamodb-hadoop/src/main/java/org/apache/hadoop/dynamodb/read0% (0 lines)
emr-dynamodb-hadoop/src/main/java/org/apache/hadoop/dynamodb/key0% (0 lines)
emr-dynamodb-hadoop/src/main/java/org/apache/hadoop/dynamodb/write0% (0 lines)
emr-dynamodb-hadoop/src/main/java/org/apache/hadoop/dynamodb/type0% (0 lines)
emr-dynamodb-tools/src/main/java/org/apache/hadoop/dynamodb/importformat0% (0 lines)
Longest Duplicates
The list of 20 longest duplicates.
See data for all 21 duplicates...
Size#FoldersFilesLinesCode
12 x 2 emr-dynamodb-tools/src/m...e/hadoop/dynamodb/tools
emr-dynamodb-tools/src/m...e/hadoop/dynamodb/tools
DynamoDBExport.java
DynamoDBImport.java
 112:124 (13%)
96:107 (18%)
 view
9 x 2 shims/hive1-shims/src/ma...oop/hive/dynamodb/shims
shims/hive2-shims/src/ma...oop/hive/dynamodb/shims
Hive1SerDeParametersShim.java
Hive2SerDeParametersShim.java
 21:33 (60%)
19:31 (64%)
 view
7 x 2 emr-dynamodb-hadoop/src/...e/hadoop/dynamodb/split
emr-dynamodb-hive/src/ma...oop/hive/dynamodb/split
DynamoDBSegmentsSplit.java
HiveDynamoDBSegmentsSplit.java
 66:75 (12%)
80:89 (12%)
 view
7 x 2 emr-dynamodb-hadoop/src/.../apache/hadoop/dynamodb
emr-dynamodb-hadoop/src/.../apache/hadoop/dynamodb
DynamoDBClient.java
DynamoDBClient.java
 423:432 (2%)
435:444 (2%)
 view
7 x 2 emr-dynamodb-tools/src/m...e/hadoop/dynamodb/tools
emr-dynamodb-tools/src/m...e/hadoop/dynamodb/tools
DynamoDBExport.java
DynamoDBImport.java
 88:94 (7%)
77:83 (10%)
 view
6 x 2 shims/hive1-shims/src/ma...oop/hive/dynamodb/shims
shims/hive1.2-shims/src/...oop/hive/dynamodb/shims
DynamoDbHive1Shims.java
DynamoDbHive1Dot2Shims.java
 37:44 (42%)
24:34 (35%)
 view
6 x 2 emr-dynamodb-hive/src/ma...doop/hive/dynamodb/type
emr-dynamodb-hive/src/ma...doop/hive/dynamodb/type
HiveDynamoDBListType.java
HiveDynamoDBNumberSetType.java
 34:41 (22%)
36:43 (28%)
 view
6 x 2 emr-dynamodb-tools/src/m...e/hadoop/dynamodb/tools
emr-dynamodb-tools/src/m...e/hadoop/dynamodb/tools
DynamoDBExport.java
DynamoDBImport.java
 42:49 (6%)
42:49 (9%)
 view
6 x 2 emr-dynamodb-hive/src/ma...doop/hive/dynamodb/util
emr-dynamodb-hive/src/ma...doop/hive/dynamodb/util
DynamoDBDataParser.java
DynamoDBDataParser.java
 162:170 (3%)
193:201 (3%)
 view
6 x 2 emr-dynamodb-hive/src/ma...doop/hive/dynamodb/type
emr-dynamodb-hive/src/ma...doop/hive/dynamodb/type
HiveDynamoDBMapType.java
HiveDynamoDBNullType.java
 36:43 (13%)
19:26 (35%)
 view
6 x 2 emr-dynamodb-hive/src/ma...doop/hive/dynamodb/type
emr-dynamodb-hive/src/ma...doop/hive/dynamodb/type
HiveDynamoDBListType.java
HiveDynamoDBNullType.java
 34:41 (22%)
19:26 (35%)
 view
6 x 2 emr-dynamodb-hive/src/ma...doop/hive/dynamodb/type
emr-dynamodb-hive/src/ma...doop/hive/dynamodb/type
HiveDynamoDBNullType.java
HiveDynamoDBNumberSetType.java
 19:26 (35%)
36:43 (28%)
 view
6 x 2 emr-dynamodb-hadoop/src/...e/hadoop/dynamodb/split
emr-dynamodb-hive/src/ma...oop/hive/dynamodb/split
DynamoDBSegmentsSplit.java
HiveDynamoDBSegmentsSplit.java
 39:46 (10%)
54:61 (10%)
 view
6 x 2 emr-dynamodb-hive/src/ma...doop/hive/dynamodb/type
emr-dynamodb-hive/src/ma...doop/hive/dynamodb/type
HiveDynamoDBNullType.java
HiveDynamoDBNumberType.java
 19:26 (35%)
34:41 (30%)
 view
6 x 2 emr-dynamodb-hive/src/ma...doop/hive/dynamodb/type
emr-dynamodb-hive/src/ma...doop/hive/dynamodb/type
HiveDynamoDBMapType.java
HiveDynamoDBNumberSetType.java
 36:43 (13%)
36:43 (28%)
 view
6 x 2 emr-dynamodb-hive/src/ma...doop/hive/dynamodb/type
emr-dynamodb-hive/src/ma...doop/hive/dynamodb/type
HiveDynamoDBNumberSetType.java
HiveDynamoDBNumberType.java
 36:43 (28%)
34:41 (30%)
 view
6 x 2 emr-dynamodb-hive/src/ma...doop/hive/dynamodb/type
emr-dynamodb-hive/src/ma...doop/hive/dynamodb/type
HiveDynamoDBListType.java
HiveDynamoDBNumberType.java
 34:41 (22%)
34:41 (30%)
 view
6 x 2 emr-dynamodb-hive/src/ma...doop/hive/dynamodb/type
emr-dynamodb-hive/src/ma...doop/hive/dynamodb/type
HiveDynamoDBMapType.java
HiveDynamoDBNumberType.java
 36:43 (13%)
34:41 (30%)
 view
6 x 2 emr-dynamodb-tools/src/m...p/dynamodb/exportformat
emr-dynamodb-tools/src/m...p/dynamodb/exportformat
ExportFileHeader.java
ExportManifestEntry.java
 28:35 (54%)
31:38 (16%)
 view
6 x 2 emr-dynamodb-hive/src/ma...op/hive/dynamodb/filter
emr-dynamodb-hive/src/ma...op/hive/dynamodb/filter
DynamoDBFilterPushdown.java
DynamoDBFilterPushdown.java
 259:264 (2%)
278:283 (2%)
 view
Duplicated Units
The list of top 1 duplicated units.
See data for all 1 unit duplicate
Size#FoldersFilesLinesCode
6 x 2 emr-dynamodb-tools/src/m...e/hadoop/dynamodb/tools
emr-dynamodb-tools/src/m...e/hadoop/dynamodb/tools
DynamoDBExport.java
DynamoDBImport.java
 151:158 
120:127 
 view