aws-samples / amazon-kinesis-data-analytics-java-examples
Duplication

Places in code with 6 or more lines that are exactly the same.

Intro
  • For duplication, we look at places in code where there are 6 or more lines of code that are exactly the same.
  • Before duplication is calculated, the code is cleaned to remove empty lines, comments, and frequently duplicated constructs such as imports.
  • You should aim at having as little as possible (<5%) of duplicated code as high-level of duplication can lead to maintenance difficulties, poor factoring, and logical contradictions.
Learn more...
Duplication Overall
  • 40% duplication:
    • 1,521 cleaned lines of cleaned code (without empty lines, comments, and frequently duplicated constructs such as imports)
    • 617 duplicated lines
  • 56 duplicates
system40% (617 lines)
Duplication per Extension
java35% (419 lines)
py60% (198 lines)
Duplication per Component (primary)
python65% (198 lines)
GettingStartedTable27% (41 lines)
S3ParquetSink78% (39 lines)
LambdaSink23% (34 lines)
GettingStarted_1_8100% (33 lines)
GettingStarted100% (33 lines)
GettingStarted_1_11100% (33 lines)
GettingStarted_1_6100% (33 lines)
TumblingWindow82% (29 lines)
SlidingWindow72% (26 lines)
S3Sink77% (24 lines)
SQSSink47% (19 lines)
CustomMetrics16% (18 lines)
CustomKeystore27% (14 lines)
KafkaConnectors70% (14 lines)
HudiConnector20% (13 lines)
CloudWatchSink9% (8 lines)
FirehoseSink24% (8 lines)
KeyspacesSink0% (0 lines)
EfoConsumer0% (0 lines)
Beam0% (0 lines)

Duplication Between Components (50+ lines)

G LambdaSink LambdaSink S3ParquetSink S3ParquetSink LambdaSink--S3ParquetSink 68 GettingStarted_1_11 GettingStarted_1_11 GettingStarted_1_6 GettingStarted_1_6 GettingStarted_1_11--GettingStarted_1_6 66 GettingStarted_1_8 GettingStarted_1_8 GettingStarted_1_11--GettingStarted_1_8 66 GettingStarted_1_6--GettingStarted_1_8 66 GettingStarted GettingStarted GettingStarted--GettingStarted_1_11 66 GettingStarted--GettingStarted_1_6 66 GettingStarted--GettingStarted_1_8 66 SlidingWindow SlidingWindow TumblingWindow TumblingWindow SlidingWindow--TumblingWindow 52

Download: SVG DOT (open online Graphviz editor)

Open 3D force graph...

Show more details on duplication between components...
Longest Duplicates
The list of 20 longest duplicates.
See data for all 56 duplicates...
Size#FoldersFilesLinesCode
33 x 2 python/SlidingWindow
python/TumblingWindow
sliding-windows.py
tumbling-windows.py
 21:80 (44%)
21:80 (45%)
 view
33 x 2 python/GettingStarted
python/SlidingWindow
getting-started.py
sliding-windows.py
 18:77 (47%)
21:80 (44%)
 view
33 x 2 GettingStarted_1_11/src/...rvices/kinesisanalytics
GettingStarted_1_6/src/m...rvices/kinesisanalytics
BasicStreamingJob.java
BasicStreamingJob.java
 19:73 (100%)
19:73 (100%)
 view
33 x 2 GettingStarted_1_6/src/m...rvices/kinesisanalytics
GettingStarted_1_8/src/m...rvices/kinesisanalytics
BasicStreamingJob.java
BasicStreamingJob.java
 19:73 (100%)
19:73 (100%)
 view
33 x 2 python/GettingStarted
python/TumblingWindow
getting-started.py
tumbling-windows.py
 18:77 (47%)
21:80 (45%)
 view
33 x 2 GettingStarted/src/main/...rvices/kinesisanalytics
GettingStarted_1_6/src/m...rvices/kinesisanalytics
BasicStreamingJob.java
BasicStreamingJob.java
 19:73 (100%)
19:73 (100%)
 view
33 x 2 GettingStarted_1_11/src/...rvices/kinesisanalytics
GettingStarted_1_8/src/m...rvices/kinesisanalytics
BasicStreamingJob.java
BasicStreamingJob.java
 19:73 (100%)
19:73 (100%)
 view
33 x 2 GettingStarted/src/main/...rvices/kinesisanalytics
GettingStarted_1_8/src/m...rvices/kinesisanalytics
BasicStreamingJob.java
BasicStreamingJob.java
 19:73 (100%)
19:73 (100%)
 view
33 x 2 GettingStarted/src/main/...rvices/kinesisanalytics
GettingStarted_1_11/src/...rvices/kinesisanalytics
BasicStreamingJob.java
BasicStreamingJob.java
 19:73 (100%)
19:73 (100%)
 view
26 x 2 SlidingWindow/src/main/j...rvices/kinesisanalytics
TumblingWindow/src/main/...rvices/kinesisanalytics
SlidingWindowStreamingJobWithParallel...
TumblingWindowStreamingJob.java
 37:70 (72%)
26:62 (74%)
 view
24 x 2 python/GettingStarted
python/SlidingWindow
getting-started.py
sliding-windows.py
 94:131 (34%)
101:138 (32%)
 view
24 x 2 python/GettingStarted
python/TumblingWindow
getting-started.py
tumbling-windows.py
 94:131 (34%)
98:135 (33%)
 view
24 x 2 python/SlidingWindow
python/TumblingWindow
sliding-windows.py
tumbling-windows.py
 101:138 (32%)
98:135 (33%)
 view
14 x 2 CustomKeystore/src/main/...rvices/kinesisanalytics
KafkaConnectors/src/main...rvices/kinesisanalytics
KDAFlinkStreamingJob.java
KafkaGettingStartedJob.java
 33:55 (56%)
25:47 (70%)
 view
14 x 2 python/S3Sink
python/SlidingWindow
streaming-file-sink.py
sliding-windows.py
 21:40 (16%)
21:39 (18%)
 view
14 x 2 GettingStartedTable/src/...rvices/kinesisanalytics
GettingStartedTable/src/...rvices/kinesisanalytics
StreamingJob.java
StreamingJob.java
 106:119 (11%)
154:167 (11%)
 view
14 x 2 LambdaSink/src/main/java...rvices/kinesisanalytics
S3ParquetSink/src/main/j...rvices/kinesisanalytics
TickCount.java
TickCount.java
 3:25 (100%)
3:25 (100%)
 view
14 x 2 python/GettingStarted
python/S3Sink
getting-started.py
streaming-file-sink.py
 18:36 (20%)
21:40 (16%)
 view
14 x 2 python/S3Sink
python/TumblingWindow
streaming-file-sink.py
tumbling-windows.py
 21:40 (16%)
21:39 (19%)
 view
13 x 2 python/S3Sink
python/TumblingWindow
streaming-file-sink.py
tumbling-windows.py
 53:69 (14%)
41:57 (18%)
 view
Duplicated Units
The list of top 7 duplicated units.
See data for all 7 unit duplicates...
Size#FoldersFilesLinesCode
10 x 2 CustomKeystore/src/main/...rvices/kinesisanalytics
KafkaConnectors/src/main...rvices/kinesisanalytics
KDAFlinkStreamingJob.java
KafkaGettingStartedJob.java
 33:46 
25:38 
 view
8 x 4 SlidingWindow/src/main/j...rvices/kinesisanalytics
CloudWatchSink/src/main/...rvices/kinesisanalytics
TumblingWindow/src/main/...rvices/kinesisanalytics
FirehoseSink/src/main/ja...rvices/kinesisanalytics
SlidingWindowStreamingJobWithParallel...
CustomSinkStreamingJob.java
TumblingWindowStreamingJob.java
FirehoseSinkStreamingJob.java
 41:50 
70:79 
30:39 
37:46 
 view
8 x 2 SlidingWindow/src/main/j...rvices/kinesisanalytics
TumblingWindow/src/main/...rvices/kinesisanalytics
SlidingWindowStreamingJobWithParallel...
TumblingWindowStreamingJob.java
 52:61 
41:50 
 view
8 x 4 SQSSink/src/main/java/co...rvices/kinesisanalytics
S3ParquetSink/src/main/j...rvices/kinesisanalytics
LambdaSink/src/main/java...rvices/kinesisanalytics
S3Sink/src/main/java/com...rvices/kinesisanalytics
SQSSinkJob.java
S3StreamingParquetSinkJob.java
LambdaSinkJob.java
S3StreamingSinkJob.java
 24:33 
31:40 
22:31 
27:36 
 view
8 x 5 GettingStarted_1_8/src/m...rvices/kinesisanalytics
CustomMetrics/WordCount/...rvices/kinesisanalytics
GettingStarted_1_6/src/m...rvices/kinesisanalytics
GettingStarted/src/main/...rvices/kinesisanalytics
GettingStarted_1_11/src/...rvices/kinesisanalytics
BasicStreamingJob.java
WordCountApplication.java
BasicStreamingJob.java
BasicStreamingJob.java
BasicStreamingJob.java
 38:47 
34:43 
38:47 
38:47 
38:47 
 view
7 x 4 GettingStarted_1_8/src/m...rvices/kinesisanalytics
GettingStarted_1_6/src/m...rvices/kinesisanalytics
GettingStarted/src/main/...rvices/kinesisanalytics
GettingStarted_1_11/src/...rvices/kinesisanalytics
BasicStreamingJob.java
BasicStreamingJob.java
BasicStreamingJob.java
BasicStreamingJob.java
 49:57 
49:57 
49:57 
49:57 
 view
7 x 4 python/SlidingWindow
python/GettingStarted
python/S3Sink
python/TumblingWindow
sliding-windows.py
getting-started.py
streaming-file-sink.py
tumbling-windows.py
 0:0 
0:0 
0:0 
0:0 
 view