Apache Kafka
Duplication

Intro
  • For duplication, we look at places in code where there are six or more lines of code that are exactly the same.
  • Before duplication is calculated, the code is cleaned to remove empty lines, comments, and frequently duplicated constructs such as imports.
  • You should aim at having as little as possible (<5%) of duplicated code as high-level of duplication can lead to maintenance difficulties, poor factoring, and logical contradictions.
Learn more...
Duplication Overall
  • 6% duplication:
    • 160,403 cleaned lines of cleaned code (without empty lines, comments, and frequently duplicated constructs such as imports)
    • 11,119 duplicated lines
  • 647 duplicates
system6% (11,119 lines)
Duplication per Extension
java8% (10,345 lines)
scala2% (774 lines)
Duplication per Component (primary)
streams14% (5,431 lines)
clients5% (2,796 lines)
connect5% (1,102 lines)
tools9% (706 lines)
core1% (662 lines)
generator5% (220 lines)
jmh-benchmarks20% (196 lines)
examples2% (6 lines)
log4j-appender0% (0 lines)

Duplication Between Components (50+ lines)

G clients clients connect connect clients--connect 308

Download: SVG DOT (open online Graphviz editor)

Show more details on duplication between components...
Longest Duplicates
The list of 20 longest duplicates.
Size#FoldersFilesLinesCode
70 x 2 streams/src/main/java/or...ams/processor/internals
streams/src/main/java/or...ams/processor/internals
 ProcessorContextAdapter.java
ProcessorContextReverseAdapter.java		 53:168 (58%)
49:164 (54%)
 view
38 x 2 streams/src/main/java/or...reams/kstream/internals
streams/src/main/java/or...reams/kstream/internals
 TimeWindowedCogroupedKStreamImpl.java
SlidingWindowedCogroupedKStreamImpl.java		 51:98 (37%)
46:92 (46%)
 view
36 x 2 clients/src/main/java/or...ache/kafka/common/utils
clients/src/main/java/or...ache/kafka/common/utils
 LogContext.java
LogContext.java		 80:137 (8%)
446:503 (8%)
 view
32 x 2 streams/src/main/java/or...streams/state/internals
streams/src/main/java/or...streams/state/internals
 CachingSessionStore.java
CachingWindowStore.java		 294:346 (13%)
487:539 (8%)
 view
31 x 2 streams/src/main/java/or...reams/kstream/internals
streams/src/main/java/or...reams/kstream/internals
 SlidingWindowedKStreamImpl.java
TimeWindowedKStreamImpl.java		 101:141 (22%)
114:158 (19%)
 view
31 x 2 streams/src/main/java/or...reams/kstream/internals
streams/src/main/java/or...reams/kstream/internals
 KStreamWindowAggregate.java
KStreamSlidingWindowAggregate.java		 162:207 (24%)
394:437 (10%)
 view
30 x 2 clients/src/main/java/or...ache/kafka/common/utils
clients/src/main/java/or...ache/kafka/common/utils
 PureJavaCrc32C.java
Crc32.java		 77:121 (5%)
114:157 (10%)
 view
29 x 2 streams/src/main/java/or...reams/kstream/internals
streams/src/main/java/or...reams/kstream/internals
 KStreamWindowAggregate.java
KStreamSlidingWindowAggregate.java		 78:109 (22%)
79:110 (10%)
 view
28 x 2 clients/src/main/java/or...e/kafka/common/protocol
clients/src/main/java/or...e/kafka/common/protocol
 RecordsReadable.java
ByteBufferAccessor.java		 35:80 (75%)
28:73 (50%)
 view
28 x 2 streams/src/main/java/or...reams/kstream/internals
streams/src/main/java/or...reams/kstream/internals
 SlidingWindowedKStreamImpl.java
TimeWindowedKStreamImpl.java		 145:186 (20%)
162:205 (17%)
 view
27 x 2 streams/streams-scala/sr...ams/scala/serialization
streams/streams-scala/sr...che/kafka/streams/scala
 Serdes.scala
Serdes.scala		 46:76 (52%)
47:77 (60%)
 view
24 x 3 streams/src/main/java/or...ams/processor/internals
streams/src/main/java/or...ams/processor/internals
streams/src/main/java/or...ams/processor/internals
 ForwardingDisabledProcessorContext.java
ProcessorContextAdapter.java
ProcessorContextReverseAdapter.java		 130:167 (29%)
192:229 (20%)
205:242 (18%)
 view
24 x 2 streams/src/main/java/or...ams/processor/internals
streams/src/main/java/or...ams/processor/internals
 AbstractReadWriteDecorator.java
AbstractReadOnlyDecorator.java		 73:110 (14%)
80:117 (14%)
 view
24 x 2 streams/src/main/java/or...kstream/internals/graph
streams/src/main/java/or...kstream/internals/graph
 OptimizableRepartitionNode.java
UnoptimizableRepartitionNode.java		 66:92 (35%)
52:78 (38%)
 view
23 x 2 streams/src/main/java/or...ams/processor/internals
streams/src/main/java/or...ams/processor/internals
 InternalProcessorContext.java
InternalApiProcessorContext.java		 40:122 (88%)
36:118 (95%)
 view
23 x 2 clients/src/main/java/or...he/kafka/common/network
clients/src/main/java/or...he/kafka/common/network
 PlaintextTransportLayer.java
SslTransportLayer.java		 48:82 (30%)
135:175 (4%)
 view
22 x 2 streams/src/main/java/or...reams/kstream/internals
streams/src/main/java/or...reams/kstream/internals
 SlidingWindowedKStreamImpl.java
TimeWindowedKStreamImpl.java		 48:77 (15%)
53:83 (14%)
 view
21 x 2 connect/runtime/src/main...che/kafka/connect/tools
connect/runtime/src/main...che/kafka/connect/tools
 MockSinkConnector.java
MockSourceConnector.java		 33:66 (65%)
33:66 (65%)
 view
21 x 2 clients/src/main/java/or...y/oauthbearer/internals
clients/src/main/java/or...ecurity/scram/internals
 OAuthBearerSaslClient.java
ScramSaslClient.java		 124:154 (20%)
154:184 (14%)
 view
20 x 2 streams/src/main/java/or...reams/kstream/internals
streams/src/main/java/or...reams/kstream/internals
 KTableKTableLeftJoin.java
KTableKTableInnerJoin.java		 73:100 (22%)
74:101 (23%)
 view
Most Frequent Duplicates
The list of 20 most frequently found duplicates.
Size#FoldersFilesLinesCode
6 x 33 streams/src/main/java/or...kafka/streams/processor
streams/src/main/java/or...streams/state/internals
streams/src/main/java/or...ream/internals/suppress
streams/src/main/java/or...reams/kstream/internals
streams/src/main/java/or...ream/internals/suppress
streams/src/main/java/or...e/kafka/streams/kstream
streams/src/main/java/or...ams/processor/internals
streams/src/main/java/or...ream/internals/suppress
streams/src/main/java/or...ams/processor/internals
streams/src/main/java/or...e/kafka/streams/kstream
...
 TaskMetadata.java
ContextualRecord.java
FinalResultsSuppressionBuilder.java
Change.java
SuppressedInternal.java
JoinWindows.java
UnwindowedChangelogTopicConfig.java
StrictBufferConfigImpl.java
ProcessorRecordContext.java
TimeWindows.java
...		 49:55 (20%)
54:60 (17%)
49:55 (15%)
36:42 (28%)
87:93 (10%)
276:282 (6%)
61:67 (17%)
105:111 (8%)
190:196 (4%)
252:258 (5%)
...
 view
6 x 27 clients/src/main/java/or...che/kafka/common/config
connect/api/src/main/jav...he/kafka/connect/health
connect/api/src/main/jav...he/kafka/connect/source
clients/src/main/java/or...che/kafka/common/record
connect/api/src/main/jav...he/kafka/connect/health
clients/src/main/java/or...che/kafka/common/record
clients/src/main/java/or...che/kafka/clients/admin
connect/runtime/src/main...e/kafka/connect/runtime
clients/src/main/java/or...che/kafka/clients/admin
clients/src/main/java/or...che/kafka/common/record
...
 ConfigResource.java
AbstractState.java
SourceRecord.java
SimpleRecord.java
ConnectorHealth.java
FileLogInputStream.java
Config.java
SessionKey.java
ConfigEntry.java
AbstractLegacyRecordBatch.java
...		 95:100 (14%)
77:82 (19%)
106:111 (9%)
88:93 (10%)
86:91 (12%)
222:227 (4%)
60:65 (25%)
58:63 (27%)
164:169 (4%)
434:439 (1%)
...
 view
6 x 22 streams/src/main/java/or...state/internals/metrics
streams/src/main/java/or...state/internals/metrics
streams/src/main/java/or...state/internals/metrics
streams/src/main/java/or...state/internals/metrics
streams/src/main/java/or...state/internals/metrics
streams/src/main/java/or...state/internals/metrics
streams/src/main/java/or...state/internals/metrics
streams/src/main/java/or...state/internals/metrics
streams/src/main/java/or...state/internals/metrics
streams/src/main/java/or...state/internals/metrics
...
 RocksDBMetrics.java
RocksDBMetrics.java
RocksDBMetrics.java
RocksDBMetrics.java
RocksDBMetrics.java
RocksDBMetrics.java
RocksDBMetrics.java
RocksDBMetrics.java
RocksDBMetrics.java
RocksDBMetrics.java
...		 515:520 (<1%)
527:532 (<1%)
539:544 (<1%)
551:556 (<1%)
563:568 (<1%)
575:580 (<1%)
587:592 (<1%)
599:604 (<1%)
611:616 (<1%)
623:628 (<1%)
...
 view
6 x 14 clients/src/main/java/or...e/kafka/common/requests
clients/src/main/java/or...e/kafka/common/requests
clients/src/main/java/or...e/kafka/common/requests
clients/src/main/java/or...e/kafka/common/requests
clients/src/main/java/or...e/kafka/common/requests
clients/src/main/java/or...e/kafka/common/requests
clients/src/main/java/or...e/kafka/common/requests
clients/src/main/java/or...e/kafka/common/requests
clients/src/main/java/or...e/kafka/common/requests
clients/src/main/java/or...e/kafka/common/requests
...
 AddOffsetsToTxnResponse.java
DeleteTopicsResponse.java
CreateTopicsResponse.java
DescribeGroupsResponse.java
TxnOffsetCommitResponse.java
DescribeLogDirsResponse.java
ExpireDelegationTokenResponse.java
RenewDelegationTokenResponse.java
CreateDelegationTokenResponse.java
ElectLeadersResponse.java
...		 57:64 (26%)
51:58 (23%)
60:67 (22%)
115:122 (6%)
82:89 (12%)
49:56 (9%)
56:63 (23%)
48:55 (23%)
78:85 (12%)
80:87 (10%)
...
 view
8 x 10 streams/src/main/java/or...state/internals/metrics
streams/src/main/java/or...state/internals/metrics
streams/src/main/java/or...state/internals/metrics
streams/src/main/java/or...state/internals/metrics
streams/src/main/java/or...state/internals/metrics
streams/src/main/java/or...state/internals/metrics
streams/src/main/java/or...state/internals/metrics
streams/src/main/java/or...state/internals/metrics
streams/src/main/java/or...state/internals/metrics
streams/src/main/java/or...state/internals/metrics
 RocksDBMetrics.java
RocksDBMetrics.java
RocksDBMetrics.java
RocksDBMetrics.java
RocksDBMetrics.java
RocksDBMetrics.java
RocksDBMetrics.java
RocksDBMetrics.java
RocksDBMetrics.java
RocksDBMetrics.java		 260:267 (1%)
277:284 (1%)
294:301 (1%)
311:318 (1%)
346:353 (1%)
363:370 (1%)
380:387 (1%)
431:438 (1%)
448:455 (1%)
465:472 (1%)
 view
9 x 10 streams/src/main/java/or...state/internals/metrics
streams/src/main/java/or...state/internals/metrics
streams/src/main/java/or...state/internals/metrics
streams/src/main/java/or...state/internals/metrics
streams/src/main/java/or...state/internals/metrics
streams/src/main/java/or...state/internals/metrics
streams/src/main/java/or...state/internals/metrics
streams/src/main/java/or...state/internals/metrics
streams/src/main/java/or...state/internals/metrics
streams/src/main/java/or...state/internals/metrics
 StateStoreMetrics.java
StateStoreMetrics.java
StateStoreMetrics.java
StateStoreMetrics.java
StateStoreMetrics.java
StateStoreMetrics.java
StateStoreMetrics.java
StateStoreMetrics.java
StateStoreMetrics.java
StateStoreMetrics.java		 171:179 (1%)
191:199 (1%)
211:219 (1%)
231:239 (1%)
251:259 (1%)
271:279 (1%)
291:299 (1%)
311:319 (1%)
331:339 (1%)
351:359 (1%)
 view
8 x 9 streams/quickstart/java/...resources/src/main/java
streams/examples/src/mai...eams/examples/wordcount
streams/examples/src/mai...a/streams/examples/pipe
streams/examples/src/mai...eams/examples/wordcount
streams/quickstart/java/...resources/src/main/java
streams/examples/src/mai...reams/examples/pageview
streams/examples/src/mai...ms/examples/temperature
streams/quickstart/java/...resources/src/main/java
streams/examples/src/mai...eams/examples/wordcount
 WordCount.java
WordCountDemo.java
PipeDemo.java
WordCountTransformerDemo.java
LineSplit.java
PageViewTypedDemo.java
TemperatureDemo.java
Pipe.java
WordCountProcessorDemo.java		 66:75 (27%)
97:106 (19%)
60:69 (33%)
155:164 (11%)
57:66 (30%)
233:242 (7%)
112:121 (19%)
52:61 (33%)
144:153 (11%)
 view
6 x 8 jmh-benchmarks/src/main/...apache/kafka/jmh/common
jmh-benchmarks/src/main/...che/kafka/jmh/partition
jmh-benchmarks/src/main/...apache/kafka/jmh/common
jmh-benchmarks/src/main/...ache/kafka/jmh/metadata
jmh-benchmarks/src/main/...ache/kafka/jmh/producer
jmh-benchmarks/src/main/...che/kafka/jmh/partition
jmh-benchmarks/src/main/...pache/kafka/jmh/fetcher
jmh-benchmarks/src/main/...apache/kafka/jmh/common
 TopicBenchmark.java
UpdateFollowerFetchStateBenchmark.java
FetchResponseBenchmark.java
MetadataRequestBenchmark.java
ProducerRecordBenchmark.java
PartitionMakeFollowerBenchmark.java
ReplicaFetcherThreadBenchmark.java
FetchRequestBenchmark.java		 33:38 (40%)
62:67 (6%)
51:56 (15%)
78:83 (5%)
33:38 (54%)
70:75 (6%)
91:96 (3%)
48:53 (10%)
 view
7 x 8 connect/transforms/src/m...afka/connect/transforms
connect/transforms/src/m...afka/connect/transforms
connect/transforms/src/m...afka/connect/transforms
connect/transforms/src/m...afka/connect/transforms
connect/transforms/src/m...afka/connect/transforms
connect/transforms/src/m...afka/connect/transforms
connect/transforms/src/m...afka/connect/transforms
connect/transforms/src/m...afka/connect/transforms
 MaskField.java
InsertField.java
Flatten.java
HoistField.java
ReplaceField.java
TimestampConverter.java
ExtractField.java
Cast.java		 193:203 (6%)
260:270 (4%)
274:284 (4%)
110:120 (12%)
217:227 (5%)
310:320 (2%)
105:115 (12%)
447:457 (2%)
 view
7 x 8 connect/transforms/src/m...afka/connect/transforms
connect/transforms/src/m...afka/connect/transforms
connect/transforms/src/m...afka/connect/transforms
connect/transforms/src/m...afka/connect/transforms
connect/transforms/src/m...afka/connect/transforms
connect/transforms/src/m...afka/connect/transforms
connect/transforms/src/m...afka/connect/transforms
connect/transforms/src/m...afka/connect/transforms
 MaskField.java
InsertField.java
Flatten.java
HoistField.java
ReplaceField.java
TimestampConverter.java
ExtractField.java
Cast.java		 176:186 (6%)
241:251 (4%)
257:267 (4%)
93:103 (12%)
198:208 (5%)
293:303 (2%)
88:98 (12%)
430:440 (2%)
 view
9 x 7 connect/transforms/src/m...afka/connect/transforms
connect/transforms/src/m...afka/connect/transforms
connect/transforms/src/m...afka/connect/transforms
connect/transforms/src/m...afka/connect/transforms
connect/transforms/src/m...afka/connect/transforms
connect/transforms/src/m...afka/connect/transforms
connect/transforms/src/m...afka/connect/transforms
 Flatten.java
ReplaceField.java
HoistField.java
InsertField.java
Cast.java
TimestampConverter.java
ExtractField.java		 274:286 (5%)
217:229 (7%)
110:122 (15%)
260:272 (6%)
447:459 (3%)
310:322 (3%)
105:117 (16%)
 view
9 x 7 streams/src/main/java/or...e/kafka/streams/kstream
streams/src/main/java/or...e/kafka/streams/kstream
streams/src/main/java/or...e/kafka/streams/kstream
streams/src/main/java/or...e/kafka/streams/kstream
streams/src/main/java/or...e/kafka/streams/kstream
streams/src/main/java/or...e/kafka/streams/kstream
streams/src/main/java/or...e/kafka/streams/kstream
 StreamJoined.java
StreamJoined.java
StreamJoined.java
StreamJoined.java
StreamJoined.java
StreamJoined.java
StreamJoined.java		 150:158 (6%)
172:180 (6%)
189:197 (6%)
206:214 (6%)
223:231 (6%)
243:251 (6%)
263:271 (6%)
 view
6 x 7 streams/src/main/java/or...reams/kstream/internals
streams/src/main/java/or...reams/kstream/internals
streams/src/main/java/or...reams/kstream/internals
streams/src/main/java/or...reams/kstream/internals
streams/src/main/java/or...reams/kstream/internals
streams/src/main/java/or...reams/kstream/internals
streams/src/main/java/or...reams/kstream/internals
 KTableKTableJoinMerger.java
KStreamReduce.java
KTableAggregate.java
KTableTransformValues.java
KTableFilter.java
KStreamAggregate.java
KTableReduce.java		 107:115 (8%)
70:77 (8%)
68:78 (9%)
97:105 (7%)
88:96 (7%)
75:82 (7%)
63:73 (10%)
 view
9 x 7 connect/transforms/src/m...afka/connect/transforms
connect/transforms/src/m...afka/connect/transforms
connect/transforms/src/m...afka/connect/transforms
connect/transforms/src/m...afka/connect/transforms
connect/transforms/src/m...afka/connect/transforms
connect/transforms/src/m...afka/connect/transforms
connect/transforms/src/m...afka/connect/transforms
 Flatten.java
ReplaceField.java
HoistField.java
InsertField.java
Cast.java
TimestampConverter.java
ExtractField.java		 257:269 (5%)
198:210 (7%)
93:105 (15%)
241:253 (6%)
430:442 (3%)
293:305 (3%)
88:100 (16%)
 view
6 x 7 clients/src/main/java/or...e/kafka/common/requests
clients/src/main/java/or...e/kafka/common/requests
clients/src/main/java/or...e/kafka/common/requests
clients/src/main/java/or...e/kafka/common/requests
clients/src/main/java/or...e/kafka/common/requests
clients/src/main/java/or...e/kafka/common/requests
clients/src/main/java/or...e/kafka/common/requests
 DescribeDelegationTokenResponse.java
HeartbeatResponse.java
RenewDelegationTokenResponse.java
EndTxnResponse.java
FindCoordinatorResponse.java
CreateDelegationTokenResponse.java
ExpireDelegationTokenResponse.java		 74:81 (10%)
59:66 (27%)
43:50 (23%)
66:73 (22%)
72:79 (15%)
73:80 (12%)
51:58 (23%)
 view
6 x 6 streams/src/main/java/or...reams/kstream/internals
streams/src/main/java/or...reams/kstream/internals
streams/src/main/java/or...reams/kstream/internals
streams/src/main/java/or...reams/kstream/internals
streams/src/main/java/or...reams/kstream/internals
streams/src/main/java/or...reams/kstream/internals
 TimestampedKeyValueStoreMaterializer....
SlidingWindowedKStreamImpl.java
TimeWindowedCogroupedKStreamImpl.java
SlidingWindowedCogroupedKStreamImpl.java
SessionWindowedCogroupedKStreamImpl.java
TimeWindowedKStreamImpl.java		 47:54 (30%)
220:226 (4%)
167:174 (5%)
131:137 (7%)
139:146 (7%)
266:273 (3%)
 view
10 x 6 streams/src/main/java/or...reams/kstream/internals
streams/src/main/java/or...reams/kstream/internals
streams/src/main/java/or...reams/kstream/internals
streams/src/main/java/or...reams/kstream/internals
streams/src/main/java/or...reams/kstream/internals
streams/src/main/java/or...reams/kstream/internals
 SlidingWindowedKStreamImpl.java
SessionWindowedKStreamImpl.java
SessionWindowedCogroupedKStreamImpl.java
SlidingWindowedCogroupedKStreamImpl.java
TimeWindowedCogroupedKStreamImpl.java
TimeWindowedKStreamImpl.java		 215:226 (7%)
248:260 (5%)
134:146 (12%)
126:137 (12%)
162:174 (9%)
261:273 (6%)
 view
6 x 6 clients/src/main/java/or...curity/token/delegation
connect/runtime/src/main...t/runtime/rest/entities
clients/src/main/java/or...curity/token/delegation
clients/src/main/java/or...he/kafka/common/network
clients/src/main/java/or...che/kafka/common/memory
clients/src/main/java/or...common/header/internals
 TokenInformation.java
ConnectorPluginInfo.java
DelegationToken.java
KafkaChannel.java
GarbageCollectedMemoryPool.java
RecordHeaders.java		 106:112 (8%)
61:67 (13%)
51:57 (18%)
464:470 (2%)
136:142 (7%)
149:155 (5%)
 view
6 x 6 clients/src/main/java/or...e/kafka/common/requests
clients/src/main/java/or...e/kafka/common/requests
clients/src/main/java/or...e/kafka/common/requests
clients/src/main/java/or...e/kafka/common/requests
clients/src/main/java/or...e/kafka/common/requests
clients/src/main/java/or...e/kafka/common/requests
 AddOffsetsToTxnResponse.java
ListPartitionReassignmentsResponse.java
InitProducerIdResponse.java
ListGroupsResponse.java
SyncGroupResponse.java
DescribeClientQuotasResponse.java		 52:59 (26%)
61:68 (25%)
54:61 (24%)
48:55 (27%)
53:60 (21%)
112:119 (10%)
 view
9 x 6 streams/examples/src/mai...reams/examples/pageview
streams/examples/src/mai...ms/examples/temperature
streams/examples/src/mai...eams/examples/wordcount
streams/examples/src/mai...a/streams/examples/pipe
streams/examples/src/mai...eams/examples/wordcount
streams/examples/src/mai...eams/examples/wordcount
 PageViewTypedDemo.java
TemperatureDemo.java
WordCountProcessorDemo.java
PipeDemo.java
WordCountTransformerDemo.java
WordCountDemo.java		 233:243 (7%)
112:122 (21%)
144:154 (13%)
60:70 (37%)
155:165 (12%)
97:107 (21%)
 view