apache / pulsar-connectorsDuplication

Intro

For duplication, we look at places in code where there are 6 or more lines of code that are exactly the same.
Before duplication is calculated, the code is cleaned to remove empty lines, comments, and frequently duplicated constructs such as imports.
You should aim at having as little as possible (<5%) of duplicated code as high-level of duplication can lead to maintenance difficulties, poor factoring, and logical contradictions.

Learn more...

Duplication Overall

15% duplication:

7,584 cleaned lines of cleaned code (without empty lines, comments, and frequently duplicated constructs such as imports)
1,202 duplicated lines

40 duplicates

dependency graphs: 2D graph | 3D graph | 3D graph (with duplicates)...

Duplication per Extension

Duplication per Component (primary)

Longest Duplicates

The list of 40 longest duplicates.

See data for all 40 duplicates...

Size	#	Folders	Files	Lines	Code
108	x 2	pulsar-io/hdfs2/src/main.../apache/pulsar/io/hdfs2 pulsar-io/hdfs3/src/main.../apache/pulsar/io/hdfs3	AbstractHdfsConnector.java AbstractHdfsConnector.java	51:248 (100%) 51:248 (100%)	view
35	x 2	pulsar-io/hdfs2/src/main...ulsar/io/hdfs2/sink/seq pulsar-io/hdfs3/src/main...ulsar/io/hdfs3/sink/seq	HdfsAbstractSequenceFileSink.java HdfsAbstractSequenceFileSink.java	45:94 (100%) 45:94 (100%)	view
30	x 2	pulsar-io/dynamodb/src/m...ache/pulsar/io/dynamodb pulsar-io/kinesis/src/ma...pache/pulsar/io/kinesis	DynamoDBSourceConfig.java KinesisSourceConfig.java	87:118 (20%) 52:83 (25%)	view
29	x 2	pulsar-io/hdfs2/src/main...he/pulsar/io/hdfs2/sink pulsar-io/hdfs3/src/main...he/pulsar/io/hdfs3/sink	HdfsSyncThread.java HdfsSyncThread.java	34:77 (100%) 34:77 (100%)	view
29	x 2	pulsar-io/dynamodb/src/m...ache/pulsar/io/dynamodb pulsar-io/kinesis/src/ma...pache/pulsar/io/kinesis	DynamoDBSourceConfig.java KinesisSourceConfig.java	137:169 (19%) 102:134 (24%)	view
26	x 2	pulsar-io/hdfs2/src/main...lsar/io/hdfs2/sink/text pulsar-io/hdfs3/src/main...lsar/io/hdfs3/sink/text	HdfsAbstractTextFileSink.java HdfsAbstractTextFileSink.java	39:76 (100%) 39:76 (100%)	view
22	x 2	pulsar-io/hdfs2/src/main...ulsar/io/hdfs2/sink/seq pulsar-io/hdfs3/src/main...ulsar/io/hdfs3/sink/seq	HdfsSequentialTextSink.java HdfsSequentialTextSink.java	39:69 (100%) 39:69 (100%)	view
21	x 2	pulsar-io/dynamodb/src/m...ache/pulsar/io/dynamodb pulsar-io/kinesis/src/ma...pache/pulsar/io/kinesis	StreamsRecordProcessor.java KinesisRecordProcessor.java	55:85 (41%) 53:83 (36%)	view
18	x 2	pulsar-io/hdfs2/src/main.../apache/pulsar/io/hdfs2 pulsar-io/hdfs3/src/main.../apache/pulsar/io/hdfs3	SecurityUtil.java SecurityUtil.java	31:89 (100%) 31:89 (100%)	view
16	x 2	pulsar-io/hdfs2/src/main...he/pulsar/io/hdfs2/sink pulsar-io/hdfs3/src/main...he/pulsar/io/hdfs3/sink	HdfsAbstractSink.java HdfsAbstractSink.java	67:88 (28%) 62:83 (30%)	view
16	x 2	pulsar-io/hdfs2/src/main.../apache/pulsar/io/hdfs2 pulsar-io/hdfs3/src/main.../apache/pulsar/io/hdfs3	AbstractHdfsConfig.java AbstractHdfsConfig.java	31:75 (100%) 31:75 (100%)	view
16	x 2	pulsar-io/hdfs2/src/main...he/pulsar/io/hdfs2/sink pulsar-io/hdfs3/src/main...he/pulsar/io/hdfs3/sink	HdfsSinkConfig.java HdfsSinkConfig.java	86:109 (50%) 76:99 (61%)	view
15	x 2	pulsar-io/hdfs2/src/main...ulsar/io/hdfs2/sink/seq pulsar-io/hdfs3/src/main...ulsar/io/hdfs3/sink/seq	HdfsTextSink.java HdfsTextSink.java	33:52 (100%) 33:52 (100%)	view
14	x 2	pulsar-io/influxdb/src/m...e/pulsar/io/influxdb/v1 pulsar-io/influxdb/src/m...e/pulsar/io/influxdb/v2	InfluxDBSinkConfig.java InfluxDBSinkConfig.java	102:120 (19%) 96:114 (20%)	view
14	x 2	pulsar-io/hdfs2/src/main.../apache/pulsar/io/hdfs2 pulsar-io/hdfs3/src/main.../apache/pulsar/io/hdfs3	HdfsResources.java HdfsResources.java	28:49 (100%) 28:49 (100%)	view
12	x 2	pulsar-io/kafka/src/main.../apache/pulsar/io/kafka pulsar-io/kafka/src/main.../apache/pulsar/io/kafka	KafkaSinkConfig.java KafkaSourceConfig.java	37:49 (21%) 37:49 (16%)	view
11	x 2	pulsar-io/dynamodb/src/m...ache/pulsar/io/dynamodb pulsar-io/kinesis/src/ma...pache/pulsar/io/kinesis	DynamoDBSourceConfig.java BaseKinesisConfig.java	55:67 (7%) 38:50 (30%)	view
11	x 2	pulsar-io/influxdb/src/m...e/pulsar/io/influxdb/v1 pulsar-io/influxdb/src/m...e/pulsar/io/influxdb/v2	InfluxDBSinkConfig.java InfluxDBSinkConfig.java	36:49 (15%) 37:50 (16%)	view
10	x 2	pulsar-io/flume/src/main...he/pulsar/io/flume/node pulsar-io/flume/src/main...he/pulsar/io/flume/node	AbstractConfigurationProvider.java AbstractConfigurationProvider.java	300:310 (2%) 346:356 (2%)	view
10	x 2	pulsar-io/hdfs2/src/main...he/pulsar/io/hdfs2/sink pulsar-io/hdfs3/src/main...he/pulsar/io/hdfs3/sink	HdfsSinkConfig.java HdfsSinkConfig.java	41:76 (31%) 39:74 (38%)	view
10	x 2	pulsar-io/file/src/main/...g/apache/pulsar/io/file pulsar-io/kinesis/src/ma...pache/pulsar/io/kinesis	FileRecord.java KinesisRecord.java	58:73 (41%) 74:89 (26%)	view
9	x 2	pulsar-io/twitter/src/ma.../pulsar/io/twitter/data pulsar-io/twitter/src/ma.../pulsar/io/twitter/data	TweetData.java TweetData.java	37:45 (10%) 101:109 (10%)	view
9	x 2	pulsar-io/influxdb/src/m...e/pulsar/io/influxdb/v1 pulsar-io/influxdb/src/m...e/pulsar/io/influxdb/v2	InfluxDBSinkConfig.java InfluxDBSinkConfig.java	90:99 (12%) 84:93 (13%)	view
8	x 2	pulsar-io/hdfs2/src/main...he/pulsar/io/hdfs2/sink pulsar-io/hdfs3/src/main...he/pulsar/io/hdfs3/sink	HdfsAbstractSink.java HdfsAbstractSink.java	98:107 (14%) 99:108 (15%)	view
8	x 2	pulsar-io/hdfs2/src/main...he/pulsar/io/hdfs2/sink pulsar-io/hdfs3/src/main...he/pulsar/io/hdfs3/sink	HdfsAbstractSink.java HdfsAbstractSink.java	55:63 (14%) 53:61 (15%)	view
7	x 2	pulsar-io/dynamodb/src/m...ache/pulsar/io/dynamodb pulsar-io/kinesis/src/ma...pache/pulsar/io/kinesis	DynamoDBSource.java KinesisSource.java	124:131 (10%) 133:140 (9%)	view
7	x 2	pulsar-io/dynamodb/src/m...ache/pulsar/io/dynamodb pulsar-io/kinesis/src/ma...pache/pulsar/io/kinesis	DynamoDBSourceConfig.java BaseKinesisConfig.java	78:85 (4%) 61:68 (19%)	view
7	x 2	pulsar-io/kafka/src/main.../apache/pulsar/io/kafka pulsar-io/kafka/src/main.../apache/pulsar/io/kafka	KafkaSinkConfig.java KafkaSourceConfig.java	63:69 (12%) 79:85 (9%)	view
7	x 2	pulsar-io/twitter/src/ma.../pulsar/io/twitter/data pulsar-io/twitter/src/ma.../pulsar/io/twitter/data	TweetData.java TweetData.java	29:35 (8%) 94:100 (8%)	view
7	x 2	pulsar-io/influxdb/src/m...e/pulsar/io/influxdb/v1 pulsar-io/influxdb/src/m...e/pulsar/io/influxdb/v2	InfluxDBSinkConfig.java InfluxDBSinkConfig.java	78:85 (9%) 78:85 (10%)	view
6	x 2	pulsar-io/hdfs2/src/main...he/pulsar/io/hdfs2/sink pulsar-io/hdfs3/src/main...he/pulsar/io/hdfs3/sink	HdfsAbstractSink.java HdfsAbstractSink.java	46:52 (10%) 45:51 (11%)	view
6	x 2	pulsar-io/canal/src/main.../apache/pulsar/io/canal pulsar-io/flume/src/main.../pulsar/io/flume/source	CanalAbstractSource.java AbstractSource.java	156:163 (6%) 119:126 (10%)	view
6	x 2	pulsar-io/flume/src/main...he/pulsar/io/flume/node pulsar-io/flume/src/main...he/pulsar/io/flume/node	AbstractConfigurationProvider.java AbstractConfigurationProvider.java	427:433 (1%) 458:464 (1%)	view
6	x 2	pulsar-io/canal/src/main.../apache/pulsar/io/canal pulsar-io/flume/src/main.../pulsar/io/flume/source	CanalAbstractSource.java AbstractSource.java	89:95 (6%) 76:82 (10%)	view
6	x 2	pulsar-io/debezium/core/...ache/pulsar/io/debezium pulsar-io/debezium/core/...ache/pulsar/io/debezium	PulsarDatabaseHistory.java PulsarDatabaseHistory.java	194:199 (3%) 241:246 (3%)	view
6	x 2	pulsar-io/dynamodb/src/m...ache/pulsar/io/dynamodb pulsar-io/kinesis/src/ma...pache/pulsar/io/kinesis	DynamoDBSourceConfig.java KinesisSourceConfig.java	222:230 (4%) 190:198 (5%)	view
6	x 2	pulsar-io/rabbitmq/src/m...ache/pulsar/io/rabbitmq pulsar-io/rabbitmq/src/m...ache/pulsar/io/rabbitmq	RabbitMQAbstractConfig.java RabbitMQSourceConfig.java	79:85 (7%) 43:49 (19%)	view
6	x 2	pulsar-io/file/src/main/...he/pulsar/io/file/utils pulsar-io/file/src/main/...he/pulsar/io/file/utils	GZipFiles.java ZipFiles.java	83:90 (21%) 72:79 (28%)	view
6	x 2	pulsar-io/dynamodb/src/m...ache/pulsar/io/dynamodb pulsar-io/kinesis/src/ma...pache/pulsar/io/kinesis	DynamoDBSourceConfig.java KinesisSourceConfig.java	129:135 (4%) 94:100 (5%)	view
6	x 2	pulsar-io/elastic-search...pulsar/io/elasticsearch pulsar-io/influxdb/src/m...e/pulsar/io/influxdb/v1	ElasticSearchConfig.java InfluxDBSinkConfig.java	83:89 (9%) 54:60 (8%)	view

Duplicated Units

The list of top 17 duplicated units.

See data for all 17 unit duplicates...

Size	#	Folders	Files	Lines	Code
25	x 2	pulsar-io/hdfs2/src/main.../apache/pulsar/io/hdfs2 pulsar-io/hdfs3/src/main.../apache/pulsar/io/hdfs3	AbstractHdfsConnector.java AbstractHdfsConnector.java	219:250 219:250	view
23	x 2	pulsar-io/kinesis/src/ma...pache/pulsar/io/kinesis pulsar-io/dynamodb/src/m...ache/pulsar/io/dynamodb	KinesisRecordProcessor.java StreamsRecordProcessor.java	52:80 54:82	view
22	x 2	pulsar-io/hdfs2/src/main.../apache/pulsar/io/hdfs2 pulsar-io/hdfs3/src/main.../apache/pulsar/io/hdfs3	AbstractHdfsConnector.java AbstractHdfsConnector.java	67:100 67:100	view
21	x 2	pulsar-io/hdfs2/src/main.../apache/pulsar/io/hdfs2 pulsar-io/hdfs3/src/main.../apache/pulsar/io/hdfs3	AbstractHdfsConnector.java AbstractHdfsConnector.java	102:127 102:127	view
15	x 2	pulsar-io/hdfs2/src/main.../apache/pulsar/io/hdfs2 pulsar-io/hdfs3/src/main.../apache/pulsar/io/hdfs3	AbstractHdfsConnector.java AbstractHdfsConnector.java	132:148 132:148	view
11	x 2	pulsar-io/hdfs2/src/main...lsar/io/hdfs2/sink/text pulsar-io/hdfs3/src/main...lsar/io/hdfs3/sink/text	HdfsAbstractTextFileSink.java HdfsAbstractTextFileSink.java	57:70 57:70	view
10	x 2	pulsar-io/hdfs2/src/main...he/pulsar/io/hdfs2/sink pulsar-io/hdfs3/src/main...he/pulsar/io/hdfs3/sink	HdfsSyncThread.java HdfsSyncThread.java	48:59 48:59	view
10	x 2	pulsar-io/hdfs2/src/main...he/pulsar/io/hdfs2/sink pulsar-io/hdfs3/src/main...he/pulsar/io/hdfs3/sink	HdfsAbstractSink.java HdfsAbstractSink.java	78:90 73:85	view
9	x 2	pulsar-io/kinesis/src/ma...pache/pulsar/io/kinesis pulsar-io/dynamodb/src/m...ache/pulsar/io/dynamodb	KinesisSource.java DynamoDBSource.java	132:142 123:133	view
9	x 2	pulsar-io/hdfs2/src/main...he/pulsar/io/hdfs2/sink pulsar-io/hdfs3/src/main...he/pulsar/io/hdfs3/sink	HdfsSyncThread.java HdfsSyncThread.java	66:79 66:79	view
9	x 2	pulsar-io/hdfs2/src/main...ulsar/io/hdfs2/sink/seq pulsar-io/hdfs3/src/main...ulsar/io/hdfs3/sink/seq	HdfsAbstractSequenceFileSink.java HdfsAbstractSequenceFileSink.java	68:78 68:78	view
7	x 2	pulsar-io/hdfs2/src/main.../apache/pulsar/io/hdfs2 pulsar-io/hdfs3/src/main.../apache/pulsar/io/hdfs3	SecurityUtil.java SecurityUtil.java	48:57 48:57	view
7	x 2	pulsar-io/hdfs2/src/main.../apache/pulsar/io/hdfs2 pulsar-io/hdfs3/src/main.../apache/pulsar/io/hdfs3	AbstractHdfsConfig.java AbstractHdfsConfig.java	68:77 68:77	view
6	x 2	pulsar-io/kinesis/src/ma...pache/pulsar/io/kinesis pulsar-io/dynamodb/src/m...ache/pulsar/io/dynamodb	KinesisSourceConfig.java DynamoDBSourceConfig.java	193:200 225:232	view
6	x 2	pulsar-io/hdfs2/src/main...ulsar/io/hdfs2/sink/seq pulsar-io/hdfs3/src/main...ulsar/io/hdfs3/sink/seq	HdfsAbstractSequenceFileSink.java HdfsAbstractSequenceFileSink.java	87:95 87:95	view
6	x 2	pulsar-io/hdfs2/src/main.../apache/pulsar/io/hdfs2 pulsar-io/hdfs3/src/main.../apache/pulsar/io/hdfs3	AbstractHdfsConnector.java AbstractHdfsConnector.java	189:198 189:198	view
6	x 2	pulsar-io/file/src/main/...he/pulsar/io/file/utils pulsar-io/file/src/main/...he/pulsar/io/file/utils	GZipFiles.java ZipFiles.java	86:94 75:83	view