opensearch-project / data-prepper
Duplication

Places in code with 6 or more lines that are exactly the same.

Intro
  • For duplication, we look at places in code where there are 6 or more lines of code that are exactly the same.
  • Before duplication is calculated, the code is cleaned to remove empty lines, comments, and frequently duplicated constructs such as imports.
  • You should aim at having as little as possible (<5%) of duplicated code as high-level of duplication can lead to maintenance difficulties, poor factoring, and logical contradictions.
Learn more...
Duplication Overall
  • 4% duplication:
    • 9,203 cleaned lines of cleaned code (without empty lines, comments, and frequently duplicated constructs such as imports)
    • 427 duplicated lines
  • 41 duplicates
system4% (427 lines)
Duplication per Extension
java3% (324 lines)
yaml12% (52 lines)
yml56% (51 lines)
Duplication per Component (primary)
e2e-test/trace27% (135 lines)
data-prepper-api/src3% (40 lines)
data-prepper-plugins/peer-forwarder5% (37 lines)
deployment-template/k8s26% (32 lines)
data-prepper-plugins/common5% (30 lines)
data-prepper-plugins/otel-trace-source8% (24 lines)
data-prepper-plugins/http-source8% (24 lines)
data-prepper-benchmarks/service-map-stateful-benchmarks20% (24 lines)
deployment-template/ec29% (20 lines)
data-prepper-plugins/otel-trace-group-prepper8% (14 lines)
data-prepper-plugins/armeria-common15% (13 lines)
data-prepper-core/src<1% (13 lines)
data-prepper-plugins/key-value-processor4% (7 lines)
data-prepper-plugins/drop-events-processor43% (7 lines)
data-prepper-plugins/otel-trace-raw-prepper1% (7 lines)
e2e-test/log0% (0 lines)
data-prepper-plugins/aggregate-processor0% (0 lines)
data-prepper-plugins/opensearch0% (0 lines)
data-prepper-plugins/grok-prepper0% (0 lines)
data-prepper-plugins/service-map-stateful0% (0 lines)
data-prepper-plugins/mapdb-prepper-state0% (0 lines)
data-prepper-plugins/blocking-buffer0% (0 lines)
data-prepper-benchmarks/mapdb-benchmarks0% (0 lines)
release/docker0% (0 lines)
deployment-template/eks0% (0 lines)
research/zipkin-opensearch-to-otel0% (0 lines)
data-prepper-logstash-configuration/src0% (0 lines)

Duplication Between Components (50+ lines)

G data-prepper-benchmarks/service-map-stateful-benchmarks data-prepper-benchmarks/service-map-stateful-benchmarks e2e-test/trace e2e-test/trace data-prepper-benchmarks/service-map-stateful-benchmarks--e2e-test/trace 66 data-prepper-plugins/common data-prepper-plugins/common data-prepper-plugins/peer-forwarder data-prepper-plugins/peer-forwarder data-prepper-plugins/common--data-prepper-plugins/peer-forwarder 53

Download: SVG DOT (open online Graphviz editor)

Open 3D force graph...

Show more details on duplication between components...
Longest Duplicates
The list of 20 longest duplicates.
See data for all 41 duplicates...
Size#FoldersFilesLinesCode
24 x 2 data-prepper-benchmarks/...pper/benchmarks/prepper
e2e-test/trace/src/integ...epper/integration/trace
ServiceMapStatefulPrepperBenchmarks.java
EndToEndServiceMapTest.java
165:188 (20%)
156:179 (14%)
view
18 x 2 e2e-test/trace/src/integ...epper/integration/trace
e2e-test/trace/src/integ...epper/integration/trace
EndToEndRawSpanTest.java
EndToEndServiceMapTest.java
177:194 (9%)
157:174 (10%)
view
18 x 2 data-prepper-benchmarks/...pper/benchmarks/prepper
e2e-test/trace/src/integ...epper/integration/trace
ServiceMapStatefulPrepperBenchmarks.java
EndToEndRawSpanTest.java
166:183 (15%)
177:194 (9%)
view
13 x 2 data-prepper-plugins/htt.../plugins/source/loghttp
data-prepper-plugins/ote...lugins/source/oteltrace
HTTPSource.java
OTelTraceSource.java
131:144 (15%)
144:157 (12%)
view
13 x 2 data-prepper-core/src/ma...prepper/pipeline/server
data-prepper-plugins/arm.../armeria/authentication
HttpBasicAuthenticationConfig.java
HttpBasicAuthenticationConfig.java
16:33 (100%)
16:33 (100%)
view
12 x 2 deployment-template/ec2
deployment-template/k8s
data-prepper-ec2-deployment-cfn.yaml
data-prepper-k8s.yaml
120:131 (5%)
20:31 (10%)
view
11 x 2 e2e-test/trace/src/integrationTest/resources
e2e-test/trace/src/integrationTest/resources
raw-span-e2e-pipeline-latest-release.yml
service-map-e2e-pipeline.yml
1:11 (42%)
1:11 (44%)
view
11 x 2 data-prepper-plugins/htt.../plugins/source/loghttp
data-prepper-plugins/ote...lugins/source/oteltrace
HTTPSource.java
OTelTraceSource.java
113:126 (12%)
127:139 (10%)
view
10 x 2 e2e-test/trace/src/integrationTest/resources
e2e-test/trace/src/integrationTest/resources
raw-span-e2e-pipeline-latest-release.yml
raw-span-e2e-pipeline.yml
9:18 (38%)
4:13 (40%)
view
9 x 2 e2e-test/trace/src/integ...epper/integration/trace
e2e-test/trace/src/integ...epper/integration/trace
EndToEndRawSpanTest.java
EndToEndServiceMapTest.java
142:154 (4%)
122:134 (5%)
view
9 x 2 data-prepper-plugins/com...plugins/certificate/acm
data-prepper-plugins/pee...rwarder/certificate/acm
ACMCertificateProvider.java
ACMCertificateProvider.java
88:98 (9%)
45:55 (27%)
view
8 x 2 data-prepper-plugins/htt.../plugins/source/loghttp
data-prepper-plugins/ote...lugins/source/oteltrace
HTTPSource.java
OTelTraceSource.java
118:126 (9%)
149:157 (7%)
view
8 x 2 data-prepper-plugins/ote...lugins/source/oteltrace
data-prepper-plugins/ote...lugins/source/oteltrace
OTelTraceSource.java
OTelTraceSource.java
131:139 (7%)
149:157 (7%)
view
8 x 2 deployment-template/ec2
deployment-template/k8s
data-prepper-ec2-deployment-cfn.yaml
data-prepper-k8s.yaml
133:140 (3%)
33:40 (6%)
view
8 x 2 data-prepper-plugins/htt.../plugins/source/loghttp
data-prepper-plugins/ote...lugins/source/oteltrace
HTTPSource.java
OTelTraceSource.java
136:144 (9%)
131:139 (7%)
view
8 x 2 data-prepper-plugins/htt.../plugins/source/loghttp
data-prepper-plugins/htt.../plugins/source/loghttp
HTTPSource.java
HTTPSource.java
118:126 (9%)
136:144 (9%)
view
7 x 2 data-prepper-plugins/com...prepper/plugins/prepper
data-prepper-plugins/ote.../prepper/oteltracegroup
StringPrepper.java
OTelTraceGroupPrepper.java
98:110 (14%)
182:193 (5%)
view
7 x 2 data-prepper-plugins/key...gins/processor/keyvalue
data-prepper-plugins/ote.../prepper/oteltracegroup
KeyValueProcessor.java
OTelTraceGroupPrepper.java
179:189 (6%)
182:193 (5%)
view
7 x 2 data-prepper-plugins/dro.../plugins/processor/drop
data-prepper-plugins/ote.../prepper/oteltracegroup
DropEventsProcessor.java
OTelTraceGroupPrepper.java
35:45 (43%)
182:193 (5%)
view
7 x 2 data-prepper-plugins/com...prepper/plugins/prepper
data-prepper-plugins/dro.../plugins/processor/drop
NoOpPrepper.java
DropEventsProcessor.java
27:39 (53%)
35:45 (43%)
view
Duplicated Units
The list of top 2 duplicated units.
See data for all 2 unit duplicates...
Size#FoldersFilesLinesCode
16 x 2 data-prepper-plugins/ote...lugins/source/oteltrace
data-prepper-plugins/htt.../plugins/source/loghttp
OTelTraceSource.java
HTTPSource.java
145:161 
132:148 
view
8 x 2 data-prepper-plugins/pee...orwarder/certificate/s3
data-prepper-plugins/com.../plugins/certificate/s3
S3CertificateProvider.java
S3CertificateProvider.java
38:48 
43:53 
view