apache / celeborn
Duplication

Places in code with 6 or more lines that are exactly the same.

Intro
Learn more...
Duplication Overall
system28% (24,287 lines)
dependency graphs: 2D graph | 3D graph | 3D graph (with duplicates)...
Duplication per Extension
java32% (13,174 lines)
scala21% (7,193 lines)
proto89% (1,613 lines)
yaml58% (1,591 lines)
vue32% (368 lines)
cpp6% (118 lines)
sql97% (98 lines)
tpl25% (74 lines)
h3% (58 lines)
Duplication per Component (primary)
client-spark79% (7,284 lines)
openapi63% (5,819 lines)
client-flink51% (3,949 lines)
common8% (1,793 lines)
cpp22% (960 lines)
client13% (911 lines)
worker6% (630 lines)
client-tez21% (623 lines)
charts41% (514 lines)
master9% (491 lines)
service15% (420 lines)
web22% (368 lines)
project17% (282 lines)
client-mr22% (159 lines)
multipart-uploader31% (66 lines)
toolkit8% (18 lines)
spi0% (0 lines)
build0% (0 lines)
dev0% (0 lines)
ROOT0% (0 lines)
cli0% (0 lines)

Duplication Between Components (50+ lines)

G common common cpp cpp common--cpp 1652 master master common--master 193 worker worker common--worker 66 client-mr client-mr client-tez client-tez client-mr--client-tez 318 master--worker 211 service service master--service 84 client-flink client-flink client-flink--common 84 client-spark client-spark client-flink--client-spark 56

Download: SVG DOT (open online Graphviz editor)

Open 2D force graph... Open 3D force graph...

Show more details on duplication between components...
Longest Duplicates
The list of 50 longest duplicates.
See data for all 8,046 duplicates...
Size#FoldersFilesLinesCode
784 x 2 common/src/main/proto
cpp/celeborn/proto
1:908 (100%)
1:908 (100%)
view
419 x 2 openapi/openapi-client/s.../rest/v1/master/invoker
openapi/openapi-client/s.../rest/v1/worker/invoker
96:1025 (98%)
96:1025 (98%)
view
350 x 2 client-flink/flink-1.20/...orn/plugin/flink/tiered
client-flink/flink-2.0/s...orn/plugin/flink/tiered
68:552 (100%)
68:552 (100%)
view
304 x 2 client-flink/flink-1.20/...orn/plugin/flink/tiered
client-flink/flink-2.0/s...orn/plugin/flink/tiered
61:492 (100%)
61:492 (100%)
view
292 x 2 client-spark/spark-3.5-c.../sql/execution/columnar
client-spark/spark-4-col.../sql/execution/columnar
31:430 (100%)
31:430 (100%)
view
241 x 2 client-spark/spark-3.5-c.../sql/execution/columnar
client-spark/spark-4-col.../sql/execution/columnar
31:287 (100%)
31:287 (100%)
view
241 x 2 client-spark/spark-3-col.../sql/execution/columnar
client-spark/spark-4-col.../sql/execution/columnar
31:287 (100%)
31:287 (100%)
view
241 x 2 client-spark/spark-3-col.../sql/execution/columnar
client-spark/spark-3.5-c.../sql/execution/columnar
31:287 (100%)
31:287 (100%)
view
234 x 2 client-spark/spark-3.5-c.../sql/execution/columnar
client-spark/spark-4-col.../sql/execution/columnar
27:356 (95%)
27:356 (95%)
view
210 x 2 client-spark/spark-3.5-c.../sql/execution/columnar
client-spark/spark-4-col.../sql/execution/columnar
41:440 (61%)
42:441 (61%)
view
202 x 2 client-flink/flink-1.20/...orn/plugin/flink/tiered
client-flink/flink-2.0/s...orn/plugin/flink/tiered
52:335 (100%)
52:335 (100%)
view
179 x 2 client-spark/spark-3-col.../sql/execution/columnar
client-spark/spark-4-col.../sql/execution/columnar
98:338 (73%)
99:339 (73%)
view
179 x 2 client-spark/spark-3-col.../sql/execution/columnar
client-spark/spark-3.5-c.../sql/execution/columnar
98:338 (73%)
99:339 (73%)
view
165 x 2 openapi/openapi-client/src/main/openapi3
openapi/openapi-client/src/main/openapi3
475:646 (14%)
205:376 (24%)
view
162 x 2 client-spark/spark-3-col.../sql/execution/columnar
client-spark/spark-4-col.../sql/execution/columnar
30:280 (100%)
30:280 (100%)
view
162 x 2 client-spark/spark-3-col.../sql/execution/columnar
client-spark/spark-3.5-c.../sql/execution/columnar
30:280 (100%)
30:280 (100%)
view
162 x 2 client-spark/spark-3.5-c.../sql/execution/columnar
client-spark/spark-4-col.../sql/execution/columnar
30:280 (100%)
30:280 (100%)
view
130 x 2 client-spark/spark-3.5-c.../sql/execution/columnar
client-spark/spark-4-col.../sql/execution/columnar
442:656 (38%)
445:660 (37%)
view
122 x 2 client-spark/spark-3-col.../sql/execution/columnar
client-spark/spark-4-col.../sql/execution/columnar
59:215 (42%)
61:217 (41%)
view
122 x 2 client-spark/spark-3-col.../sql/execution/columnar
client-spark/spark-3.5-c.../sql/execution/columnar
59:215 (42%)
61:217 (41%)
view
117 x 2 openapi/openapi-client/s...celeborn/rest/v1/master
openapi/openapi-client/s...celeborn/rest/v1/worker
40:231 (100%)
40:231 (100%)
view
116 x 2 openapi/openapi-client/s...celeborn/rest/v1/master
openapi/openapi-client/s...celeborn/rest/v1/worker
39:231 (100%)
39:231 (100%)
view
113 x 2 openapi/openapi-client/s...celeborn/rest/v1/master
openapi/openapi-client/s...celeborn/rest/v1/worker
39:222 (100%)
39:222 (100%)
view
97 x 2 openapi/openapi-client/src/main/openapi3
openapi/openapi-client/src/main/openapi3
802:900 (8%)
497:595 (14%)
view
95 x 2 client-spark/spark-3.5-c.../sql/execution/columnar
client-spark/spark-4-col.../sql/execution/columnar
35:173 (100%)
35:173 (100%)
view
95 x 2 client-spark/spark-3-col.../sql/execution/columnar
client-spark/spark-3.5-c.../sql/execution/columnar
35:173 (100%)
35:173 (100%)
view
95 x 2 client-spark/spark-3-col.../sql/execution/columnar
client-spark/spark-4-col.../sql/execution/columnar
35:173 (100%)
35:173 (100%)
view
92 x 2 client-spark/spark-3.5-c.../spark/shuffle/celeborn
client-spark/spark-4-col.../spark/shuffle/celeborn
43:161 (100%)
43:161 (100%)
view
92 x 2 client-spark/spark-3-col.../spark/shuffle/celeborn
client-spark/spark-4-col.../spark/shuffle/celeborn
43:161 (100%)
43:161 (100%)
view
92 x 2 client-spark/spark-3-col.../spark/shuffle/celeborn
client-spark/spark-3.5-c.../spark/shuffle/celeborn
43:161 (100%)
43:161 (100%)
view
89 x 2 openapi/openapi-client/src/main/openapi3
openapi/openapi-client/src/main/openapi3
20:112 (8%)
20:112 (12%)
view
89 x 2 client-flink/flink-1.19/...e/celeborn/plugin/flink
client-flink/flink-1.20/...e/celeborn/plugin/flink
59:178 (100%)
59:178 (100%)
view
85 x 2 client-spark/spark-3.5-c.../sql/execution/columnar
client-spark/spark-4-col.../sql/execution/columnar
27:148 (100%)
27:148 (100%)
view
82 x 2 client-flink/flink-1.20/...orn/plugin/flink/tiered
client-flink/flink-2.0/s...orn/plugin/flink/tiered
37:163 (100%)
37:163 (100%)
view
74 x 2 client-flink/flink-1.20/...orn/plugin/flink/tiered
client-flink/flink-2.0/s...orn/plugin/flink/tiered
123:217 (63%)
123:217 (63%)
view
71 x 2 client-spark/spark-3.5-c.../sql/execution/columnar
client-spark/spark-4-col.../sql/execution/columnar
27:113 (100%)
27:113 (100%)
view
70 x 2 openapi/openapi-client/s...celeborn/rest/v1/master
openapi/openapi-client/s...celeborn/rest/v1/worker
55:154 (92%)
123:222 (61%)
view
67 x 2 openapi/openapi-client/s...celeborn/rest/v1/master
openapi/openapi-client/s...celeborn/rest/v1/worker
282:363 (35%)
150:231 (57%)
view
67 x 2 openapi/openapi-client/s...celeborn/rest/v1/master
openapi/openapi-client/s...celeborn/rest/v1/master
150:231 (57%)
692:773 (16%)
view
67 x 2 openapi/openapi-client/s...celeborn/rest/v1/master
openapi/openapi-client/s...celeborn/rest/v1/master
282:363 (35%)
692:773 (16%)
view
67 x 2 openapi/openapi-client/s...celeborn/rest/v1/master
openapi/openapi-client/s...celeborn/rest/v1/worker
420:501 (25%)
212:293 (44%)
view
67 x 2 openapi/openapi-client/s...celeborn/rest/v1/worker
openapi/openapi-client/s...celeborn/rest/v1/worker
150:231 (57%)
212:293 (44%)
view
67 x 2 openapi/openapi-client/s...celeborn/rest/v1/master
openapi/openapi-client/s...celeborn/rest/v1/worker
150:231 (57%)
212:293 (44%)
view
67 x 2 openapi/openapi-client/s...celeborn/rest/v1/master
openapi/openapi-client/s...celeborn/rest/v1/worker
282:363 (35%)
212:293 (44%)
view
67 x 2 openapi/openapi-client/s...celeborn/rest/v1/master
openapi/openapi-client/s...celeborn/rest/v1/master
150:231 (57%)
420:501 (25%)
view
67 x 2 openapi/openapi-client/s...celeborn/rest/v1/master
openapi/openapi-client/s...celeborn/rest/v1/worker
420:501 (25%)
150:231 (57%)
view
67 x 2 openapi/openapi-client/s...celeborn/rest/v1/master
openapi/openapi-client/s...celeborn/rest/v1/master
282:363 (35%)
420:501 (25%)
view
67 x 2 openapi/openapi-client/s...celeborn/rest/v1/master
openapi/openapi-client/s...celeborn/rest/v1/master
282:363 (35%)
150:231 (57%)
view
67 x 2 openapi/openapi-client/s...celeborn/rest/v1/master
openapi/openapi-client/s...celeborn/rest/v1/worker
692:773 (16%)
150:231 (57%)
view
67 x 2 openapi/openapi-client/s...celeborn/rest/v1/master
openapi/openapi-client/s...celeborn/rest/v1/worker
692:773 (16%)
212:293 (44%)
view
Duplicated Units
The list of top 50 duplicated units.
See data for all 176 unit duplicates...
Size#FoldersFilesLinesCode
183 x 3 client-spark/spark-3-col.../sql/execution/columnar
client-spark/spark-4-col.../sql/execution/columnar
client-spark/spark-3.5-c.../sql/execution/columnar
94:278 
94:278 
94:278 
view
64 x 3 client-spark/spark-3-col.../sql/execution/columnar
client-spark/spark-4-col.../sql/execution/columnar
client-spark/spark-3.5-c.../sql/execution/columnar
93:167 
93:167 
93:167 
view
57 x 2 openapi/openapi-client/s.../rest/v1/worker/invoker
openapi/openapi-client/s.../rest/v1/master/invoker
944:1011 
944:1011 
view
48 x 3 client-spark/spark-3-col.../sql/execution/columnar
client-spark/spark-4-col.../sql/execution/columnar
client-spark/spark-3.5-c.../sql/execution/columnar
35:88 
35:88 
35:88 
view
41 x 2 openapi/openapi-client/s.../rest/v1/worker/invoker
openapi/openapi-client/s.../rest/v1/master/invoker
679:722 
679:722 
view
40 x 2 client-flink/flink-2.0/s...orn/plugin/flink/tiered
client-flink/flink-1.20/...orn/plugin/flink/tiered
270:312 
270:312 
view
38 x 2 openapi/openapi-client/s.../rest/v1/worker/invoker
openapi/openapi-client/s.../rest/v1/master/invoker
842:889 
842:889 
view
37 x 2 openapi/openapi-client/s...celeborn/rest/v1/worker
openapi/openapi-client/s...celeborn/rest/v1/master
141:191 
141:191 
view
36 x 2 client-flink/flink-2.0/s...orn/plugin/flink/tiered
client-flink/flink-1.20/...orn/plugin/flink/tiered
325:368 
325:368 
view
36 x 2 client-flink/flink-2.0/s...orn/plugin/flink/tiered
client-flink/flink-1.20/...orn/plugin/flink/tiered
226:267 
226:267 
view
36 x 2 openapi/openapi-client/s...celeborn/rest/v1/worker
openapi/openapi-client/s...celeborn/rest/v1/master
73:122 
73:122 
view
35 x 2 openapi/openapi-client/s...celeborn/rest/v1/worker
openapi/openapi-client/s...celeborn/rest/v1/master
144:191 
144:191 
view
34 x 12 openapi/openapi-client/s...celeborn/rest/v1/worker
openapi/openapi-client/s...celeborn/rest/v1/worker
openapi/openapi-client/s...celeborn/rest/v1/worker
openapi/openapi-client/s...celeborn/rest/v1/worker
openapi/openapi-client/s...celeborn/rest/v1/worker
openapi/openapi-client/s...celeborn/rest/v1/master
openapi/openapi-client/s...celeborn/rest/v1/master
openapi/openapi-client/s...celeborn/rest/v1/master
openapi/openapi-client/s...celeborn/rest/v1/master
openapi/openapi-client/s...celeborn/rest/v1/master
...
67:114 
68:115 
68:115 
68:115 
70:117 
140:187 
68:115 
67:114 
67:114 
68:115 
...
view
33 x 2 client-flink/flink-2.0/s...orn/plugin/flink/tiered
client-flink/flink-1.20/...orn/plugin/flink/tiered
86:122 
86:122 
view
33 x 3 client-spark/spark-3-col.../spark/shuffle/celeborn
client-spark/spark-4-col.../spark/shuffle/celeborn
client-spark/spark-3.5-c.../spark/shuffle/celeborn
95:132 
95:132 
95:132 
view
32 x 2 client-spark/spark-4-col.../sql/execution/columnar
client-spark/spark-3.5-c.../sql/execution/columnar
399:432 
399:432 
view
32 x 6 openapi/openapi-client/s...celeborn/rest/v1/worker
openapi/openapi-client/s...celeborn/rest/v1/worker
openapi/openapi-client/s...celeborn/rest/v1/master
openapi/openapi-client/s...celeborn/rest/v1/master
openapi/openapi-client/s...celeborn/rest/v1/master
openapi/openapi-client/s...celeborn/rest/v1/master
194:232 
256:294 
326:364 
194:232 
464:502 
736:774 
view
31 x 2 client-flink/flink-2.0/s...orn/plugin/flink/tiered
client-flink/flink-1.20/...orn/plugin/flink/tiered
159:196 
159:196 
view
31 x 3 client-spark/spark-3-col.../sql/execution/columnar
client-spark/spark-4-col.../sql/execution/columnar
client-spark/spark-3.5-c.../sql/execution/columnar
168:200 
170:202 
170:202 
view
31 x 8 openapi/openapi-client/s...celeborn/rest/v1/worker
openapi/openapi-client/s...celeborn/rest/v1/worker
openapi/openapi-client/s...celeborn/rest/v1/worker
openapi/openapi-client/s...celeborn/rest/v1/worker
openapi/openapi-client/s...celeborn/rest/v1/master
openapi/openapi-client/s...celeborn/rest/v1/master
openapi/openapi-client/s...celeborn/rest/v1/master
openapi/openapi-client/s...celeborn/rest/v1/master
117:155 
194:232 
185:223 
185:223 
194:232 
117:155 
117:155 
185:223 
view
30 x 3 client-spark/spark-3-col.../sql/execution/columnar
client-spark/spark-4-col.../sql/execution/columnar
client-spark/spark-3.5-c.../sql/execution/columnar
135:166 
137:168 
137:168 
view
29 x 6 client-flink/flink-1.16/...e/celeborn/plugin/flink
client-flink/flink-1.17/...e/celeborn/plugin/flink
client-flink/flink-2.0/s...e/celeborn/plugin/flink
client-flink/flink-1.19/...e/celeborn/plugin/flink
client-flink/flink-1.20/...e/celeborn/plugin/flink
client-flink/flink-1.18/...e/celeborn/plugin/flink
49:79 
49:79 
50:80 
49:79 
50:80 
49:79 
view
28 x 2 client-flink/flink-2.0/s...orn/plugin/flink/tiered
client-flink/flink-1.20/...orn/plugin/flink/tiered
390:429 
390:429 
view
28 x 2 openapi/openapi-client/s.../rest/v1/worker/invoker
openapi/openapi-client/s.../rest/v1/master/invoker
735:769 
735:769 
view
27 x 2 client-tez/tez/src/main/.../runtime/library/output
client-tez/tez/src/main/.../runtime/library/output
117:145 
127:155 
view
27 x 2 client-flink/flink-2.0/s...orn/plugin/flink/tiered
client-flink/flink-1.20/...orn/plugin/flink/tiered
276:304 
276:304 
view
27 x 2 client-spark/spark-4-col.../sql/execution/columnar
client-spark/spark-3.5-c.../sql/execution/columnar
216:244 
216:244 
view
27 x 2 openapi/openapi-client/s.../rest/v1/worker/invoker
openapi/openapi-client/s.../rest/v1/master/invoker
779:811 
779:811 
view
26 x 2 client-tez/tez/src/main/.../runtime/library/output
client-tez/tez/src/main/.../runtime/library/output
157:188 
167:198 
view
25 x 2 client-flink/flink-2.0/s...orn/plugin/flink/tiered
client-flink/flink-1.20/...orn/plugin/flink/tiered
395:421 
395:421 
view
25 x 2 client-flink/flink-2.0/s...orn/plugin/flink/tiered
client-flink/flink-1.20/...orn/plugin/flink/tiered
493:519 
493:519 
view
25 x 2 client-flink/flink-2.0/s...orn/plugin/flink/tiered
client-flink/flink-1.20/...orn/plugin/flink/tiered
64:90 
64:90 
view
25 x 2 openapi/openapi-client/s.../rest/v1/worker/invoker
openapi/openapi-client/s.../rest/v1/master/invoker
517:555 
517:555 
view
24 x 2 client-flink/flink-2.0/s...orn/plugin/flink/tiered
client-flink/flink-1.20/...orn/plugin/flink/tiered
193:227 
193:227 
view
24 x 2 client-flink/flink-2.0/s...orn/plugin/flink/tiered
client-flink/flink-1.20/...orn/plugin/flink/tiered
122:147 
122:147 
view
24 x 2 client-flink/flink-2.0/s...orn/plugin/flink/tiered
client-flink/flink-1.20/...orn/plugin/flink/tiered
346:371 
346:371 
view
24 x 2 client-flink/flink-2.0/s...orn/plugin/flink/tiered
client-flink/flink-1.20/...orn/plugin/flink/tiered
466:491 
466:491 
view
23 x 6 client-flink/flink-1.16/...e/celeborn/plugin/flink
client-flink/flink-1.17/...e/celeborn/plugin/flink
client-flink/flink-2.0/s...e/celeborn/plugin/flink
client-flink/flink-1.19/...e/celeborn/plugin/flink
client-flink/flink-1.20/...e/celeborn/plugin/flink
client-flink/flink-1.18/...e/celeborn/plugin/flink
43:67 
43:67 
43:67 
43:67 
43:67 
43:67 
view
23 x 2 client-flink/flink-2.0/s...orn/plugin/flink/tiered
client-flink/flink-1.20/...orn/plugin/flink/tiered
148:178 
148:178 
view
23 x 2 client-flink/flink-2.0/s...orn/plugin/flink/tiered
client-flink/flink-1.20/...orn/plugin/flink/tiered
423:449 
423:449 
view
23 x 2 client-flink/flink-2.0/s...orn/plugin/flink/tiered
client-flink/flink-1.20/...orn/plugin/flink/tiered
189:218 
189:218 
view
22 x 2 client-tez/tez/src/main/...ez/runtime/library/sort
client-mr/mr/src/main/ja...rg/apache/hadoop/mapred
204:232 
217:245 
view
22 x 2 client-spark/spark-3/src.../spark/shuffle/celeborn
client-spark/spark-2/src.../spark/shuffle/celeborn
162:185 
166:189 
view
21 x 3 client-spark/spark-3-col.../spark/shuffle/celeborn
client-spark/spark-4-col.../spark/shuffle/celeborn
client-spark/spark-3.5-c.../spark/shuffle/celeborn
60:82 
60:82 
60:82 
view
20 x 2 client-tez/tez/src/main/...ez/runtime/library/sort
client-mr/mr/src/main/ja...rg/apache/hadoop/mapred
245:266 
258:279 
view
20 x 2 client-flink/flink-2.0/s...orn/plugin/flink/tiered
client-flink/flink-1.20/...orn/plugin/flink/tiered
449:477 
449:477 
view
20 x 2 client-flink/flink-2.0/s...orn/plugin/flink/tiered
client-flink/flink-1.20/...orn/plugin/flink/tiered
199:223 
199:223 
view
19 x 6 client-flink/flink-1.16/...e/celeborn/plugin/flink
client-flink/flink-1.17/...e/celeborn/plugin/flink
client-flink/flink-2.0/s...e/celeborn/plugin/flink
client-flink/flink-1.19/...e/celeborn/plugin/flink
client-flink/flink-1.20/...e/celeborn/plugin/flink
client-flink/flink-1.18/...e/celeborn/plugin/flink
34:55 
34:55 
34:55 
34:55 
34:55 
34:55 
view
19 x 3 client-spark/spark-3-col.../sql/execution/columnar
client-spark/spark-4-col.../sql/execution/columnar
client-spark/spark-3.5-c.../sql/execution/columnar
64:90 
64:90 
64:90 
view
18 x 2 client-flink/flink-2.0/s...orn/plugin/flink/tiered
client-flink/flink-1.20/...orn/plugin/flink/tiered
228:247 
228:247 
view