apache / incubator-celeborn
Duplication

Places in code with 6 or more lines that are exactly the same.

Intro
Learn more...
Duplication Overall
system12% (4,817 lines)
dependency graphs: 2D graph | 3D graph | 3D graph (with duplicates)...
Duplication per Extension
java18% (3,330 lines)
scala6% (1,194 lines)
yaml39% (223 lines)
proto12% (70 lines)
Duplication per Component (primary)
client-spark37% (1,557 lines)
client-flink27% (1,156 lines)
client11% (543 lines)
common3% (513 lines)
worker6% (401 lines)
master12% (332 lines)
charts34% (223 lines)
project13% (74 lines)
toolkit8% (18 lines)
ROOT0% (0 lines)
dev0% (0 lines)
build0% (0 lines)
service0% (0 lines)

Duplication Between Components (50+ lines)

G common common master master common--master 136 worker worker master--worker 71

Download: SVG DOT (open online Graphviz editor)

Open 3D force graph...

Show more details on duplication between components...
Longest Duplicates
The list of 50 longest duplicates.
See data for all 542 duplicates...
Size#FoldersFilesLinesCode
94 x 2 client-flink/flink-1.15/...e/celeborn/plugin/flink
client-flink/flink-1.17/...e/celeborn/plugin/flink
63:217 (65%)
63:217 (62%)
view
75 x 2 client-flink/flink-1.14/...e/celeborn/plugin/flink
client-flink/flink-1.15/...e/celeborn/plugin/flink
58:160 (68%)
53:155 (65%)
view
64 x 2 client-spark/spark-2/src.../spark/shuffle/celeborn
client-spark/spark-3/src.../spark/shuffle/celeborn
280:356 (23%)
302:378 (22%)
view
59 x 2 client-flink/flink-1.15/...e/celeborn/plugin/flink
client-flink/flink-1.17/...e/celeborn/plugin/flink
117:205 (51%)
129:217 (47%)
view
50 x 2 client-spark/spark-2/src.../spark/shuffle/celeborn
client-spark/spark-3/src.../spark/shuffle/celeborn
206:269 (19%)
283:346 (15%)
view
40 x 2 client-flink/flink-1.15/...e/celeborn/plugin/flink
client-flink/flink-1.17/...e/celeborn/plugin/flink
49:96 (35%)
50:97 (32%)
view
39 x 2 client-flink/flink-1.14/...e/celeborn/plugin/flink
client-flink/flink-1.15/...e/celeborn/plugin/flink
35:76 (100%)
35:77 (100%)
view
38 x 2 client-flink/flink-1.15/...e/celeborn/plugin/flink
client-flink/flink-1.17/...e/celeborn/plugin/flink
219:270 (26%)
219:270 (25%)
view
38 x 2 client-flink/flink-1.14/...e/celeborn/plugin/flink
client-flink/flink-1.17/...e/celeborn/plugin/flink
58:101 (34%)
54:97 (30%)
view
34 x 2 client-flink/flink-1.14/...e/celeborn/plugin/flink
client-flink/flink-1.15/...e/celeborn/plugin/flink
216:268 (24%)
233:285 (23%)
view
32 x 2 client-spark/spark-2/src.../spark/shuffle/celeborn
client-spark/spark-3/src.../spark/shuffle/celeborn
271:310 (12%)
348:387 (9%)
view
31 x 2 client-flink/flink-1.14/...e/celeborn/plugin/flink
client-flink/flink-1.15/...e/celeborn/plugin/flink
160:205 (22%)
172:217 (21%)
view
31 x 2 client-flink/flink-1.14/...e/celeborn/plugin/flink
client-flink/flink-1.17/...e/celeborn/plugin/flink
160:205 (22%)
172:217 (20%)
view
29 x 2 client-flink/flink-1.14/...e/celeborn/plugin/flink
client-flink/flink-1.17/...e/celeborn/plugin/flink
48:76 (74%)
49:77 (74%)
view
29 x 2 client-flink/flink-1.15/...e/celeborn/plugin/flink
client-flink/flink-1.17/...e/celeborn/plugin/flink
49:77 (74%)
49:77 (74%)
view
28 x 2 client-spark/spark-2/src.../spark/shuffle/celeborn
client-spark/spark-3/src.../spark/shuffle/celeborn
201:231 (10%)
250:281 (9%)
view
28 x 2 client-flink/flink-1.15/...e/celeborn/plugin/flink
client-flink/flink-1.17/...e/celeborn/plugin/flink
36:80 (100%)
36:80 (100%)
view
28 x 2 client-flink/flink-1.14/...e/celeborn/plugin/flink
client-flink/flink-1.15/...e/celeborn/plugin/flink
36:79 (100%)
36:80 (100%)
view
28 x 2 client-spark/spark-3/src.../spark/shuffle/celeborn
client-spark/spark-3/src.../spark/shuffle/celeborn
234:262 (12%)
291:319 (12%)
view
28 x 2 client-flink/flink-1.14/...e/celeborn/plugin/flink
client-flink/flink-1.17/...e/celeborn/plugin/flink
36:79 (100%)
36:80 (100%)
view
27 x 2 client-flink/flink-1.15/...e/celeborn/plugin/flink
client-flink/flink-1.17/...e/celeborn/plugin/flink
24:54 (100%)
28:58 (100%)
view
27 x 2 client-flink/flink-1.14/...e/celeborn/plugin/flink
client-flink/flink-1.17/...e/celeborn/plugin/flink
24:54 (100%)
28:58 (100%)
view
27 x 2 client-flink/flink-1.14/...e/celeborn/plugin/flink
client-flink/flink-1.15/...e/celeborn/plugin/flink
24:54 (100%)
24:54 (100%)
view
24 x 2 client-flink/flink-1.14/...e/celeborn/plugin/flink
client-flink/flink-1.17/...e/celeborn/plugin/flink
216:253 (17%)
233:270 (16%)
view
24 x 2 client-flink/flink-1.14/...e/celeborn/plugin/flink
client-flink/flink-1.17/...e/celeborn/plugin/flink
122:160 (21%)
129:167 (19%)
view
23 x 2 client-spark/spark-2/src.../spark/shuffle/celeborn
client-spark/spark-3/src.../spark/shuffle/celeborn
133:158 (9%)
153:178 (6%)
view
22 x 2 client-spark/spark-2/src.../spark/shuffle/celeborn
client-spark/spark-3/src.../spark/shuffle/celeborn
86:119 (28%)
103:136 (24%)
view
22 x 2 client-flink/flink-1.14/...e/celeborn/plugin/flink
client-flink/flink-1.17/...e/celeborn/plugin/flink
175:205 (20%)
187:217 (17%)
view
22 x 2 charts/celeborn/templates
charts/celeborn/templates
55:76 (17%)
55:76 (17%)
view
22 x 2 client-flink/flink-1.14/...e/celeborn/plugin/flink
client-flink/flink-1.15/...e/celeborn/plugin/flink
175:205 (20%)
175:205 (19%)
view
21 x 2 client-spark/spark-2/src.../spark/shuffle/celeborn
client-spark/spark-3/src.../spark/shuffle/celeborn
135:168 (14%)
143:176 (9%)
view
20 x 2 client/src/main/java/org/apache/celeborn/client
client/src/main/java/org/apache/celeborn/client
1145:1164 (1%)
1179:1198 (1%)
view
19 x 2 client-flink/flink-1.14/...e/celeborn/plugin/flink
client-flink/flink-1.15/...e/celeborn/plugin/flink
109:147 (13%)
114:152 (13%)
view
19 x 2 client-spark/spark-2/src.../spark/shuffle/celeborn
client-spark/spark-3/src.../spark/shuffle/celeborn
125:145 (7%)
125:145 (6%)
view
19 x 2 client/src/main/java/org/apache/celeborn/client
client/src/main/java/org/apache/celeborn/client
363:381 (1%)
399:417 (1%)
view
19 x 2 client-flink/flink-1.14/...e/celeborn/plugin/flink
client-flink/flink-1.17/...e/celeborn/plugin/flink
109:147 (13%)
114:152 (12%)
view
18 x 2 client-flink/flink-1.14/...e/celeborn/plugin/flink
client-flink/flink-1.15/...e/celeborn/plugin/flink
58:75 (13%)
67:85 (12%)
view
18 x 2 client-flink/flink-1.14/...e/celeborn/plugin/flink
client-flink/flink-1.17/...e/celeborn/plugin/flink
58:75 (13%)
67:85 (12%)
view
18 x 2 client-spark/spark-2/src.../spark/shuffle/celeborn
client-spark/spark-3/src.../spark/shuffle/celeborn
147:165 (6%)
147:165 (6%)
view
18 x 2 client-flink/flink-1.15/...e/celeborn/plugin/flink
client-flink/flink-1.17/...e/celeborn/plugin/flink
32:52 (100%)
32:53 (100%)
view
17 x 2 client/src/main/java/org/apache/celeborn/client
client/src/main/java/org/apache/celeborn/client
947:963 (1%)
1035:1051 (1%)
view
17 x 2 worker/src/main/java/org...e/deploy/worker/storage
worker/src/main/java/org...e/deploy/worker/storage
56:72 (8%)
42:58 (30%)
view
17 x 2 common/src/main/scala/or.../common/internal/config
common/src/main/scala/or.../common/internal/config
155:171 (10%)
188:204 (10%)
view
17 x 2 common/src/main/scala/or.../common/internal/config
common/src/main/scala/or.../common/internal/config
122:138 (10%)
188:204 (10%)
view
17 x 2 common/src/main/scala/or.../common/internal/config
common/src/main/scala/or.../common/internal/config
122:138 (10%)
155:171 (10%)
view
16 x 2 client-spark/spark-2/src.../spark/shuffle/celeborn
client-spark/spark-3/src.../spark/shuffle/celeborn
49:69 (10%)
50:70 (6%)
view
16 x 2 common/src/main/scala/or...n/common/metrics/source
common/src/main/scala/or...n/common/metrics/source
315:331 (6%)
341:357 (6%)
view
16 x 2 client-spark/spark-2/src.../spark/shuffle/celeborn
client-spark/spark-3/src.../spark/shuffle/celeborn
326:342 (6%)
426:442 (4%)
view
15 x 2 client-spark/spark-2/src.../spark/shuffle/celeborn
client-spark/spark-3/src.../spark/shuffle/celeborn
232:253 (10%)
322:343 (6%)
view
15 x 2 client-flink/flink-1.14/...e/celeborn/plugin/flink
client-flink/flink-1.17/...e/celeborn/plugin/flink
101:122 (13%)
102:123 (12%)
view
Duplicated Units
The list of top 22 duplicated units.
See data for all 22 unit duplicates...
Size#FoldersFilesLinesCode
37 x 2 client-spark/spark-2/src.../spark/shuffle/celeborn
client-spark/spark-3/src.../spark/shuffle/celeborn
280:321 
302:343 
view
29 x 3 client-flink/flink-1.17/...e/celeborn/plugin/flink
client-flink/flink-1.14/...e/celeborn/plugin/flink
client-flink/flink-1.15/...e/celeborn/plugin/flink
48:78 
47:77 
48:78 
view
27 x 3 client-flink/flink-1.17/...e/celeborn/plugin/flink
client-flink/flink-1.14/...e/celeborn/plugin/flink
client-flink/flink-1.15/...e/celeborn/plugin/flink
54:84 
58:88 
53:83 
view
21 x 2 client-spark/spark-2/src.../spark/shuffle/celeborn
client-spark/spark-3/src.../spark/shuffle/celeborn
232:257 
309:334 
view
19 x 2 client-flink/flink-1.17/...e/celeborn/plugin/flink
client-flink/flink-1.15/...e/celeborn/plugin/flink
67:88 
67:88 
view
18 x 3 client-flink/flink-1.17/...e/celeborn/plugin/flink
client-flink/flink-1.14/...e/celeborn/plugin/flink
client-flink/flink-1.15/...e/celeborn/plugin/flink
39:59 
35:55 
35:55 
view
17 x 2 client-spark/spark-2/src.../spark/shuffle/celeborn
client-spark/spark-3/src.../spark/shuffle/celeborn
286:306 
363:383 
view
14 x 2 client-spark/spark-2/src.../spark/shuffle/celeborn
client-spark/spark-3/src.../spark/shuffle/celeborn
323:338 
345:360 
view
13 x 2 client-flink/flink-1.17/...e/celeborn/plugin/flink
client-flink/flink-1.15/...e/celeborn/plugin/flink
40:54 
39:53 
view
12 x 2 client-spark/spark-2/src.../spark/shuffle/celeborn
client-spark/spark-3/src.../spark/shuffle/celeborn
173:186 
220:233 
view
10 x 2 client-spark/spark-2/src.../spark/shuffle/celeborn
client-spark/spark-3/src.../spark/shuffle/celeborn
140:153 
148:161 
view
9 x 2 client-spark/spark-2/src.../spark/shuffle/celeborn
client-spark/spark-3/src.../spark/shuffle/celeborn
240:250 
290:300 
view
9 x 2 client-spark/spark-2/src.../spark/shuffle/celeborn
client-spark/spark-3/src.../spark/shuffle/celeborn
238:249 
328:339 
view
9 x 2 client-spark/spark-2/src.../spark/shuffle/celeborn
client-spark/spark-3/src.../spark/shuffle/celeborn
58:69 
64:75 
view
9 x 3 client-flink/flink-1.17/...e/celeborn/plugin/flink
client-flink/flink-1.14/...e/celeborn/plugin/flink
client-flink/flink-1.15/...e/celeborn/plugin/flink
197:207 
185:195 
185:195 
view
8 x 3 client-flink/flink-1.17/...e/celeborn/plugin/flink
client-flink/flink-1.14/...e/celeborn/plugin/flink
client-flink/flink-1.15/...e/celeborn/plugin/flink
50:60 
50:59 
50:60 
view
7 x 2 client-spark/spark-2/src.../spark/shuffle/celeborn
client-spark/spark-3/src.../spark/shuffle/celeborn
100:108 
79:87 
view
7 x 3 client-flink/flink-1.17/...e/celeborn/plugin/flink
client-flink/flink-1.14/...e/celeborn/plugin/flink
client-flink/flink-1.15/...e/celeborn/plugin/flink
87:95 
91:99 
86:94 
view
7 x 3 client-flink/flink-1.17/...e/celeborn/plugin/flink
client-flink/flink-1.14/...e/celeborn/plugin/flink
client-flink/flink-1.15/...e/celeborn/plugin/flink
113:121 
112:120 
107:115 
view
6 x 2 client-spark/spark-3/src.../spark/shuffle/celeborn
client-spark/spark-3/src.../spark/shuffle/celeborn
217:225 
236:244 
view
6 x 2 master/src/main/java/org...y/master/clustermeta/ha
master/src/main/java/org...ploy/master/clustermeta
48:55 
38:45 
view
6 x 3 client-flink/flink-1.17/...e/celeborn/plugin/flink
client-flink/flink-1.14/...e/celeborn/plugin/flink
client-flink/flink-1.15/...e/celeborn/plugin/flink
63:70 
62:69 
63:70 
view