duplicated block id: 1 size: 94 cleaned lines of code in 2 files: - client-flink/flink-1.15/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleInputGate.java (63:217) - client-flink/flink-1.17/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleInputGate.java (63:217) duplicated block id: 2 size: 75 cleaned lines of code in 2 files: - client-flink/flink-1.14/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleResultPartition.java (58:160) - client-flink/flink-1.15/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleResultPartition.java (53:155) duplicated block id: 3 size: 64 cleaned lines of code in 2 files: - client-spark/spark-2/src/main/java/org/apache/spark/shuffle/celeborn/SortBasedShuffleWriter.java (280:356) - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/SortBasedShuffleWriter.java (302:378) duplicated block id: 4 size: 59 cleaned lines of code in 2 files: - client-flink/flink-1.15/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleResultPartition.java (117:205) - client-flink/flink-1.17/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleResultPartition.java (129:217) duplicated block id: 5 size: 50 cleaned lines of code in 2 files: - client-spark/spark-2/src/main/java/org/apache/spark/shuffle/celeborn/HashBasedShuffleWriter.java (206:269) - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/HashBasedShuffleWriter.java (283:346) duplicated block id: 6 size: 40 cleaned lines of code in 2 files: - client-flink/flink-1.15/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleResultPartition.java (49:96) - client-flink/flink-1.17/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleResultPartition.java (50:97) duplicated block id: 7 size: 39 cleaned lines of code in 2 files: - client-flink/flink-1.14/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleResultPartitionFactory.java (35:76) - client-flink/flink-1.15/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleResultPartitionFactory.java (35:77) duplicated block id: 8 size: 38 cleaned lines of code in 2 files: - client-flink/flink-1.15/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleInputGate.java (219:270) - client-flink/flink-1.17/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleInputGate.java (219:270) duplicated block id: 9 size: 38 cleaned lines of code in 2 files: - client-flink/flink-1.14/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleResultPartition.java (58:101) - client-flink/flink-1.17/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleResultPartition.java (54:97) duplicated block id: 10 size: 34 cleaned lines of code in 2 files: - client-flink/flink-1.14/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleInputGate.java (216:268) - client-flink/flink-1.15/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleInputGate.java (233:285) duplicated block id: 11 size: 32 cleaned lines of code in 2 files: - client-spark/spark-2/src/main/java/org/apache/spark/shuffle/celeborn/HashBasedShuffleWriter.java (271:310) - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/HashBasedShuffleWriter.java (348:387) duplicated block id: 12 size: 31 cleaned lines of code in 2 files: - client-flink/flink-1.14/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleInputGate.java (160:205) - client-flink/flink-1.15/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleInputGate.java (172:217) duplicated block id: 13 size: 31 cleaned lines of code in 2 files: - client-flink/flink-1.14/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleInputGate.java (160:205) - client-flink/flink-1.17/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleInputGate.java (172:217) duplicated block id: 14 size: 29 cleaned lines of code in 2 files: - client-flink/flink-1.14/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleResultPartitionFactory.java (48:76) - client-flink/flink-1.17/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleResultPartitionFactory.java (49:77) duplicated block id: 15 size: 29 cleaned lines of code in 2 files: - client-flink/flink-1.15/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleResultPartitionFactory.java (49:77) - client-flink/flink-1.17/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleResultPartitionFactory.java (49:77) duplicated block id: 16 size: 28 cleaned lines of code in 2 files: - client-spark/spark-2/src/main/java/org/apache/spark/shuffle/celeborn/SortBasedShuffleWriter.java (201:231) - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/SortBasedShuffleWriter.java (250:281) duplicated block id: 17 size: 28 cleaned lines of code in 2 files: - client-flink/flink-1.15/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleEnvironment.java (36:80) - client-flink/flink-1.17/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleEnvironment.java (36:80) duplicated block id: 18 size: 28 cleaned lines of code in 2 files: - client-flink/flink-1.14/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleEnvironment.java (36:79) - client-flink/flink-1.15/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleEnvironment.java (36:80) duplicated block id: 19 size: 28 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/SparkShuffleManager.java (234:262) - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/SparkShuffleManager.java (291:319) duplicated block id: 20 size: 28 cleaned lines of code in 2 files: - client-flink/flink-1.14/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleEnvironment.java (36:79) - client-flink/flink-1.17/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleEnvironment.java (36:80) duplicated block id: 21 size: 27 cleaned lines of code in 2 files: - client-flink/flink-1.15/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleServiceFactory.java (24:54) - client-flink/flink-1.17/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleServiceFactory.java (28:58) duplicated block id: 22 size: 27 cleaned lines of code in 2 files: - client-flink/flink-1.14/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleServiceFactory.java (24:54) - client-flink/flink-1.17/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleServiceFactory.java (28:58) duplicated block id: 23 size: 27 cleaned lines of code in 2 files: - client-flink/flink-1.14/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleServiceFactory.java (24:54) - client-flink/flink-1.15/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleServiceFactory.java (24:54) duplicated block id: 24 size: 24 cleaned lines of code in 2 files: - client-flink/flink-1.14/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleInputGate.java (216:253) - client-flink/flink-1.17/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleInputGate.java (233:270) duplicated block id: 25 size: 24 cleaned lines of code in 2 files: - client-flink/flink-1.14/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleResultPartition.java (122:160) - client-flink/flink-1.17/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleResultPartition.java (129:167) duplicated block id: 26 size: 23 cleaned lines of code in 2 files: - client-spark/spark-2/src/main/java/org/apache/spark/shuffle/celeborn/HashBasedShuffleWriter.java (133:158) - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/HashBasedShuffleWriter.java (153:178) duplicated block id: 27 size: 22 cleaned lines of code in 2 files: - client-spark/spark-2/src/main/scala/org/apache/spark/shuffle/celeborn/CelebornShuffleReader.scala (86:119) - client-spark/spark-3/src/main/scala/org/apache/spark/shuffle/celeborn/CelebornShuffleReader.scala (103:136) duplicated block id: 28 size: 22 cleaned lines of code in 2 files: - client-flink/flink-1.14/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleResultPartition.java (175:205) - client-flink/flink-1.17/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleResultPartition.java (187:217) duplicated block id: 29 size: 22 cleaned lines of code in 2 files: - charts/celeborn/templates/master-statefulset.yaml (55:76) - charts/celeborn/templates/worker-statefulset.yaml (55:76) duplicated block id: 30 size: 22 cleaned lines of code in 2 files: - client-flink/flink-1.14/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleResultPartition.java (175:205) - client-flink/flink-1.15/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleResultPartition.java (175:205) duplicated block id: 31 size: 21 cleaned lines of code in 2 files: - client-spark/spark-2/src/main/java/org/apache/spark/shuffle/celeborn/SparkShuffleManager.java (135:168) - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/SparkShuffleManager.java (143:176) duplicated block id: 32 size: 20 cleaned lines of code in 2 files: - client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java (1145:1164) - client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java (1179:1198) duplicated block id: 33 size: 19 cleaned lines of code in 2 files: - client-flink/flink-1.14/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleInputGate.java (109:147) - client-flink/flink-1.15/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleInputGate.java (114:152) duplicated block id: 34 size: 19 cleaned lines of code in 2 files: - client-spark/spark-2/src/main/java/org/apache/spark/shuffle/celeborn/SortBasedShuffleWriter.java (125:145) - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/SortBasedShuffleWriter.java (125:145) duplicated block id: 35 size: 19 cleaned lines of code in 2 files: - client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java (363:381) - client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java (399:417) duplicated block id: 36 size: 19 cleaned lines of code in 2 files: - client-flink/flink-1.14/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleInputGate.java (109:147) - client-flink/flink-1.17/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleInputGate.java (114:152) duplicated block id: 37 size: 18 cleaned lines of code in 2 files: - client-flink/flink-1.14/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleInputGate.java (58:75) - client-flink/flink-1.15/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleInputGate.java (67:85) duplicated block id: 38 size: 18 cleaned lines of code in 2 files: - client-flink/flink-1.14/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleInputGate.java (58:75) - client-flink/flink-1.17/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleInputGate.java (67:85) duplicated block id: 39 size: 18 cleaned lines of code in 2 files: - client-spark/spark-2/src/main/java/org/apache/spark/shuffle/celeborn/SortBasedShuffleWriter.java (147:165) - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/SortBasedShuffleWriter.java (147:165) duplicated block id: 40 size: 18 cleaned lines of code in 2 files: - client-flink/flink-1.15/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleInputGateFactory.java (32:52) - client-flink/flink-1.17/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleInputGateFactory.java (32:53) duplicated block id: 41 size: 17 cleaned lines of code in 2 files: - client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java (947:963) - client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java (1035:1051) duplicated block id: 42 size: 17 cleaned lines of code in 2 files: - worker/src/main/java/org/apache/celeborn/service/deploy/worker/storage/MapPartitionFileWriter.java (56:72) - worker/src/main/java/org/apache/celeborn/service/deploy/worker/storage/ReducePartitionFileWriter.java (42:58) duplicated block id: 43 size: 17 cleaned lines of code in 2 files: - common/src/main/scala/org/apache/celeborn/common/internal/config/ConfigEntry.scala (155:171) - common/src/main/scala/org/apache/celeborn/common/internal/config/ConfigEntry.scala (188:204) duplicated block id: 44 size: 17 cleaned lines of code in 2 files: - common/src/main/scala/org/apache/celeborn/common/internal/config/ConfigEntry.scala (122:138) - common/src/main/scala/org/apache/celeborn/common/internal/config/ConfigEntry.scala (188:204) duplicated block id: 45 size: 17 cleaned lines of code in 2 files: - common/src/main/scala/org/apache/celeborn/common/internal/config/ConfigEntry.scala (122:138) - common/src/main/scala/org/apache/celeborn/common/internal/config/ConfigEntry.scala (155:171) duplicated block id: 46 size: 16 cleaned lines of code in 2 files: - client-spark/spark-2/src/main/java/org/apache/spark/shuffle/celeborn/SparkShuffleManager.java (49:69) - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/SparkShuffleManager.java (50:70) duplicated block id: 47 size: 16 cleaned lines of code in 2 files: - common/src/main/scala/org/apache/celeborn/common/metrics/source/AbstractSource.scala (315:331) - common/src/main/scala/org/apache/celeborn/common/metrics/source/AbstractSource.scala (341:357) duplicated block id: 48 size: 16 cleaned lines of code in 2 files: - client-spark/spark-2/src/main/java/org/apache/spark/shuffle/celeborn/HashBasedShuffleWriter.java (326:342) - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/HashBasedShuffleWriter.java (426:442) duplicated block id: 49 size: 15 cleaned lines of code in 2 files: - client-spark/spark-2/src/main/java/org/apache/spark/shuffle/celeborn/SparkShuffleManager.java (232:253) - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/SparkShuffleManager.java (322:343) duplicated block id: 50 size: 15 cleaned lines of code in 2 files: - client-flink/flink-1.14/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleResultPartition.java (101:122) - client-flink/flink-1.17/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleResultPartition.java (102:123) duplicated block id: 51 size: 15 cleaned lines of code in 2 files: - client-flink/flink-1.15/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleResultPartition.java (96:117) - client-flink/flink-1.17/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleResultPartition.java (102:123) duplicated block id: 52 size: 14 cleaned lines of code in 2 files: - client-spark/spark-2/src/main/java/org/apache/spark/shuffle/celeborn/SparkShuffleManager.java (74:91) - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/SparkShuffleManager.java (81:98) duplicated block id: 53 size: 14 cleaned lines of code in 2 files: - worker/src/main/scala/org/apache/celeborn/service/deploy/worker/PushDataHandler.scala (368:382) - worker/src/main/scala/org/apache/celeborn/service/deploy/worker/PushDataHandler.scala (813:827) duplicated block id: 54 size: 14 cleaned lines of code in 2 files: - common/src/main/proto/TransportMessages.proto (1:14) - master/src/main/proto/Resource.proto (1:14) duplicated block id: 55 size: 14 cleaned lines of code in 2 files: - client-spark/spark-2/src/main/java/org/apache/spark/shuffle/celeborn/SortBasedShuffleWriter.java (172:189) - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/SortBasedShuffleWriter.java (219:236) duplicated block id: 56 size: 14 cleaned lines of code in 2 files: - master/src/main/scala/org/apache/celeborn/service/deploy/master/MasterArguments.scala (58:74) - worker/src/main/scala/org/apache/celeborn/service/deploy/worker/WorkerArguments.scala (46:62) duplicated block id: 57 size: 13 cleaned lines of code in 2 files: - client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java (998:1019) - client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java (1378:1397) duplicated block id: 58 size: 13 cleaned lines of code in 2 files: - client-spark/spark-2/src/main/java/org/apache/spark/shuffle/celeborn/SparkUtils.java (90:110) - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/SparkUtils.java (69:89) duplicated block id: 59 size: 13 cleaned lines of code in 2 files: - client-spark/spark-2/src/main/java/org/apache/spark/shuffle/celeborn/HashBasedShuffleWriter.java (166:183) - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/HashBasedShuffleWriter.java (199:217) duplicated block id: 60 size: 13 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/HashBasedShuffleWriter.java (348:360) - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/SortBasedShuffleWriter.java (347:359) duplicated block id: 61 size: 13 cleaned lines of code in 2 files: - client/src/main/java/org/apache/celeborn/client/compress/Lz4Decompressor.java (38:53) - client/src/main/java/org/apache/celeborn/client/compress/ZstdDecompressor.java (35:50) duplicated block id: 62 size: 13 cleaned lines of code in 2 files: - client-flink/flink-1.14/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleInputGate.java (81:104) - client-flink/flink-1.15/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleInputGate.java (91:114) duplicated block id: 63 size: 13 cleaned lines of code in 2 files: - client-spark/spark-2/src/main/java/org/apache/spark/shuffle/celeborn/HashBasedShuffleWriter.java (271:283) - client-spark/spark-2/src/main/java/org/apache/spark/shuffle/celeborn/SortBasedShuffleWriter.java (325:337) duplicated block id: 64 size: 13 cleaned lines of code in 2 files: - client-flink/flink-1.14/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleInputGate.java (81:104) - client-flink/flink-1.17/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleInputGate.java (91:114) duplicated block id: 65 size: 13 cleaned lines of code in 2 files: - charts/celeborn/templates/prometheus-podmonitor.yaml (24:36) - charts/celeborn/templates/prometheus-podmonitor.yaml (49:61) duplicated block id: 66 size: 13 cleaned lines of code in 2 files: - worker/src/main/scala/org/apache/celeborn/service/deploy/worker/PushDataHandler.scala (148:160) - worker/src/main/scala/org/apache/celeborn/service/deploy/worker/PushDataHandler.scala (396:408) duplicated block id: 67 size: 13 cleaned lines of code in 2 files: - client-spark/spark-2/src/main/java/org/apache/spark/shuffle/celeborn/HashBasedShuffleWriter.java (271:283) - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/SortBasedShuffleWriter.java (347:359) duplicated block id: 68 size: 13 cleaned lines of code in 2 files: - client-spark/spark-2/src/main/java/org/apache/spark/shuffle/celeborn/SortBasedShuffleWriter.java (325:337) - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/HashBasedShuffleWriter.java (348:360) duplicated block id: 69 size: 12 cleaned lines of code in 2 files: - worker/src/main/scala/org/apache/celeborn/service/deploy/worker/PushDataHandler.scala (302:316) - worker/src/main/scala/org/apache/celeborn/service/deploy/worker/PushDataHandler.scala (558:573) duplicated block id: 70 size: 12 cleaned lines of code in 2 files: - client-spark/spark-2/src/main/java/org/apache/spark/shuffle/celeborn/SortBasedShuffleWriter.java (133:144) - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/SortBasedShuffleWriter.java (153:164) duplicated block id: 71 size: 12 cleaned lines of code in 2 files: - worker/src/main/scala/org/apache/celeborn/service/deploy/worker/PushDataHandler.scala (350:363) - worker/src/main/scala/org/apache/celeborn/service/deploy/worker/PushDataHandler.scala (605:618) duplicated block id: 72 size: 12 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnType.scala (521:539) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnType.scala (567:585) duplicated block id: 73 size: 12 cleaned lines of code in 2 files: - client-spark/spark-2/src/main/java/org/apache/spark/shuffle/celeborn/SortBasedShuffleWriter.java (153:164) - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/SortBasedShuffleWriter.java (133:144) duplicated block id: 74 size: 12 cleaned lines of code in 2 files: - charts/celeborn/templates/master-statefulset.yaml (116:127) - charts/celeborn/templates/worker-statefulset.yaml (121:132) duplicated block id: 75 size: 12 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/SortBasedShuffleWriter.java (133:144) - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/SortBasedShuffleWriter.java (153:164) duplicated block id: 76 size: 12 cleaned lines of code in 2 files: - client-spark/spark-2/src/main/java/org/apache/spark/shuffle/celeborn/SortBasedShuffleWriter.java (133:144) - client-spark/spark-2/src/main/java/org/apache/spark/shuffle/celeborn/SortBasedShuffleWriter.java (153:164) duplicated block id: 77 size: 12 cleaned lines of code in 2 files: - master/src/main/java/org/apache/celeborn/service/deploy/master/clustermeta/SingleMasterMetaManager.java (84:95) - master/src/main/java/org/apache/celeborn/service/deploy/master/clustermeta/ha/HAMasterMetaManager.java (189:200) duplicated block id: 78 size: 11 cleaned lines of code in 2 files: - client-spark/spark-2/src/main/java/org/apache/spark/shuffle/celeborn/SortBasedShuffleWriter.java (281:293) - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/HashBasedShuffleWriter.java (310:322) duplicated block id: 79 size: 11 cleaned lines of code in 2 files: - client-flink/flink-1.15/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleInputGate.java (270:285) - client-flink/flink-1.17/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleInputGate.java (278:293) duplicated block id: 80 size: 11 cleaned lines of code in 2 files: - client-flink/flink-1.14/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleInputGateFactory.java (32:45) - client-flink/flink-1.15/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleInputGateFactory.java (32:45) duplicated block id: 81 size: 11 cleaned lines of code in 2 files: - client-spark/spark-2/src/main/java/org/apache/spark/shuffle/celeborn/HashBasedShuffleWriter.java (233:245) - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/SortBasedShuffleWriter.java (303:315) duplicated block id: 82 size: 11 cleaned lines of code in 2 files: - master/src/main/java/org/apache/celeborn/service/deploy/master/clustermeta/ha/HAMasterMetaManager.java (153:166) - master/src/main/java/org/apache/celeborn/service/deploy/master/clustermeta/ha/HAMasterMetaManager.java (176:189) duplicated block id: 83 size: 11 cleaned lines of code in 2 files: - client-flink/flink-1.14/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleInputGateFactory.java (32:45) - client-flink/flink-1.17/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleInputGateFactory.java (32:46) duplicated block id: 84 size: 11 cleaned lines of code in 2 files: - client/src/main/java/org/apache/celeborn/client/compress/Lz4Compressor.java (35:49) - client/src/main/java/org/apache/celeborn/client/compress/ZstdCompressor.java (34:48) duplicated block id: 85 size: 11 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/HashBasedShuffleWriter.java (310:322) - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/SortBasedShuffleWriter.java (303:315) duplicated block id: 86 size: 11 cleaned lines of code in 2 files: - client-spark/spark-2/src/main/scala/org/apache/spark/shuffle/celeborn/CelebornShuffleFallbackPolicyRunner.scala (47:68) - client-spark/spark-3/src/main/scala/org/apache/spark/shuffle/celeborn/CelebornShuffleFallbackPolicyRunner.scala (53:74) duplicated block id: 87 size: 11 cleaned lines of code in 2 files: - common/src/main/java/org/apache/celeborn/common/protocol/PartitionLocation.java (105:117) - common/src/main/java/org/apache/celeborn/common/protocol/PartitionLocation.java (129:141) duplicated block id: 88 size: 11 cleaned lines of code in 2 files: - charts/celeborn/templates/master-statefulset.yaml (91:101) - charts/celeborn/templates/worker-statefulset.yaml (95:105) duplicated block id: 89 size: 11 cleaned lines of code in 2 files: - client-spark/spark-2/src/main/java/org/apache/spark/shuffle/celeborn/SparkShuffleManager.java (183:193) - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/SparkShuffleManager.java (193:203) duplicated block id: 90 size: 11 cleaned lines of code in 2 files: - client-spark/spark-2/src/main/java/org/apache/spark/shuffle/celeborn/HashBasedShuffleWriter.java (233:245) - client-spark/spark-2/src/main/java/org/apache/spark/shuffle/celeborn/SortBasedShuffleWriter.java (281:293) duplicated block id: 91 size: 11 cleaned lines of code in 2 files: - client-flink/flink-1.14/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleInputGate.java (253:268) - client-flink/flink-1.17/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleInputGate.java (278:293) duplicated block id: 92 size: 11 cleaned lines of code in 2 files: - charts/celeborn/values.yaml (147:157) - charts/celeborn/values.yaml (161:171) duplicated block id: 93 size: 11 cleaned lines of code in 2 files: - client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java (779:790) - client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java (817:828) duplicated block id: 94 size: 11 cleaned lines of code in 2 files: - client/src/main/java/org/apache/celeborn/client/compress/Lz4Compressor.java (65:80) - client/src/main/java/org/apache/celeborn/client/compress/ZstdCompressor.java (72:87) duplicated block id: 95 size: 11 cleaned lines of code in 2 files: - client-spark/spark-2/src/main/java/org/apache/spark/shuffle/celeborn/HashBasedShuffleWriter.java (191:203) - client-spark/spark-2/src/main/java/org/apache/spark/shuffle/celeborn/SortBasedShuffleWriter.java (196:209) duplicated block id: 96 size: 11 cleaned lines of code in 2 files: - client-flink/common/src/main/java/org/apache/celeborn/plugin/flink/protocol/ReadData.java (47:63) - common/src/main/java/org/apache/celeborn/common/network/protocol/ReadData.java (43:59) duplicated block id: 97 size: 10 cleaned lines of code in 2 files: - client-flink/common/src/main/java/org/apache/celeborn/plugin/flink/buffer/FlinkNettyManagedBuffer.java (48:63) - common/src/main/java/org/apache/celeborn/common/network/buffer/NettyManagedBuffer.java (58:73) duplicated block id: 98 size: 10 cleaned lines of code in 2 files: - client-spark/spark-2/src/main/java/org/apache/spark/shuffle/celeborn/SortBasedShuffleWriter.java (236:249) - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/SortBasedShuffleWriter.java (286:299) duplicated block id: 99 size: 10 cleaned lines of code in 2 files: - client-flink/common/src/main/java/org/apache/celeborn/plugin/flink/AbstractRemoteShuffleResultPartitionFactory.java (163:172) - client-flink/flink-1.17/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleResultPartitionFactory.java (49:58) duplicated block id: 100 size: 10 cleaned lines of code in 2 files: - master/src/main/java/org/apache/celeborn/service/deploy/master/clustermeta/SingleMasterMetaManager.java (108:117) - master/src/main/java/org/apache/celeborn/service/deploy/master/clustermeta/ha/HAMasterMetaManager.java (226:235) duplicated block id: 101 size: 10 cleaned lines of code in 2 files: - project/CelebornBuild.scala (610:624) - project/CelebornBuild.scala (811:825) duplicated block id: 102 size: 10 cleaned lines of code in 2 files: - client-flink/common/src/main/java/org/apache/celeborn/plugin/flink/readclient/FlinkShuffleClientImpl.java (315:324) - client-flink/common/src/main/java/org/apache/celeborn/plugin/flink/readclient/FlinkShuffleClientImpl.java (426:435) duplicated block id: 103 size: 10 cleaned lines of code in 2 files: - client-spark/spark-2/src/main/java/org/apache/spark/shuffle/celeborn/HashBasedShuffleWriter.java (165:176) - client-spark/spark-2/src/main/java/org/apache/spark/shuffle/celeborn/SortBasedShuffleWriter.java (174:185) duplicated block id: 104 size: 10 cleaned lines of code in 2 files: - client-spark/spark-2/src/main/java/org/apache/spark/shuffle/celeborn/HashBasedShuffleWriter.java (207:216) - client-spark/spark-2/src/main/java/org/apache/spark/shuffle/celeborn/SortBasedShuffleWriter.java (213:222) duplicated block id: 105 size: 10 cleaned lines of code in 2 files: - client-flink/common/src/main/java/org/apache/celeborn/plugin/flink/AbstractRemoteShuffleResultPartitionFactory.java (163:172) - client-flink/flink-1.15/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleResultPartitionFactory.java (49:58) duplicated block id: 106 size: 10 cleaned lines of code in 2 files: - client-flink/common/src/main/java/org/apache/celeborn/plugin/flink/AbstractRemoteShuffleResultPartitionFactory.java (163:172) - client-flink/flink-1.14/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleResultPartitionFactory.java (48:57) duplicated block id: 107 size: 10 cleaned lines of code in 2 files: - client-spark/spark-2/src/main/java/org/apache/spark/shuffle/celeborn/SortBasedShuffleWriter.java (213:222) - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/HashBasedShuffleWriter.java (284:293) duplicated block id: 108 size: 10 cleaned lines of code in 2 files: - client/src/main/java/org/apache/celeborn/client/ShuffleClient.java (119:128) - client/src/main/java/org/apache/celeborn/client/ShuffleClient.java (134:143) duplicated block id: 109 size: 10 cleaned lines of code in 2 files: - client-spark/spark-2/src/main/java/org/apache/spark/shuffle/celeborn/HashBasedShuffleWriter.java (165:176) - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/SortBasedShuffleWriter.java (221:232) duplicated block id: 110 size: 10 cleaned lines of code in 2 files: - client-spark/spark-2/src/main/java/org/apache/spark/shuffle/celeborn/SortBasedShuffleWriter.java (259:277) - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/SortBasedShuffleWriter.java (198:216) duplicated block id: 111 size: 10 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornCompressionSchemes.scala (153:163) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornCompressionSchemes.scala (187:197) duplicated block id: 112 size: 10 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/HashBasedShuffleWriter.java (284:293) - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/SortBasedShuffleWriter.java (263:272) duplicated block id: 113 size: 10 cleaned lines of code in 2 files: - client/src/main/java/org/apache/celeborn/client/read/CelebornInputStream.java (50:59) - client/src/main/java/org/apache/celeborn/client/read/CelebornInputStream.java (140:149) duplicated block id: 114 size: 10 cleaned lines of code in 2 files: - worker/src/main/scala/org/apache/celeborn/service/deploy/worker/PushDataHandler.scala (258:270) - worker/src/main/scala/org/apache/celeborn/service/deploy/worker/PushDataHandler.scala (520:533) duplicated block id: 115 size: 10 cleaned lines of code in 2 files: - charts/celeborn/templates/master-statefulset.yaml (40:49) - charts/celeborn/templates/worker-statefulset.yaml (40:49) duplicated block id: 116 size: 10 cleaned lines of code in 2 files: - client-spark/spark-2/src/main/java/org/apache/spark/shuffle/celeborn/HashBasedShuffleWriter.java (207:216) - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/SortBasedShuffleWriter.java (263:272) duplicated block id: 117 size: 9 cleaned lines of code in 2 files: - master/src/main/java/org/apache/celeborn/service/deploy/master/clustermeta/IMetadataHandler.java (49:57) - master/src/main/java/org/apache/celeborn/service/deploy/master/clustermeta/ha/HAMasterMetaManager.java (191:199) duplicated block id: 118 size: 9 cleaned lines of code in 2 files: - worker/src/main/scala/org/apache/celeborn/service/deploy/worker/Controller.scala (84:92) - worker/src/main/scala/org/apache/celeborn/service/deploy/worker/Controller.scala (101:109) duplicated block id: 119 size: 9 cleaned lines of code in 2 files: - common/src/main/java/org/apache/celeborn/common/network/protocol/PushDataHandShake.java (99:110) - common/src/main/java/org/apache/celeborn/common/network/protocol/RegionStart.java (101:112) duplicated block id: 120 size: 9 cleaned lines of code in 2 files: - client/src/main/java/org/apache/celeborn/client/read/DfsPartitionReader.java (82:90) - client/src/main/java/org/apache/celeborn/client/read/WorkerPartitionReader.java (111:119) duplicated block id: 121 size: 9 cleaned lines of code in 2 files: - master/src/main/scala/org/apache/celeborn/service/deploy/master/Master.scala (324:332) - master/src/main/scala/org/apache/celeborn/service/deploy/master/Master.scala (340:348) duplicated block id: 122 size: 9 cleaned lines of code in 2 files: - common/src/main/java/org/apache/celeborn/common/network/protocol/PushDataHandShake.java (99:110) - common/src/main/java/org/apache/celeborn/common/network/protocol/RegionFinish.java (81:92) duplicated block id: 123 size: 9 cleaned lines of code in 2 files: - client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java (1349:1357) - client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java (1361:1369) duplicated block id: 124 size: 9 cleaned lines of code in 2 files: - common/src/main/java/org/apache/celeborn/common/network/protocol/RegionFinish.java (81:92) - common/src/main/java/org/apache/celeborn/common/network/protocol/RegionStart.java (101:112) duplicated block id: 125 size: 9 cleaned lines of code in 2 files: - client-flink/flink-1.14/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleResultPartition.java (157:171) - client-flink/flink-1.17/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleResultPartition.java (169:183) duplicated block id: 126 size: 9 cleaned lines of code in 2 files: - client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java (967:975) - client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java (979:987) duplicated block id: 127 size: 9 cleaned lines of code in 2 files: - toolkit/scripts/genSkewData.scala (27:35) - toolkit/scripts/genSkewData.scala (44:52) duplicated block id: 128 size: 9 cleaned lines of code in 2 files: - client-spark/spark-2/src/main/java/org/apache/spark/shuffle/celeborn/SortBasedShuffleWriter.java (111:121) - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/SortBasedShuffleWriter.java (112:122) duplicated block id: 129 size: 9 cleaned lines of code in 2 files: - charts/celeborn/templates/master-service.yaml (22:30) - charts/celeborn/templates/worker-service.yaml (22:30) duplicated block id: 130 size: 9 cleaned lines of code in 2 files: - worker/src/main/scala/org/apache/celeborn/service/deploy/worker/Controller.scala (162:170) - worker/src/main/scala/org/apache/celeborn/service/deploy/worker/Controller.scala (201:209) duplicated block id: 131 size: 9 cleaned lines of code in 2 files: - master/src/main/java/org/apache/celeborn/service/deploy/master/clustermeta/IMetadataHandler.java (49:57) - master/src/main/java/org/apache/celeborn/service/deploy/master/clustermeta/SingleMasterMetaManager.java (86:94) duplicated block id: 132 size: 9 cleaned lines of code in 2 files: - worker/src/main/scala/org/apache/celeborn/service/deploy/worker/PushDataHandler.scala (646:655) - worker/src/main/scala/org/apache/celeborn/service/deploy/worker/PushDataHandler.scala (818:827) duplicated block id: 133 size: 9 cleaned lines of code in 2 files: - client/src/main/java/org/apache/celeborn/client/ShuffleClient.java (134:142) - client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java (1145:1153) duplicated block id: 134 size: 9 cleaned lines of code in 2 files: - master/src/main/scala/org/apache/celeborn/service/deploy/master/Master.scala (530:538) - master/src/main/scala/org/apache/celeborn/service/deploy/master/Master.scala (541:549) duplicated block id: 135 size: 9 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/HashBasedShuffleWriter.java (495:509) - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/SortBasedShuffleWriter.java (412:427) duplicated block id: 136 size: 9 cleaned lines of code in 2 files: - client/src/main/java/org/apache/celeborn/client/ShuffleClient.java (134:142) - client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java (1179:1187) duplicated block id: 137 size: 9 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/HashBasedShuffleWriter.java (268:279) - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/SortBasedShuffleWriter.java (247:258) duplicated block id: 138 size: 9 cleaned lines of code in 2 files: - client-flink/flink-1.15/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleResultPartitionFactory.java (35:47) - client-flink/flink-1.17/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleResultPartitionFactory.java (35:47) duplicated block id: 139 size: 9 cleaned lines of code in 2 files: - client-spark/spark-2/src/main/java/org/apache/spark/shuffle/celeborn/SortBasedShuffleWriter.java (175:185) - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/HashBasedShuffleWriter.java (199:210) duplicated block id: 140 size: 9 cleaned lines of code in 2 files: - master/src/main/scala/org/apache/celeborn/service/deploy/master/Master.scala (516:524) - master/src/main/scala/org/apache/celeborn/service/deploy/master/Master.scala (530:538) duplicated block id: 141 size: 9 cleaned lines of code in 2 files: - common/src/main/java/org/apache/celeborn/common/protocol/PartitionLocation.java (95:103) - common/src/main/java/org/apache/celeborn/common/protocol/PartitionLocation.java (119:127) duplicated block id: 142 size: 9 cleaned lines of code in 2 files: - common/src/main/scala/org/apache/celeborn/common/protocol/message/ControlMessages.scala (107:115) - master/src/main/scala/org/apache/celeborn/service/deploy/master/Master.scala (422:430) duplicated block id: 143 size: 9 cleaned lines of code in 2 files: - client-spark/spark-2/src/main/java/org/apache/spark/shuffle/celeborn/HashBasedShuffleWriter.java (56:67) - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/HashBasedShuffleWriter.java (60:71) duplicated block id: 144 size: 9 cleaned lines of code in 2 files: - worker/src/main/scala/org/apache/celeborn/service/deploy/worker/PushDataHandler.scala (373:382) - worker/src/main/scala/org/apache/celeborn/service/deploy/worker/PushDataHandler.scala (646:655) duplicated block id: 145 size: 9 cleaned lines of code in 2 files: - client-flink/flink-1.14/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleResultPartition.java (157:171) - client-flink/flink-1.15/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleResultPartition.java (157:171) duplicated block id: 146 size: 9 cleaned lines of code in 2 files: - client-flink/flink-1.14/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleResultPartitionFactory.java (35:46) - client-flink/flink-1.17/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleResultPartitionFactory.java (35:47) duplicated block id: 147 size: 9 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/HashBasedShuffleWriter.java (70:78) - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/SortBasedShuffleWriter.java (61:69) duplicated block id: 148 size: 9 cleaned lines of code in 2 files: - client/src/main/java/org/apache/celeborn/client/ShuffleClient.java (119:127) - client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java (1145:1153) duplicated block id: 149 size: 9 cleaned lines of code in 2 files: - client-spark/spark-2/src/main/java/org/apache/spark/shuffle/celeborn/HashBasedShuffleWriter.java (66:74) - client-spark/spark-2/src/main/java/org/apache/spark/shuffle/celeborn/SortBasedShuffleWriter.java (61:69) duplicated block id: 150 size: 9 cleaned lines of code in 2 files: - master/src/main/scala/org/apache/celeborn/service/deploy/master/Master.scala (516:524) - master/src/main/scala/org/apache/celeborn/service/deploy/master/Master.scala (541:549) duplicated block id: 151 size: 9 cleaned lines of code in 2 files: - client/src/main/java/org/apache/celeborn/client/ShuffleClient.java (119:127) - client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java (1179:1187) duplicated block id: 152 size: 9 cleaned lines of code in 2 files: - client-spark/spark-2/src/main/scala/org/apache/spark/shuffle/celeborn/CelebornShuffleReader.scala (122:134) - client-spark/spark-3/src/main/scala/org/apache/spark/shuffle/celeborn/CelebornShuffleReader.scala (139:151) duplicated block id: 153 size: 9 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/HashBasedShuffleWriter.java (199:210) - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/SortBasedShuffleWriter.java (222:232) duplicated block id: 154 size: 8 cleaned lines of code in 2 files: - client-spark/spark-2/src/main/java/org/apache/spark/shuffle/celeborn/HashBasedShuffleWriter.java (195:203) - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/SortBasedShuffleWriter.java (250:259) duplicated block id: 155 size: 8 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/SparkShuffleManager.java (268:275) - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/SparkShuffleManager.java (294:301) duplicated block id: 156 size: 8 cleaned lines of code in 2 files: - charts/celeborn/templates/prometheus-podmonitor.yaml (49:56) - charts/celeborn/templates/worker-service.yaml (22:29) duplicated block id: 157 size: 8 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/SparkShuffleManager.java (237:244) - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/SparkShuffleManager.java (268:275) duplicated block id: 158 size: 8 cleaned lines of code in 2 files: - client-flink/common/src/main/java/org/apache/celeborn/plugin/flink/readclient/FlinkShuffleClientImpl.java (319:326) - client-flink/common/src/main/java/org/apache/celeborn/plugin/flink/readclient/FlinkShuffleClientImpl.java (357:364) duplicated block id: 159 size: 8 cleaned lines of code in 2 files: - master/src/main/java/org/apache/celeborn/service/deploy/master/clustermeta/ha/HAMasterMetaManager.java (208:215) - master/src/main/java/org/apache/celeborn/service/deploy/master/clustermeta/ha/HAMasterMetaManager.java (243:250) duplicated block id: 160 size: 8 cleaned lines of code in 2 files: - worker/src/main/scala/org/apache/celeborn/service/deploy/worker/storage/StorageManager.scala (337:344) - worker/src/main/scala/org/apache/celeborn/service/deploy/worker/storage/StorageManager.scala (384:391) duplicated block id: 161 size: 8 cleaned lines of code in 2 files: - client/src/main/scala/org/apache/celeborn/client/commit/CommitHandler.scala (446:453) - client/src/main/scala/org/apache/celeborn/client/commit/CommitHandler.scala (467:474) duplicated block id: 162 size: 8 cleaned lines of code in 2 files: - worker/src/main/scala/org/apache/celeborn/service/deploy/worker/storage/StorageManager.scala (346:354) - worker/src/main/scala/org/apache/celeborn/service/deploy/worker/storage/StorageManager.scala (393:401) duplicated block id: 163 size: 8 cleaned lines of code in 2 files: - master/src/main/java/org/apache/celeborn/service/deploy/master/clustermeta/AbstractMetaManager.java (138:145) - master/src/main/java/org/apache/celeborn/service/deploy/master/clustermeta/ha/HAMasterMetaManager.java (191:198) duplicated block id: 164 size: 8 cleaned lines of code in 2 files: - charts/celeborn/templates/prometheus-podmonitor.yaml (24:31) - charts/celeborn/templates/worker-service.yaml (22:29) duplicated block id: 165 size: 8 cleaned lines of code in 2 files: - worker/src/main/scala/org/apache/celeborn/service/deploy/worker/storage/StorageManager.scala (335:342) - worker/src/main/scala/org/apache/celeborn/service/deploy/worker/storage/StorageManager.scala (344:351) duplicated block id: 166 size: 8 cleaned lines of code in 2 files: - client-spark/spark-2/src/main/java/org/apache/spark/shuffle/celeborn/HashBasedShuffleWriter.java (366:377) - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/HashBasedShuffleWriter.java (479:490) duplicated block id: 167 size: 8 cleaned lines of code in 2 files: - worker/src/main/java/org/apache/celeborn/service/deploy/worker/storage/PartitionFilesSorter.java (178:185) - worker/src/main/java/org/apache/celeborn/service/deploy/worker/storage/PartitionFilesSorter.java (225:232) duplicated block id: 168 size: 8 cleaned lines of code in 2 files: - client/src/main/java/org/apache/celeborn/client/ShuffleClient.java (119:126) - client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java (765:772) duplicated block id: 169 size: 8 cleaned lines of code in 2 files: - master/src/main/java/org/apache/celeborn/service/deploy/master/clustermeta/AbstractMetaManager.java (138:145) - master/src/main/java/org/apache/celeborn/service/deploy/master/clustermeta/SingleMasterMetaManager.java (86:93) duplicated block id: 170 size: 8 cleaned lines of code in 2 files: - common/src/main/scala/org/apache/celeborn/common/internal/config/ConfigBuilder.scala (183:190) - common/src/main/scala/org/apache/celeborn/common/internal/config/ConfigBuilder.scala (204:211) duplicated block id: 171 size: 8 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (266:273) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (282:289) duplicated block id: 172 size: 8 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornCompressionSchemes.scala (391:399) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornCompressionSchemes.scala (419:427) duplicated block id: 173 size: 8 cleaned lines of code in 2 files: - common/src/main/java/org/apache/celeborn/common/write/InFlightRequestTracker.java (106:118) - common/src/main/java/org/apache/celeborn/common/write/InFlightRequestTracker.java (146:158) duplicated block id: 174 size: 8 cleaned lines of code in 2 files: - master/src/main/java/org/apache/celeborn/service/deploy/master/clustermeta/AbstractMetaManager.java (138:145) - master/src/main/java/org/apache/celeborn/service/deploy/master/clustermeta/IMetadataHandler.java (49:56) duplicated block id: 175 size: 8 cleaned lines of code in 2 files: - common/src/main/java/org/apache/celeborn/common/protocol/PartitionLocation.java (86:93) - common/src/main/java/org/apache/celeborn/common/protocol/PartitionLocation.java (133:140) duplicated block id: 176 size: 8 cleaned lines of code in 2 files: - common/src/main/java/org/apache/celeborn/common/protocol/PartitionLocation.java (86:93) - common/src/main/java/org/apache/celeborn/common/protocol/PartitionLocation.java (109:116) duplicated block id: 177 size: 8 cleaned lines of code in 2 files: - client/src/main/java/org/apache/celeborn/client/compress/Lz4Trait.java (22:31) - client/src/main/java/org/apache/celeborn/client/compress/ZstdTrait.java (22:31) duplicated block id: 178 size: 8 cleaned lines of code in 2 files: - worker/src/main/scala/org/apache/celeborn/service/deploy/worker/storage/StorageManager.scala (382:389) - worker/src/main/scala/org/apache/celeborn/service/deploy/worker/storage/StorageManager.scala (391:398) duplicated block id: 179 size: 8 cleaned lines of code in 2 files: - worker/src/main/scala/org/apache/celeborn/service/deploy/worker/FetchHandler.scala (113:120) - worker/src/main/scala/org/apache/celeborn/service/deploy/worker/FetchHandler.scala (155:162) duplicated block id: 180 size: 8 cleaned lines of code in 2 files: - common/src/main/scala/org/apache/celeborn/common/internal/config/ConfigBuilder.scala (140:147) - common/src/main/scala/org/apache/celeborn/common/internal/config/ConfigBuilder.scala (204:211) duplicated block id: 181 size: 8 cleaned lines of code in 2 files: - client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java (1330:1337) - client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java (1411:1418) duplicated block id: 182 size: 8 cleaned lines of code in 2 files: - master/src/main/scala/org/apache/celeborn/service/deploy/master/Master.scala (296:303) - master/src/main/scala/org/apache/celeborn/service/deploy/master/Master.scala (339:346) duplicated block id: 183 size: 8 cleaned lines of code in 2 files: - client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java (765:772) - client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java (1145:1152) duplicated block id: 184 size: 8 cleaned lines of code in 2 files: - worker/src/main/scala/org/apache/celeborn/service/deploy/worker/PushDataHandler.scala (285:297) - worker/src/main/scala/org/apache/celeborn/service/deploy/worker/PushDataHandler.scala (541:553) duplicated block id: 185 size: 8 cleaned lines of code in 2 files: - client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java (765:772) - client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java (1179:1186) duplicated block id: 186 size: 8 cleaned lines of code in 2 files: - common/src/main/scala/org/apache/celeborn/common/internal/config/ConfigBuilder.scala (164:171) - common/src/main/scala/org/apache/celeborn/common/internal/config/ConfigBuilder.scala (183:190) duplicated block id: 187 size: 8 cleaned lines of code in 2 files: - common/src/main/scala/org/apache/celeborn/common/protocol/message/ControlMessages.scala (423:430) - master/src/main/scala/org/apache/celeborn/service/deploy/master/Master.scala (323:330) duplicated block id: 188 size: 8 cleaned lines of code in 2 files: - common/src/main/scala/org/apache/celeborn/common/internal/config/ConfigBuilder.scala (164:171) - common/src/main/scala/org/apache/celeborn/common/internal/config/ConfigBuilder.scala (204:211) duplicated block id: 189 size: 8 cleaned lines of code in 2 files: - charts/celeborn/templates/master-service.yaml (22:29) - charts/celeborn/templates/prometheus-podmonitor.yaml (49:56) duplicated block id: 190 size: 8 cleaned lines of code in 2 files: - common/src/main/java/org/apache/celeborn/common/network/protocol/PushData.java (100:110) - common/src/main/java/org/apache/celeborn/common/network/protocol/PushMergedData.java (114:124) duplicated block id: 191 size: 8 cleaned lines of code in 2 files: - common/src/main/java/org/apache/celeborn/common/protocol/PartitionLocation.java (202:209) - common/src/main/java/org/apache/celeborn/common/protocol/PartitionLocation.java (305:312) duplicated block id: 192 size: 8 cleaned lines of code in 2 files: - common/src/main/scala/org/apache/celeborn/common/internal/config/ConfigBuilder.scala (140:147) - common/src/main/scala/org/apache/celeborn/common/internal/config/ConfigBuilder.scala (164:171) duplicated block id: 193 size: 8 cleaned lines of code in 2 files: - client/src/main/java/org/apache/celeborn/client/ShuffleClient.java (134:141) - client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java (765:772) duplicated block id: 194 size: 8 cleaned lines of code in 2 files: - common/src/main/scala/org/apache/celeborn/common/internal/config/ConfigBuilder.scala (140:147) - common/src/main/scala/org/apache/celeborn/common/internal/config/ConfigBuilder.scala (183:190) duplicated block id: 195 size: 8 cleaned lines of code in 2 files: - client-spark/spark-2/src/main/java/org/apache/spark/shuffle/celeborn/SortBasedShuffleWriter.java (102:109) - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/SortBasedShuffleWriter.java (103:110) duplicated block id: 196 size: 8 cleaned lines of code in 2 files: - worker/src/main/scala/org/apache/celeborn/service/deploy/worker/PushDataHandler.scala (246:253) - worker/src/main/scala/org/apache/celeborn/service/deploy/worker/PushDataHandler.scala (508:515) duplicated block id: 197 size: 8 cleaned lines of code in 2 files: - common/src/main/java/org/apache/celeborn/common/network/buffer/FileSegmentManagedBuffer.java (114:125) - common/src/main/java/org/apache/celeborn/common/network/buffer/NioManagedBuffer.java (51:62) duplicated block id: 198 size: 8 cleaned lines of code in 2 files: - charts/celeborn/templates/master-service.yaml (22:29) - charts/celeborn/templates/prometheus-podmonitor.yaml (24:31) duplicated block id: 199 size: 7 cleaned lines of code in 2 files: - client-spark/spark-2/src/main/scala/org/apache/spark/shuffle/celeborn/CelebornShuffleReader.scala (67:73) - client-spark/spark-3/src/main/scala/org/apache/spark/shuffle/celeborn/CelebornShuffleReader.scala (84:90) duplicated block id: 200 size: 7 cleaned lines of code in 2 files: - client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java (293:299) - client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java (1081:1087) duplicated block id: 201 size: 7 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (188:194) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (233:239) duplicated block id: 202 size: 7 cleaned lines of code in 2 files: - client/src/main/java/org/apache/celeborn/client/compress/Lz4Decompressor.java (61:72) - client/src/main/java/org/apache/celeborn/client/compress/ZstdDecompressor.java (64:73) duplicated block id: 203 size: 7 cleaned lines of code in 2 files: - common/src/main/scala/org/apache/celeborn/common/protocol/message/ControlMessages.scala (424:430) - master/src/main/scala/org/apache/celeborn/service/deploy/master/Master.scala (542:548) duplicated block id: 204 size: 7 cleaned lines of code in 2 files: - client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java (1285:1291) - client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java (1432:1438) duplicated block id: 205 size: 7 cleaned lines of code in 2 files: - common/src/main/scala/org/apache/celeborn/common/protocol/message/ControlMessages.scala (639:645) - worker/src/main/scala/org/apache/celeborn/service/deploy/worker/Controller.scala (83:89) duplicated block id: 206 size: 7 cleaned lines of code in 2 files: - client-spark/spark-2/src/main/scala/org/apache/spark/shuffle/celeborn/CelebornShuffleReader.scala (31:37) - client-spark/spark-3/src/main/scala/org/apache/spark/shuffle/celeborn/CelebornShuffleReader.scala (32:38) duplicated block id: 207 size: 7 cleaned lines of code in 2 files: - master/src/main/java/org/apache/celeborn/service/deploy/master/clustermeta/SingleMasterMetaManager.java (97:103) - master/src/main/scala/org/apache/celeborn/service/deploy/master/Master.scala (297:303) duplicated block id: 208 size: 7 cleaned lines of code in 2 files: - master/src/main/java/org/apache/celeborn/service/deploy/master/clustermeta/SingleMasterMetaManager.java (86:92) - master/src/main/java/org/apache/celeborn/service/deploy/master/clustermeta/SingleMasterMetaManager.java (110:116) duplicated block id: 209 size: 7 cleaned lines of code in 2 files: - master/src/main/java/org/apache/celeborn/service/deploy/master/clustermeta/IMetadataHandler.java (49:55) - master/src/main/java/org/apache/celeborn/service/deploy/master/clustermeta/ha/HAMasterMetaManager.java (228:234) duplicated block id: 210 size: 7 cleaned lines of code in 2 files: - charts/celeborn/templates/configmap.yaml (22:28) - charts/celeborn/templates/prometheus-podmonitor.yaml (49:55) duplicated block id: 211 size: 7 cleaned lines of code in 2 files: - common/src/main/scala/org/apache/celeborn/common/protocol/message/ControlMessages.scala (424:430) - master/src/main/scala/org/apache/celeborn/service/deploy/master/Master.scala (517:523) duplicated block id: 212 size: 7 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (158:164) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (188:194) duplicated block id: 213 size: 7 cleaned lines of code in 2 files: - common/src/main/scala/org/apache/celeborn/common/protocol/message/ControlMessages.scala (424:430) - master/src/main/scala/org/apache/celeborn/service/deploy/master/Master.scala (531:537) duplicated block id: 214 size: 7 cleaned lines of code in 2 files: - master/src/main/java/org/apache/celeborn/service/deploy/master/clustermeta/AbstractMetaManager.java (138:144) - master/src/main/java/org/apache/celeborn/service/deploy/master/clustermeta/ha/HAMasterMetaManager.java (228:234) duplicated block id: 215 size: 7 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (158:164) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (173:179) duplicated block id: 216 size: 7 cleaned lines of code in 2 files: - master/src/main/scala/org/apache/celeborn/service/deploy/master/Master.scala (297:303) - master/src/main/scala/org/apache/celeborn/service/deploy/master/Master.scala (542:548) duplicated block id: 217 size: 7 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/SortBasedShuffleWriter.java (321:327) - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/SortBasedShuffleWriter.java (330:336) duplicated block id: 218 size: 7 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (188:194) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (250:256) duplicated block id: 219 size: 7 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (158:164) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (233:239) duplicated block id: 220 size: 7 cleaned lines of code in 2 files: - master/src/main/java/org/apache/celeborn/service/deploy/master/clustermeta/IMetadataHandler.java (61:67) - master/src/main/java/org/apache/celeborn/service/deploy/master/clustermeta/ha/HAMasterMetaManager.java (191:197) duplicated block id: 221 size: 7 cleaned lines of code in 2 files: - worker/src/main/scala/org/apache/celeborn/service/deploy/worker/Controller.scala (482:488) - worker/src/main/scala/org/apache/celeborn/service/deploy/worker/Controller.scala (502:508) duplicated block id: 222 size: 7 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (158:164) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (218:224) duplicated block id: 223 size: 7 cleaned lines of code in 2 files: - client-flink/common/src/main/java/org/apache/celeborn/plugin/flink/AbstractRemoteShuffleResultPartitionFactory.java (127:133) - client-flink/flink-1.17/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleResultPartitionFactory.java (49:55) duplicated block id: 224 size: 7 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (158:164) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (203:209) duplicated block id: 225 size: 7 cleaned lines of code in 2 files: - master/src/main/java/org/apache/celeborn/service/deploy/master/clustermeta/IMetadataHandler.java (61:67) - master/src/main/java/org/apache/celeborn/service/deploy/master/clustermeta/ha/HAMasterMetaManager.java (228:234) duplicated block id: 226 size: 7 cleaned lines of code in 2 files: - master/src/main/java/org/apache/celeborn/service/deploy/master/clustermeta/ha/MetaHandler.java (200:206) - master/src/main/java/org/apache/celeborn/service/deploy/master/clustermeta/ha/MetaHandler.java (208:214) duplicated block id: 227 size: 7 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (188:194) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (283:289) duplicated block id: 228 size: 7 cleaned lines of code in 2 files: - client-spark/spark-2/src/main/java/org/apache/spark/shuffle/celeborn/SortBasedShuffleWriter.java (299:305) - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/SortBasedShuffleWriter.java (330:336) duplicated block id: 229 size: 7 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (188:194) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (267:273) duplicated block id: 230 size: 7 cleaned lines of code in 2 files: - client-spark/spark-2/src/main/java/org/apache/spark/shuffle/celeborn/SortBasedShuffleWriter.java (308:314) - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/SortBasedShuffleWriter.java (321:327) duplicated block id: 231 size: 7 cleaned lines of code in 2 files: - master/src/main/java/org/apache/celeborn/service/deploy/master/clustermeta/ha/HAMasterMetaManager.java (191:197) - master/src/main/java/org/apache/celeborn/service/deploy/master/clustermeta/ha/HAMasterMetaManager.java (228:234) duplicated block id: 232 size: 7 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (218:224) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (233:239) duplicated block id: 233 size: 7 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (218:224) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (283:289) duplicated block id: 234 size: 7 cleaned lines of code in 2 files: - client-spark/spark-2/src/main/java/org/apache/spark/shuffle/celeborn/HashBasedShuffleWriter.java (146:152) - client-spark/spark-2/src/main/java/org/apache/spark/shuffle/celeborn/SortBasedShuffleWriter.java (156:162) duplicated block id: 235 size: 7 cleaned lines of code in 2 files: - client-spark/spark-2/src/main/java/org/apache/spark/shuffle/celeborn/SortBasedShuffleWriter.java (64:71) - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/SortBasedShuffleWriter.java (64:71) duplicated block id: 236 size: 7 cleaned lines of code in 2 files: - client-spark/spark-2/src/main/java/org/apache/spark/shuffle/celeborn/HashBasedShuffleWriter.java (146:152) - client-spark/spark-2/src/main/java/org/apache/spark/shuffle/celeborn/SortBasedShuffleWriter.java (136:142) duplicated block id: 237 size: 7 cleaned lines of code in 2 files: - charts/celeborn/templates/configmap.yaml (22:28) - charts/celeborn/templates/master-service.yaml (22:28) duplicated block id: 238 size: 7 cleaned lines of code in 2 files: - common/src/main/java/org/apache/celeborn/common/network/protocol/RpcRequest.java (45:57) - common/src/main/java/org/apache/celeborn/common/network/protocol/RpcResponse.java (40:52) duplicated block id: 239 size: 7 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (218:224) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (250:256) duplicated block id: 240 size: 7 cleaned lines of code in 2 files: - client-spark/spark-2/src/main/java/org/apache/spark/shuffle/celeborn/HashBasedShuffleWriter.java (146:152) - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/SortBasedShuffleWriter.java (156:162) duplicated block id: 241 size: 7 cleaned lines of code in 2 files: - client-spark/spark-2/src/main/java/org/apache/spark/shuffle/celeborn/HashBasedShuffleWriter.java (146:152) - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/SortBasedShuffleWriter.java (136:142) duplicated block id: 242 size: 7 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (128:134) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (250:256) duplicated block id: 243 size: 7 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (218:224) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (267:273) duplicated block id: 244 size: 7 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (128:134) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (267:273) duplicated block id: 245 size: 7 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (233:239) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (267:273) duplicated block id: 246 size: 7 cleaned lines of code in 2 files: - worker/src/main/scala/org/apache/celeborn/service/deploy/worker/Worker.scala (583:589) - worker/src/main/scala/org/apache/celeborn/service/deploy/worker/Worker.scala (613:619) duplicated block id: 247 size: 7 cleaned lines of code in 2 files: - common/src/main/scala/org/apache/celeborn/common/protocol/message/ControlMessages.scala (579:585) - master/src/main/scala/org/apache/celeborn/service/deploy/master/Master.scala (259:265) duplicated block id: 248 size: 7 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (233:239) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (250:256) duplicated block id: 249 size: 7 cleaned lines of code in 2 files: - charts/celeborn/templates/configmap.yaml (22:28) - charts/celeborn/templates/prometheus-podmonitor.yaml (24:30) duplicated block id: 250 size: 7 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (128:134) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (283:289) duplicated block id: 251 size: 7 cleaned lines of code in 2 files: - client/src/main/scala/org/apache/celeborn/client/commit/MapPartitionCommitHandler.scala (175:181) - client/src/main/scala/org/apache/celeborn/client/commit/ReducePartitionCommitHandler.scala (225:231) duplicated block id: 252 size: 7 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (233:239) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (283:289) duplicated block id: 253 size: 7 cleaned lines of code in 2 files: - master/src/main/java/org/apache/celeborn/service/deploy/master/clustermeta/SingleMasterMetaManager.java (86:92) - master/src/main/java/org/apache/celeborn/service/deploy/master/clustermeta/ha/HAMasterMetaManager.java (228:234) duplicated block id: 254 size: 7 cleaned lines of code in 2 files: - common/src/main/java/org/apache/celeborn/common/network/protocol/BufferStreamEnd.java (26:36) - common/src/main/java/org/apache/celeborn/common/network/protocol/ReadData.java (33:43) duplicated block id: 255 size: 7 cleaned lines of code in 2 files: - master/src/main/java/org/apache/celeborn/service/deploy/master/clustermeta/IMetadataHandler.java (61:67) - master/src/main/java/org/apache/celeborn/service/deploy/master/clustermeta/SingleMasterMetaManager.java (110:116) duplicated block id: 256 size: 7 cleaned lines of code in 2 files: - common/src/main/java/org/apache/celeborn/common/network/protocol/PushDataHandShake.java (59:67) - common/src/main/java/org/apache/celeborn/common/network/protocol/RegionFinish.java (49:57) duplicated block id: 257 size: 7 cleaned lines of code in 2 files: - master/src/main/java/org/apache/celeborn/service/deploy/master/clustermeta/IMetadataHandler.java (61:67) - master/src/main/java/org/apache/celeborn/service/deploy/master/clustermeta/SingleMasterMetaManager.java (86:92) duplicated block id: 258 size: 7 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (143:149) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (250:256) duplicated block id: 259 size: 7 cleaned lines of code in 2 files: - master/src/main/scala/org/apache/celeborn/service/deploy/master/Master.scala (245:251) - master/src/main/scala/org/apache/celeborn/service/deploy/master/Master.scala (356:362) duplicated block id: 260 size: 7 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (188:194) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (218:224) duplicated block id: 261 size: 7 cleaned lines of code in 2 files: - master/src/main/java/org/apache/celeborn/service/deploy/master/clustermeta/SingleMasterMetaManager.java (97:103) - master/src/main/scala/org/apache/celeborn/service/deploy/master/Master.scala (542:548) duplicated block id: 262 size: 7 cleaned lines of code in 2 files: - worker/src/main/java/org/apache/celeborn/service/deploy/worker/storage/FileWriter.java (90:96) - worker/src/main/java/org/apache/celeborn/service/deploy/worker/storage/ReducePartitionFileWriter.java (42:48) duplicated block id: 263 size: 7 cleaned lines of code in 2 files: - master/src/main/java/org/apache/celeborn/service/deploy/master/clustermeta/ha/MetaHandler.java (168:174) - master/src/main/java/org/apache/celeborn/service/deploy/master/clustermeta/ha/MetaHandler.java (200:206) duplicated block id: 264 size: 7 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (188:194) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (203:209) duplicated block id: 265 size: 7 cleaned lines of code in 2 files: - master/src/main/java/org/apache/celeborn/service/deploy/master/clustermeta/ha/MetaHandler.java (168:174) - master/src/main/java/org/apache/celeborn/service/deploy/master/clustermeta/ha/MetaHandler.java (208:214) duplicated block id: 266 size: 7 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (143:149) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (267:273) duplicated block id: 267 size: 7 cleaned lines of code in 2 files: - client-spark/spark-2/src/main/java/org/apache/spark/shuffle/celeborn/SortBasedShuffleWriter.java (201:208) - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/HashBasedShuffleWriter.java (271:279) duplicated block id: 268 size: 7 cleaned lines of code in 2 files: - master/src/main/java/org/apache/celeborn/service/deploy/master/clustermeta/AbstractMetaManager.java (138:144) - master/src/main/java/org/apache/celeborn/service/deploy/master/clustermeta/IMetadataHandler.java (61:67) duplicated block id: 269 size: 7 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (143:149) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (283:289) duplicated block id: 270 size: 7 cleaned lines of code in 2 files: - charts/celeborn/templates/configmap.yaml (22:28) - charts/celeborn/templates/worker-service.yaml (22:28) duplicated block id: 271 size: 7 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/HashBasedShuffleWriter.java (166:172) - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/SortBasedShuffleWriter.java (156:162) duplicated block id: 272 size: 7 cleaned lines of code in 2 files: - client-spark/spark-2/src/main/scala/org/apache/spark/shuffle/celeborn/CelebornShuffleReader.scala (76:84) - client-spark/spark-3/src/main/scala/org/apache/spark/shuffle/celeborn/CelebornShuffleReader.scala (93:101) duplicated block id: 273 size: 7 cleaned lines of code in 2 files: - master/src/main/java/org/apache/celeborn/service/deploy/master/clustermeta/IMetadataHandler.java (49:55) - master/src/main/java/org/apache/celeborn/service/deploy/master/clustermeta/SingleMasterMetaManager.java (110:116) duplicated block id: 274 size: 7 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/HashBasedShuffleWriter.java (166:172) - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/SortBasedShuffleWriter.java (136:142) duplicated block id: 275 size: 7 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (203:209) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (218:224) duplicated block id: 276 size: 7 cleaned lines of code in 2 files: - client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java (438:444) - client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java (1330:1336) duplicated block id: 277 size: 7 cleaned lines of code in 2 files: - client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java (1266:1272) - client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java (1349:1355) duplicated block id: 278 size: 7 cleaned lines of code in 2 files: - client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java (1266:1272) - client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java (1319:1325) duplicated block id: 279 size: 7 cleaned lines of code in 2 files: - client-spark/spark-2/src/main/java/org/apache/spark/shuffle/celeborn/SortBasedShuffleWriter.java (53:62) - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/SortBasedShuffleWriter.java (53:62) duplicated block id: 280 size: 7 cleaned lines of code in 2 files: - client-spark/spark-2/src/main/java/org/apache/spark/shuffle/celeborn/SparkShuffleManager.java (127:133) - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/SparkShuffleManager.java (135:141) duplicated block id: 281 size: 7 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (128:134) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (143:149) duplicated block id: 282 size: 7 cleaned lines of code in 2 files: - master/src/main/java/org/apache/celeborn/service/deploy/master/clustermeta/SingleMasterMetaManager.java (97:103) - master/src/main/scala/org/apache/celeborn/service/deploy/master/Master.scala (517:523) duplicated block id: 283 size: 7 cleaned lines of code in 2 files: - client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java (1266:1272) - client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java (1361:1367) duplicated block id: 284 size: 7 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (128:134) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (158:164) duplicated block id: 285 size: 7 cleaned lines of code in 2 files: - master/src/main/java/org/apache/celeborn/service/deploy/master/clustermeta/SingleMasterMetaManager.java (97:103) - master/src/main/scala/org/apache/celeborn/service/deploy/master/Master.scala (531:537) duplicated block id: 286 size: 7 cleaned lines of code in 2 files: - worker/src/main/java/org/apache/celeborn/service/deploy/worker/storage/MapPartitionFileWriter.java (131:137) - worker/src/main/java/org/apache/celeborn/service/deploy/worker/storage/ReducePartitionFileWriter.java (96:102) duplicated block id: 287 size: 7 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (128:134) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (173:179) duplicated block id: 288 size: 7 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (128:134) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (188:194) duplicated block id: 289 size: 7 cleaned lines of code in 2 files: - client-spark/spark-2/src/main/java/org/apache/spark/shuffle/celeborn/HashBasedShuffleWriter.java (76:85) - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/HashBasedShuffleWriter.java (81:90) duplicated block id: 290 size: 7 cleaned lines of code in 2 files: - common/src/main/scala/org/apache/celeborn/common/protocol/message/ControlMessages.scala (424:430) - master/src/main/scala/org/apache/celeborn/service/deploy/master/Master.scala (340:346) duplicated block id: 291 size: 7 cleaned lines of code in 2 files: - master/src/main/java/org/apache/celeborn/service/deploy/master/clustermeta/IMetadataHandler.java (49:55) - master/src/main/java/org/apache/celeborn/service/deploy/master/clustermeta/IMetadataHandler.java (61:67) duplicated block id: 292 size: 7 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (128:134) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (203:209) duplicated block id: 293 size: 7 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (128:134) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (218:224) duplicated block id: 294 size: 7 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (143:149) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (218:224) duplicated block id: 295 size: 7 cleaned lines of code in 2 files: - client-spark/spark-2/src/main/java/org/apache/spark/shuffle/celeborn/SortBasedShuffleWriter.java (156:162) - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/HashBasedShuffleWriter.java (166:172) duplicated block id: 296 size: 7 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (250:256) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (283:289) duplicated block id: 297 size: 7 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (143:149) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (188:194) duplicated block id: 298 size: 7 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (128:134) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (233:239) duplicated block id: 299 size: 7 cleaned lines of code in 2 files: - client-spark/spark-2/src/main/java/org/apache/spark/shuffle/celeborn/SortBasedShuffleWriter.java (299:305) - client-spark/spark-2/src/main/java/org/apache/spark/shuffle/celeborn/SortBasedShuffleWriter.java (308:314) duplicated block id: 300 size: 7 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (143:149) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (203:209) duplicated block id: 301 size: 7 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (250:256) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (267:273) duplicated block id: 302 size: 7 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (173:179) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (267:273) duplicated block id: 303 size: 7 cleaned lines of code in 2 files: - client-spark/spark-2/src/main/java/org/apache/spark/shuffle/celeborn/SortBasedShuffleWriter.java (136:142) - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/HashBasedShuffleWriter.java (166:172) duplicated block id: 304 size: 7 cleaned lines of code in 2 files: - master/src/main/scala/org/apache/celeborn/service/deploy/master/Master.scala (324:330) - master/src/main/scala/org/apache/celeborn/service/deploy/master/Master.scala (542:548) duplicated block id: 305 size: 7 cleaned lines of code in 2 files: - common/src/main/scala/org/apache/celeborn/common/protocol/message/ControlMessages.scala (424:430) - master/src/main/scala/org/apache/celeborn/service/deploy/master/Master.scala (297:303) duplicated block id: 306 size: 7 cleaned lines of code in 2 files: - master/src/main/scala/org/apache/celeborn/service/deploy/master/Master.scala (297:303) - master/src/main/scala/org/apache/celeborn/service/deploy/master/Master.scala (324:330) duplicated block id: 307 size: 7 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/HashBasedShuffleWriter.java (216:224) - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/SortBasedShuffleWriter.java (235:243) duplicated block id: 308 size: 7 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (173:179) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (250:256) duplicated block id: 309 size: 7 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (143:149) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (233:239) duplicated block id: 310 size: 7 cleaned lines of code in 2 files: - client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java (1319:1325) - client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java (1349:1355) duplicated block id: 311 size: 7 cleaned lines of code in 2 files: - master/src/main/java/org/apache/celeborn/service/deploy/master/clustermeta/AbstractMetaManager.java (138:144) - master/src/main/java/org/apache/celeborn/service/deploy/master/clustermeta/SingleMasterMetaManager.java (110:116) duplicated block id: 312 size: 7 cleaned lines of code in 2 files: - client-spark/spark-2/src/main/java/org/apache/spark/shuffle/celeborn/HashBasedShuffleWriter.java (379:386) - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/HashBasedShuffleWriter.java (492:499) duplicated block id: 313 size: 7 cleaned lines of code in 2 files: - client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java (1319:1325) - client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java (1361:1367) duplicated block id: 314 size: 7 cleaned lines of code in 2 files: - master/src/main/java/org/apache/celeborn/service/deploy/master/clustermeta/SingleMasterMetaManager.java (110:116) - master/src/main/java/org/apache/celeborn/service/deploy/master/clustermeta/ha/HAMasterMetaManager.java (191:197) duplicated block id: 315 size: 7 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (173:179) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (283:289) duplicated block id: 316 size: 7 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (203:209) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (233:239) duplicated block id: 317 size: 7 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (143:149) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (173:179) duplicated block id: 318 size: 7 cleaned lines of code in 2 files: - client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java (438:444) - client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java (1411:1417) duplicated block id: 319 size: 7 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/SortBasedShuffleWriter.java (99:105) - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/SortBasedShuffleWriter.java (174:180) duplicated block id: 320 size: 7 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (143:149) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (158:164) duplicated block id: 321 size: 7 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (203:209) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (283:289) duplicated block id: 322 size: 7 cleaned lines of code in 2 files: - master/src/main/scala/org/apache/celeborn/service/deploy/master/Master.scala (340:346) - master/src/main/scala/org/apache/celeborn/service/deploy/master/Master.scala (531:537) duplicated block id: 323 size: 7 cleaned lines of code in 2 files: - client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java (1284:1290) - client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java (1399:1405) duplicated block id: 324 size: 7 cleaned lines of code in 2 files: - master/src/main/scala/org/apache/celeborn/service/deploy/master/Master.scala (340:346) - master/src/main/scala/org/apache/celeborn/service/deploy/master/Master.scala (517:523) duplicated block id: 325 size: 7 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (203:209) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (267:273) duplicated block id: 326 size: 7 cleaned lines of code in 2 files: - charts/celeborn/templates/master-statefulset.yaml (29:35) - charts/celeborn/templates/worker-statefulset.yaml (29:35) duplicated block id: 327 size: 7 cleaned lines of code in 2 files: - worker/src/main/java/org/apache/celeborn/service/deploy/worker/storage/FileWriter.java (90:96) - worker/src/main/java/org/apache/celeborn/service/deploy/worker/storage/MapPartitionFileWriter.java (56:62) duplicated block id: 328 size: 7 cleaned lines of code in 2 files: - client-flink/common/src/main/java/org/apache/celeborn/plugin/flink/AbstractRemoteShuffleResultPartitionFactory.java (127:133) - client-flink/common/src/main/java/org/apache/celeborn/plugin/flink/AbstractRemoteShuffleResultPartitionFactory.java (163:169) duplicated block id: 329 size: 7 cleaned lines of code in 2 files: - common/src/main/scala/org/apache/celeborn/common/protocol/message/ControlMessages.scala (424:430) - master/src/main/java/org/apache/celeborn/service/deploy/master/clustermeta/SingleMasterMetaManager.java (97:103) duplicated block id: 330 size: 7 cleaned lines of code in 2 files: - client-spark/spark-2/src/main/java/org/apache/spark/shuffle/celeborn/SortBasedShuffleWriter.java (372:381) - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/SortBasedShuffleWriter.java (393:403) duplicated block id: 331 size: 7 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornCompressionSchemes.scala (139:145) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornCompressionSchemes.scala (171:177) duplicated block id: 332 size: 7 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (203:209) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (250:256) duplicated block id: 333 size: 7 cleaned lines of code in 2 files: - client-flink/common/src/main/java/org/apache/celeborn/plugin/flink/AbstractRemoteShuffleResultPartitionFactory.java (127:133) - client-flink/flink-1.14/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleResultPartitionFactory.java (48:54) duplicated block id: 334 size: 7 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (158:164) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (283:289) duplicated block id: 335 size: 7 cleaned lines of code in 2 files: - client-spark/spark-2/src/main/java/org/apache/spark/shuffle/celeborn/HashBasedShuffleWriter.java (195:202) - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/HashBasedShuffleWriter.java (271:279) duplicated block id: 336 size: 7 cleaned lines of code in 2 files: - master/src/main/java/org/apache/celeborn/service/deploy/master/clustermeta/SingleMasterMetaManager.java (97:103) - master/src/main/scala/org/apache/celeborn/service/deploy/master/Master.scala (324:330) duplicated block id: 337 size: 7 cleaned lines of code in 2 files: - master/src/main/java/org/apache/celeborn/service/deploy/master/clustermeta/SingleMasterMetaManager.java (97:103) - master/src/main/scala/org/apache/celeborn/service/deploy/master/Master.scala (340:346) duplicated block id: 338 size: 7 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (173:179) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (188:194) duplicated block id: 339 size: 7 cleaned lines of code in 2 files: - master/src/main/scala/org/apache/celeborn/service/deploy/master/Master.scala (297:303) - master/src/main/scala/org/apache/celeborn/service/deploy/master/Master.scala (531:537) duplicated block id: 340 size: 7 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (173:179) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (203:209) duplicated block id: 341 size: 7 cleaned lines of code in 2 files: - master/src/main/scala/org/apache/celeborn/service/deploy/master/Master.scala (297:303) - master/src/main/scala/org/apache/celeborn/service/deploy/master/Master.scala (517:523) duplicated block id: 342 size: 7 cleaned lines of code in 2 files: - master/src/main/scala/org/apache/celeborn/service/deploy/master/Master.scala (340:346) - master/src/main/scala/org/apache/celeborn/service/deploy/master/Master.scala (542:548) duplicated block id: 343 size: 7 cleaned lines of code in 2 files: - charts/celeborn/templates/master-statefulset.yaml (80:86) - charts/celeborn/templates/worker-statefulset.yaml (80:86) duplicated block id: 344 size: 7 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (173:179) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (233:239) duplicated block id: 345 size: 7 cleaned lines of code in 2 files: - client-flink/common/src/main/java/org/apache/celeborn/plugin/flink/AbstractRemoteShuffleResultPartitionFactory.java (127:133) - client-flink/flink-1.15/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleResultPartitionFactory.java (49:55) duplicated block id: 346 size: 7 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (158:164) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (267:273) duplicated block id: 347 size: 7 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (158:164) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (250:256) duplicated block id: 348 size: 7 cleaned lines of code in 2 files: - master/src/main/scala/org/apache/celeborn/service/deploy/master/Master.scala (324:330) - master/src/main/scala/org/apache/celeborn/service/deploy/master/Master.scala (517:523) duplicated block id: 349 size: 7 cleaned lines of code in 2 files: - common/src/main/java/org/apache/celeborn/common/network/protocol/PushDataHandShake.java (52:58) - common/src/main/java/org/apache/celeborn/common/network/protocol/RegionStart.java (54:60) duplicated block id: 350 size: 7 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (173:179) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (218:224) duplicated block id: 351 size: 7 cleaned lines of code in 2 files: - master/src/main/scala/org/apache/celeborn/service/deploy/master/Master.scala (324:330) - master/src/main/scala/org/apache/celeborn/service/deploy/master/Master.scala (531:537) duplicated block id: 352 size: 6 cleaned lines of code in 2 files: - client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java (1285:1290) - client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java (1459:1464) duplicated block id: 353 size: 6 cleaned lines of code in 2 files: - master/src/main/scala/org/apache/celeborn/service/deploy/master/Master.scala (505:510) - master/src/main/scala/org/apache/celeborn/service/deploy/master/Master.scala (542:547) duplicated block id: 354 size: 6 cleaned lines of code in 2 files: - master/src/main/java/org/apache/celeborn/service/deploy/master/clustermeta/AbstractMetaManager.java (138:143) - master/src/main/java/org/apache/celeborn/service/deploy/master/clustermeta/AbstractMetaManager.java (174:179) duplicated block id: 355 size: 6 cleaned lines of code in 2 files: - client-flink/flink-1.14/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleInputGateFactory.java (47:52) - client-flink/flink-1.17/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleInputGateFactory.java (48:53) duplicated block id: 356 size: 6 cleaned lines of code in 2 files: - client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java (1432:1437) - client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java (1459:1464) duplicated block id: 357 size: 6 cleaned lines of code in 2 files: - master/src/main/java/org/apache/celeborn/service/deploy/master/clustermeta/ha/MetaHandler.java (168:173) - worker/src/main/scala/org/apache/celeborn/service/deploy/worker/Worker.scala (317:322) duplicated block id: 358 size: 6 cleaned lines of code in 2 files: - client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java (1361:1366) - client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java (1399:1404) duplicated block id: 359 size: 6 cleaned lines of code in 2 files: - common/src/main/scala/org/apache/celeborn/common/meta/WorkerPartitionLocationInfo.scala (66:71) - common/src/main/scala/org/apache/celeborn/common/meta/WorkerPartitionLocationInfo.scala (83:88) duplicated block id: 360 size: 6 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (114:119) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (268:273) duplicated block id: 361 size: 6 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (195:200) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (210:215) duplicated block id: 362 size: 6 cleaned lines of code in 2 files: - charts/celeborn/templates/master-statefulset.yaml (22:27) - charts/celeborn/templates/prometheus-podmonitor.yaml (24:29) duplicated block id: 363 size: 6 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (114:119) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (251:256) duplicated block id: 364 size: 6 cleaned lines of code in 2 files: - client-spark/spark-2/src/main/java/org/apache/spark/shuffle/celeborn/SortBasedShuffleWriter.java (96:101) - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/SortBasedShuffleWriter.java (96:101) duplicated block id: 365 size: 6 cleaned lines of code in 2 files: - worker/src/main/scala/org/apache/celeborn/service/deploy/worker/storage/StorageManager.scala (337:342) - worker/src/main/scala/org/apache/celeborn/service/deploy/worker/storage/StorageManager.scala (393:398) duplicated block id: 366 size: 6 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (195:200) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (255:260) duplicated block id: 367 size: 6 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (195:200) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (272:277) duplicated block id: 368 size: 6 cleaned lines of code in 2 files: - common/src/main/java/org/apache/celeborn/common/network/protocol/PushDataHandShake.java (62:67) - common/src/main/java/org/apache/celeborn/common/network/protocol/RegionStart.java (64:69) duplicated block id: 369 size: 6 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (195:200) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (225:230) duplicated block id: 370 size: 6 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (195:200) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (240:245) duplicated block id: 371 size: 6 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (114:119) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (129:134) duplicated block id: 372 size: 6 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/HashBasedShuffleWriter.java (402:407) - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/HashBasedShuffleWriter.java (428:433) duplicated block id: 373 size: 6 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (114:119) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (219:224) duplicated block id: 374 size: 6 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (195:200) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (289:294) duplicated block id: 375 size: 6 cleaned lines of code in 2 files: - master/src/main/scala/org/apache/celeborn/service/deploy/master/Master.scala (421:426) - master/src/main/scala/org/apache/celeborn/service/deploy/master/Master.scala (494:499) duplicated block id: 376 size: 6 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (195:200) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (307:312) duplicated block id: 377 size: 6 cleaned lines of code in 2 files: - worker/src/main/scala/org/apache/celeborn/service/deploy/worker/storage/StorageManager.scala (346:351) - worker/src/main/scala/org/apache/celeborn/service/deploy/worker/storage/StorageManager.scala (384:389) duplicated block id: 378 size: 6 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (114:119) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (204:209) duplicated block id: 379 size: 6 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (114:119) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (174:179) duplicated block id: 380 size: 6 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (114:119) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (189:194) duplicated block id: 381 size: 6 cleaned lines of code in 2 files: - master/src/main/scala/org/apache/celeborn/service/deploy/master/Master.scala (421:426) - master/src/main/scala/org/apache/celeborn/service/deploy/master/Master.scala (463:468) duplicated block id: 382 size: 6 cleaned lines of code in 2 files: - master/src/main/scala/org/apache/celeborn/service/deploy/master/Master.scala (505:510) - master/src/main/scala/org/apache/celeborn/service/deploy/master/Master.scala (531:536) duplicated block id: 383 size: 6 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (114:119) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (159:164) duplicated block id: 384 size: 6 cleaned lines of code in 2 files: - client-flink/common/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleInputGateDelegation.java (129:134) - client-flink/flink-1.14/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleInputGate.java (59:64) duplicated block id: 385 size: 6 cleaned lines of code in 2 files: - master/src/main/scala/org/apache/celeborn/service/deploy/master/Master.scala (505:510) - master/src/main/scala/org/apache/celeborn/service/deploy/master/Master.scala (517:522) duplicated block id: 386 size: 6 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (150:155) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (165:170) duplicated block id: 387 size: 6 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (240:245) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (272:277) duplicated block id: 388 size: 6 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (165:170) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (272:277) duplicated block id: 389 size: 6 cleaned lines of code in 2 files: - client-flink/common/src/main/java/org/apache/celeborn/plugin/flink/readclient/CelebornBufferStream.java (58:63) - client-flink/common/src/main/java/org/apache/celeborn/plugin/flink/readclient/CelebornBufferStream.java (153:158) duplicated block id: 390 size: 6 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (240:245) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (289:294) duplicated block id: 391 size: 6 cleaned lines of code in 2 files: - client/src/main/java/org/apache/celeborn/client/compress/Lz4Compressor.java (56:61) - client/src/main/java/org/apache/celeborn/client/compress/ZstdCompressor.java (63:68) duplicated block id: 392 size: 6 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (165:170) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (255:260) duplicated block id: 393 size: 6 cleaned lines of code in 2 files: - client-spark/spark-2/src/main/scala/org/apache/spark/shuffle/celeborn/CelebornShuffleReader.scala (43:49) - client-spark/spark-3/src/main/scala/org/apache/spark/shuffle/celeborn/CelebornShuffleReader.scala (45:51) duplicated block id: 394 size: 6 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (240:245) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (307:312) duplicated block id: 395 size: 6 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (165:170) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (240:245) duplicated block id: 396 size: 6 cleaned lines of code in 2 files: - common/src/main/scala/org/apache/celeborn/common/metrics/sink/CsvSink.scala (60:69) - common/src/main/scala/org/apache/celeborn/common/metrics/sink/GraphiteSink.scala (79:88) duplicated block id: 397 size: 6 cleaned lines of code in 2 files: - charts/celeborn/templates/master-service.yaml (22:27) - charts/celeborn/templates/master-statefulset.yaml (22:27) duplicated block id: 398 size: 6 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnType.scala (486:491) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnType.scala (534:539) duplicated block id: 399 size: 6 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (165:170) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (307:312) duplicated block id: 400 size: 6 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnType.scala (486:491) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnType.scala (580:585) duplicated block id: 401 size: 6 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (114:119) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (234:239) duplicated block id: 402 size: 6 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (240:245) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (255:260) duplicated block id: 403 size: 6 cleaned lines of code in 2 files: - client-spark/spark-2/src/main/java/org/apache/spark/shuffle/celeborn/HashBasedShuffleWriter.java (328:333) - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/HashBasedShuffleWriter.java (402:407) duplicated block id: 404 size: 6 cleaned lines of code in 2 files: - charts/celeborn/templates/configmap.yaml (22:27) - charts/celeborn/templates/worker-statefulset.yaml (22:27) duplicated block id: 405 size: 6 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (165:170) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (289:294) duplicated block id: 406 size: 6 cleaned lines of code in 2 files: - client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java (1349:1354) - client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java (1399:1404) duplicated block id: 407 size: 6 cleaned lines of code in 2 files: - client-spark/common/src/main/java/org/apache/spark/shuffle/celeborn/SortBasedPusher.java (118:123) - client-spark/spark-2/src/main/java/org/apache/spark/shuffle/celeborn/HashBasedShuffleWriter.java (142:147) duplicated block id: 408 size: 6 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (255:260) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (307:312) duplicated block id: 409 size: 6 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (255:260) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (289:294) duplicated block id: 410 size: 6 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (165:170) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (225:230) duplicated block id: 411 size: 6 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (255:260) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (272:277) duplicated block id: 412 size: 6 cleaned lines of code in 2 files: - client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java (1319:1324) - client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java (1399:1404) duplicated block id: 413 size: 6 cleaned lines of code in 2 files: - project/CelebornBuild.scala (291:299) - project/CelebornBuild.scala (314:322) duplicated block id: 414 size: 6 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (165:170) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (210:215) duplicated block id: 415 size: 6 cleaned lines of code in 2 files: - client-spark/spark-2/src/main/java/org/apache/spark/shuffle/celeborn/SparkShuffleManager.java (207:213) - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/SparkShuffleManager.java (218:224) duplicated block id: 416 size: 6 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (165:170) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (195:200) duplicated block id: 417 size: 6 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (165:170) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (180:185) duplicated block id: 418 size: 6 cleaned lines of code in 2 files: - master/src/main/java/org/apache/celeborn/service/deploy/master/clustermeta/SingleMasterMetaManager.java (47:52) - master/src/main/java/org/apache/celeborn/service/deploy/master/clustermeta/ha/HAMasterMetaManager.java (65:70) duplicated block id: 419 size: 6 cleaned lines of code in 2 files: - master/src/main/java/org/apache/celeborn/service/deploy/master/clustermeta/ha/MetaHandler.java (168:173) - worker/src/main/scala/org/apache/celeborn/service/deploy/worker/Worker.scala (211:216) duplicated block id: 420 size: 6 cleaned lines of code in 2 files: - worker/src/main/scala/org/apache/celeborn/service/deploy/worker/PushDataHandler.scala (223:229) - worker/src/main/scala/org/apache/celeborn/service/deploy/worker/PushDataHandler.scala (487:493) duplicated block id: 421 size: 6 cleaned lines of code in 2 files: - client-spark/spark-2/src/main/scala/org/apache/spark/shuffle/celeborn/CelebornShuffleHandle.scala (25:30) - client-spark/spark-3/src/main/scala/org/apache/spark/shuffle/celeborn/CelebornShuffleHandle.scala (25:30) duplicated block id: 422 size: 6 cleaned lines of code in 2 files: - charts/celeborn/templates/master-service.yaml (22:27) - charts/celeborn/templates/worker-statefulset.yaml (22:27) duplicated block id: 423 size: 6 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (180:185) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (255:260) duplicated block id: 424 size: 6 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (114:119) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (144:149) duplicated block id: 425 size: 6 cleaned lines of code in 2 files: - client-spark/spark-2/src/main/java/org/apache/spark/shuffle/celeborn/SortBasedShuffleWriter.java (64:69) - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/HashBasedShuffleWriter.java (73:78) duplicated block id: 426 size: 6 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (180:185) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (240:245) duplicated block id: 427 size: 6 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (180:185) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (225:230) duplicated block id: 428 size: 6 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (180:185) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (210:215) duplicated block id: 429 size: 6 cleaned lines of code in 2 files: - master/src/main/proto/Resource.proto (115:120) - master/src/main/proto/Resource.proto (127:132) duplicated block id: 430 size: 6 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (180:185) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (195:200) duplicated block id: 431 size: 6 cleaned lines of code in 2 files: - master/src/main/java/org/apache/celeborn/service/deploy/master/clustermeta/SingleMasterMetaManager.java (39:44) - master/src/main/java/org/apache/celeborn/service/deploy/master/clustermeta/ha/HAMasterMetaManager.java (49:54) duplicated block id: 432 size: 6 cleaned lines of code in 2 files: - client-flink/common/src/main/java/org/apache/celeborn/plugin/flink/AbstractRemoteShuffleResultPartitionFactory.java (147:152) - client-flink/flink-1.17/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleResultPartitionFactory.java (61:66) duplicated block id: 433 size: 6 cleaned lines of code in 2 files: - charts/celeborn/templates/master-statefulset.yaml (22:27) - charts/celeborn/templates/prometheus-podmonitor.yaml (49:54) duplicated block id: 434 size: 6 cleaned lines of code in 2 files: - client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java (448:453) - client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java (1421:1426) duplicated block id: 435 size: 6 cleaned lines of code in 2 files: - common/src/main/scala/org/apache/celeborn/common/serializer/Serializer.scala (167:175) - common/src/main/scala/org/apache/celeborn/common/serializer/Serializer.scala (187:195) duplicated block id: 436 size: 6 cleaned lines of code in 2 files: - common/src/main/java/org/apache/celeborn/common/network/protocol/ChunkFetchRequest.java (39:46) - common/src/main/java/org/apache/celeborn/common/network/protocol/ChunkFetchSuccess.java (46:54) duplicated block id: 437 size: 6 cleaned lines of code in 2 files: - charts/celeborn/templates/master-statefulset.yaml (22:27) - charts/celeborn/templates/worker-statefulset.yaml (22:27) duplicated block id: 438 size: 6 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (272:277) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (289:294) duplicated block id: 439 size: 6 cleaned lines of code in 2 files: - project/CelebornBuild.scala (314:322) - project/CelebornBuild.scala (737:745) duplicated block id: 440 size: 6 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (180:185) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (307:312) duplicated block id: 441 size: 6 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (289:294) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (307:312) duplicated block id: 442 size: 6 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (180:185) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (289:294) duplicated block id: 443 size: 6 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (272:277) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (307:312) duplicated block id: 444 size: 6 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (180:185) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (272:277) duplicated block id: 445 size: 6 cleaned lines of code in 2 files: - master/src/main/java/org/apache/celeborn/service/deploy/master/clustermeta/AbstractMetaManager.java (174:179) - master/src/main/java/org/apache/celeborn/service/deploy/master/clustermeta/ha/HAMasterMetaManager.java (228:233) duplicated block id: 446 size: 6 cleaned lines of code in 2 files: - master/src/main/java/org/apache/celeborn/service/deploy/master/clustermeta/AbstractMetaManager.java (174:179) - master/src/main/java/org/apache/celeborn/service/deploy/master/clustermeta/ha/HAMasterMetaManager.java (191:196) duplicated block id: 447 size: 6 cleaned lines of code in 2 files: - client-spark/spark-2/src/main/java/org/apache/spark/shuffle/celeborn/SortBasedShuffleWriter.java (362:370) - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/SortBasedShuffleWriter.java (384:392) duplicated block id: 448 size: 6 cleaned lines of code in 2 files: - common/src/main/java/org/apache/celeborn/reflect/DynConstructors.java (58:65) - common/src/main/java/org/apache/celeborn/reflect/DynMethods.java (62:69) duplicated block id: 449 size: 6 cleaned lines of code in 2 files: - client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java (1266:1271) - client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java (1284:1289) duplicated block id: 450 size: 6 cleaned lines of code in 2 files: - master/src/main/java/org/apache/celeborn/service/deploy/master/clustermeta/ha/MetaHandler.java (168:173) - master/src/main/java/org/apache/celeborn/service/deploy/master/clustermeta/ha/MetaHandler.java (177:182) duplicated block id: 451 size: 6 cleaned lines of code in 2 files: - project/CelebornBuild.scala (401:407) - project/CelebornBuild.scala (415:421) duplicated block id: 452 size: 6 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (150:155) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (180:185) duplicated block id: 453 size: 6 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/SparkUtils.java (116:121) - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/SparkUtils.java (129:134) duplicated block id: 454 size: 6 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (150:155) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (195:200) duplicated block id: 455 size: 6 cleaned lines of code in 2 files: - master/src/main/proto/Resource.proto (99:104) - master/src/main/proto/Resource.proto (107:112) duplicated block id: 456 size: 6 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (150:155) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (210:215) duplicated block id: 457 size: 6 cleaned lines of code in 2 files: - master/src/main/scala/org/apache/celeborn/service/deploy/master/Master.scala (463:468) - master/src/main/scala/org/apache/celeborn/service/deploy/master/Master.scala (494:499) duplicated block id: 458 size: 6 cleaned lines of code in 2 files: - client-flink/common/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleInputGateDelegation.java (129:134) - client-flink/flink-1.17/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleInputGate.java (68:73) duplicated block id: 459 size: 6 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (150:155) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (225:230) duplicated block id: 460 size: 6 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (150:155) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (240:245) duplicated block id: 461 size: 6 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (150:155) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (255:260) duplicated block id: 462 size: 6 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (150:155) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (272:277) duplicated block id: 463 size: 6 cleaned lines of code in 2 files: - client-spark/spark-2/src/main/java/org/apache/spark/shuffle/celeborn/HashBasedShuffleWriter.java (123:129) - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/HashBasedShuffleWriter.java (145:151) duplicated block id: 464 size: 6 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (150:155) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (289:294) duplicated block id: 465 size: 6 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/HashBasedShuffleWriter.java (228:234) - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/HashBasedShuffleWriter.java (268:274) duplicated block id: 466 size: 6 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (150:155) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (307:312) duplicated block id: 467 size: 6 cleaned lines of code in 2 files: - client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java (1266:1271) - client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java (1399:1404) duplicated block id: 468 size: 6 cleaned lines of code in 2 files: - client/src/main/java/org/apache/celeborn/client/read/DfsPartitionReader.java (252:260) - client/src/main/java/org/apache/celeborn/client/read/WorkerPartitionReader.java (157:165) duplicated block id: 469 size: 6 cleaned lines of code in 2 files: - client-flink/common/src/main/java/org/apache/celeborn/plugin/flink/readclient/FlinkShuffleClientImpl.java (357:362) - client-flink/common/src/main/java/org/apache/celeborn/plugin/flink/readclient/FlinkShuffleClientImpl.java (430:435) duplicated block id: 470 size: 6 cleaned lines of code in 2 files: - worker/src/main/scala/org/apache/celeborn/service/deploy/worker/Worker.scala (211:216) - worker/src/main/scala/org/apache/celeborn/service/deploy/worker/Worker.scala (317:322) duplicated block id: 471 size: 6 cleaned lines of code in 2 files: - master/src/main/java/org/apache/celeborn/service/deploy/master/clustermeta/ha/MetaHandler.java (208:213) - worker/src/main/scala/org/apache/celeborn/service/deploy/worker/Worker.scala (317:322) duplicated block id: 472 size: 6 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/HashBasedShuffleWriter.java (228:234) - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/SortBasedShuffleWriter.java (247:253) duplicated block id: 473 size: 6 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/SparkUtils.java (110:115) - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/SparkUtils.java (125:130) duplicated block id: 474 size: 6 cleaned lines of code in 2 files: - client-flink/common/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleInputGateDelegation.java (129:134) - client-flink/flink-1.15/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleInputGate.java (68:73) duplicated block id: 475 size: 6 cleaned lines of code in 2 files: - common/src/main/java/org/apache/celeborn/common/network/protocol/RegionFinish.java (52:57) - common/src/main/java/org/apache/celeborn/common/network/protocol/RegionStart.java (64:69) duplicated block id: 476 size: 6 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (210:215) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (307:312) duplicated block id: 477 size: 6 cleaned lines of code in 2 files: - client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java (1021:1026) - client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java (1056:1061) duplicated block id: 478 size: 6 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (210:215) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (255:260) duplicated block id: 479 size: 6 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (210:215) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (240:245) duplicated block id: 480 size: 6 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (210:215) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (289:294) duplicated block id: 481 size: 6 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (210:215) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (272:277) duplicated block id: 482 size: 6 cleaned lines of code in 2 files: - charts/celeborn/templates/configmap.yaml (22:27) - charts/celeborn/templates/master-statefulset.yaml (22:27) duplicated block id: 483 size: 6 cleaned lines of code in 2 files: - project/CelebornBuild.scala (518:523) - project/CelebornBuild.scala (538:543) duplicated block id: 484 size: 6 cleaned lines of code in 2 files: - master/src/main/java/org/apache/celeborn/service/deploy/master/clustermeta/SingleMasterMetaManager.java (97:102) - master/src/main/scala/org/apache/celeborn/service/deploy/master/Master.scala (505:510) duplicated block id: 485 size: 6 cleaned lines of code in 2 files: - master/src/main/java/org/apache/celeborn/service/deploy/master/clustermeta/ha/MetaHandler.java (200:205) - worker/src/main/scala/org/apache/celeborn/service/deploy/worker/Worker.scala (317:322) duplicated block id: 486 size: 6 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (210:215) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (225:230) duplicated block id: 487 size: 6 cleaned lines of code in 2 files: - common/src/main/scala/org/apache/celeborn/common/meta/WorkerInfo.scala (52:57) - master/src/main/scala/org/apache/celeborn/service/deploy/master/Master.scala (471:476) duplicated block id: 488 size: 6 cleaned lines of code in 2 files: - client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java (919:924) - client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java (979:984) duplicated block id: 489 size: 6 cleaned lines of code in 2 files: - client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java (1400:1405) - client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java (1459:1464) duplicated block id: 490 size: 6 cleaned lines of code in 2 files: - client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java (919:924) - client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java (967:972) duplicated block id: 491 size: 6 cleaned lines of code in 2 files: - client-flink/flink-1.14/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleInputGateFactory.java (47:52) - client-flink/flink-1.15/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleInputGateFactory.java (47:52) duplicated block id: 492 size: 6 cleaned lines of code in 2 files: - common/src/main/java/org/apache/celeborn/common/network/protocol/ReadAddCredit.java (32:39) - common/src/main/java/org/apache/celeborn/common/network/protocol/StreamHandle.java (42:49) duplicated block id: 493 size: 6 cleaned lines of code in 2 files: - client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java (932:937) - client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java (967:972) duplicated block id: 494 size: 6 cleaned lines of code in 2 files: - charts/celeborn/templates/prometheus-podmonitor.yaml (24:29) - charts/celeborn/templates/worker-statefulset.yaml (22:27) duplicated block id: 495 size: 6 cleaned lines of code in 2 files: - client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java (1400:1405) - client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java (1432:1437) duplicated block id: 496 size: 6 cleaned lines of code in 2 files: - client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java (932:937) - client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java (979:984) duplicated block id: 497 size: 6 cleaned lines of code in 2 files: - master/src/main/java/org/apache/celeborn/service/deploy/master/clustermeta/ha/MetaHandler.java (177:182) - worker/src/main/scala/org/apache/celeborn/service/deploy/worker/Worker.scala (317:322) duplicated block id: 498 size: 6 cleaned lines of code in 2 files: - master/src/main/java/org/apache/celeborn/service/deploy/master/clustermeta/AbstractMetaManager.java (174:179) - master/src/main/java/org/apache/celeborn/service/deploy/master/clustermeta/SingleMasterMetaManager.java (86:91) duplicated block id: 499 size: 6 cleaned lines of code in 2 files: - worker/src/main/scala/org/apache/celeborn/service/deploy/worker/Worker.scala (587:593) - worker/src/main/scala/org/apache/celeborn/service/deploy/worker/Worker.scala (657:663) duplicated block id: 500 size: 6 cleaned lines of code in 2 files: - charts/celeborn/templates/worker-service.yaml (22:27) - charts/celeborn/templates/worker-statefulset.yaml (22:27) duplicated block id: 501 size: 6 cleaned lines of code in 2 files: - client-spark/common/src/main/java/org/apache/spark/shuffle/celeborn/SortBasedPusher.java (118:123) - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/HashBasedShuffleWriter.java (162:167) duplicated block id: 502 size: 6 cleaned lines of code in 2 files: - master/src/main/scala/org/apache/celeborn/service/deploy/master/Master.scala (340:345) - master/src/main/scala/org/apache/celeborn/service/deploy/master/Master.scala (505:510) duplicated block id: 503 size: 6 cleaned lines of code in 2 files: - master/src/main/java/org/apache/celeborn/service/deploy/master/clustermeta/AbstractMetaManager.java (174:179) - master/src/main/java/org/apache/celeborn/service/deploy/master/clustermeta/SingleMasterMetaManager.java (110:115) duplicated block id: 504 size: 6 cleaned lines of code in 2 files: - project/CelebornBuild.scala (291:299) - project/CelebornBuild.scala (737:745) duplicated block id: 505 size: 6 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnStats.scala (254:262) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnStats.scala (305:313) duplicated block id: 506 size: 6 cleaned lines of code in 2 files: - master/src/main/java/org/apache/celeborn/service/deploy/master/clustermeta/ha/MetaHandler.java (208:213) - worker/src/main/scala/org/apache/celeborn/service/deploy/worker/Worker.scala (211:216) duplicated block id: 507 size: 6 cleaned lines of code in 2 files: - client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java (1284:1289) - client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java (1349:1354) duplicated block id: 508 size: 6 cleaned lines of code in 2 files: - master/src/main/scala/org/apache/celeborn/service/deploy/master/Master.scala (504:509) - worker/src/main/scala/org/apache/celeborn/service/deploy/worker/Worker.scala (210:215) duplicated block id: 509 size: 6 cleaned lines of code in 2 files: - client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java (1284:1289) - client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java (1361:1366) duplicated block id: 510 size: 6 cleaned lines of code in 2 files: - client-flink/common/src/main/java/org/apache/celeborn/plugin/flink/AbstractRemoteShuffleResultPartitionFactory.java (147:152) - client-flink/flink-1.15/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleResultPartitionFactory.java (61:66) duplicated block id: 511 size: 6 cleaned lines of code in 2 files: - client-flink/common/src/main/java/org/apache/celeborn/plugin/flink/AbstractRemoteShuffleResultPartitionFactory.java (147:152) - client-flink/flink-1.14/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleResultPartitionFactory.java (60:65) duplicated block id: 512 size: 6 cleaned lines of code in 2 files: - client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java (1284:1289) - client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java (1319:1324) duplicated block id: 513 size: 6 cleaned lines of code in 2 files: - common/src/main/java/org/apache/celeborn/common/network/protocol/OpenStream.java (62:67) - common/src/main/java/org/apache/celeborn/common/network/protocol/OpenStreamWithCredit.java (59:64) duplicated block id: 514 size: 6 cleaned lines of code in 2 files: - common/src/main/proto/TransportMessages.proto (117:122) - common/src/main/proto/TransportMessages.proto (142:147) duplicated block id: 515 size: 6 cleaned lines of code in 2 files: - common/src/main/scala/org/apache/celeborn/common/protocol/message/ControlMessages.scala (640:645) - worker/src/main/scala/org/apache/celeborn/service/deploy/worker/Controller.scala (101:106) duplicated block id: 516 size: 6 cleaned lines of code in 2 files: - common/src/main/proto/TransportMessages.proto (117:122) - common/src/main/proto/TransportMessages.proto (131:136) duplicated block id: 517 size: 6 cleaned lines of code in 2 files: - client-spark/spark-2/src/main/java/org/apache/spark/shuffle/celeborn/HashBasedShuffleWriter.java (69:74) - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/SortBasedShuffleWriter.java (64:69) duplicated block id: 518 size: 6 cleaned lines of code in 2 files: - common/src/main/scala/org/apache/celeborn/common/protocol/message/ControlMessages.scala (81:86) - common/src/main/scala/org/apache/celeborn/common/protocol/message/ControlMessages.scala (259:264) duplicated block id: 519 size: 6 cleaned lines of code in 2 files: - common/src/main/scala/org/apache/celeborn/common/protocol/message/ControlMessages.scala (424:429) - master/src/main/scala/org/apache/celeborn/service/deploy/master/Master.scala (505:510) duplicated block id: 520 size: 6 cleaned lines of code in 2 files: - common/src/main/scala/org/apache/celeborn/common/util/Utils.scala (139:144) - common/src/main/scala/org/apache/celeborn/common/util/Utils.scala (160:165) duplicated block id: 521 size: 6 cleaned lines of code in 2 files: - charts/celeborn/templates/master-statefulset.yaml (22:27) - charts/celeborn/templates/worker-service.yaml (22:27) duplicated block id: 522 size: 6 cleaned lines of code in 2 files: - client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java (919:924) - client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java (932:937) duplicated block id: 523 size: 6 cleaned lines of code in 2 files: - master/src/main/java/org/apache/celeborn/service/deploy/master/clustermeta/ha/MetaHandler.java (200:205) - worker/src/main/scala/org/apache/celeborn/service/deploy/worker/Worker.scala (211:216) duplicated block id: 524 size: 6 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (225:230) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (240:245) duplicated block id: 525 size: 6 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (225:230) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (255:260) duplicated block id: 526 size: 6 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (225:230) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (272:277) duplicated block id: 527 size: 6 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (225:230) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (289:294) duplicated block id: 528 size: 6 cleaned lines of code in 2 files: - charts/celeborn/templates/prometheus-podmonitor.yaml (49:54) - charts/celeborn/templates/worker-statefulset.yaml (22:27) duplicated block id: 529 size: 6 cleaned lines of code in 2 files: - common/src/main/proto/TransportMessages.proto (131:136) - common/src/main/proto/TransportMessages.proto (142:147) duplicated block id: 530 size: 6 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (225:230) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnBuilder.scala (307:312) duplicated block id: 531 size: 6 cleaned lines of code in 2 files: - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (114:119) - client-spark/spark-3/src/main/scala/org/apache/spark/sql/execution/columnar/CelebornColumnarBatchCodeGenBuild.scala (284:289) duplicated block id: 532 size: 6 cleaned lines of code in 2 files: - project/CelebornBuild.scala (591:599) - project/CelebornBuild.scala (787:795) duplicated block id: 533 size: 6 cleaned lines of code in 2 files: - client/src/main/scala/org/apache/celeborn/client/ChangePartitionManager.scala (99:106) - client/src/main/scala/org/apache/celeborn/client/ReleasePartitionManager.scala (95:102) duplicated block id: 534 size: 6 cleaned lines of code in 2 files: - client-spark/spark-2/src/main/java/org/apache/spark/shuffle/celeborn/SortBasedShuffleWriter.java (383:388) - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/SortBasedShuffleWriter.java (405:410) duplicated block id: 535 size: 6 cleaned lines of code in 2 files: - master/src/main/scala/org/apache/celeborn/service/deploy/master/Master.scala (297:302) - master/src/main/scala/org/apache/celeborn/service/deploy/master/Master.scala (505:510) duplicated block id: 536 size: 6 cleaned lines of code in 2 files: - master/src/main/java/org/apache/celeborn/service/deploy/master/clustermeta/ha/MetaHandler.java (177:182) - worker/src/main/scala/org/apache/celeborn/service/deploy/worker/Worker.scala (211:216) duplicated block id: 537 size: 6 cleaned lines of code in 2 files: - master/src/main/java/org/apache/celeborn/service/deploy/master/clustermeta/AbstractMetaManager.java (174:179) - master/src/main/java/org/apache/celeborn/service/deploy/master/clustermeta/IMetadataHandler.java (49:54) duplicated block id: 538 size: 6 cleaned lines of code in 2 files: - master/src/main/java/org/apache/celeborn/service/deploy/master/clustermeta/ha/MetaHandler.java (177:182) - master/src/main/java/org/apache/celeborn/service/deploy/master/clustermeta/ha/MetaHandler.java (208:213) duplicated block id: 539 size: 6 cleaned lines of code in 2 files: - master/src/main/java/org/apache/celeborn/service/deploy/master/clustermeta/ha/MetaHandler.java (177:182) - master/src/main/java/org/apache/celeborn/service/deploy/master/clustermeta/ha/MetaHandler.java (200:205) duplicated block id: 540 size: 6 cleaned lines of code in 2 files: - master/src/main/scala/org/apache/celeborn/service/deploy/master/Master.scala (324:329) - master/src/main/scala/org/apache/celeborn/service/deploy/master/Master.scala (505:510) duplicated block id: 541 size: 6 cleaned lines of code in 2 files: - client-spark/spark-2/src/main/java/org/apache/spark/shuffle/celeborn/HashBasedShuffleWriter.java (69:74) - client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/HashBasedShuffleWriter.java (73:78) duplicated block id: 542 size: 6 cleaned lines of code in 2 files: - master/src/main/java/org/apache/celeborn/service/deploy/master/clustermeta/AbstractMetaManager.java (174:179) - master/src/main/java/org/apache/celeborn/service/deploy/master/clustermeta/IMetadataHandler.java (61:66)