duplicated block id: 1 size: 23 cleaned lines of code in 2 files: - runtime/master/src/main/java/org/apache/nemo/runtime/master/scheduler/BatchScheduler.java (203:235) - runtime/master/src/main/java/org/apache/nemo/runtime/master/scheduler/SimulationScheduler.java (371:403) duplicated block id: 2 size: 21 cleaned lines of code in 2 files: - runtime/common/src/main/proto/ControlMessage.proto (1:23) - runtime/common/src/main/proto/GrcpMessageService.proto (1:23) duplicated block id: 3 size: 15 cleaned lines of code in 2 files: - runtime/master/src/main/java/org/apache/nemo/runtime/master/scheduler/BatchScheduler.java (308:328) - runtime/master/src/main/java/org/apache/nemo/runtime/master/scheduler/SimulationScheduler.java (251:271) duplicated block id: 4 size: 14 cleaned lines of code in 2 files: - runtime/executor/src/main/java/org/apache/nemo/runtime/executor/task/MultiThreadParentTaskDataFetcher.java (159:176) - runtime/executor/src/main/java/org/apache/nemo/runtime/executor/task/ParentTaskDataFetcher.java (181:198) duplicated block id: 5 size: 12 cleaned lines of code in 2 files: - runtime/master/src/main/java/org/apache/nemo/runtime/master/scheduler/BatchScheduler.java (97:119) - runtime/master/src/main/java/org/apache/nemo/runtime/master/scheduler/SimulationScheduler.java (275:297) duplicated block id: 6 size: 11 cleaned lines of code in 2 files: - runtime/master/src/main/java/org/apache/nemo/runtime/master/scheduler/BatchScheduler.java (179:191) - runtime/master/src/main/java/org/apache/nemo/runtime/master/scheduler/SimulationScheduler.java (346:359) duplicated block id: 7 size: 11 cleaned lines of code in 2 files: - webui/components/stages/StagesView.vue (48:60) - webui/components/stages/StagesView.vue (136:148) duplicated block id: 8 size: 11 cleaned lines of code in 2 files: - runtime/master/src/main/java/org/apache/nemo/runtime/master/RuntimeMaster.java (452:463) - runtime/master/src/main/java/org/apache/nemo/runtime/master/scheduler/SimulationScheduler.java (545:556) duplicated block id: 9 size: 11 cleaned lines of code in 2 files: - webui/components/stages/StagesView.vue (48:60) - webui/components/stages/StagesView.vue (114:126) duplicated block id: 10 size: 11 cleaned lines of code in 2 files: - webui/components/stages/StagesView.vue (48:60) - webui/components/stages/StagesView.vue (92:104) duplicated block id: 11 size: 11 cleaned lines of code in 2 files: - common/src/main/java/org/apache/nemo/common/dag/DAGBuilder.java (233:245) - common/src/main/java/org/apache/nemo/common/dag/DAGBuilder.java (290:302) duplicated block id: 12 size: 11 cleaned lines of code in 2 files: - webui/components/stages/StagesView.vue (70:82) - webui/components/stages/StagesView.vue (136:148) duplicated block id: 13 size: 11 cleaned lines of code in 2 files: - webui/components/stages/StagesView.vue (114:126) - webui/components/stages/StagesView.vue (136:148) duplicated block id: 14 size: 11 cleaned lines of code in 2 files: - runtime/executor/src/main/java/org/apache/nemo/runtime/executor/data/block/FileBlock.java (130:152) - runtime/executor/src/main/java/org/apache/nemo/runtime/executor/data/block/SerializedMemoryBlock.java (84:107) duplicated block id: 15 size: 11 cleaned lines of code in 2 files: - compiler/frontend/spark/src/main/java/org/apache/nemo/compiler/frontend/spark/source/SparkDatasetBoundedSourceVertex.java (150:165) - compiler/frontend/spark/src/main/java/org/apache/nemo/compiler/frontend/spark/source/SparkTextFileBoundedSourceVertex.java (134:149) duplicated block id: 16 size: 11 cleaned lines of code in 2 files: - webui/components/stages/StagesView.vue (70:82) - webui/components/stages/StagesView.vue (92:104) duplicated block id: 17 size: 11 cleaned lines of code in 2 files: - webui/components/stages/StagesView.vue (70:82) - webui/components/stages/StagesView.vue (114:126) duplicated block id: 18 size: 11 cleaned lines of code in 2 files: - common/src/main/java/org/apache/nemo/common/ir/vertex/transform/StreamTransform.java (41:58) - compiler/frontend/beam/src/main/java/org/apache/nemo/compiler/frontend/beam/transform/FlattenTransform.java (38:55) duplicated block id: 19 size: 11 cleaned lines of code in 2 files: - webui/components/stages/StagesView.vue (48:60) - webui/components/stages/StagesView.vue (70:82) duplicated block id: 20 size: 11 cleaned lines of code in 2 files: - webui/components/stages/StagesView.vue (92:104) - webui/components/stages/StagesView.vue (114:126) duplicated block id: 21 size: 11 cleaned lines of code in 2 files: - webui/components/stages/StagesView.vue (92:104) - webui/components/stages/StagesView.vue (136:148) duplicated block id: 22 size: 11 cleaned lines of code in 2 files: - runtime/master/src/main/java/org/apache/nemo/runtime/master/scheduler/BatchScheduler.java (144:159) - runtime/master/src/main/java/org/apache/nemo/runtime/master/scheduler/SimulationScheduler.java (210:225) duplicated block id: 23 size: 10 cleaned lines of code in 2 files: - runtime/master/src/main/java/org/apache/nemo/runtime/master/scheduler/BatchScheduler.java (173:183) - runtime/master/src/main/java/org/apache/nemo/runtime/master/scheduler/StreamingScheduler.java (120:130) duplicated block id: 24 size: 10 cleaned lines of code in 2 files: - runtime/executor/src/main/java/org/apache/nemo/runtime/executor/data/block/FileBlock.java (344:362) - runtime/executor/src/main/java/org/apache/nemo/runtime/executor/data/block/SerializedMemoryBlock.java (199:216) duplicated block id: 25 size: 10 cleaned lines of code in 2 files: - webui/components/jobs/detail/DAG.vue (121:132) - webui/components/jobs/detail/DAG.vue (179:190) duplicated block id: 26 size: 10 cleaned lines of code in 2 files: - compiler/frontend/beam/src/main/java/org/apache/nemo/compiler/frontend/beam/transform/AbstractDoFnTransform.java (106:115) - compiler/frontend/beam/src/main/java/org/apache/nemo/compiler/frontend/beam/transform/PushBackDoFnTransform.java (67:76) duplicated block id: 27 size: 9 cleaned lines of code in 2 files: - webui/components/jobs/detail/DAG.vue (151:160) - webui/components/jobs/detail/DAG.vue (205:214) duplicated block id: 28 size: 9 cleaned lines of code in 2 files: - runtime/executor/src/main/java/org/apache/nemo/runtime/executor/data/BlockManagerWorker.java (306:316) - runtime/executor/src/main/java/org/apache/nemo/runtime/executor/data/BlockManagerWorker.java (340:350) duplicated block id: 29 size: 9 cleaned lines of code in 2 files: - compiler/frontend/beam/src/main/java/org/apache/nemo/compiler/frontend/beam/BeamKeyExtractor.java (38:53) - compiler/frontend/spark/src/main/java/org/apache/nemo/compiler/frontend/spark/SparkKeyExtractor.java (34:49) duplicated block id: 30 size: 9 cleaned lines of code in 2 files: - runtime/master/src/main/java/org/apache/nemo/runtime/master/scheduler/BatchScheduler.java (193:201) - runtime/master/src/main/java/org/apache/nemo/runtime/master/scheduler/SimulationScheduler.java (361:369) duplicated block id: 31 size: 8 cleaned lines of code in 2 files: - common/src/main/java/org/apache/nemo/common/dag/DAGBuilder.java (168:176) - common/src/main/java/org/apache/nemo/common/dag/DAGBuilder.java (294:302) duplicated block id: 32 size: 8 cleaned lines of code in 2 files: - compiler/frontend/beam/src/main/java/org/apache/nemo/compiler/frontend/beam/transform/FlattenTransform.java (43:55) - compiler/frontend/beam/src/main/java/org/apache/nemo/compiler/frontend/beam/transform/SideInputTransform.java (50:61) duplicated block id: 33 size: 8 cleaned lines of code in 2 files: - compiler/frontend/beam/src/main/java/org/apache/nemo/compiler/frontend/beam/PipelineTranslator.java (222:230) - compiler/frontend/beam/src/main/java/org/apache/nemo/compiler/frontend/beam/PipelineTranslator.java (243:250) duplicated block id: 34 size: 8 cleaned lines of code in 2 files: - compiler/frontend/beam/src/main/java/org/apache/nemo/compiler/frontend/beam/source/BeamBoundedSourceVertex.java (115:124) - compiler/frontend/beam/src/main/java/org/apache/nemo/compiler/frontend/beam/source/BeamUnboundedSourceVertex.java (105:114) duplicated block id: 35 size: 8 cleaned lines of code in 2 files: - common/src/main/java/org/apache/nemo/common/ir/vertex/transform/StreamTransform.java (46:58) - compiler/frontend/beam/src/main/java/org/apache/nemo/compiler/frontend/beam/transform/WindowFnTransform.java (91:102) duplicated block id: 36 size: 8 cleaned lines of code in 2 files: - compiler/frontend/beam/src/main/java/org/apache/nemo/compiler/frontend/beam/transform/SideInputTransform.java (50:61) - compiler/frontend/beam/src/main/java/org/apache/nemo/compiler/frontend/beam/transform/WindowFnTransform.java (91:102) duplicated block id: 37 size: 8 cleaned lines of code in 2 files: - common/src/main/java/org/apache/nemo/common/ir/vertex/transform/StreamTransform.java (46:58) - compiler/frontend/beam/src/main/java/org/apache/nemo/compiler/frontend/beam/transform/SideInputTransform.java (50:61) duplicated block id: 38 size: 8 cleaned lines of code in 2 files: - common/src/main/java/org/apache/nemo/common/dag/DAGBuilder.java (168:176) - common/src/main/java/org/apache/nemo/common/dag/DAGBuilder.java (237:245) duplicated block id: 39 size: 8 cleaned lines of code in 2 files: - common/src/main/java/org/apache/nemo/common/ir/vertex/CachedSourceVertex.java (63:76) - compiler/frontend/spark/src/main/java/org/apache/nemo/compiler/frontend/spark/source/SparkDatasetBoundedSourceVertex.java (82:93) duplicated block id: 40 size: 8 cleaned lines of code in 2 files: - common/src/main/java/org/apache/nemo/common/ir/IRDAG.java (149:157) - runtime/common/src/main/java/org/apache/nemo/runtime/common/metric/JobMetric.java (101:109) duplicated block id: 41 size: 8 cleaned lines of code in 2 files: - runtime/executor/src/main/java/org/apache/nemo/runtime/executor/data/DataUtil.java (404:415) - runtime/executor/src/main/java/org/apache/nemo/runtime/executor/data/DataUtil.java (454:465) duplicated block id: 42 size: 8 cleaned lines of code in 2 files: - compiler/frontend/beam/src/main/java/org/apache/nemo/compiler/frontend/beam/transform/FlattenTransform.java (43:55) - compiler/frontend/beam/src/main/java/org/apache/nemo/compiler/frontend/beam/transform/WindowFnTransform.java (91:102) duplicated block id: 43 size: 8 cleaned lines of code in 2 files: - webui/components/jobs/detail/MetricTimeline.vue (153:161) - webui/components/jobs/detail/StageSelect.vue (194:202) duplicated block id: 44 size: 7 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/BasicPullPolicy.java (41:51) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/DisaggregationPolicy.java (42:52) duplicated block id: 45 size: 7 cleaned lines of code in 2 files: - webui/components/jobs/detail/JobView.vue (342:349) - webui/components/jobs/detail/StageSelect.vue (195:202) duplicated block id: 46 size: 7 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/DataSkewPolicy.java (46:56) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/DisaggregationPolicy.java (42:52) duplicated block id: 47 size: 7 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/BasicPullPolicy.java (41:51) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/SamplingLargeShuffleSkewPolicy.java (48:58) duplicated block id: 48 size: 7 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/BasicPullPolicy.java (41:51) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/BasicPushPolicy.java (43:53) duplicated block id: 49 size: 7 cleaned lines of code in 2 files: - compiler/frontend/spark/src/main/java/org/apache/nemo/compiler/frontend/spark/transform/FlatMapTransform.java (47:58) - compiler/frontend/spark/src/main/java/org/apache/nemo/compiler/frontend/spark/transform/MapTransform.java (47:58) duplicated block id: 50 size: 7 cleaned lines of code in 2 files: - webui/components/jobs/detail/DAG.vue (617:624) - webui/components/jobs/detail/DAG.vue (705:712) duplicated block id: 51 size: 7 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/DefaultPolicy.java (38:48) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/DynamicTaskSizingPolicy.java (51:60) duplicated block id: 52 size: 7 cleaned lines of code in 2 files: - common/src/main/java/org/apache/nemo/common/PairKeyExtractor.java (37:47) - compiler/frontend/beam/src/main/java/org/apache/nemo/compiler/frontend/beam/BeamKeyExtractor.java (43:53) duplicated block id: 53 size: 7 cleaned lines of code in 2 files: - runtime/executor/src/main/java/org/apache/nemo/runtime/executor/data/block/FileBlock.java (370:387) - runtime/executor/src/main/java/org/apache/nemo/runtime/executor/data/block/SerializedMemoryBlock.java (222:239) duplicated block id: 54 size: 7 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/DefaultPolicy.java (38:48) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/DefaultPolicyWithSeparatePass.java (46:56) duplicated block id: 55 size: 7 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/DefaultPolicy.java (38:48) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/XGBoostPolicy.java (42:52) duplicated block id: 56 size: 7 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/DefaultPolicy.java (38:48) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/TransientResourcePolicy.java (42:52) duplicated block id: 57 size: 7 cleaned lines of code in 2 files: - compiler/frontend/spark/src/main/java/org/apache/nemo/compiler/frontend/spark/transform/FlatMapTransform.java (47:58) - compiler/frontend/spark/src/main/java/org/apache/nemo/compiler/frontend/spark/transform/MapToPairTransform.java (50:61) duplicated block id: 58 size: 7 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/DefaultPolicyWithSeparatePass.java (46:56) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/SamplingLargeShuffleSkewPolicy.java (48:58) duplicated block id: 59 size: 7 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/BasicPullPolicy.java (41:51) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/LargeShufflePolicy.java (42:52) duplicated block id: 60 size: 7 cleaned lines of code in 2 files: - compiler/frontend/spark/src/main/java/org/apache/nemo/compiler/frontend/spark/transform/MapToPairTransform.java (50:61) - compiler/frontend/spark/src/main/java/org/apache/nemo/compiler/frontend/spark/transform/MapTransform.java (47:58) duplicated block id: 61 size: 7 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/DefaultPolicy.java (38:48) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/DisaggregationPolicy.java (42:52) duplicated block id: 62 size: 7 cleaned lines of code in 2 files: - compiler/frontend/beam/src/main/java/org/apache/nemo/compiler/frontend/beam/transform/CombineFnFinalTransform.java (99:108) - compiler/frontend/beam/src/main/java/org/apache/nemo/compiler/frontend/beam/transform/CombineFnPartialTransform.java (96:105) duplicated block id: 63 size: 7 cleaned lines of code in 2 files: - webui/components/jobs/detail/JobView.vue (287:293) - webui/components/jobs/detail/JobView.vue (308:314) duplicated block id: 64 size: 7 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/DisaggregationPolicy.java (42:52) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/SamplingLargeShuffleSkewPolicy.java (48:58) duplicated block id: 65 size: 7 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/BasicPushPolicy.java (43:53) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/LargeShufflePolicy.java (42:52) duplicated block id: 66 size: 7 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/DynamicTaskSizingPolicy.java (51:60) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/SamplingLargeShuffleSkewPolicy.java (48:58) duplicated block id: 67 size: 7 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/BasicPushPolicy.java (43:53) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/DataSkewPolicy.java (46:56) duplicated block id: 68 size: 7 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/DataSkewPolicy.java (46:56) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/SamplingLargeShuffleSkewPolicy.java (48:58) duplicated block id: 69 size: 7 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/BasicPullPolicy.java (41:51) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/DynamicTaskSizingPolicy.java (51:60) duplicated block id: 70 size: 7 cleaned lines of code in 2 files: - runtime/executor/src/main/java/org/apache/nemo/runtime/executor/task/MultiThreadParentTaskDataFetcher.java (187:196) - runtime/executor/src/main/java/org/apache/nemo/runtime/executor/task/ParentTaskDataFetcher.java (205:214) duplicated block id: 71 size: 7 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/DefaultPolicy.java (38:48) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/LargeShufflePolicy.java (42:52) duplicated block id: 72 size: 7 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/BasicPullPolicy.java (41:51) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/DefaultPolicyWithSeparatePass.java (46:56) duplicated block id: 73 size: 7 cleaned lines of code in 2 files: - compiler/frontend/beam/src/main/java/org/apache/nemo/compiler/frontend/beam/transform/CombineFnFinalTransform.java (90:96) - compiler/frontend/beam/src/main/java/org/apache/nemo/compiler/frontend/beam/transform/CombineFnPartialTransform.java (87:93) duplicated block id: 74 size: 7 cleaned lines of code in 2 files: - webui/components/jobs/JobsView.vue (668:674) - webui/components/jobs/JobsView.vue (697:704) duplicated block id: 75 size: 7 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/DataSkewPolicy.java (46:56) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/DefaultPolicy.java (38:48) duplicated block id: 76 size: 7 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/DataSkewPolicy.java (46:56) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/DefaultPolicyWithSeparatePass.java (46:56) duplicated block id: 77 size: 7 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/DisaggregationPolicy.java (42:52) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/LargeShufflePolicy.java (42:52) duplicated block id: 78 size: 7 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/BasicPushPolicy.java (43:53) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/XGBoostPolicy.java (42:52) duplicated block id: 79 size: 7 cleaned lines of code in 2 files: - webui/components/jobs/detail/JobView.vue (342:349) - webui/components/jobs/detail/MetricTimeline.vue (154:161) duplicated block id: 80 size: 7 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/BasicPullPolicy.java (41:51) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/DataSkewPolicy.java (46:56) duplicated block id: 81 size: 7 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/BasicPullPolicy.java (41:51) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/DefaultPolicy.java (38:48) duplicated block id: 82 size: 7 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/LargeShufflePolicy.java (42:52) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/TransientResourcePolicy.java (42:52) duplicated block id: 83 size: 7 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/DynamicTaskSizingPolicy.java (51:60) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/TransientResourcePolicy.java (42:52) duplicated block id: 84 size: 7 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/DisaggregationPolicy.java (42:52) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/TransientResourcePolicy.java (42:52) duplicated block id: 85 size: 7 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/DisaggregationPolicy.java (42:52) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/XGBoostPolicy.java (42:52) duplicated block id: 86 size: 7 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/BasicPushPolicy.java (43:53) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/DefaultPolicyWithSeparatePass.java (46:56) duplicated block id: 87 size: 7 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/DataSkewPolicy.java (46:56) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/TransientResourcePolicy.java (42:52) duplicated block id: 88 size: 7 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/LargeShufflePolicy.java (42:52) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/SamplingLargeShuffleSkewPolicy.java (48:58) duplicated block id: 89 size: 7 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/BasicPushPolicy.java (43:53) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/TransientResourcePolicy.java (42:52) duplicated block id: 90 size: 7 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/BasicPullPolicy.java (41:51) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/TransientResourcePolicy.java (42:52) duplicated block id: 91 size: 7 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/LargeShufflePolicy.java (42:52) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/XGBoostPolicy.java (42:52) duplicated block id: 92 size: 7 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/BasicPushPolicy.java (43:53) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/DisaggregationPolicy.java (42:52) duplicated block id: 93 size: 7 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/DefaultPolicyWithSeparatePass.java (46:56) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/XGBoostPolicy.java (42:52) duplicated block id: 94 size: 7 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/SamplingLargeShuffleSkewPolicy.java (48:58) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/TransientResourcePolicy.java (42:52) duplicated block id: 95 size: 7 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/SamplingLargeShuffleSkewPolicy.java (48:58) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/XGBoostPolicy.java (42:52) duplicated block id: 96 size: 7 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/TransientResourcePolicy.java (42:52) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/XGBoostPolicy.java (42:52) duplicated block id: 97 size: 7 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/DataSkewPolicy.java (46:56) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/LargeShufflePolicy.java (42:52) duplicated block id: 98 size: 7 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/DefaultPolicyWithSeparatePass.java (46:56) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/TransientResourcePolicy.java (42:52) duplicated block id: 99 size: 7 cleaned lines of code in 2 files: - compiler/frontend/spark/src/main/scala/org/apache/nemo/compiler/frontend/spark/core/rdd/RDD.scala (169:176) - compiler/frontend/spark/src/main/scala/org/apache/nemo/compiler/frontend/spark/core/rdd/RDD.scala (226:233) duplicated block id: 100 size: 7 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/DynamicTaskSizingPolicy.java (51:60) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/XGBoostPolicy.java (42:52) duplicated block id: 101 size: 7 cleaned lines of code in 2 files: - common/src/main/java/org/apache/nemo/common/PairKeyExtractor.java (37:47) - compiler/frontend/spark/src/main/java/org/apache/nemo/compiler/frontend/spark/SparkKeyExtractor.java (39:49) duplicated block id: 102 size: 7 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/BasicPushPolicy.java (43:53) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/SamplingLargeShuffleSkewPolicy.java (48:58) duplicated block id: 103 size: 7 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/DefaultPolicyWithSeparatePass.java (46:56) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/DynamicTaskSizingPolicy.java (51:60) duplicated block id: 104 size: 7 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/BasicPushPolicy.java (43:53) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/DynamicTaskSizingPolicy.java (51:60) duplicated block id: 105 size: 7 cleaned lines of code in 2 files: - common/src/main/java/org/apache/nemo/common/ir/vertex/LoopVertex.java (323:330) - common/src/main/java/org/apache/nemo/common/ir/vertex/utility/TaskSizeSplitterVertex.java (179:185) duplicated block id: 106 size: 7 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/LambdaPolicy.java (40:50) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/StreamingPolicy.java (39:49) duplicated block id: 107 size: 7 cleaned lines of code in 2 files: - runtime/common/src/main/java/org/apache/nemo/runtime/common/state/PlanState.java (50:64) - runtime/common/src/main/java/org/apache/nemo/runtime/common/state/TaskState.java (65:78) duplicated block id: 108 size: 7 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/DefaultPolicyWithSeparatePass.java (46:56) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/DisaggregationPolicy.java (42:52) duplicated block id: 109 size: 7 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/DataSkewPolicy.java (46:56) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/XGBoostPolicy.java (42:52) duplicated block id: 110 size: 7 cleaned lines of code in 2 files: - compiler/frontend/beam/src/main/java/org/apache/nemo/compiler/frontend/beam/transform/FinalCombineFn.java (41:51) - compiler/frontend/beam/src/main/java/org/apache/nemo/compiler/frontend/beam/transform/PartialCombineFn.java (40:50) duplicated block id: 111 size: 7 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/BasicPushPolicy.java (43:53) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/DefaultPolicy.java (38:48) duplicated block id: 112 size: 7 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/DynamicTaskSizingPolicy.java (51:60) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/LargeShufflePolicy.java (42:52) duplicated block id: 113 size: 7 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/DefaultPolicyWithSeparatePass.java (46:56) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/LargeShufflePolicy.java (42:52) duplicated block id: 114 size: 7 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/DisaggregationPolicy.java (42:52) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/DynamicTaskSizingPolicy.java (51:60) duplicated block id: 115 size: 7 cleaned lines of code in 2 files: - compiler/frontend/spark/src/main/java/org/apache/nemo/compiler/frontend/spark/transform/HDFSTextFileTransform.java (56:66) - compiler/frontend/spark/src/main/java/org/apache/nemo/compiler/frontend/spark/transform/LocalTextFileTransform.java (52:62) duplicated block id: 116 size: 7 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/DataSkewPolicy.java (46:56) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/DynamicTaskSizingPolicy.java (51:60) duplicated block id: 117 size: 7 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/DefaultPolicy.java (38:48) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/SamplingLargeShuffleSkewPolicy.java (48:58) duplicated block id: 118 size: 7 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/BasicPullPolicy.java (41:51) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/XGBoostPolicy.java (42:52) duplicated block id: 119 size: 6 cleaned lines of code in 2 files: - common/src/main/java/org/apache/nemo/common/punctuation/Watermark.java (45:51) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/pass/compiletime/annotating/DefaultScheduleGroupPass.java (215:221) duplicated block id: 120 size: 6 cleaned lines of code in 2 files: - runtime/executor/src/main/java/org/apache/nemo/runtime/executor/data/stores/LocalFileStore.java (83:97) - runtime/executor/src/main/java/org/apache/nemo/runtime/executor/data/stores/SerializedMemoryStore.java (69:80) duplicated block id: 121 size: 6 cleaned lines of code in 2 files: - common/src/main/java/org/apache/nemo/common/ir/vertex/executionproperty/ClonedSchedulingProperty.java (128:135) - common/src/main/java/org/apache/nemo/common/punctuation/Watermark.java (45:51) duplicated block id: 122 size: 6 cleaned lines of code in 2 files: - webui/components/jobs/detail/DAG.vue (81:87) - webui/components/jobs/detail/StageSelect.vue (195:201) duplicated block id: 123 size: 6 cleaned lines of code in 2 files: - common/src/main/java/org/apache/nemo/common/punctuation/Watermark.java (45:51) - runtime/executor/src/main/java/org/apache/nemo/runtime/executor/transfer/ByteTransport.java (254:260) duplicated block id: 124 size: 6 cleaned lines of code in 2 files: - common/src/main/java/org/apache/nemo/common/ir/edge/IREdge.java (112:118) - runtime/common/src/main/java/org/apache/nemo/runtime/common/plan/StageEdge.java (118:124) duplicated block id: 125 size: 6 cleaned lines of code in 2 files: - webui/components/jobs/detail/DAG.vue (626:631) - webui/components/jobs/detail/DAG.vue (714:719) duplicated block id: 126 size: 6 cleaned lines of code in 2 files: - common/src/main/java/org/apache/nemo/common/ir/edge/IREdge.java (112:118) - common/src/main/java/org/apache/nemo/common/punctuation/LatencyMark.java (95:101) duplicated block id: 127 size: 6 cleaned lines of code in 2 files: - runtime/master/src/main/java/org/apache/nemo/runtime/master/servlet/AllMetricServlet.java (33:38) - runtime/master/src/main/java/org/apache/nemo/runtime/master/servlet/JobMetricServlet.java (34:39) duplicated block id: 128 size: 6 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/ConditionalLargeShufflePolicy.java (58:65) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/LargeShufflePolicy.java (45:52) duplicated block id: 129 size: 6 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/DisaggregationPolicy.java (45:52) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/LambdaPolicy.java (43:50) duplicated block id: 130 size: 6 cleaned lines of code in 2 files: - runtime/master/src/main/java/org/apache/nemo/runtime/master/servlet/AllMetricServlet.java (33:38) - runtime/master/src/main/java/org/apache/nemo/runtime/master/servlet/StageMetricServlet.java (34:39) duplicated block id: 131 size: 6 cleaned lines of code in 2 files: - compiler/frontend/beam/src/main/java/org/apache/nemo/compiler/frontend/beam/transform/WindowFnTransform.java (87:97) - compiler/frontend/spark/src/main/java/org/apache/nemo/compiler/frontend/spark/transform/MapTransform.java (48:58) duplicated block id: 132 size: 6 cleaned lines of code in 2 files: - common/src/main/java/org/apache/nemo/common/punctuation/Watermark.java (45:51) - runtime/executor/src/main/java/org/apache/nemo/runtime/executor/data/FileArea.java (65:71) duplicated block id: 133 size: 6 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/DisaggregationPolicy.java (45:52) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/StreamingPolicy.java (42:49) duplicated block id: 134 size: 6 cleaned lines of code in 2 files: - compiler/frontend/spark/src/main/java/org/apache/nemo/compiler/frontend/spark/sql/DataFrameReader.java (284:289) - compiler/frontend/spark/src/main/java/org/apache/nemo/compiler/frontend/spark/sql/SparkSession.java (251:256) duplicated block id: 135 size: 6 cleaned lines of code in 2 files: - common/src/main/java/org/apache/nemo/common/ir/edge/IREdge.java (112:118) - runtime/executor/src/main/java/org/apache/nemo/runtime/executor/data/FileArea.java (65:71) duplicated block id: 136 size: 6 cleaned lines of code in 2 files: - common/src/main/java/org/apache/nemo/common/ir/vertex/executionproperty/ClonedSchedulingProperty.java (128:135) - common/src/main/java/org/apache/nemo/common/punctuation/LatencyMark.java (95:101) duplicated block id: 137 size: 6 cleaned lines of code in 2 files: - runtime/master/src/main/java/org/apache/nemo/runtime/master/servlet/StageMetricServlet.java (34:39) - runtime/master/src/main/java/org/apache/nemo/runtime/master/servlet/TaskMetricServlet.java (34:39) duplicated block id: 138 size: 6 cleaned lines of code in 2 files: - compiler/frontend/beam/src/main/java/org/apache/nemo/compiler/frontend/beam/transform/WindowFnTransform.java (87:97) - compiler/frontend/spark/src/main/java/org/apache/nemo/compiler/frontend/spark/transform/FlatMapTransform.java (48:58) duplicated block id: 139 size: 6 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/DefaultPolicyWithSeparatePass.java (49:56) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/StreamingPolicy.java (42:49) duplicated block id: 140 size: 6 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/pass/compiletime/annotating/ShuffleEdgePushPass.java (43:48) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/pass/compiletime/annotating/TransientResourceDataTransferPass.java (44:49) duplicated block id: 141 size: 6 cleaned lines of code in 2 files: - webui/components/jobs/detail/DAG.vue (81:87) - webui/components/jobs/detail/JobView.vue (342:348) duplicated block id: 142 size: 6 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/LambdaPolicy.java (43:50) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/XGBoostPolicy.java (45:52) duplicated block id: 143 size: 6 cleaned lines of code in 2 files: - common/src/main/java/org/apache/nemo/common/ir/edge/IREdge.java (112:118) - common/src/main/java/org/apache/nemo/common/punctuation/Watermark.java (45:51) duplicated block id: 144 size: 6 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/ConditionalLargeShufflePolicy.java (58:65) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/DynamicTaskSizingPolicy.java (53:60) duplicated block id: 145 size: 6 cleaned lines of code in 2 files: - common/src/main/java/org/apache/nemo/common/ir/vertex/executionproperty/ClonedSchedulingProperty.java (128:135) - runtime/executor/src/main/java/org/apache/nemo/runtime/executor/data/FileArea.java (65:71) duplicated block id: 146 size: 6 cleaned lines of code in 2 files: - common/src/main/java/org/apache/nemo/common/ir/edge/IREdge.java (112:118) - common/src/main/java/org/apache/nemo/common/ir/vertex/executionproperty/ClonedSchedulingProperty.java (128:135) duplicated block id: 147 size: 6 cleaned lines of code in 2 files: - common/src/main/java/org/apache/nemo/common/ir/vertex/executionproperty/ClonedSchedulingProperty.java (128:135) - runtime/executor/src/main/java/org/apache/nemo/runtime/executor/transfer/ByteTransport.java (254:260) duplicated block id: 148 size: 6 cleaned lines of code in 2 files: - runtime/master/src/main/java/org/apache/nemo/runtime/master/servlet/JobMetricServlet.java (34:39) - runtime/master/src/main/java/org/apache/nemo/runtime/master/servlet/TaskMetricServlet.java (34:39) duplicated block id: 149 size: 6 cleaned lines of code in 2 files: - runtime/executor/src/main/java/org/apache/nemo/runtime/executor/data/partition/SerializedPartition.java (101:106) - runtime/executor/src/main/java/org/apache/nemo/runtime/executor/data/partition/SerializedPartition.java (125:130) duplicated block id: 150 size: 6 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/pass/compiletime/annotating/DefaultScheduleGroupPass.java (215:221) - runtime/executor/src/main/java/org/apache/nemo/runtime/executor/transfer/ByteTransport.java (254:260) duplicated block id: 151 size: 6 cleaned lines of code in 2 files: - common/src/main/java/org/apache/nemo/common/HashRange.java (104:110) - common/src/main/java/org/apache/nemo/common/punctuation/LatencyMark.java (95:101) duplicated block id: 152 size: 6 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/ConditionalLargeShufflePolicy.java (58:65) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/LambdaPolicy.java (43:50) duplicated block id: 153 size: 6 cleaned lines of code in 2 files: - common/src/main/java/org/apache/nemo/common/punctuation/Watermark.java (45:51) - runtime/common/src/main/java/org/apache/nemo/runtime/common/plan/StageEdge.java (118:124) duplicated block id: 154 size: 6 cleaned lines of code in 2 files: - common/src/main/java/org/apache/nemo/common/HashRange.java (104:110) - runtime/executor/src/main/java/org/apache/nemo/runtime/executor/transfer/ByteTransport.java (254:260) duplicated block id: 155 size: 6 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/StreamingPolicy.java (42:49) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/TransientResourcePolicy.java (45:52) duplicated block id: 156 size: 6 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/DynamicTaskSizingPolicy.java (53:60) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/LambdaPolicy.java (43:50) duplicated block id: 157 size: 6 cleaned lines of code in 2 files: - common/src/main/java/org/apache/nemo/common/HashRange.java (104:110) - common/src/main/java/org/apache/nemo/common/ir/vertex/executionproperty/ClonedSchedulingProperty.java (128:135) duplicated block id: 158 size: 6 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/DefaultPolicyWithSeparatePass.java (49:56) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/LambdaPolicy.java (43:50) duplicated block id: 159 size: 6 cleaned lines of code in 2 files: - runtime/executor/src/main/java/org/apache/nemo/runtime/executor/data/block/NonSerializedMemoryBlock.java (64:80) - runtime/executor/src/main/java/org/apache/nemo/runtime/executor/data/block/SerializedMemoryBlock.java (64:80) duplicated block id: 160 size: 6 cleaned lines of code in 2 files: - common/src/main/java/org/apache/nemo/common/ir/edge/IREdge.java (112:118) - runtime/executor/src/main/java/org/apache/nemo/runtime/executor/transfer/ByteTransport.java (254:260) duplicated block id: 161 size: 6 cleaned lines of code in 2 files: - runtime/executor/src/main/java/org/apache/nemo/runtime/executor/data/stores/MemoryStore.java (70:81) - runtime/executor/src/main/java/org/apache/nemo/runtime/executor/data/stores/SerializedMemoryStore.java (69:80) duplicated block id: 162 size: 6 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/BasicPullPolicy.java (44:51) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/LambdaPolicy.java (43:50) duplicated block id: 163 size: 6 cleaned lines of code in 2 files: - common/src/main/java/org/apache/nemo/common/ir/edge/IREdge.java (112:118) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/pass/compiletime/annotating/DefaultParallelismPass.java (138:144) duplicated block id: 164 size: 6 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/ConditionalLargeShufflePolicy.java (58:65) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/XGBoostPolicy.java (45:52) duplicated block id: 165 size: 6 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/ConditionalLargeShufflePolicy.java (58:65) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/DisaggregationPolicy.java (45:52) duplicated block id: 166 size: 6 cleaned lines of code in 2 files: - common/src/main/java/org/apache/nemo/common/HashRange.java (104:110) - runtime/common/src/main/java/org/apache/nemo/runtime/common/plan/StageEdge.java (118:124) duplicated block id: 167 size: 6 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/ConditionalLargeShufflePolicy.java (58:65) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/DataSkewPolicy.java (49:56) duplicated block id: 168 size: 6 cleaned lines of code in 2 files: - common/src/main/java/org/apache/nemo/common/punctuation/LatencyMark.java (95:101) - runtime/common/src/main/java/org/apache/nemo/runtime/common/plan/StageEdge.java (118:124) duplicated block id: 169 size: 6 cleaned lines of code in 2 files: - runtime/master/src/main/java/org/apache/nemo/runtime/master/servlet/AllMetricServlet.java (33:38) - runtime/master/src/main/java/org/apache/nemo/runtime/master/servlet/TaskMetricServlet.java (34:39) duplicated block id: 170 size: 6 cleaned lines of code in 2 files: - runtime/common/src/main/java/org/apache/nemo/runtime/common/plan/StageEdge.java (118:124) - runtime/executor/src/main/java/org/apache/nemo/runtime/executor/transfer/ByteTransport.java (254:260) duplicated block id: 171 size: 6 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/pass/compiletime/annotating/DuplicateEdgeGroupSizePass.java (44:49) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/pass/compiletime/annotating/DuplicateEdgeGroupSizePass.java (55:60) duplicated block id: 172 size: 6 cleaned lines of code in 2 files: - runtime/executor/src/main/java/org/apache/nemo/runtime/executor/TaskStateManager.java (125:130) - runtime/master/src/main/java/org/apache/nemo/runtime/master/scheduler/SimulatedTaskExecutor.java (219:224) duplicated block id: 173 size: 6 cleaned lines of code in 2 files: - common/src/main/java/org/apache/nemo/common/ir/vertex/executionproperty/ClonedSchedulingProperty.java (128:135) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/pass/compiletime/annotating/DefaultScheduleGroupPass.java (215:221) duplicated block id: 174 size: 6 cleaned lines of code in 2 files: - common/src/main/java/org/apache/nemo/common/HashRange.java (104:110) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/pass/compiletime/annotating/DefaultParallelismPass.java (138:144) duplicated block id: 175 size: 6 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/pass/compiletime/annotating/DefaultParallelismPass.java (138:144) - runtime/executor/src/main/java/org/apache/nemo/runtime/executor/transfer/ByteTransport.java (254:260) duplicated block id: 176 size: 6 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/BasicPushPolicy.java (46:53) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/ConditionalLargeShufflePolicy.java (58:65) duplicated block id: 177 size: 6 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/LargeShufflePolicy.java (45:52) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/StreamingPolicy.java (42:49) duplicated block id: 178 size: 6 cleaned lines of code in 2 files: - common/src/main/java/org/apache/nemo/common/HashRange.java (104:110) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/pass/compiletime/annotating/DefaultScheduleGroupPass.java (215:221) duplicated block id: 179 size: 6 cleaned lines of code in 2 files: - common/src/main/java/org/apache/nemo/common/punctuation/LatencyMark.java (95:101) - runtime/executor/src/main/java/org/apache/nemo/runtime/executor/data/FileArea.java (65:71) duplicated block id: 180 size: 6 cleaned lines of code in 2 files: - common/src/main/java/org/apache/nemo/common/ir/vertex/executionproperty/ClonedSchedulingProperty.java (128:135) - runtime/common/src/main/java/org/apache/nemo/runtime/common/plan/StageEdge.java (118:124) duplicated block id: 181 size: 6 cleaned lines of code in 2 files: - compiler/frontend/spark/src/main/java/org/apache/nemo/compiler/frontend/spark/core/rdd/SparkJavaPairRDD.java (497:504) - compiler/frontend/spark/src/main/java/org/apache/nemo/compiler/frontend/spark/core/rdd/SparkJavaPairRDD.java (542:549) duplicated block id: 182 size: 6 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/BasicPullPolicy.java (44:51) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/StreamingPolicy.java (42:49) duplicated block id: 183 size: 6 cleaned lines of code in 2 files: - common/src/main/java/org/apache/nemo/common/ir/vertex/executionproperty/ClonedSchedulingProperty.java (128:135) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/pass/compiletime/annotating/DefaultParallelismPass.java (138:144) duplicated block id: 184 size: 6 cleaned lines of code in 2 files: - compiler/frontend/beam/src/main/java/org/apache/nemo/compiler/frontend/beam/source/BeamBoundedSourceVertex.java (102:111) - compiler/frontend/beam/src/main/java/org/apache/nemo/compiler/frontend/beam/source/BeamUnboundedSourceVertex.java (92:101) duplicated block id: 185 size: 6 cleaned lines of code in 2 files: - webui/components/jobs/JobsView.vue (49:56) - webui/components/jobs/JobsView.vue (102:109) duplicated block id: 186 size: 6 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/ConditionalLargeShufflePolicy.java (58:65) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/SamplingLargeShuffleSkewPolicy.java (51:58) duplicated block id: 187 size: 6 cleaned lines of code in 2 files: - common/src/main/java/org/apache/nemo/common/HashRange.java (104:110) - common/src/main/java/org/apache/nemo/common/punctuation/Watermark.java (45:51) duplicated block id: 188 size: 6 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/SamplingLargeShuffleSkewPolicy.java (51:58) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/StreamingPolicy.java (42:49) duplicated block id: 189 size: 6 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/ConditionalLargeShufflePolicy.java (58:65) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/TransientResourcePolicy.java (45:52) duplicated block id: 190 size: 6 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/BasicPushPolicy.java (46:53) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/LambdaPolicy.java (43:50) duplicated block id: 191 size: 6 cleaned lines of code in 2 files: - common/src/main/java/org/apache/nemo/common/HashRange.java (104:110) - common/src/main/java/org/apache/nemo/common/ir/edge/IREdge.java (112:118) duplicated block id: 192 size: 6 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/pass/compiletime/annotating/DefaultScheduleGroupPass.java (215:221) - runtime/common/src/main/java/org/apache/nemo/runtime/common/plan/StageEdge.java (118:124) duplicated block id: 193 size: 6 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/DefaultPolicy.java (41:48) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/LambdaPolicy.java (43:50) duplicated block id: 194 size: 6 cleaned lines of code in 2 files: - common/src/main/java/org/apache/nemo/common/punctuation/LatencyMark.java (95:101) - runtime/executor/src/main/java/org/apache/nemo/runtime/executor/transfer/ByteTransport.java (254:260) duplicated block id: 195 size: 6 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/DataSkewPolicy.java (49:56) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/StreamingPolicy.java (42:49) duplicated block id: 196 size: 6 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/pass/compiletime/annotating/DefaultParallelismPass.java (138:144) - runtime/executor/src/main/java/org/apache/nemo/runtime/executor/data/FileArea.java (65:71) duplicated block id: 197 size: 6 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/BasicPushPolicy.java (46:53) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/StreamingPolicy.java (42:49) duplicated block id: 198 size: 6 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/pass/compiletime/annotating/DefaultParallelismPass.java (138:144) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/pass/compiletime/annotating/DefaultScheduleGroupPass.java (215:221) duplicated block id: 199 size: 6 cleaned lines of code in 2 files: - compiler/frontend/beam/src/main/java/org/apache/nemo/compiler/frontend/beam/transform/WindowFnTransform.java (87:97) - compiler/frontend/spark/src/main/java/org/apache/nemo/compiler/frontend/spark/transform/MapToPairTransform.java (51:61) duplicated block id: 200 size: 6 cleaned lines of code in 2 files: - common/src/main/java/org/apache/nemo/common/punctuation/LatencyMark.java (95:101) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/pass/compiletime/annotating/DefaultParallelismPass.java (138:144) duplicated block id: 201 size: 6 cleaned lines of code in 2 files: - runtime/master/src/main/java/org/apache/nemo/runtime/master/servlet/JobMetricServlet.java (34:39) - runtime/master/src/main/java/org/apache/nemo/runtime/master/servlet/StageMetricServlet.java (34:39) duplicated block id: 202 size: 6 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/DynamicTaskSizingPolicy.java (53:60) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/StreamingPolicy.java (42:49) duplicated block id: 203 size: 6 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/DataSkewPolicy.java (49:56) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/LambdaPolicy.java (43:50) duplicated block id: 204 size: 6 cleaned lines of code in 2 files: - runtime/executor/src/main/java/org/apache/nemo/runtime/executor/data/FileArea.java (65:71) - runtime/executor/src/main/java/org/apache/nemo/runtime/executor/transfer/ByteTransport.java (254:260) duplicated block id: 205 size: 6 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/ConditionalLargeShufflePolicy.java (58:65) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/DefaultPolicyWithSeparatePass.java (49:56) duplicated block id: 206 size: 6 cleaned lines of code in 2 files: - common/src/main/java/org/apache/nemo/common/ir/edge/IREdge.java (112:118) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/pass/compiletime/annotating/DefaultScheduleGroupPass.java (215:221) duplicated block id: 207 size: 6 cleaned lines of code in 2 files: - runtime/executor/src/main/java/org/apache/nemo/runtime/executor/datatransfer/BlockInputReader.java (72:79) - runtime/executor/src/main/java/org/apache/nemo/runtime/executor/datatransfer/PipeInputReader.java (62:69) duplicated block id: 208 size: 6 cleaned lines of code in 2 files: - common/src/main/java/org/apache/nemo/common/HashRange.java (104:110) - runtime/executor/src/main/java/org/apache/nemo/runtime/executor/data/FileArea.java (65:71) duplicated block id: 209 size: 6 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/StreamingPolicy.java (42:49) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/XGBoostPolicy.java (45:52) duplicated block id: 210 size: 6 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/ConditionalLargeShufflePolicy.java (58:65) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/StreamingPolicy.java (42:49) duplicated block id: 211 size: 6 cleaned lines of code in 2 files: - webui/components/jobs/detail/DAG.vue (81:87) - webui/components/jobs/detail/MetricTimeline.vue (154:160) duplicated block id: 212 size: 6 cleaned lines of code in 2 files: - runtime/executor/src/main/java/org/apache/nemo/runtime/executor/datatransfer/BlockInputReader.java (152:157) - runtime/executor/src/main/java/org/apache/nemo/runtime/executor/datatransfer/BlockInputReader.java (183:188) duplicated block id: 213 size: 6 cleaned lines of code in 2 files: - runtime/executor/src/main/java/org/apache/nemo/runtime/executor/data/stores/LocalFileStore.java (83:97) - runtime/executor/src/main/java/org/apache/nemo/runtime/executor/data/stores/MemoryStore.java (70:81) duplicated block id: 214 size: 6 cleaned lines of code in 2 files: - common/src/main/java/org/apache/nemo/common/punctuation/LatencyMark.java (95:101) - common/src/main/java/org/apache/nemo/common/punctuation/Watermark.java (45:51) duplicated block id: 215 size: 6 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/BasicPullPolicy.java (44:51) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/ConditionalLargeShufflePolicy.java (58:65) duplicated block id: 216 size: 6 cleaned lines of code in 2 files: - runtime/master/src/main/java/org/apache/nemo/runtime/master/metric/MetricManagerMaster.java (66:73) - runtime/master/src/main/java/org/apache/nemo/runtime/master/scheduler/SimulationScheduler.java (416:423) duplicated block id: 217 size: 6 cleaned lines of code in 2 files: - runtime/master/src/main/java/org/apache/nemo/runtime/master/metric/MetricStore.java (192:199) - runtime/master/src/main/java/org/apache/nemo/runtime/master/metric/MetricStore.java (218:225) duplicated block id: 218 size: 6 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/LambdaPolicy.java (43:50) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/SamplingLargeShuffleSkewPolicy.java (51:58) duplicated block id: 219 size: 6 cleaned lines of code in 2 files: - common/src/main/java/org/apache/nemo/common/punctuation/LatencyMark.java (95:101) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/pass/compiletime/annotating/DefaultScheduleGroupPass.java (215:221) duplicated block id: 220 size: 6 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/DefaultPolicy.java (41:48) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/StreamingPolicy.java (42:49) duplicated block id: 221 size: 6 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/pass/compiletime/annotating/DefaultParallelismPass.java (138:144) - runtime/common/src/main/java/org/apache/nemo/runtime/common/plan/StageEdge.java (118:124) duplicated block id: 222 size: 6 cleaned lines of code in 2 files: - common/src/main/java/org/apache/nemo/common/punctuation/Watermark.java (45:51) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/pass/compiletime/annotating/DefaultParallelismPass.java (138:144) duplicated block id: 223 size: 6 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/LambdaPolicy.java (43:50) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/LargeShufflePolicy.java (45:52) duplicated block id: 224 size: 6 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/pass/compiletime/annotating/DefaultScheduleGroupPass.java (215:221) - runtime/executor/src/main/java/org/apache/nemo/runtime/executor/data/FileArea.java (65:71) duplicated block id: 225 size: 6 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/ConditionalLargeShufflePolicy.java (58:65) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/DefaultPolicy.java (41:48) duplicated block id: 226 size: 6 cleaned lines of code in 2 files: - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/LambdaPolicy.java (43:50) - compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/policy/TransientResourcePolicy.java (45:52) duplicated block id: 227 size: 6 cleaned lines of code in 2 files: - runtime/common/src/main/java/org/apache/nemo/runtime/common/plan/StageEdge.java (118:124) - runtime/executor/src/main/java/org/apache/nemo/runtime/executor/data/FileArea.java (65:71) duplicated block id: 228 size: 6 cleaned lines of code in 2 files: - compiler/frontend/beam/src/main/java/org/apache/nemo/compiler/frontend/beam/PipelineTranslator.java (502:507) - compiler/frontend/beam/src/main/java/org/apache/nemo/compiler/frontend/beam/PipelineTranslator.java (514:519)