duplicated block id: 1 size: 32 cleaned lines of code in 2 files: - workload_generator/AIOB_simAI_workload_generator.py (504:542) - workload_generator/AIOB_simAI_workload_generator.py (744:776) duplicated block id: 2 size: 24 cleaned lines of code in 2 files: - workload_generator/generate_megatron_workload.py (157:184) - workload_generator/generate_megatron_workload.py (210:235) duplicated block id: 3 size: 21 cleaned lines of code in 2 files: - workload_generator/generate_megatron_workload.py (157:179) - workload_generator/generate_megatron_workload.py (333:353) duplicated block id: 4 size: 21 cleaned lines of code in 2 files: - workload_generator/generate_megatron_workload.py (210:231) - workload_generator/generate_megatron_workload.py (333:353) duplicated block id: 5 size: 17 cleaned lines of code in 2 files: - workload_generator/AIOB_simAI_workload_generator.py (212:229) - workload_generator/AIOB_simAI_workload_generator.py (578:594) duplicated block id: 6 size: 15 cleaned lines of code in 2 files: - workload_generator/AIOB_simAI_workload_generator.py (249:263) - workload_generator/AIOB_simAI_workload_generator.py (596:610) duplicated block id: 7 size: 15 cleaned lines of code in 2 files: - workload_generator/AIOB_simAI_workload_generator.py (615:629) - workload_generator/AIOB_simAI_workload_generator.py (702:716) duplicated block id: 8 size: 14 cleaned lines of code in 2 files: - workload_generator/AIOB_simAI_workload_generator.py (348:361) - workload_generator/AIOB_simAI_workload_generator.py (492:505) duplicated block id: 9 size: 13 cleaned lines of code in 2 files: - workload_generator/AIOB_simAI_workload_generator.py (409:421) - workload_generator/AIOB_simAI_workload_generator.py (684:697) duplicated block id: 10 size: 13 cleaned lines of code in 2 files: - workload_generator/AIOB_simAI_workload_generator.py (389:401) - workload_generator/AIOB_simAI_workload_generator.py (664:676) duplicated block id: 11 size: 12 cleaned lines of code in 2 files: - workload_generator/AIOB_simAI_workload_generator.py (149:160) - workload_generator/AIOB_simAI_workload_generator.py (546:557) duplicated block id: 12 size: 11 cleaned lines of code in 2 files: - workload_generator/generate_megatron_workload.py (263:273) - workload_generator/generate_megatron_workload.py (318:328) duplicated block id: 13 size: 11 cleaned lines of code in 2 files: - workload_generator/generate_megatron_workload.py (146:156) - workload_generator/generate_megatron_workload.py (196:206) duplicated block id: 14 size: 11 cleaned lines of code in 2 files: - workload_generator/AIOB_simAI_workload_generator.py (475:485) - workload_generator/AIOB_simAI_workload_generator.py (707:717) duplicated block id: 15 size: 11 cleaned lines of code in 2 files: - workload_generator/generate_deepspeed_stage3_workload.py (126:136) - workload_generator/generate_deepspeed_stage3_workload.py (139:149) duplicated block id: 16 size: 11 cleaned lines of code in 2 files: - workload_generator/generate_megatron_workload.py (276:286) - workload_generator/generate_megatron_workload.py (318:328) duplicated block id: 17 size: 11 cleaned lines of code in 2 files: - workload_generator/generate_megatron_workload.py (183:193) - workload_generator/generate_megatron_workload.py (246:256) duplicated block id: 18 size: 11 cleaned lines of code in 2 files: - workload_generator/AIOB_simAI_workload_generator.py (386:396) - workload_generator/AIOB_simAI_workload_generator.py (422:432) duplicated block id: 19 size: 11 cleaned lines of code in 2 files: - workload_generator/generate_megatron_workload.py (263:273) - workload_generator/generate_megatron_workload.py (276:286) duplicated block id: 20 size: 10 cleaned lines of code in 2 files: - workload_generator/AIOB_simAI_workload_generator.py (475:484) - workload_generator/AIOB_simAI_workload_generator.py (620:629) duplicated block id: 21 size: 10 cleaned lines of code in 2 files: - workload_generator/AIOB_simAI_workload_generator.py (295:305) - workload_generator/AIOB_simAI_workload_generator.py (329:339) duplicated block id: 22 size: 9 cleaned lines of code in 2 files: - workload_generator/generate_megatron_workload.py (49:57) - workload_generator/generate_megatron_workload.py (59:67) duplicated block id: 23 size: 9 cleaned lines of code in 2 files: - workload_generator/AIOB_simAI_workload_generator.py (703:711) - workload_generator/AIOB_simAI_workload_generator.py (718:726) duplicated block id: 24 size: 9 cleaned lines of code in 2 files: - workload_generator/AIOB_simAI_workload_generator.py (285:293) - workload_generator/AIOB_simAI_workload_generator.py (622:631) duplicated block id: 25 size: 9 cleaned lines of code in 2 files: - workload_generator/AIOB_simAI_workload_generator.py (616:624) - workload_generator/AIOB_simAI_workload_generator.py (718:726) duplicated block id: 26 size: 9 cleaned lines of code in 2 files: - log_analyzer/ds_comm_log_analyzer.py (39:47) - workload_generator/analysis_pytorch_trace.py (37:45) duplicated block id: 27 size: 9 cleaned lines of code in 2 files: - workload_generator/generate_deepspeed_stage1_2_workload.py (56:64) - workload_generator/generate_deepspeed_stage3_workload.py (78:86) duplicated block id: 28 size: 9 cleaned lines of code in 2 files: - workload_generator/generate_deepspeed_stage1_2_workload.py (124:132) - workload_generator/generate_deepspeed_stage3_workload.py (148:158) duplicated block id: 29 size: 9 cleaned lines of code in 2 files: - workload_generator/generate_deepspeed_stage3_workload.py (315:324) - workload_generator/generate_megatron_workload.py (438:447) duplicated block id: 30 size: 8 cleaned lines of code in 2 files: - workload_generator/AIOB_simAI_workload_generator.py (753:760) - workload_generator/AIOB_simAI_workload_generator.py (769:776) duplicated block id: 31 size: 8 cleaned lines of code in 2 files: - workload_generator/generate_megatron_workload.py (232:240) - workload_generator/generate_megatron_workload.py (301:308) duplicated block id: 32 size: 8 cleaned lines of code in 2 files: - workload_generator/generate_megatron_workload.py (196:203) - workload_generator/generate_megatron_workload.py (234:241) duplicated block id: 33 size: 8 cleaned lines of code in 2 files: - workload_generator/generate_deepspeed_stage1_2_workload.py (83:90) - workload_generator/generate_deepspeed_stage3_workload.py (126:133) duplicated block id: 34 size: 8 cleaned lines of code in 2 files: - workload_generator/AIOB_simAI_workload_generator.py (425:432) - workload_generator/AIOB_simAI_workload_generator.py (664:671) duplicated block id: 35 size: 8 cleaned lines of code in 2 files: - workload_generator/AIOB_simAI_workload_generator.py (313:320) - workload_generator/AIOB_simAI_workload_generator.py (348:355) duplicated block id: 36 size: 8 cleaned lines of code in 2 files: - workload_generator/generate_megatron_workload.py (146:153) - workload_generator/generate_megatron_workload.py (234:241) duplicated block id: 37 size: 8 cleaned lines of code in 2 files: - workload_generator/AIOB_simAI_workload_generator.py (519:526) - workload_generator/AIOB_simAI_workload_generator.py (535:542) duplicated block id: 38 size: 8 cleaned lines of code in 2 files: - workload_generator/AIOB_simAI_workload_generator.py (285:292) - workload_generator/AIOB_simAI_workload_generator.py (477:484) duplicated block id: 39 size: 8 cleaned lines of code in 2 files: - workload_generator/generate_megatron_workload.py (75:83) - workload_generator/generate_megatron_workload.py (108:116) duplicated block id: 40 size: 8 cleaned lines of code in 2 files: - workload_generator/generate_deepspeed_stage1_2_workload.py (83:90) - workload_generator/generate_deepspeed_stage1_2_workload.py (115:122) duplicated block id: 41 size: 8 cleaned lines of code in 2 files: - workload_generator/AIOB_simAI_workload_generator.py (313:320) - workload_generator/AIOB_simAI_workload_generator.py (492:499) duplicated block id: 42 size: 8 cleaned lines of code in 2 files: - workload_generator/AIOB_simAI_workload_generator.py (535:542) - workload_generator/AIOB_simAI_workload_generator.py (753:760) duplicated block id: 43 size: 8 cleaned lines of code in 2 files: - workload_applyer.py (298:305) - workload_applyer.py (321:328) duplicated block id: 44 size: 8 cleaned lines of code in 2 files: - workload_generator/generate_deepspeed_stage1_2_workload.py (115:122) - workload_generator/generate_deepspeed_stage3_workload.py (139:146) duplicated block id: 45 size: 8 cleaned lines of code in 2 files: - workload_generator/AIOB_simAI_workload_generator.py (404:411) - workload_generator/AIOB_simAI_workload_generator.py (440:447) duplicated block id: 46 size: 8 cleaned lines of code in 2 files: - workload_generator/AIOB_simAI_workload_generator.py (285:292) - workload_generator/AIOB_simAI_workload_generator.py (709:716) duplicated block id: 47 size: 8 cleaned lines of code in 2 files: - workload_generator/AIOB_simAI_workload_generator.py (450:457) - workload_generator/AIOB_simAI_workload_generator.py (689:697) duplicated block id: 48 size: 8 cleaned lines of code in 2 files: - workload_generator/generate_deepspeed_stage1_2_workload.py (83:90) - workload_generator/generate_deepspeed_stage3_workload.py (139:146) duplicated block id: 49 size: 8 cleaned lines of code in 2 files: - workload_generator/generate_megatron_workload.py (287:294) - workload_generator/generate_megatron_workload.py (302:309) duplicated block id: 50 size: 8 cleaned lines of code in 2 files: - workload_generator/AIOB_simAI_workload_generator.py (519:526) - workload_generator/AIOB_simAI_workload_generator.py (769:776) duplicated block id: 51 size: 8 cleaned lines of code in 2 files: - workload_generator/generate_deepspeed_stage1_2_workload.py (115:122) - workload_generator/generate_deepspeed_stage3_workload.py (126:133) duplicated block id: 52 size: 8 cleaned lines of code in 2 files: - workload_generator/AIOB_simAI_workload_generator.py (414:421) - workload_generator/AIOB_simAI_workload_generator.py (450:457) duplicated block id: 53 size: 8 cleaned lines of code in 2 files: - workload_generator/AIOB_simAI_workload_generator.py (276:283) - workload_generator/AIOB_simAI_workload_generator.py (468:475) duplicated block id: 54 size: 7 cleaned lines of code in 2 files: - workload_generator/generate_megatron_workload.py (410:417) - workload_generator/generate_megatron_workload.py (420:427) duplicated block id: 55 size: 7 cleaned lines of code in 2 files: - workload_generator/generate_deepspeed_stage1_2_workload.py (259:266) - workload_generator/generate_deepspeed_stage3_workload.py (317:324) duplicated block id: 56 size: 7 cleaned lines of code in 2 files: - workload_generator/generate_deepspeed_stage1_2_workload.py (66:73) - workload_generator/generate_deepspeed_stage3_workload.py (89:96) duplicated block id: 57 size: 7 cleaned lines of code in 2 files: - workload_generator/generate_megatron_workload.py (234:240) - workload_generator/generate_megatron_workload.py (287:293) duplicated block id: 58 size: 7 cleaned lines of code in 2 files: - workload_generator/generate_megatron_workload.py (109:116) - workload_generator/generate_megatron_workload.py (390:396) duplicated block id: 59 size: 7 cleaned lines of code in 2 files: - workload_generator/generate_megatron_workload.py (196:202) - workload_generator/generate_megatron_workload.py (287:293) duplicated block id: 60 size: 7 cleaned lines of code in 2 files: - workload_generator/generate_megatron_workload.py (246:252) - workload_generator/generate_megatron_workload.py (276:282) duplicated block id: 61 size: 7 cleaned lines of code in 2 files: - workload_generator/generate_megatron_workload.py (196:202) - workload_generator/generate_megatron_workload.py (302:308) duplicated block id: 62 size: 7 cleaned lines of code in 2 files: - workload_generator/generate_megatron_workload.py (246:252) - workload_generator/generate_megatron_workload.py (263:269) duplicated block id: 63 size: 7 cleaned lines of code in 2 files: - workload_generator/AIOB_simAI_workload_generator.py (499:505) - workload_generator/AIOB_simAI_workload_generator.py (710:716) duplicated block id: 64 size: 7 cleaned lines of code in 2 files: - workload_generator/generate_megatron_workload.py (246:252) - workload_generator/generate_megatron_workload.py (318:324) duplicated block id: 65 size: 7 cleaned lines of code in 2 files: - workload_generator/generate_megatron_workload.py (146:152) - workload_generator/generate_megatron_workload.py (302:308) duplicated block id: 66 size: 7 cleaned lines of code in 2 files: - workload_generator/AIOB_simAI_workload_generator.py (378:384) - workload_generator/AIOB_simAI_workload_generator.py (654:660) duplicated block id: 67 size: 7 cleaned lines of code in 2 files: - workload_generator/AIOB_simAI_workload_generator.py (175:181) - workload_generator/AIOB_simAI_workload_generator.py (832:838) duplicated block id: 68 size: 7 cleaned lines of code in 2 files: - workload_generator/AIOB_simAI_workload_generator.py (499:505) - workload_generator/AIOB_simAI_workload_generator.py (623:629) duplicated block id: 69 size: 7 cleaned lines of code in 2 files: - workload_generator/AIOB_simAI_workload_generator.py (286:292) - workload_generator/AIOB_simAI_workload_generator.py (499:505) duplicated block id: 70 size: 7 cleaned lines of code in 2 files: - workload_generator/generate_megatron_workload.py (146:152) - workload_generator/generate_megatron_workload.py (287:293) duplicated block id: 71 size: 7 cleaned lines of code in 2 files: - visualize/generate.py (405:411) - visualize/generate.py (455:461) duplicated block id: 72 size: 7 cleaned lines of code in 2 files: - workload_generator/generate_megatron_workload.py (183:189) - workload_generator/generate_megatron_workload.py (318:324) duplicated block id: 73 size: 7 cleaned lines of code in 2 files: - workload_generator/AIOB_simAI_workload_generator.py (355:361) - workload_generator/AIOB_simAI_workload_generator.py (710:716) duplicated block id: 74 size: 7 cleaned lines of code in 2 files: - workload_generator/AIOB_simAI_workload_generator.py (478:484) - workload_generator/AIOB_simAI_workload_generator.py (499:505) duplicated block id: 75 size: 7 cleaned lines of code in 2 files: - workload_generator/generate_megatron_workload.py (86:93) - workload_generator/generate_megatron_workload.py (218:224) duplicated block id: 76 size: 7 cleaned lines of code in 2 files: - workload_generator/generate_megatron_workload.py (76:83) - workload_generator/generate_megatron_workload.py (390:396) duplicated block id: 77 size: 7 cleaned lines of code in 2 files: - workload_generator/generate_megatron_workload.py (86:93) - workload_generator/generate_megatron_workload.py (341:347) duplicated block id: 78 size: 7 cleaned lines of code in 2 files: - workload_generator/generate_deepspeed_stage1_2_workload.py (259:266) - workload_generator/generate_megatron_workload.py (440:447) duplicated block id: 79 size: 7 cleaned lines of code in 2 files: - workload_generator/AIOB_simAI_workload_generator.py (189:195) - workload_generator/AIOB_simAI_workload_generator.py (832:838) duplicated block id: 80 size: 7 cleaned lines of code in 2 files: - workload_generator/AIOB_simAI_workload_generator.py (286:292) - workload_generator/AIOB_simAI_workload_generator.py (355:361) duplicated block id: 81 size: 7 cleaned lines of code in 2 files: - workload_generator/generate_megatron_workload.py (86:93) - workload_generator/generate_megatron_workload.py (165:171) duplicated block id: 82 size: 7 cleaned lines of code in 2 files: - workload_generator/generate_megatron_workload.py (86:93) - workload_generator/generate_megatron_workload.py (155:161) duplicated block id: 83 size: 7 cleaned lines of code in 2 files: - workload_generator/AIOB_simAI_workload_generator.py (175:181) - workload_generator/AIOB_simAI_workload_generator.py (189:195) duplicated block id: 84 size: 7 cleaned lines of code in 2 files: - workload_generator/generate_megatron_workload.py (155:161) - workload_generator/generate_megatron_workload.py (218:224) duplicated block id: 85 size: 7 cleaned lines of code in 2 files: - workload_generator/AIOB_simAI_workload_generator.py (355:361) - workload_generator/AIOB_simAI_workload_generator.py (478:484) duplicated block id: 86 size: 7 cleaned lines of code in 2 files: - workload_generator/generate_megatron_workload.py (183:189) - workload_generator/generate_megatron_workload.py (276:282) duplicated block id: 87 size: 7 cleaned lines of code in 2 files: - workload_generator/AIOB_simAI_workload_generator.py (207:213) - workload_generator/AIOB_simAI_workload_generator.py (739:745) duplicated block id: 88 size: 7 cleaned lines of code in 2 files: - workload_generator/AIOB_simAI_workload_generator.py (355:361) - workload_generator/AIOB_simAI_workload_generator.py (623:629) duplicated block id: 89 size: 7 cleaned lines of code in 2 files: - workload_generator/generate_megatron_workload.py (155:161) - workload_generator/generate_megatron_workload.py (341:347) duplicated block id: 90 size: 7 cleaned lines of code in 2 files: - workload_generator/generate_megatron_workload.py (183:189) - workload_generator/generate_megatron_workload.py (263:269) duplicated block id: 91 size: 7 cleaned lines of code in 2 files: - workload_generator/generate_megatron_workload.py (155:161) - workload_generator/generate_megatron_workload.py (165:171) duplicated block id: 92 size: 6 cleaned lines of code in 2 files: - visualize/generate.py (207:212) - visualize/generate.py (217:222) duplicated block id: 93 size: 6 cleaned lines of code in 2 files: - workload_generator/generate_deepspeed_stage3_workload.py (114:119) - workload_generator/generate_deepspeed_stage3_workload.py (164:169) duplicated block id: 94 size: 6 cleaned lines of code in 2 files: - workload_generator/generate_deepspeed_stage3_workload.py (92:97) - workload_generator/generate_deepspeed_stage3_workload.py (103:108) duplicated block id: 95 size: 6 cleaned lines of code in 2 files: - workload_generator/generate_deepspeed_stage1_2_workload.py (141:146) - workload_generator/generate_deepspeed_stage3_workload.py (284:289) duplicated block id: 96 size: 6 cleaned lines of code in 2 files: - log_analyzer/log.py (211:216) - log_analyzer/log.py (296:301) duplicated block id: 97 size: 6 cleaned lines of code in 2 files: - workload_generator/generate_deepspeed_stage3_workload.py (291:296) - workload_generator/generate_megatron_workload.py (363:369) duplicated block id: 98 size: 6 cleaned lines of code in 2 files: - visualize/generate.py (298:304) - visualize/generate.py (395:401) duplicated block id: 99 size: 6 cleaned lines of code in 2 files: - workload_generator/AIOB_simAI_workload_generator.py (295:300) - workload_generator/AIOB_simAI_workload_generator.py (363:368) duplicated block id: 100 size: 6 cleaned lines of code in 2 files: - workload_generator/AIOB_simAI_workload_generator.py (800:805) - workload_generator/AIOB_simAI_workload_generator.py (851:856) duplicated block id: 101 size: 6 cleaned lines of code in 2 files: - workload_generator/AIOB_simAI_workload_generator.py (851:856) - workload_generator/AIOB_simAI_workload_generator.py (864:869) duplicated block id: 102 size: 6 cleaned lines of code in 2 files: - visualize/generate.py (223:229) - visualize/generate.py (298:304) duplicated block id: 103 size: 6 cleaned lines of code in 2 files: - workload_generator/AIOB_simAI_workload_generator.py (800:805) - workload_generator/AIOB_simAI_workload_generator.py (864:869) duplicated block id: 104 size: 6 cleaned lines of code in 2 files: - visualize/generate.py (223:229) - visualize/generate.py (395:401) duplicated block id: 105 size: 6 cleaned lines of code in 2 files: - workload_generator/AIOB_simAI_workload_generator.py (270:275) - workload_generator/AIOB_simAI_workload_generator.py (458:464) duplicated block id: 106 size: 6 cleaned lines of code in 2 files: - workload_generator/AIOB_simAI_workload_generator.py (295:300) - workload_generator/AIOB_simAI_workload_generator.py (486:491) duplicated block id: 107 size: 6 cleaned lines of code in 2 files: - workload_generator/AIOB_simAI_workload_generator.py (329:334) - workload_generator/AIOB_simAI_workload_generator.py (486:491) duplicated block id: 108 size: 6 cleaned lines of code in 2 files: - workload_generator/AIOB_simAI_workload_generator.py (329:334) - workload_generator/AIOB_simAI_workload_generator.py (363:368) duplicated block id: 109 size: 6 cleaned lines of code in 2 files: - workload_generator/AIOB_simAI_workload_generator.py (363:368) - workload_generator/AIOB_simAI_workload_generator.py (486:491)