duplicated block id: 1 size: 42 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/tokenization_openai.py (111:156) - pytorch-transformers/pytorch_transformers/tokenization_xlm.py (611:656) duplicated block id: 2 size: 33 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/modeling_utils.py (780:831) - pytorch-transformers/pytorch_transformers/modeling_xlnet.py (1231:1282) duplicated block id: 3 size: 30 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/tokenization_xlm.py (99:128) - pytorch-transformers/pytorch_transformers/tokenization_xlm.py (130:159) duplicated block id: 4 size: 28 cleaned lines of code in 2 files: - XLM/src/model/transformer.py (180:215) - pytorch-transformers/pytorch_transformers/modeling_xlm.py (311:346) duplicated block id: 5 size: 26 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/modeling_gpt2.py (189:221) - pytorch-transformers/pytorch_transformers/modeling_openai.py (206:237) duplicated block id: 6 size: 26 cleaned lines of code in 2 files: - XLM/src/evaluation/evaluator.py (639:664) - pytorch-transformers/add_umt_subqs_subas_to_q_squad_format_new.py (44:69) duplicated block id: 7 size: 26 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/tokenization_gpt2.py (146:172) - pytorch-transformers/pytorch_transformers/tokenization_xlm.py (626:652) duplicated block id: 8 size: 26 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/tokenization_gpt2.py (146:172) - pytorch-transformers/pytorch_transformers/tokenization_openai.py (126:152) duplicated block id: 9 size: 23 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/modeling_transfo_xl.py (152:175) - pytorch-transformers/pytorch_transformers/modeling_xlnet.py (153:176) duplicated block id: 10 size: 21 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/modeling_bert.py (1209:1235) - pytorch-transformers/pytorch_transformers/modeling_roberta.py (413:439) duplicated block id: 11 size: 20 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/modeling_transfo_xl_utilities.py (121:144) - pytorch-transformers/pytorch_transformers/modeling_transfo_xl_utilities.py (215:237) duplicated block id: 12 size: 19 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/tokenization_gpt2.py (202:224) - pytorch-transformers/pytorch_transformers/tokenization_openai.py (186:208) duplicated block id: 13 size: 19 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/modeling_distilbert.py (177:198) - pytorch-transformers/pytorch_transformers/modeling_xlm.py (274:295) duplicated block id: 14 size: 19 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/modeling_gpt2.py (268:291) - pytorch-transformers/pytorch_transformers/modeling_openai.py (282:305) duplicated block id: 15 size: 17 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/modeling_gpt2.py (732:751) - pytorch-transformers/pytorch_transformers/modeling_openai.py (694:713) duplicated block id: 16 size: 16 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/modeling_transfo_xl.py (598:626) - pytorch-transformers/pytorch_transformers/modeling_transfo_xl.py (690:717) duplicated block id: 17 size: 15 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/modeling_transfo_xl.py (294:312) - pytorch-transformers/pytorch_transformers/modeling_xlnet.py (315:333) duplicated block id: 18 size: 15 cleaned lines of code in 2 files: - XLM/src/evaluation/evaluator.py (335:354) - XLM/src/evaluation/evaluator.py (410:429) duplicated block id: 19 size: 13 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/modeling_gpt2.py (246:263) - pytorch-transformers/pytorch_transformers/modeling_openai.py (261:276) duplicated block id: 20 size: 13 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/tokenization_gpt2.py (17:45) - pytorch-transformers/pytorch_transformers/tokenization_roberta.py (17:45) duplicated block id: 21 size: 13 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/modeling_gpt2.py (170:182) - pytorch-transformers/pytorch_transformers/modeling_openai.py (185:197) duplicated block id: 22 size: 13 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/modeling_gpt2.py (484:502) - pytorch-transformers/pytorch_transformers/modeling_openai.py (480:498) duplicated block id: 23 size: 13 cleaned lines of code in 2 files: - XLM/src/data/dataset.py (212:230) - XLM/src/data/dataset.py (377:395) duplicated block id: 24 size: 12 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/modeling_gpt2.py (607:625) - pytorch-transformers/pytorch_transformers/modeling_openai.py (586:603) duplicated block id: 25 size: 12 cleaned lines of code in 2 files: - XLM/src/optim.py (45:61) - pytorch-transformers/pytorch_transformers/optimization.py (130:149) duplicated block id: 26 size: 12 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/modeling_transfo_xl.py (386:401) - pytorch-transformers/pytorch_transformers/modeling_transfo_xl.py (475:490) duplicated block id: 27 size: 12 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/modeling_xlm.py (858:874) - pytorch-transformers/pytorch_transformers/modeling_xlnet.py (1128:1144) duplicated block id: 28 size: 11 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/modeling_gpt2.py (85:95) - pytorch-transformers/pytorch_transformers/modeling_openai.py (98:108) duplicated block id: 29 size: 11 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/modeling_bert.py (971:985) - pytorch-transformers/pytorch_transformers/modeling_xlm.py (860:874) duplicated block id: 30 size: 11 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/modeling_gpt2.py (232:243) - pytorch-transformers/pytorch_transformers/modeling_openai.py (248:259) duplicated block id: 31 size: 11 cleaned lines of code in 2 files: - XLM/src/model/memory/query.py (164:174) - XLM/src/model/memory/query.py (216:226) duplicated block id: 32 size: 11 cleaned lines of code in 2 files: - XLM/src/optim.py (107:119) - XLM/src/optim.py (161:173) duplicated block id: 33 size: 11 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/modeling_bert.py (971:985) - pytorch-transformers/pytorch_transformers/modeling_xlnet.py (1130:1144) duplicated block id: 34 size: 11 cleaned lines of code in 2 files: - XLM/src/model/transformer.py (60:79) - pytorch-transformers/pytorch_transformers/modeling_xlm.py (210:229) duplicated block id: 35 size: 10 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/tokenization_openai.py (186:198) - pytorch-transformers/pytorch_transformers/tokenization_xlm.py (769:781) duplicated block id: 36 size: 10 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/modeling_gpt2.py (91:102) - pytorch-transformers/pytorch_transformers/modeling_openai.py (109:120) duplicated block id: 37 size: 10 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/modeling_bert.py (969:981) - pytorch-transformers/pytorch_transformers/modeling_roberta.py (323:334) duplicated block id: 38 size: 10 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/modeling_transfo_xl.py (425:438) - pytorch-transformers/pytorch_transformers/modeling_transfo_xl.py (680:694) duplicated block id: 39 size: 10 cleaned lines of code in 2 files: - XLM/src/evaluation/evaluator.py (666:675) - pytorch-transformers/add_umt_subqs_subas_to_q_squad_format_new.py (71:80) duplicated block id: 40 size: 10 cleaned lines of code in 2 files: - pytorch-transformers/pseudoalignment/pseudo_decomp_bert.py (48:57) - pytorch-transformers/pseudoalignment/pseudo_decomp_bert_nsp.py (52:61) duplicated block id: 41 size: 10 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/tokenization_gpt2.py (202:214) - pytorch-transformers/pytorch_transformers/tokenization_xlm.py (769:781) duplicated block id: 42 size: 10 cleaned lines of code in 2 files: - XLM/src/model/memory/memory.py (431:442) - XLM/src/model/memory/memory.py (543:554) duplicated block id: 43 size: 9 cleaned lines of code in 2 files: - pytorch-transformers/pseudoalignment/pseudo_decomp_bert.py (19:27) - pytorch-transformers/pseudoalignment/pseudo_decomp_fasttext.py (18:26) duplicated block id: 44 size: 9 cleaned lines of code in 2 files: - pytorch-transformers/pseudoalignment/pseudo_decomp_fasttext.py (36:44) - pytorch-transformers/pseudoalignment/pseudo_decomp_variable.py (72:81) duplicated block id: 45 size: 9 cleaned lines of code in 2 files: - pytorch-transformers/pseudoalignment/pseudo_decomp_bert.py (19:27) - pytorch-transformers/pseudoalignment/pseudo_decomp_variable.py (108:116) duplicated block id: 46 size: 9 cleaned lines of code in 2 files: - XLM/src/model/transformer.py (60:68) - pytorch-transformers/pytorch_transformers/modeling_distilbert.py (113:121) duplicated block id: 47 size: 9 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/modeling_distilbert.py (113:121) - pytorch-transformers/pytorch_transformers/modeling_xlm.py (210:218) duplicated block id: 48 size: 9 cleaned lines of code in 2 files: - pytorch-transformers/pseudoalignment/pseudo_decomp_fasttext.py (18:26) - pytorch-transformers/pseudoalignment/pseudo_decomp_tfidf.py (18:26) duplicated block id: 49 size: 9 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/tokenization_openai.py (17:37) - pytorch-transformers/pytorch_transformers/tokenization_xlm.py (17:41) duplicated block id: 50 size: 9 cleaned lines of code in 2 files: - XLM/src/model/transformer.py (88:101) - pytorch-transformers/pytorch_transformers/modeling_xlm.py (241:254) duplicated block id: 51 size: 9 cleaned lines of code in 2 files: - pytorch-transformers/pseudoalignment/pseudo_decomp_fasttext.py (18:26) - pytorch-transformers/pseudoalignment/pseudo_decomp_variable.py (108:116) duplicated block id: 52 size: 9 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/modeling_roberta.py (324:334) - pytorch-transformers/pytorch_transformers/modeling_xlm.py (860:870) duplicated block id: 53 size: 9 cleaned lines of code in 2 files: - pytorch-transformers/pseudoalignment/pseudo_decomp_bert.py (82:92) - pytorch-transformers/pseudoalignment/pseudo_decomp_bert_nsp.py (71:80) duplicated block id: 54 size: 9 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/convert_gpt2_checkpoint_to_pytorch.py (45:55) - pytorch-transformers/pytorch_transformers/convert_openai_checkpoint_to_pytorch.py (45:55) duplicated block id: 55 size: 9 cleaned lines of code in 2 files: - pytorch-transformers/pseudoalignment/pseudo_decomp_tfidf.py (18:26) - pytorch-transformers/pseudoalignment/pseudo_decomp_variable.py (108:116) duplicated block id: 56 size: 9 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/modeling_bert.py (126:136) - pytorch-transformers/pytorch_transformers/modeling_openai.py (109:119) duplicated block id: 57 size: 9 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/convert_gpt2_checkpoint_to_pytorch.py (58:66) - pytorch-transformers/pytorch_transformers/convert_openai_checkpoint_to_pytorch.py (58:66) duplicated block id: 58 size: 9 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/modeling_gpt2.py (323:334) - pytorch-transformers/pytorch_transformers/modeling_openai.py (332:343) duplicated block id: 59 size: 9 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/modeling_roberta.py (324:334) - pytorch-transformers/pytorch_transformers/modeling_xlnet.py (1130:1140) duplicated block id: 60 size: 9 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/convert_roberta_checkpoint_to_pytorch.py (162:170) - pytorch-transformers/pytorch_transformers/convert_xlm_checkpoint_to_pytorch.py (65:73) duplicated block id: 61 size: 9 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/modeling_bert.py (697:705) - pytorch-transformers/pytorch_transformers/modeling_distilbert.py (514:522) duplicated block id: 62 size: 9 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/modeling_bert.py (126:136) - pytorch-transformers/pytorch_transformers/modeling_gpt2.py (91:101) duplicated block id: 63 size: 9 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/convert_transfo_xl_checkpoint_to_pytorch.py (84:94) - pytorch-transformers/pytorch_transformers/convert_xlnet_checkpoint_to_pytorch.py (66:76) duplicated block id: 64 size: 9 cleaned lines of code in 2 files: - pytorch-transformers/pseudoalignment/pseudo_decomp_bert.py (19:27) - pytorch-transformers/pseudoalignment/pseudo_decomp_tfidf.py (18:26) duplicated block id: 65 size: 9 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/modeling_transfo_xl.py (1120:1128) - pytorch-transformers/pytorch_transformers/modeling_xlnet.py (935:943) duplicated block id: 66 size: 8 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/modeling_distilbert.py (739:748) - pytorch-transformers/pytorch_transformers/modeling_roberta.py (422:431) duplicated block id: 67 size: 8 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/tokenization_gpt2.py (36:45) - pytorch-transformers/pytorch_transformers/tokenization_xlm.py (32:41) duplicated block id: 68 size: 8 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/modeling_auto.py (386:481) - pytorch-transformers/pytorch_transformers/modeling_auto.py (497:592) duplicated block id: 69 size: 8 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/tokenization_openai.py (171:184) - pytorch-transformers/pytorch_transformers/tokenization_xlm.py (738:751) duplicated block id: 70 size: 8 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/tokenization_openai.py (48:60) - pytorch-transformers/pytorch_transformers/tokenization_xlm.py (413:425) duplicated block id: 71 size: 8 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/tokenization_roberta.py (36:45) - pytorch-transformers/pytorch_transformers/tokenization_xlm.py (32:41) duplicated block id: 72 size: 8 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/tokenization_gpt2.py (36:45) - pytorch-transformers/pytorch_transformers/tokenization_openai.py (28:37) duplicated block id: 73 size: 8 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/modeling_bert.py (1218:1227) - pytorch-transformers/pytorch_transformers/modeling_distilbert.py (739:748) duplicated block id: 74 size: 8 cleaned lines of code in 2 files: - XLM/tools/lowercase_and_remove_accent.py (33:40) - pytorch-transformers/pytorch_transformers/tokenization_bert.py (280:287) duplicated block id: 75 size: 8 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/tokenization_gpt2.py (216:224) - pytorch-transformers/pytorch_transformers/tokenization_xlm.py (782:790) duplicated block id: 76 size: 8 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/tokenization_openai.py (28:37) - pytorch-transformers/pytorch_transformers/tokenization_roberta.py (36:45) duplicated block id: 77 size: 8 cleaned lines of code in 2 files: - XLM/src/evaluation/glue.py (163:172) - XLM/src/evaluation/xnli.py (141:150) duplicated block id: 78 size: 8 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/modeling_transfo_xl.py (248:255) - pytorch-transformers/pytorch_transformers/modeling_xlnet.py (271:278) duplicated block id: 79 size: 8 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/modeling_bert.py (81:89) - pytorch-transformers/pytorch_transformers/modeling_gpt2.py (59:67) duplicated block id: 80 size: 8 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/modeling_auto.py (262:363) - pytorch-transformers/pytorch_transformers/modeling_auto.py (497:592) duplicated block id: 81 size: 8 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/modeling_bert.py (198:205) - pytorch-transformers/pytorch_transformers/modeling_distilbert.py (72:79) duplicated block id: 82 size: 8 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/modeling_gpt2.py (364:375) - pytorch-transformers/pytorch_transformers/modeling_openai.py (370:381) duplicated block id: 83 size: 8 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/modeling_auto.py (262:363) - pytorch-transformers/pytorch_transformers/modeling_auto.py (386:481) duplicated block id: 84 size: 8 cleaned lines of code in 2 files: - XLM/src/evaluation/glue.py (81:92) - XLM/src/evaluation/xnli.py (61:72) duplicated block id: 85 size: 8 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/modeling_bert.py (655:668) - pytorch-transformers/pytorch_transformers/modeling_distilbert.py (488:501) duplicated block id: 86 size: 8 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/modeling_gpt2.py (141:148) - pytorch-transformers/pytorch_transformers/modeling_openai.py (173:180) duplicated block id: 87 size: 8 cleaned lines of code in 2 files: - XLM/src/evaluation/glue.py (117:129) - XLM/src/evaluation/xnli.py (96:108) duplicated block id: 88 size: 8 cleaned lines of code in 2 files: - pytorch-transformers/pseudoalignment/pseudo_decomp_bert.py (47:54) - pytorch-transformers/pseudoalignment/pseudo_decomp_tfidf.py (37:44) duplicated block id: 89 size: 8 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/tokenization_openai.py (200:208) - pytorch-transformers/pytorch_transformers/tokenization_xlm.py (782:790) duplicated block id: 90 size: 7 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/modeling_transfo_xl.py (248:254) - pytorch-transformers/pytorch_transformers/modeling_xlm.py (147:153) duplicated block id: 91 size: 7 cleaned lines of code in 2 files: - XLM/src/evaluation/xnli.py (123:129) - XLM/src/evaluation/xnli.py (186:192) duplicated block id: 92 size: 7 cleaned lines of code in 2 files: - XLM/src/model/memory/memory.py (582:592) - XLM/src/model/memory/memory.py (653:663) duplicated block id: 93 size: 7 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/modeling_bert.py (198:204) - pytorch-transformers/pytorch_transformers/modeling_transfo_xl.py (248:254) duplicated block id: 94 size: 7 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/modeling_gpt2.py (313:321) - pytorch-transformers/pytorch_transformers/modeling_openai.py (322:330) duplicated block id: 95 size: 7 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/modeling_xlm.py (147:153) - pytorch-transformers/pytorch_transformers/modeling_xlnet.py (271:277) duplicated block id: 96 size: 7 cleaned lines of code in 2 files: - XLM/src/data/dataset.py (178:189) - XLM/src/data/dataset.py (345:356) duplicated block id: 97 size: 7 cleaned lines of code in 2 files: - XLM/tools/lowercase_and_remove_accent.py (33:39) - pytorch-transformers/pytorch_transformers/tokenization_xlm.py (435:441) duplicated block id: 98 size: 7 cleaned lines of code in 2 files: - XLM/src/evaluation/evaluator.py (518:528) - XLM/src/trainer.py (851:861) duplicated block id: 99 size: 7 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/modeling_gpt2.py (91:97) - pytorch-transformers/pytorch_transformers/modeling_xlnet.py (164:170) duplicated block id: 100 size: 7 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/__main__.py (47:54) - pytorch-transformers/pytorch_transformers/__main__.py (71:79) duplicated block id: 101 size: 7 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/modeling_bert.py (126:132) - pytorch-transformers/pytorch_transformers/modeling_xlnet.py (164:170) duplicated block id: 102 size: 7 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/modeling_openai.py (109:115) - pytorch-transformers/pytorch_transformers/modeling_transfo_xl.py (163:169) duplicated block id: 103 size: 7 cleaned lines of code in 2 files: - pytorch-transformers/pseudoalignment/pseudo_decomp_fasttext.py (40:46) - pytorch-transformers/pseudoalignment/pseudo_decomp_tfidf.py (36:42) duplicated block id: 104 size: 7 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/tokenization_bert.py (280:286) - pytorch-transformers/pytorch_transformers/tokenization_xlm.py (435:441) duplicated block id: 105 size: 7 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/modeling_distilbert.py (72:78) - pytorch-transformers/pytorch_transformers/modeling_transfo_xl.py (248:254) duplicated block id: 106 size: 7 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/tokenization_bert.py (188:204) - pytorch-transformers/pytorch_transformers/tokenization_xlm.py (751:767) duplicated block id: 107 size: 7 cleaned lines of code in 2 files: - pytorch-transformers/pseudoalignment/pseudo_decomp_bert_nsp.py (52:58) - pytorch-transformers/pseudoalignment/pseudo_decomp_tfidf.py (38:44) duplicated block id: 108 size: 7 cleaned lines of code in 2 files: - XLM/src/evaluation/evaluator.py (335:343) - XLM/src/evaluation/evaluator.py (499:507) duplicated block id: 109 size: 7 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/modeling_bert.py (198:204) - pytorch-transformers/pytorch_transformers/modeling_xlnet.py (271:277) duplicated block id: 110 size: 7 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/modeling_gpt2.py (91:97) - pytorch-transformers/pytorch_transformers/modeling_transfo_xl.py (163:169) duplicated block id: 111 size: 7 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/modeling_transfo_xl.py (1014:1020) - pytorch-transformers/pytorch_transformers/modeling_transfo_xl.py (1025:1031) duplicated block id: 112 size: 7 cleaned lines of code in 2 files: - XLM/src/evaluation/evaluator.py (410:418) - XLM/src/evaluation/evaluator.py (499:507) duplicated block id: 113 size: 7 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/modeling_xlnet.py (1045:1051) - pytorch-transformers/pytorch_transformers/modeling_xlnet.py (1116:1122) duplicated block id: 114 size: 7 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/convert_tf_checkpoint_to_pytorch.py (43:50) - pytorch-transformers/pytorch_transformers/convert_xlnet_checkpoint_to_pytorch.py (75:82) duplicated block id: 115 size: 7 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/modeling_bert.py (325:332) - pytorch-transformers/pytorch_transformers/modeling_bert.py (391:398) duplicated block id: 116 size: 7 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/tokenization_gpt2.py (88:98) - pytorch-transformers/pytorch_transformers/tokenization_openai.py (50:60) duplicated block id: 117 size: 7 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/tokenization_gpt2.py (88:98) - pytorch-transformers/pytorch_transformers/tokenization_xlm.py (415:425) duplicated block id: 118 size: 7 cleaned lines of code in 2 files: - XLM/src/trainer.py (715:721) - XLM/src/trainer.py (749:755) duplicated block id: 119 size: 7 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/modeling_bert.py (126:132) - pytorch-transformers/pytorch_transformers/modeling_transfo_xl.py (163:169) duplicated block id: 120 size: 7 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/modeling_bert.py (198:204) - pytorch-transformers/pytorch_transformers/modeling_xlm.py (147:153) duplicated block id: 121 size: 7 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/modeling_distilbert.py (72:78) - pytorch-transformers/pytorch_transformers/modeling_xlnet.py (271:277) duplicated block id: 122 size: 7 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/modeling_transfo_xl.py (557:563) - pytorch-transformers/pytorch_transformers/modeling_transfo_xl.py (645:652) duplicated block id: 123 size: 7 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/modeling_gpt2.py (72:78) - pytorch-transformers/pytorch_transformers/modeling_openai.py (86:92) duplicated block id: 124 size: 7 cleaned lines of code in 2 files: - XLM/glue-xnli.py (51:57) - XLM/train.py (125:131) duplicated block id: 125 size: 7 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/modeling_openai.py (109:115) - pytorch-transformers/pytorch_transformers/modeling_xlnet.py (164:170) duplicated block id: 126 size: 7 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/modeling_distilbert.py (72:78) - pytorch-transformers/pytorch_transformers/modeling_xlm.py (147:153) duplicated block id: 127 size: 7 cleaned lines of code in 2 files: - XLM/src/model/transformer.py (492:498) - XLM/src/model/transformer.py (595:601) duplicated block id: 128 size: 6 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/modeling_xlm.py (183:189) - pytorch-transformers/pytorch_transformers/modeling_xlnet.py (305:311) duplicated block id: 129 size: 6 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/modeling_gpt2.py (715:725) - pytorch-transformers/pytorch_transformers/modeling_openai.py (677:687) duplicated block id: 130 size: 6 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/modeling_distilbert.py (100:106) - pytorch-transformers/pytorch_transformers/modeling_xlm.py (201:207) duplicated block id: 131 size: 6 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/modeling_distilbert.py (184:189) - pytorch-transformers/pytorch_transformers/modeling_openai.py (258:263) duplicated block id: 132 size: 6 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/modeling_gpt2.py (209:215) - pytorch-transformers/pytorch_transformers/modeling_transfo_xl.py (306:312) duplicated block id: 133 size: 6 cleaned lines of code in 2 files: - XLM/src/optim.py (126:132) - XLM/src/optim.py (183:189) duplicated block id: 134 size: 6 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/modeling_openai.py (226:232) - pytorch-transformers/pytorch_transformers/modeling_xlnet.py (327:333) duplicated block id: 135 size: 6 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/__main__.py (47:53) - pytorch-transformers/pytorch_transformers/__main__.py (91:97) duplicated block id: 136 size: 6 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/__main__.py (15:21) - pytorch-transformers/pytorch_transformers/__main__.py (91:97) duplicated block id: 137 size: 6 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/__main__.py (15:21) - pytorch-transformers/pytorch_transformers/__main__.py (71:77) duplicated block id: 138 size: 6 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/modeling_gpt2.py (224:230) - pytorch-transformers/pytorch_transformers/modeling_openai.py (238:244) duplicated block id: 139 size: 6 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/modeling_transfo_xl.py (136:141) - pytorch-transformers/pytorch_transformers/modeling_xlnet.py (130:135) duplicated block id: 140 size: 6 cleaned lines of code in 2 files: - XLM/src/data/dataset.py (142:151) - XLM/src/data/dataset.py (300:309) duplicated block id: 141 size: 6 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/modeling_openai.py (258:263) - pytorch-transformers/pytorch_transformers/modeling_xlm.py (281:286) duplicated block id: 142 size: 6 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/modeling_bert.py (764:774) - pytorch-transformers/pytorch_transformers/modeling_bert.py (832:842) duplicated block id: 143 size: 6 cleaned lines of code in 2 files: - pytorch-transformers/pseudoalignment/pseudo_decomp_bert.py (89:95) - pytorch-transformers/pseudoalignment/pseudo_decomp_fasttext.py (81:87) duplicated block id: 144 size: 6 cleaned lines of code in 2 files: - XLM/src/model/transformer.py (231:236) - pytorch-transformers/pytorch_transformers/modeling_xlm.py (370:375) duplicated block id: 145 size: 6 cleaned lines of code in 2 files: - pytorch-transformers/pseudoalignment/pseudo_decomp_bert_nsp.py (133:138) - pytorch-transformers/pseudoalignment/pseudo_decomp_bert_nsp.py (154:159) duplicated block id: 146 size: 6 cleaned lines of code in 2 files: - XLM/src/evaluation/glue.py (132:140) - XLM/src/evaluation/xnli.py (111:119) duplicated block id: 147 size: 6 cleaned lines of code in 2 files: - pytorch-transformers/pseudoalignment/embed_questions_with_bert.py (81:86) - pytorch-transformers/pseudoalignment/pseudo_decomp_bert_nsp.py (51:56) duplicated block id: 148 size: 6 cleaned lines of code in 2 files: - pytorch-transformers/pseudoalignment/pseudo_decomp_bert.py (47:52) - pytorch-transformers/pseudoalignment/pseudo_decomp_fasttext.py (41:46) duplicated block id: 149 size: 6 cleaned lines of code in 2 files: - XLM/src/evaluation/evaluator.py (370:375) - XLM/src/evaluation/evaluator.py (542:547) duplicated block id: 150 size: 6 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/__main__.py (15:21) - pytorch-transformers/pytorch_transformers/__main__.py (47:53) duplicated block id: 151 size: 6 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/modeling_bert.py (346:352) - pytorch-transformers/pytorch_transformers/modeling_gpt2.py (242:248) duplicated block id: 152 size: 6 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/__main__.py (71:77) - pytorch-transformers/pytorch_transformers/__main__.py (91:97) duplicated block id: 153 size: 6 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/modeling_bert.py (850:856) - pytorch-transformers/pytorch_transformers/modeling_roberta.py (239:246) duplicated block id: 154 size: 6 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/modeling_utils.py (186:191) - pytorch-transformers/pytorch_transformers/modeling_utils.py (512:517) duplicated block id: 155 size: 6 cleaned lines of code in 2 files: - pytorch-transformers/pseudoalignment/embed_questions_with_bert.py (94:99) - pytorch-transformers/pseudoalignment/pseudo_decomp_variable.py (36:42) duplicated block id: 156 size: 6 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/modeling_gpt2.py (209:215) - pytorch-transformers/pytorch_transformers/modeling_xlnet.py (327:333) duplicated block id: 157 size: 6 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/modeling_xlm.py (793:799) - pytorch-transformers/pytorch_transformers/modeling_xlm.py (849:855) duplicated block id: 158 size: 6 cleaned lines of code in 2 files: - XLM/train.py (192:197) - XLM/translate.py (50:55) duplicated block id: 159 size: 6 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/modeling_transfo_xl.py (549:554) - pytorch-transformers/pytorch_transformers/modeling_transfo_xl.py (639:644) duplicated block id: 160 size: 6 cleaned lines of code in 2 files: - pytorch-transformers/pytorch_transformers/modeling_openai.py (226:232) - pytorch-transformers/pytorch_transformers/modeling_transfo_xl.py (306:312)