openai / mle-bench
Duplication

Places in code with 6 or more lines that are exactly the same.

Intro
  • For duplication, we look at places in code where there are 6 or more lines of code that are exactly the same.
  • Before duplication is calculated, the code is cleaned to remove empty lines, comments, and frequently duplicated constructs such as imports.
  • You should aim at having as little as possible (<5%) of duplicated code as high-level of duplication can lead to maintenance difficulties, poor factoring, and logical contradictions.
Learn more...
Duplication Overall
  • 8% duplication:
    • 13,772 cleaned lines of cleaned code (without empty lines, comments, and frequently duplicated constructs such as imports)
    • 1,128 duplicated lines
  • 97 duplicates
system8% (1,128 lines)
dependency graphs: 2D graph | 2D graph (with duplicates)... | 3D graph | 3D graph (with duplicates)...
Duplication per Extension
py6% (790 lines)
yaml14% (338 lines)
Duplication per Component (primary)
mlebench8% (1,029 lines)
extras10% (63 lines)
agents5% (30 lines)
experiments1% (6 lines)
ROOT0% (0 lines)
environment0% (0 lines)
Longest Duplicates
The list of 50 longest duplicates.
See data for all 97 duplicates...
Size#FoldersFilesLinesCode
54 x 2 mlebench/competitions/herbarium-2020-fgvc7
mlebench/competitions/herbarium-2021-fgvc8
53:118 (32%)
53:118 (32%)
view
42 x 2 mlebench/competitions/te...llenge-english-language
mlebench/competitions/te...llenge-russian-language
44:93 (50%)
42:91 (50%)
view
41 x 2 mlebench/competitions/herbarium-2020-fgvc7
mlebench/competitions/herbarium-2021-fgvc8
206:251 (24%)
206:251 (24%)
view
31 x 2 mlebench/competitions/te...llenge-english-language
mlebench/competitions/te...llenge-russian-language
7:49 (100%)
7:49 (100%)
view
25 x 2 mlebench/competitions/herbarium-2020-fgvc7
mlebench/competitions/herbarium-2021-fgvc8
152:184 (14%)
152:184 (14%)
view
19 x 2 mlebench/competitions/rs...pine-fracture-detection
mlebench/competitions/rs...ogenomic-classification
3:21 (65%)
3:21 (65%)
view
19 x 2 mlebench/competitions/herbarium-2020-fgvc7
mlebench/competitions/herbarium-2021-fgvc8
7:34 (100%)
7:34 (100%)
view
17 x 2 mlebench/competitions/te...llenge-russian-language
mlebench/competitions
18:41 (54%)
197:220 (10%)
view
17 x 2 mlebench/competitions/te...llenge-english-language
mlebench/competitions
18:41 (54%)
197:220 (10%)
view
16 x 2 mlebench/competitions/herbarium-2021-fgvc8
mlebench/competitions/herbarium-2022-fgvc9
152:170 (9%)
139:159 (10%)
view
16 x 2 mlebench/competitions/herbarium-2020-fgvc7
mlebench/competitions/herbarium-2022-fgvc9
152:170 (9%)
139:159 (10%)
view
15 x 2 mlebench/competitions/herbarium-2021-fgvc8
mlebench/competitions/herbarium-2022-fgvc9
63:79 (8%)
56:72 (10%)
view
15 x 2 mlebench/competitions/herbarium-2020-fgvc7
mlebench/competitions/herbarium-2022-fgvc9
63:80 (8%)
56:72 (10%)
view
14 x 2 mlebench/competitions/go...s-reduce-global-warming
mlebench/competitions/h-...fashion-recommendations
4:17 (56%)
4:17 (56%)
view
13 x 2 mlebench/competitions/random-acts-of-pizza
mlebench/competitions/st...rg-classifier-challenge
63:75 (25%)
84:96 (17%)
view
13 x 2 mlebench/competitions/herbarium-2021-fgvc8
mlebench/competitions/herbarium-2022-fgvc9
238:251 (7%)
220:233 (8%)
view
13 x 2 mlebench/competitions/herbarium-2020-fgvc7
mlebench/competitions/herbarium-2022-fgvc9
238:251 (7%)
220:233 (8%)
view
12 x 2 mlebench/competitions/te...llenge-english-language
mlebench/competitions/te...llenge-russian-language
23:36 (14%)
21:34 (14%)
view
12 x 2 mlebench/competitions/herbarium-2021-fgvc8
mlebench/competitions/herbarium-2022-fgvc9
106:118 (7%)
84:96 (8%)
view
12 x 2 mlebench/competitions/herbarium-2020-fgvc7
mlebench/competitions/herbarium-2022-fgvc9
106:118 (7%)
84:96 (8%)
view
12 x 2 mlebench/competitions/ji...toxicity-classification
mlebench/competitions/ve...challenge-ink-detection
14:25 (36%)
14:25 (35%)
view
11 x 2 extras/kernels
extras/kernels
58:72 (20%)
136:150 (9%)
view
10 x 2 mlebench/competitions/ji...toxicity-classification
mlebench/competitions/rsna-breast-cancer-detection
12:21 (30%)
12:21 (34%)
view
10 x 2 mlebench/competitions/herbarium-2021-fgvc8
mlebench/competitions/inaturalist-2019-fgvc6
100:111 (5%)
91:102 (5%)
view
10 x 2 mlebench/competitions/ji...toxicity-classification
mlebench/competitions/siim-covid19-detection
16:25 (30%)
16:25 (30%)
view
10 x 2 mlebench/competitions/herbarium-2020-fgvc7
mlebench/competitions/inaturalist-2019-fgvc6
100:111 (5%)
91:102 (5%)
view
10 x 2 mlebench/competitions/siim-covid19-detection
mlebench/competitions/ve...challenge-ink-detection
16:25 (30%)
16:25 (29%)
view
10 x 2 mlebench/competitions/herbarium-2021-fgvc8
mlebench/competitions/herbarium-2022-fgvc9
22:34 (52%)
19:31 (58%)
view
10 x 2 mlebench/competitions/herbarium-2020-fgvc7
mlebench/competitions/herbarium-2022-fgvc9
22:34 (52%)
19:31 (58%)
view
9 x 2 agents/opendevin
agents/opendevin
107:116 (7%)
128:137 (7%)
view
9 x 2 mlebench/competitions/te...llenge-english-language
mlebench/competitions/te...llenge-russian-language
3:11 (47%)
3:11 (47%)
view
9 x 2 mlebench/competitions/herbarium-2020-fgvc7
mlebench/competitions/herbarium-2022-fgvc9
225:235 (5%)
207:217 (6%)
view
9 x 2 mlebench/competitions/alaska2-image-steganalysis
mlebench/competitions/stanford-covid-vaccine
3:11 (47%)
3:11 (47%)
view
9 x 2 mlebench/competitions/herbarium-2020-fgvc7
mlebench/competitions/inaturalist-2019-fgvc6
206:214 (5%)
241:250 (4%)
view
9 x 2 mlebench/competitions/stanford-covid-vaccine
mlebench/competitions/te...llenge-russian-language
3:11 (47%)
3:11 (47%)
view
9 x 2 mlebench/competitions/herbarium-2021-fgvc8
mlebench/competitions/herbarium-2022-fgvc9
225:235 (5%)
207:217 (6%)
view
9 x 2 mlebench/competitions/alaska2-image-steganalysis
mlebench/competitions/te...llenge-english-language
3:11 (47%)
3:11 (47%)
view
9 x 2 mlebench/competitions/alaska2-image-steganalysis
mlebench/competitions/te...llenge-russian-language
3:11 (47%)
3:11 (47%)
view
9 x 2 mlebench/competitions/bms-molecular-translation
mlebench/competitions/st...rg-classifier-challenge
3:11 (47%)
3:11 (47%)
view
9 x 2 mlebench/competitions/petfinder-pawpularity-score
mlebench/competitions/te...low2-question-answering
3:11 (47%)
3:11 (34%)
view
9 x 2 mlebench/competitions/herbarium-2020-fgvc7
mlebench/competitions/herbarium-2021-fgvc8
193:202 (5%)
193:202 (5%)
view
9 x 2 mlebench/competitions/herbarium-2021-fgvc8
mlebench/competitions/inaturalist-2019-fgvc6
206:214 (5%)
241:250 (4%)
view
9 x 2 mlebench/competitions/stanford-covid-vaccine
mlebench/competitions/te...llenge-english-language
3:11 (47%)
3:11 (47%)
view
9 x 2 mlebench/competitions/le...tomated-essay-scoring-2
mlebench/competitions/petfinder-pawpularity-score
3:11 (36%)
3:11 (47%)
view
9 x 2 mlebench/competitions/le...tomated-essay-scoring-2
mlebench/competitions/te...low2-question-answering
3:11 (36%)
3:11 (34%)
view
9 x 2 mlebench/competitions/us...rase-to-phrase-matching
mlebench/competitions/uw...ract-image-segmentation
3:11 (47%)
3:11 (47%)
view
8 x 2 mlebench/competitions/3d...for-autonomous-vehicles
mlebench/competitions/us...rase-to-phrase-matching
4:11 (42%)
4:11 (42%)
view
8 x 2 mlebench/competitions/alaska2-image-steganalysis
mlebench/competitions/petfinder-pawpularity-score
4:11 (42%)
4:11 (42%)
view
8 x 2 mlebench/competitions/herbarium-2020-fgvc7
mlebench/competitions/herbarium-2021-fgvc8
120:130 (4%)
120:130 (4%)
view
8 x 2 mlebench/competitions/3d...for-autonomous-vehicles
mlebench/competitions/uw...ract-image-segmentation
4:11 (42%)
4:11 (42%)
view
Most Frequent Duplicates
The list of 50 most frequently found duplicates.
Size#FoldersFilesLinesCode
6 x 7 mlebench/competitions/te...low2-question-answering
mlebench/competitions/alaska2-image-steganalysis
mlebench/competitions/petfinder-pawpularity-score
mlebench/competitions/stanford-covid-vaccine
mlebench/competitions/te...llenge-english-language
mlebench/competitions/te...llenge-russian-language
mlebench/competitions/le...tomated-essay-scoring-2
5:10 (23%)
5:10 (31%)
5:10 (31%)
5:10 (31%)
5:10 (31%)
5:10 (31%)
5:10 (24%)
view
6 x 7 mlebench/competitions/te...low2-question-answering
mlebench/competitions/alaska2-image-steganalysis
mlebench/competitions/petfinder-pawpularity-score
mlebench/competitions/stanford-covid-vaccine
mlebench/competitions/te...llenge-english-language
mlebench/competitions/te...llenge-russian-language
mlebench/competitions/le...tomated-essay-scoring-2
6:11 (23%)
6:11 (31%)
6:11 (31%)
6:11 (31%)
6:11 (31%)
6:11 (31%)
6:11 (24%)
view
6 x 7 mlebench/competitions/te...low2-question-answering
mlebench/competitions/alaska2-image-steganalysis
mlebench/competitions/petfinder-pawpularity-score
mlebench/competitions/stanford-covid-vaccine
mlebench/competitions/te...llenge-english-language
mlebench/competitions/te...llenge-russian-language
mlebench/competitions/le...tomated-essay-scoring-2
4:9 (23%)
4:9 (31%)
4:9 (31%)
4:9 (31%)
4:9 (31%)
4:9 (31%)
4:9 (24%)
view
6 x 4 mlebench/competitions/inaturalist-2019-fgvc6
mlebench/competitions/herbarium-2021-fgvc8
mlebench/competitions/herbarium-2022-fgvc9
mlebench/competitions/herbarium-2020-fgvc7
126:132 (3%)
153:159 (3%)
140:146 (4%)
153:159 (3%)
view
6 x 4 mlebench/competitions/te...llenge-english-language
mlebench/competitions/alaska2-image-steganalysis
mlebench/competitions/te...llenge-russian-language
mlebench/competitions/stanford-covid-vaccine
3:8 (31%)
3:8 (31%)
3:8 (31%)
3:8 (31%)
view
6 x 4 mlebench/competitions/te...low2-question-answering
mlebench/competitions/ji...toxicity-classification
mlebench/competitions/petfinder-pawpularity-score
mlebench/competitions/le...tomated-essay-scoring-2
3:8 (23%)
3:8 (18%)
3:8 (31%)
3:8 (24%)
view
6 x 4 mlebench/competitions/inaturalist-2019-fgvc6
mlebench/competitions/herbarium-2021-fgvc8
mlebench/competitions/herbarium-2022-fgvc9
mlebench/competitions/herbarium-2020-fgvc7
125:130 (3%)
152:157 (3%)
139:144 (4%)
152:157 (3%)
view
6 x 4 mlebench/competitions/inaturalist-2019-fgvc6
mlebench/competitions/herbarium-2021-fgvc8
mlebench/competitions/herbarium-2022-fgvc9
mlebench/competitions/herbarium-2020-fgvc7
97:102 (3%)
106:111 (3%)
84:89 (4%)
106:111 (3%)
view
6 x 4 mlebench/competitions/rsna-breast-cancer-detection
mlebench/competitions/ve...challenge-ink-detection
mlebench/competitions/siim-covid19-detection
mlebench/competitions/ji...toxicity-classification
16:21 (20%)
16:21 (17%)
16:21 (18%)
16:21 (18%)
view
6 x 4 mlebench/competitions/inaturalist-2019-fgvc6
mlebench/competitions/herbarium-2021-fgvc8
mlebench/competitions/herbarium-2022-fgvc9
mlebench/competitions/herbarium-2020-fgvc7
189:194 (3%)
243:248 (3%)
225:230 (4%)
243:248 (3%)
view
6 x 3 mlebench/competitions/herbarium-2020-fgvc7
mlebench/competitions/herbarium-2021-fgvc8
mlebench/competitions/herbarium-2022-fgvc9
7:15 (31%)
7:15 (31%)
7:15 (35%)
view
6 x 3 mlebench/competitions/seti-breakthrough-listen
mlebench/competitions/rs...ogenomic-classification
mlebench/competitions/rs...pine-fracture-detection
5:10 (31%)
5:10 (20%)
5:10 (20%)
view
6 x 3 mlebench/competitions/herbarium-2022-fgvc9
mlebench/competitions/herbarium-2021-fgvc8
mlebench/competitions/herbarium-2020-fgvc7
88:93 (4%)
110:115 (3%)
110:115 (3%)
view
6 x 3 mlebench/competitions/te...llenge-russian-language
mlebench/competitions/te...llenge-english-language
mlebench/competitions
26:33 (19%)
26:33 (19%)
205:212 (3%)
view
6 x 3 mlebench/competitions/inaturalist-2019-fgvc6
mlebench/competitions/herbarium-2021-fgvc8
mlebench/competitions/herbarium-2020-fgvc7
241:247 (3%)
206:211 (3%)
206:211 (3%)
view
6 x 3 mlebench/competitions/herbarium-2022-fgvc9
mlebench/competitions/herbarium-2021-fgvc8
mlebench/competitions/herbarium-2020-fgvc7
90:96 (4%)
112:118 (3%)
112:118 (3%)
view
6 x 3 mlebench/competitions/herbarium-2022-fgvc9
mlebench/competitions/herbarium-2021-fgvc8
mlebench/competitions/herbarium-2020-fgvc7
226:231 (4%)
244:249 (3%)
244:249 (3%)
view
6 x 3 mlebench/competitions/us...rase-to-phrase-matching
mlebench/competitions/3d...for-autonomous-vehicles
mlebench/competitions/uw...ract-image-segmentation
5:10 (31%)
5:10 (31%)
5:10 (31%)
view
6 x 3 mlebench/competitions/herbarium-2022-fgvc9
mlebench/competitions/herbarium-2021-fgvc8
mlebench/competitions/herbarium-2020-fgvc7
208:215 (4%)
226:233 (3%)
226:233 (3%)
view
6 x 3 mlebench/competitions/herbarium-2022-fgvc9
mlebench/competitions/herbarium-2021-fgvc8
mlebench/competitions/herbarium-2020-fgvc7
61:68 (4%)
68:75 (3%)
68:75 (3%)
view
6 x 3 mlebench/competitions/herbarium-2022-fgvc9
mlebench/competitions/herbarium-2021-fgvc8
mlebench/competitions/herbarium-2020-fgvc7
87:92 (4%)
109:114 (3%)
109:114 (3%)
view
6 x 3 mlebench/competitions/te...llenge-russian-language
mlebench/competitions/te...llenge-english-language
mlebench/competitions
29:34 (19%)
29:34 (19%)
208:213 (3%)
view
6 x 3 mlebench/competitions/te...llenge-russian-language
mlebench/competitions/te...llenge-english-language
mlebench/competitions
20:29 (19%)
20:29 (19%)
199:208 (3%)
view
6 x 3 mlebench/competitions/inaturalist-2019-fgvc6
mlebench/competitions/herbarium-2021-fgvc8
mlebench/competitions/herbarium-2020-fgvc7
157:162 (3%)
195:200 (3%)
195:200 (3%)
view
6 x 3 mlebench/competitions/herbarium-2022-fgvc9
mlebench/competitions/herbarium-2021-fgvc8
mlebench/competitions/herbarium-2020-fgvc7
222:228 (4%)
240:246 (3%)
240:246 (3%)
view
6 x 3 mlebench/competitions/herbarium-2020-fgvc7
mlebench/competitions/herbarium-2021-fgvc8
mlebench/competitions/herbarium-2022-fgvc9
23:29 (31%)
23:29 (31%)
20:26 (35%)
view
6 x 3 mlebench/competitions/herbarium-2022-fgvc9
mlebench/competitions/herbarium-2021-fgvc8
mlebench/competitions/herbarium-2020-fgvc7
89:94 (4%)
111:116 (3%)
111:116 (3%)
view
6 x 3 mlebench/competitions/herbarium-2022-fgvc9
mlebench/competitions/herbarium-2021-fgvc8
mlebench/competitions/herbarium-2020-fgvc7
227:232 (4%)
245:250 (3%)
245:250 (3%)
view
6 x 3 mlebench/competitions/rsna-breast-cancer-detection
mlebench/competitions/ve...challenge-ink-detection
mlebench/competitions/ji...toxicity-classification
15:20 (20%)
15:20 (17%)
15:20 (18%)
view
6 x 3 mlebench/competitions/siim-covid19-detection
mlebench/competitions/ji...toxicity-classification
mlebench/competitions/ve...challenge-ink-detection
20:25 (18%)
20:25 (18%)
20:25 (17%)
view
6 x 3 mlebench/competitions/herbarium-2020-fgvc7
mlebench/competitions/herbarium-2021-fgvc8
mlebench/competitions/herbarium-2022-fgvc9
11:16 (31%)
11:16 (31%)
11:16 (35%)
view
6 x 3 mlebench/competitions/herbarium-2022-fgvc9
mlebench/competitions/herbarium-2021-fgvc8
mlebench/competitions/herbarium-2020-fgvc7
148:153 (4%)
161:166 (3%)
161:166 (3%)
view
6 x 3 mlebench/competitions/te...llenge-russian-language
mlebench/competitions/te...llenge-english-language
mlebench/competitions
19:26 (19%)
19:26 (19%)
198:205 (3%)
view
6 x 3 mlebench/competitions/herbarium-2022-fgvc9
mlebench/competitions/herbarium-2021-fgvc8
mlebench/competitions/herbarium-2020-fgvc7
56:61 (4%)
63:68 (3%)
63:68 (3%)
view
6 x 3 mlebench/competitions/lmsys-chatbot-arena
mlebench/competitions/ve...challenge-ink-detection
mlebench/competitions/ap...019-blindness-detection
3:8 (26%)
3:8 (17%)
3:8 (28%)
view
6 x 3 mlebench/competitions/herbarium-2020-fgvc7
mlebench/competitions/herbarium-2021-fgvc8
mlebench/competitions/herbarium-2022-fgvc9
24:32 (31%)
24:32 (31%)
21:29 (35%)
view
6 x 3 mlebench/competitions/herbarium-2022-fgvc9
mlebench/competitions/herbarium-2021-fgvc8
mlebench/competitions/herbarium-2020-fgvc7
209:216 (4%)
227:234 (3%)
227:234 (3%)
view
6 x 3 mlebench/competitions/herbarium-2022-fgvc9
mlebench/competitions/herbarium-2021-fgvc8
mlebench/competitions/herbarium-2020-fgvc7
150:159 (4%)
163:170 (3%)
163:170 (3%)
view
6 x 3 mlebench/competitions/us...rase-to-phrase-matching
mlebench/competitions/3d...for-autonomous-vehicles
mlebench/competitions/uw...ract-image-segmentation
6:11 (31%)
6:11 (31%)
6:11 (31%)
view
6 x 3 mlebench/competitions/te...llenge-russian-language
mlebench/competitions/te...llenge-english-language
mlebench/competitions
18:25 (19%)
18:25 (19%)
197:204 (3%)
view
6 x 3 mlebench/competitions/herbarium-2022-fgvc9
mlebench/competitions/herbarium-2021-fgvc8
mlebench/competitions/herbarium-2020-fgvc7
228:233 (4%)
246:251 (3%)
246:251 (3%)
view
6 x 3 mlebench/competitions/inaturalist-2019-fgvc6
mlebench/competitions/herbarium-2021-fgvc8
mlebench/competitions/herbarium-2020-fgvc7
91:98 (3%)
100:107 (3%)
100:107 (3%)
view
6 x 3 mlebench/competitions/inaturalist-2019-fgvc6
mlebench/competitions/herbarium-2021-fgvc8
mlebench/competitions/herbarium-2020-fgvc7
95:100 (3%)
104:109 (3%)
104:109 (3%)
view
6 x 3 mlebench/competitions/inaturalist-2019-fgvc6
mlebench/competitions/herbarium-2021-fgvc8
mlebench/competitions/herbarium-2020-fgvc7
155:160 (3%)
193:198 (3%)
193:198 (3%)
view
6 x 3 mlebench/competitions/seti-breakthrough-listen
mlebench/competitions/rs...ogenomic-classification
mlebench/competitions/rs...pine-fracture-detection
4:9 (31%)
4:9 (20%)
4:9 (20%)
view
6 x 3 mlebench/competitions/herbarium-2022-fgvc9
mlebench/competitions/herbarium-2021-fgvc8
mlebench/competitions/herbarium-2020-fgvc7
149:154 (4%)
162:167 (3%)
162:167 (3%)
view
6 x 3 mlebench/competitions/herbarium-2022-fgvc9
mlebench/competitions/herbarium-2021-fgvc8
mlebench/competitions/herbarium-2020-fgvc7
86:91 (4%)
108:113 (3%)
108:113 (3%)
view
6 x 3 mlebench/competitions/te...llenge-russian-language
mlebench/competitions/te...llenge-english-language
mlebench/competitions
30:37 (19%)
30:37 (19%)
209:216 (3%)
view
6 x 3 mlebench/competitions/inaturalist-2019-fgvc6
mlebench/competitions/herbarium-2021-fgvc8
mlebench/competitions/herbarium-2020-fgvc7
243:249 (3%)
208:213 (3%)
208:213 (3%)
view
6 x 3 mlebench/competitions/herbarium-2022-fgvc9
mlebench/competitions/herbarium-2021-fgvc8
mlebench/competitions/herbarium-2020-fgvc7
207:214 (4%)
225:232 (3%)
225:232 (3%)
view
Duplicated Units
The list of top 2 duplicated units.
See data for all 2 unit duplicates...
Size#FoldersFilesLinesCode
18 x 2 mlebench/competitions/herbarium-2020-fgvc7
mlebench/competitions/herbarium-2021-fgvc8
0:0 
0:0 
view
6 x 2 mlebench/competitions/te...llenge-russian-language
mlebench/competitions/te...llenge-english-language
0:0 
0:0 
view