aliyun / aliyun-maxcompute-data-collectors
Duplication

Places in code with 6 or more lines that are exactly the same.

Intro
Learn more...
Duplication Overall
system15% (9,789 lines)
dependency graphs: 2D graph | 3D graph | 3D graph (with duplicates)...
Duplication per Extension
java15% (7,485 lines)
scala19% (2,082 lines)
xml17% (184 lines)
cmd22% (38 lines)
Duplication per Component (primary)
odps-sqoop12% (4,244 lines)
spark-connector12% (1,352 lines)
presto-connector75% (803 lines)
trino-connector76% (796 lines)
kettle-plugin49% (575 lines)
spark-datasource-v3.160% (519 lines)
spark-datasource-v2.362% (469 lines)
flink-connector-odps9% (450 lines)
flume-plugin15% (155 lines)
hive_data_transfer_udtf47% (145 lines)
table-api-tunnel-impl11% (100 lines)
ogg-plugin5% (99 lines)
cupid-table-api4% (70 lines)
odps-data-carrier<1% (12 lines)
common0% (0 lines)

Duplication Between Components (50+ lines)

G presto-connector presto-connector trino-connector trino-connector presto-connector--trino-connector 1599 spark-datasource-v2.3 spark-datasource-v2.3 spark-datasource-v3.1 spark-datasource-v3.1 spark-datasource-v2.3--spark-datasource-v3.1 904 spark-connector spark-connector spark-connector--spark-datasource-v2.3 166 spark-connector--spark-datasource-v3.1 290 table-api-tunnel-impl table-api-tunnel-impl spark-connector--table-api-tunnel-impl 142 cupid-table-api cupid-table-api flink-connector-odps flink-connector-odps cupid-table-api--flink-connector-odps 85

Download: SVG DOT (open online Graphviz editor)

Open 2D force graph... Open 3D force graph...

Show more details on duplication between components...
Longest Duplicates
The list of 50 longest duplicates.
See data for all 754 duplicates...
Size#FoldersFilesLinesCode
145 x 2 kettle-plugin/src/main/j...i/trans/steps/odpsinput
kettle-plugin/src/main/j.../trans/steps/odpsoutput
118:287 (46%)
112:281 (39%)
view
118 x 2 spark-datasource-v2.3/sr...park/sql/odps/converter
spark-datasource-v3.1/sr...park/sql/odps/converter
35:179 (48%)
34:178 (48%)
view
97 x 2 presto-connector/src/mai...presto/maxcompute/utils
trino-connector/src/main...plugin/maxcompute/utils
161:296 (57%)
161:296 (57%)
view
74 x 2 presto-connector/src/mai...ebook/presto/maxcompute
trino-connector/src/main...trino/plugin/maxcompute
36:153 (100%)
36:153 (100%)
view
72 x 2 presto-connector/src/mai...presto/maxcompute/utils
trino-connector/src/main...plugin/maxcompute/utils
70:157 (42%)
70:157 (42%)
view
61 x 2 spark-datasource-v2.3/sr...e/spark/sql/odps/reader
spark-datasource-v3.1/sr...e/spark/sql/odps/reader
182:265 (37%)
159:242 (41%)
view
56 x 2 odps-sqoop/src/java/org/apache/sqoop/mapreduce
odps-sqoop/src/java/org/...he/sqoop/mapreduce/odps
119:214 (18%)
693:788 (11%)
view
56 x 2 odps-sqoop/src/java/com/cloudera/sqoop/orm
odps-sqoop/src/java/org/apache/sqoop/orm
31:88 (81%)
61:118 (5%)
view
49 x 2 presto-connector/src/mai...ebook/presto/maxcompute
trino-connector/src/main...trino/plugin/maxcompute
26:103 (100%)
26:103 (100%)
view
39 x 2 spark-datasource-v2.3/sr...park/sql/odps/converter
spark-datasource-v3.1/sr...park/sql/odps/converter
195:241 (15%)
194:240 (15%)
view
39 x 2 presto-connector/src/mai...ebook/presto/maxcompute
trino-connector/src/main...trino/plugin/maxcompute
29:95 (100%)
29:95 (100%)
view
37 x 2 presto-connector/src/mai...ebook/presto/maxcompute
trino-connector/src/main...trino/plugin/maxcompute
20:86 (100%)
20:86 (100%)
view
35 x 2 spark-connector/hive/src...e/spark/sql/hive/client
spark-connector/hive/src...e/spark/sql/hive/client
1226:1260 (2%)
1510:1544 (2%)
view
34 x 2 presto-connector/src/mai...ebook/presto/maxcompute
trino-connector/src/main...trino/plugin/maxcompute
109:159 (44%)
109:159 (45%)
view
30 x 2 odps-sqoop/src/java/org/apache/sqoop/mapreduce
odps-sqoop/src/java/org/...oop/mapreduce/sqlserver
123:168 (43%)
62:109 (60%)
view
30 x 2 spark-connector/common/s...g/apache/spark/sql/odps
spark-datasource-v3.1/sr...park/sql/odps/converter
96:132 (14%)
213:249 (12%)
view
30 x 2 presto-connector/src/mai...presto/maxcompute/utils
trino-connector/src/main...plugin/maxcompute/utils
27:85 (100%)
27:85 (100%)
view
28 x 2 presto-connector/src/mai...presto/maxcompute/utils
trino-connector/src/main...plugin/maxcompute/utils
40:83 (68%)
37:80 (73%)
view
28 x 2 kettle-plugin/src/main/j...i/trans/steps/odpsinput
kettle-plugin/src/main/j.../trans/steps/odpsoutput
329:375 (8%)
348:394 (7%)
view
26 x 2 presto-connector/src/mai...presto/maxcompute/utils
trino-connector/src/main...plugin/maxcompute/utils
40:107 (55%)
40:107 (55%)
view
26 x 2 presto-connector/src/mai...ebook/presto/maxcompute
trino-connector/src/main...trino/plugin/maxcompute
33:73 (48%)
26:66 (78%)
view
26 x 2 presto-connector/src/mai...ebook/presto/maxcompute
trino-connector/src/main...trino/plugin/maxcompute
155:194 (25%)
156:195 (29%)
view
24 x 2 odps-sqoop/src/java/org/apache/sqoop/mapreduce
odps-sqoop/src/java/org/...he/sqoop/mapreduce/odps
43:78 (52%)
46:81 (48%)
view
24 x 2 spark-connector/common/s...g/apache/spark/sql/odps
spark-datasource-v2.3/sr...park/sql/odps/converter
96:123 (11%)
214:241 (9%)
view
24 x 2 odps-sqoop/src/java/org/...he/sqoop/mapreduce/odps
odps-sqoop/src/java/org/...he/sqoop/mapreduce/odps
429:459 (5%)
90:120 (16%)
view
24 x 2 odps-sqoop/src/java/org/...op/mapreduce/db/netezza
odps-sqoop/src/java/org/...op/mapreduce/db/netezza
116:143 (14%)
92:120 (17%)
view
23 x 2 odps-sqoop/src/java/org/apache/sqoop/manager
odps-sqoop/src/java/org/apache/sqoop/manager
862:890 (3%)
481:509 (3%)
view
23 x 2 odps-sqoop/src/java/org/apache/sqoop/mapreduce
odps-sqoop/src/java/org/apache/sqoop/mapreduce
81:108 (31%)
49:75 (46%)
view
23 x 2 spark-datasource-v2.3/sr...e/spark/sql/odps/writer
spark-datasource-v3.1/sr...e/spark/sql/odps/writer
76:113 (41%)
81:118 (40%)
view
23 x 2 hive_data_transfer_udtf/...ain/java/odps/data/dump
hive_data_transfer_udtf/...ain/java/odps/data/dump
140:170 (21%)
185:215 (16%)
view
22 x 2 odps-sqoop/src/java/org/apache/sqoop/tool
odps-sqoop/src/java/org/apache/sqoop/tool
814:835 (1%)
199:220 (7%)
view
22 x 2 odps-sqoop/src/java/org/apache/sqoop/mapreduce
odps-sqoop/src/java/org/...sqoop/mapreduce/netezza
157:184 (20%)
110:137 (25%)
view
22 x 2 presto-connector/src/mai...ebook/presto/maxcompute
trino-connector/src/main...trino/plugin/maxcompute
63:102 (21%)
61:100 (25%)
view
22 x 2 kettle-plugin/src/main/j...i/trans/steps/odpsinput
kettle-plugin/src/main/j.../trans/steps/odpsoutput
380:420 (7%)
397:437 (5%)
view
21 x 2 odps-sqoop/src/java/org/apache/sqoop/mapreduce
odps-sqoop/src/java/org/apache/sqoop/mapreduce
149:175 (19%)
77:103 (28%)
view
21 x 2 odps-sqoop/src/java/org/apache/sqoop/mapreduce
odps-sqoop/src/java/org/apache/sqoop/mapreduce
104:137 (38%)
139:170 (30%)
view
21 x 2 odps-sqoop/src/java/org/apache/sqoop/mapreduce
odps-sqoop/src/java/org/...he/sqoop/mapreduce/odps
91:116 (19%)
359:384 (4%)
view
21 x 2 odps-sqoop/src/java/org/apache/sqoop/mapreduce
odps-sqoop/src/java/org/apache/sqoop/mapreduce
53:88 (47%)
53:88 (47%)
view
20 x 2 odps-sqoop/src/java/org/apache/sqoop/mapreduce
odps-sqoop/src/java/org/apache/sqoop/mapreduce
153:175 (18%)
49:71 (40%)
view
20 x 2 spark-connector/datasour...ion/datasources/v2/odps
spark-connector/hive/src...g/apache/spark/sql/hive
608:635 (4%)
899:926 (3%)
view
20 x 2 odps-sqoop/src/java/org/apache/sqoop/mapreduce
odps-sqoop/src/java/org/...he/sqoop/mapreduce/odps
302:332 (6%)
198:225 (4%)
view
20 x 2 odps-sqoop/src/java/org/apache/sqoop/mapreduce
odps-sqoop/src/java/org/...oop/mapreduce/sqlserver
104:135 (37%)
78:109 (40%)
view
19 x 2 spark-connector/common/s...g/apache/spark/sql/odps
spark-datasource-v3.1/sr...park/sql/odps/converter
224:243 (9%)
40:59 (7%)
view
19 x 2 spark-connector/common/s...g/apache/spark/sql/odps
spark-datasource-v2.3/sr...park/sql/odps/converter
224:243 (9%)
41:60 (7%)
view
19 x 2 presto-connector/src/mai...presto/maxcompute/utils
trino-connector/src/main...plugin/maxcompute/utils
110:135 (40%)
110:135 (40%)
view
19 x 2 presto-connector/src/mai...presto/maxcompute/utils
trino-connector/src/main...plugin/maxcompute/utils
25:57 (100%)
25:57 (100%)
view
19 x 2 odps-sqoop/src/java/org/apache/sqoop/manager
odps-sqoop/src/java/org/apache/sqoop/manager
955:979 (3%)
1021:1046 (3%)
view
19 x 2 odps-sqoop/src/scripts
odps-sqoop/src/scripts
1:19 (38%)
1:19 (21%)
view
19 x 2 spark-connector/common/s.../odps/table/tunnel/read
spark-connector/common/s.../odps/table/tunnel/read
147:169 (11%)
115:137 (16%)
view
18 x 2 odps-sqoop/src/java/org/apache/sqoop/mapreduce
odps-sqoop/src/java/org/apache/sqoop/mapreduce
93:124 (33%)
150:181 (21%)
view
Duplicated Units
The list of top 50 duplicated units.
See data for all 68 unit duplicates...
Size#FoldersFilesLinesCode
64 x 2 spark-datasource-v2.3/sr...e/spark/sql/odps/reader
spark-datasource-v3.1/sr...e/spark/sql/odps/reader
182:267 
159:244 
view
41 x 2 presto-connector/src/mai...presto/maxcompute/utils
trino-connector/src/main...plugin/maxcompute/utils
86:128 
86:128 
view
37 x 2 odps-sqoop/src/java/org/...he/sqoop/mapreduce/odps
odps-sqoop/src/java/org/apache/sqoop/mapreduce
702:749 
128:175 
view
36 x 2 spark-datasource-v2.3/sr...park/sql/odps/converter
spark-datasource-v3.1/sr...park/sql/odps/converter
63:101 
62:100 
view
35 x 2 odps-sqoop/src/java/org/apache/sqoop/mapreduce
odps-sqoop/src/java/org/apache/sqoop/mapreduce
61:109 
61:109 
view
28 x 2 odps-sqoop/src/java/org/...he/sqoop/mapreduce/odps
odps-sqoop/src/java/org/apache/sqoop/mapreduce
755:789 
181:215 
view
22 x 2 odps-sqoop/src/java/org/apache/sqoop/mapreduce
odps-sqoop/src/java/org/...he/sqoop/mapreduce/odps
50:79 
53:82 
view
19 x 3 spark-connector/common/s...g/apache/spark/sql/odps
spark-datasource-v2.3/sr...park/sql/odps/converter
spark-datasource-v3.1/sr...park/sql/odps/converter
224:244 
41:61 
40:60 
view
19 x 2 spark-connector/hive/src...g/apache/spark/sql/hive
spark-connector/datasour...ion/datasources/v2/odps
970:990 
678:698 
view
18 x 2 presto-connector/src/mai...ebook/presto/maxcompute
trino-connector/src/main...trino/plugin/maxcompute
109:130 
109:130 
view
17 x 2 presto-connector/src/mai...ebook/presto/maxcompute
trino-connector/src/main...trino/plugin/maxcompute
163:181 
164:182 
view
17 x 2 spark-connector/hive/src...park/sql/hive/execution
spark-connector/datasour...ion/datasources/v2/odps
221:241 
107:127 
view
17 x 2 odps-sqoop/src/java/org/apache/sqoop/mapreduce
odps-sqoop/src/java/org/apache/sqoop/mapreduce
41:64 
56:79 
view
16 x 2 odps-sqoop/src/java/org/...he/sqoop/mapreduce/odps
odps-sqoop/src/java/org/apache/sqoop/mapreduce
199:221 
303:325 
view
15 x 2 spark-datasource-v2.3/sr...e/spark/sql/odps/writer
spark-datasource-v3.1/sr...e/spark/sql/odps/writer
81:100 
87:106 
view
14 x 2 presto-connector/src/mai...ebook/presto/maxcompute
trino-connector/src/main...trino/plugin/maxcompute
105:120 
105:120 
view
13 x 2 presto-connector/src/mai...presto/maxcompute/utils
trino-connector/src/main...plugin/maxcompute/utils
69:83 
71:85 
view
13 x 2 presto-connector/src/mai...presto/maxcompute/utils
trino-connector/src/main...plugin/maxcompute/utils
195:209 
195:209 
view
13 x 2 presto-connector/src/mai...ebook/presto/maxcompute
trino-connector/src/main...trino/plugin/maxcompute
47:61 
47:61 
view
13 x 2 odps-sqoop/src/java/org/apache/sqoop/mapreduce
odps-sqoop/src/java/org/apache/sqoop/mapreduce
52:68 
56:72 
view
12 x 2 presto-connector/src/mai...presto/maxcompute/utils
trino-connector/src/main...plugin/maxcompute/utils
56:70 
53:67 
view
12 x 2 presto-connector/src/mai...ebook/presto/maxcompute
trino-connector/src/main...trino/plugin/maxcompute
45:58 
45:58 
view
12 x 3 odps-sqoop/src/java/org/apache/sqoop/mapreduce
odps-sqoop/src/java/org/apache/sqoop/mapreduce
odps-sqoop/src/java/org/apache/sqoop/mapreduce
55:70 
56:71 
64:79 
view
12 x 2 spark-datasource-v2.3/sr...e/spark/sql/odps/writer
spark-datasource-v3.1/sr...e/spark/sql/odps/writer
82:97 
87:102 
view
12 x 2 spark-datasource-v2.3/sr...park/sql/odps/converter
spark-datasource-v3.1/sr...park/sql/odps/converter
103:116 
102:115 
view
12 x 2 spark-datasource-v3.1/sr...e/spark/sql/odps/writer
spark-datasource-v3.1/sr...e/spark/sql/odps/writer
72:85 
72:85 
view
11 x 2 presto-connector/src/mai...presto/maxcompute/utils
trino-connector/src/main...plugin/maxcompute/utils
32:46 
32:46 
view
11 x 2 spark-datasource-v2.3/sr...e/spark/sql/odps/writer
spark-datasource-v2.3/sr...e/spark/sql/odps/writer
68:80 
67:79 
view
11 x 2 spark-datasource-v2.3/sr...park/sql/odps/converter
spark-datasource-v3.1/sr...park/sql/odps/converter
118:130 
117:129 
view
11 x 2 spark-datasource-v2.3/sr...park/sql/odps/converter
spark-datasource-v3.1/sr...park/sql/odps/converter
132:144 
131:143 
view
11 x 2 spark-datasource-v2.3/sr...park/sql/odps/converter
spark-datasource-v3.1/sr...park/sql/odps/converter
146:158 
145:157 
view
11 x 2 spark-datasource-v2.3/sr...park/sql/odps/converter
spark-datasource-v3.1/sr...park/sql/odps/converter
160:172 
159:171 
view
10 x 2 presto-connector/src/mai...ebook/presto/maxcompute
trino-connector/src/main...trino/plugin/maxcompute
69:81 
67:79 
view
10 x 2 presto-connector/src/mai...presto/maxcompute/utils
trino-connector/src/main...plugin/maxcompute/utils
38:49 
38:49 
view
10 x 2 presto-connector/src/mai...presto/maxcompute/utils
trino-connector/src/main...plugin/maxcompute/utils
276:287 
276:287 
view
10 x 2 presto-connector/src/mai...ebook/presto/maxcompute
trino-connector/src/main...trino/plugin/maxcompute
75:87 
75:87 
view
10 x 2 presto-connector/src/mai...ebook/presto/maxcompute
trino-connector/src/main...trino/plugin/maxcompute
60:71 
60:71 
view
10 x 2 flink-connector-odps/src...flink/odps/input/reader
flink-connector-odps/src...flink/odps/input/reader
98:110 
102:114 
view
10 x 2 odps-sqoop/src/java/org/apache/sqoop/manager
odps-sqoop/src/java/org/apache/sqoop/manager
59:70 
279:290 
view
9 x 2 presto-connector/src/mai...ebook/presto/maxcompute
trino-connector/src/main...trino/plugin/maxcompute
89:100 
87:98 
view
9 x 2 presto-connector/src/mai...presto/maxcompute/utils
trino-connector/src/main...plugin/maxcompute/utils
245:255 
245:255 
view
9 x 2 presto-connector/src/mai...ebook/presto/maxcompute
trino-connector/src/main...trino/plugin/maxcompute
41:51 
42:52 
view
9 x 2 presto-connector/src/mai...ebook/presto/maxcompute
trino-connector/src/main...trino/plugin/maxcompute
129:139 
129:139 
view
9 x 2 presto-connector/src/mai...ebook/presto/maxcompute
trino-connector/src/main...trino/plugin/maxcompute
145:155 
146:156 
view
9 x 2 ogg-plugin/src/main/java...dps/ogg/handler/datahub
ogg-plugin/src/main/java...dps/ogg/handler/datahub
98:109 
206:217 
view
8 x 2 presto-connector/src/mai...ebook/presto/maxcompute
trino-connector/src/main...trino/plugin/maxcompute
34:43 
34:43 
view
8 x 2 odps-sqoop/src/java/org/apache/sqoop/tool
odps-sqoop/src/java/org/apache/sqoop/tool
385:396 
343:354 
view
8 x 2 odps-sqoop/src/java/org/apache/sqoop/mapreduce
odps-sqoop/src/java/org/apache/sqoop/mapreduce
343:354 
308:319 
view
7 x 2 presto-connector/src/mai...presto/maxcompute/utils
trino-connector/src/main...plugin/maxcompute/utils
48:56 
48:56 
view
7 x 2 presto-connector/src/mai...presto/maxcompute/utils
trino-connector/src/main...plugin/maxcompute/utils
58:66 
58:66 
view