GoogleCloudPlatform / DataflowTemplates
Duplication

Places in code with 6 or more lines that are exactly the same.

Intro
Learn more...
Duplication Overall
system31% (40,763 lines)
dependency graphs: 2D graph | 3D graph | 3D graph (with duplicates)...
Duplication per Extension
java20% (20,998 lines)
tf88% (19,653 lines)
avsc38% (68 lines)
py6% (22 lines)
proto14% (22 lines)
Duplication per Component (primary)
v233% (27,329 lines)
v133% (11,490 lines)
plugins24% (946 lines)
it8% (596 lines)
metadata38% (360 lines)
yaml16% (31 lines)
python3% (11 lines)
structured-logging0% (0 lines)
ROOT0% (0 lines)
cicd0% (0 lines)
test-infra0% (0 lines)
plaintext-logging0% (0 lines)

Duplication Between Components (50+ lines)

G v1 v1 v2 v2 v1--v2 17014 yaml yaml v2--yaml 78 plugins plugins plugins--v1 4327 plugins--v2 9621 it it it--v1 66

Download: SVG DOT (open online Graphviz editor)

Open 2D force graph... Open 3D force graph...

Show more details on duplication between components...
Longest Duplicates
The list of 50 longest duplicates.
See data for all 35,167 duplicates...
Size#FoldersFilesLinesCode
250 x 2 v2/spanner-change-stream...ms_to_Sharded_File_Sink
v2/spanner-to-sourcedb/t...ms_to_Sharded_File_Sink
 21:326 (100%)
21:326 (100%)
 view
244 x 2 v2/googlecloud-to-elasti...gQuery_to_Elasticsearch
v2/googlecloud-to-elasti...rm/GCS_to_Elasticsearch
 63:360 (72%)
93:390 (66%)
 view
218 x 2 v2/oracle-to-googlecloud...orm/Jdbc_to_PubSub_Auto
v2/sqlserver-to-googlecl...orm/Jdbc_to_PubSub_Auto
 21:284 (100%)
21:284 (100%)
 view
218 x 2 v2/postgresql-to-googlec...orm/Jdbc_to_PubSub_Auto
v2/sqlserver-to-googlecl...orm/Jdbc_to_PubSub_Auto
 21:284 (100%)
21:284 (100%)
 view
218 x 2 v2/jdbc-and-googlecloud/...orm/Jdbc_to_PubSub_Auto
v2/postgresql-to-googlec...orm/Jdbc_to_PubSub_Auto
 21:284 (100%)
21:284 (100%)
 view
218 x 2 v2/oracle-to-googlecloud...orm/Jdbc_to_PubSub_Auto
v2/postgresql-to-googlec...orm/Jdbc_to_PubSub_Auto
 21:284 (100%)
21:284 (100%)
 view
218 x 2 v2/jdbc-and-googlecloud/...orm/Jdbc_to_PubSub_Auto
v2/mysql-to-googlecloud/...orm/Jdbc_to_PubSub_Auto
 21:284 (100%)
21:284 (100%)
 view
218 x 2 v2/jdbc-and-googlecloud/...orm/Jdbc_to_PubSub_Auto
v2/oracle-to-googlecloud...orm/Jdbc_to_PubSub_Auto
 21:284 (100%)
21:284 (100%)
 view
218 x 2 v2/mysql-to-googlecloud/...orm/Jdbc_to_PubSub_Auto
v2/sqlserver-to-googlecl...orm/Jdbc_to_PubSub_Auto
 21:284 (100%)
21:284 (100%)
 view
218 x 2 v2/mysql-to-googlecloud/...orm/Jdbc_to_PubSub_Auto
v2/oracle-to-googlecloud...orm/Jdbc_to_PubSub_Auto
 21:284 (100%)
21:284 (100%)
 view
218 x 2 v2/mysql-to-googlecloud/...orm/Jdbc_to_PubSub_Auto
v2/postgresql-to-googlec...orm/Jdbc_to_PubSub_Auto
 21:284 (100%)
21:284 (100%)
 view
218 x 2 v2/jdbc-and-googlecloud/...orm/Jdbc_to_PubSub_Auto
v2/sqlserver-to-googlecl...orm/Jdbc_to_PubSub_Auto
 21:284 (100%)
21:284 (100%)
 view
209 x 2 v2/postgresql-to-googlec.../PostgreSQL_to_BigQuery
v2/sqlserver-to-googlecl...m/SQLServer_to_BigQuery
 40:294 (76%)
40:294 (76%)
 view
209 x 2 v2/jdbc-and-googlecloud/...m/Jdbc_to_BigQuery_Flex
v2/postgresql-to-googlec.../PostgreSQL_to_BigQuery
 52:306 (74%)
40:294 (76%)
 view
209 x 2 v2/oracle-to-googlecloud...form/Oracle_to_BigQuery
v2/postgresql-to-googlec.../PostgreSQL_to_BigQuery
 40:294 (76%)
40:294 (76%)
 view
209 x 2 v2/jdbc-and-googlecloud/...m/Jdbc_to_BigQuery_Flex
v2/sqlserver-to-googlecl...m/SQLServer_to_BigQuery
 52:306 (74%)
40:294 (76%)
 view
209 x 2 v2/jdbc-and-googlecloud/...m/Jdbc_to_BigQuery_Flex
v2/mysql-to-googlecloud/...aform/MySQL_to_BigQuery
 52:306 (74%)
40:294 (76%)
 view
209 x 2 v2/mysql-to-googlecloud/...aform/MySQL_to_BigQuery
v2/oracle-to-googlecloud...form/Oracle_to_BigQuery
 40:294 (76%)
40:294 (76%)
 view
209 x 2 v2/jdbc-and-googlecloud/...m/Jdbc_to_BigQuery_Flex
v2/oracle-to-googlecloud...form/Oracle_to_BigQuery
 52:306 (74%)
40:294 (76%)
 view
209 x 2 v2/mysql-to-googlecloud/...aform/MySQL_to_BigQuery
v2/sqlserver-to-googlecl...m/SQLServer_to_BigQuery
 40:294 (76%)
40:294 (76%)
 view
209 x 2 v2/mysql-to-googlecloud/...aform/MySQL_to_BigQuery
v2/postgresql-to-googlec.../PostgreSQL_to_BigQuery
 40:294 (76%)
40:294 (76%)
 view
209 x 2 v2/oracle-to-googlecloud...form/Oracle_to_BigQuery
v2/sqlserver-to-googlecl...m/SQLServer_to_BigQuery
 40:294 (76%)
40:294 (76%)
 view
200 x 2 v2/mysql-to-googlecloud/terraform/Jdbc_to_PubSub
v2/sqlserver-to-googlecl...erraform/Jdbc_to_PubSub
 21:263 (100%)
21:263 (100%)
 view
200 x 2 v2/jdbc-and-googlecloud/terraform/Jdbc_to_PubSub
v2/mysql-to-googlecloud/terraform/Jdbc_to_PubSub
 21:263 (100%)
21:263 (100%)
 view
200 x 2 v2/oracle-to-googlecloud/terraform/Jdbc_to_PubSub
v2/sqlserver-to-googlecl...erraform/Jdbc_to_PubSub
 21:263 (100%)
21:263 (100%)
 view
200 x 2 v2/oracle-to-googlecloud/terraform/Jdbc_to_PubSub
v2/postgresql-to-googlec...erraform/Jdbc_to_PubSub
 21:263 (100%)
21:263 (100%)
 view
200 x 2 v2/jdbc-and-googlecloud/terraform/Jdbc_to_PubSub
v2/oracle-to-googlecloud/terraform/Jdbc_to_PubSub
 21:263 (100%)
21:263 (100%)
 view
200 x 2 v2/mysql-to-googlecloud/terraform/Jdbc_to_PubSub
v2/oracle-to-googlecloud/terraform/Jdbc_to_PubSub
 21:263 (100%)
21:263 (100%)
 view
200 x 2 v2/jdbc-and-googlecloud/terraform/Jdbc_to_PubSub
v2/sqlserver-to-googlecl...erraform/Jdbc_to_PubSub
 21:263 (100%)
21:263 (100%)
 view
200 x 2 v2/mysql-to-googlecloud/terraform/Jdbc_to_PubSub
v2/postgresql-to-googlec...erraform/Jdbc_to_PubSub
 21:263 (100%)
21:263 (100%)
 view
200 x 2 v2/postgresql-to-googlec...erraform/Jdbc_to_PubSub
v2/sqlserver-to-googlecl...erraform/Jdbc_to_PubSub
 21:263 (100%)
21:263 (100%)
 view
200 x 2 v2/jdbc-and-googlecloud/terraform/Jdbc_to_PubSub
v2/postgresql-to-googlec...erraform/Jdbc_to_PubSub
 21:263 (100%)
21:263 (100%)
 view
160 x 2 v2/googlecloud-to-google...nge_Streams_to_BigQuery
v2/googlecloud-to-google...to_Google_Cloud_Storage
 93:290 (59%)
101:298 (58%)
 view
160 x 2 v2/googlecloud-to-google...to_Google_Cloud_Storage
v2/googlecloud-to-google...hange_Streams_to_PubSub
 101:298 (58%)
105:302 (57%)
 view
160 x 2 v2/googlecloud-to-google...nge_Streams_to_BigQuery
v2/googlecloud-to-google...hange_Streams_to_PubSub
 93:290 (59%)
105:302 (57%)
 view
142 x 2 v2/googlecloud-to-google...oud_PubSub_to_Avro_Flex
v2/googlecloud-to-google...PubSub_to_GCS_Text_Flex
 70:242 (64%)
70:242 (63%)
 view
127 x 2 v2/cdc-parent/cdc-change...dc_To_BigQuery_Template
v2/googlecloud-to-google...PubSub_to_BigQuery_Auto
 69:222 (63%)
89:242 (57%)
 view
127 x 2 v2/pubsub-binary-to-bigq...ubSub_Proto_to_BigQuery
v2/pubsub-cdc-to-bigquer.../PubSub_CDC_to_BigQuery
 111:264 (53%)
137:290 (47%)
 view
127 x 2 v2/googlecloud-to-google...PubSub_to_BigQuery_Auto
v2/pubsub-cdc-to-bigquer.../PubSub_CDC_to_BigQuery
 89:242 (57%)
137:290 (47%)
 view
127 x 2 v2/cdc-parent/cdc-change...dc_To_BigQuery_Template
v2/pubsub-binary-to-bigq...ubSub_Proto_to_BigQuery
 69:222 (63%)
111:264 (53%)
 view
127 x 2 v2/googlecloud-to-google...PubSub_to_BigQuery_Auto
v2/googlecloud-to-google...nge_Streams_to_BigQuery
 89:242 (57%)
147:300 (46%)
 view
127 x 2 v2/cdc-parent/cdc-change...dc_To_BigQuery_Template
v2/pubsub-cdc-to-bigquer.../PubSub_CDC_to_BigQuery
 69:222 (63%)
137:290 (47%)
 view
127 x 2 v2/googlecloud-to-google...PubSub_to_BigQuery_Auto
v2/pubsub-binary-to-bigq...ubSub_Proto_to_BigQuery
 89:242 (57%)
111:264 (53%)
 view
127 x 2 v2/cdc-parent/cdc-change...dc_To_BigQuery_Template
v2/googlecloud-to-google...nge_Streams_to_BigQuery
 69:222 (63%)
147:300 (46%)
 view
127 x 2 v2/googlecloud-to-google...nge_Streams_to_BigQuery
v2/pubsub-cdc-to-bigquer.../PubSub_CDC_to_BigQuery
 147:300 (46%)
137:290 (47%)
 view
127 x 2 v2/googlecloud-to-google...nge_Streams_to_BigQuery
v2/pubsub-binary-to-bigq...ubSub_Proto_to_BigQuery
 147:300 (46%)
111:264 (53%)
 view
126 x 2 v2/googlecloud-to-google...nge_Streams_to_BigQuery
v2/googlecloud-to-google...S_Text_to_BigQuery_Flex
 148:300 (45%)
70:222 (63%)
 view
126 x 2 v2/googlecloud-to-google...PubSub_to_BigQuery_Auto
v2/pubsub-binary-to-bigq...PubSub_Avro_to_BigQuery
 90:242 (57%)
70:222 (63%)
 view
126 x 2 v2/googlecloud-to-google...nge_Streams_to_BigQuery
v2/pubsub-binary-to-bigq...PubSub_Avro_to_BigQuery
 148:300 (45%)
70:222 (63%)
 view
126 x 2 v2/cdc-parent/cdc-change...dc_To_BigQuery_Template
v2/googlecloud-to-google...S_Text_to_BigQuery_Flex
 70:222 (63%)
70:222 (63%)
 view
Duplicated Units
The list of top 50 duplicated units.
See data for all 122 unit duplicates...
Size#FoldersFilesLinesCode
53 x 2 v2/jdbc-common/src/main/...cloud/teleport/v2/utils
v1/src/main/java/com/goo...leport/templates/common
 66:137 
205:276 
 view
50 x 2 v2/gcs-to-sourcedb/src/m...emplates/processing/dml
v2/spanner-to-sourcedb/s...2/templates/dbutils/dml
 384:436 
384:436 
 view
47 x 2 v2/googlecloud-to-google...cloud/teleport/v2/utils
v1/src/main/java/com/goo...leport/templates/common
 479:532 
462:515 
 view
40 x 2 v2/spanner-change-stream...v2/templates/transforms
v2/spanner-to-sourcedb/s...v2/templates/transforms
 422:463 
410:451 
 view
39 x 2 v2/spanner-change-stream...v2/templates/transforms
v2/spanner-to-sourcedb/s...v2/templates/transforms
 380:420 
368:408 
 view
38 x 2 v1/src/main/java/com/goo...loud/teleport/templates
v1/src/main/java/com/goo...loud/teleport/templates
 326:367 
321:362 
 view
37 x 2 v2/gcs-to-sourcedb/src/m...leport/v2/templates/dao
v2/spanner-change-stream...port/v2/templates/utils
 270:311 
412:452 
 view
29 x 2 v2/spanner-change-stream...v2/templates/transforms
v2/spanner-to-sourcedb/s...v2/templates/transforms
 128:158 
133:163 
 view
26 x 2 v2/spanner-common/src/ma...teleport/v2/spanner/ddl
v1/src/main/java/com/goo...ud/teleport/spanner/ddl
 202:230 
320:348 
 view
25 x 2 v2/gcs-to-sourcedb/src/m...emplates/processing/dml
v2/spanner-to-sourcedb/s...2/templates/dbutils/dml
 355:382 
355:382 
 view
25 x 2 v2/spanner-change-stream...v2/templates/transforms
v2/spanner-to-sourcedb/s...v2/templates/transforms
 259:287 
250:278 
 view
25 x 2 v2/googlecloud-to-google...cloud/teleport/v2/utils
v1/src/main/java/com/goo...leport/templates/common
 906:934 
889:917 
 view
24 x 2 v2/gcs-to-sourcedb/src/m...leport/v2/templates/dao
v2/spanner-change-stream...port/v2/templates/utils
 359:389 
454:484 
 view
23 x 2 v2/gcs-to-sourcedb/src/m...ort/v2/templates/common
v2/spanner-change-stream...ort/v2/templates/common
 150:174 
150:174 
 view
23 x 2 v2/spanner-change-stream...port/v2/templates/utils
v2/spanner-to-sourcedb/s...port/v2/templates/utils
 40:64 
40:64 
 view
23 x 2 v2/googlecloud-to-google...cloud/teleport/v2/utils
v1/src/main/java/com/goo...leport/templates/common
 716:742 
699:725 
 view
22 x 2 v2/googlecloud-to-google...cloud/teleport/v2/utils
v1/src/main/java/com/goo...leport/templates/common
 606:629 
589:612 
 view
21 x 2 v2/common/src/main/java/.../teleport/v2/transforms
v1/src/main/java/com/goo...leport/templates/common
 326:350 
305:329 
 view
20 x 4 v2/spanner-change-stream...port/v2/templates/utils
v2/spanner-change-stream...v2/templates/transforms
v2/spanner-to-sourcedb/s...port/v2/templates/utils
v2/spanner-to-sourcedb/s...v2/templates/transforms
 66:87 
465:486 
66:87 
453:474 
 view
20 x 2 v2/spanner-common/src/ma...teleport/v2/spanner/ddl
v1/src/main/java/com/goo...ud/teleport/spanner/ddl
 272:293 
617:638 
 view
20 x 2 v2/spanner-to-sourcedb/s...d/teleport/v2/templates
v2/datastream-to-spanner...d/teleport/v2/templates
 804:825 
931:952 
 view
20 x 2 v2/spanner-to-sourcedb/s...v2/templates/transforms
v2/spanner-to-sourcedb/s...v2/templates/transforms
 106:128 
58:80 
 view
20 x 2 v2/mongodb-to-googleclou...rt/v2/mongodb/templates
v2/mongodb-to-googleclou.../teleport/v2/transforms
 225:247 
170:192 
 view
18 x 2 v2/common/src/main/java/.../teleport/v2/transforms
v1/src/main/java/com/goo...leport/templates/common
 294:314 
282:302 
 view
18 x 2 v2/googlecloud-to-google...cloud/teleport/v2/utils
v1/src/main/java/com/goo...leport/templates/common
 750:769 
733:752 
 view
18 x 2 v2/googlecloud-to-google...d/teleport/v2/templates
v1/src/main/java/com/goo...loud/teleport/templates
 622:642 
619:639 
 view
17 x 2 v2/datastream-to-sql/src.../teleport/v2/transforms
v2/datastream-to-postgre.../teleport/v2/transforms
 88:112 
90:114 
 view
17 x 2 v2/spanner-change-stream.../templates/changestream
v2/spanner-to-sourcedb/s.../templates/changestream
 155:174 
139:158 
 view
17 x 2 v2/common/src/main/java/.../teleport/v2/transforms
v1/src/main/java/com/goo...leport/templates/common
 212:238 
166:192 
 view
17 x 2 v2/common/src/main/java/.../teleport/v2/transforms
v1/src/main/java/com/goo...leport/templates/common
 237:258 
225:246 
 view
17 x 2 v2/common/src/main/java/.../teleport/v2/transforms
v1/src/main/java/com/goo...leport/templates/common
 267:286 
255:274 
 view
17 x 2 v2/googlecloud-to-google...changestreamstobigquery
v2/googlecloud-to-google...lechangestreamstopubsub
 114:135 
145:166 
 view
17 x 3 v2/googlecloud-to-google...d/teleport/v2/templates
v1/src/main/java/com/goo...loud/teleport/templates
v1/src/main/java/com/goo...loud/teleport/templates
 400:421 
314:335 
246:267 
 view
17 x 2 v2/googlecloud-to-neo4j/...2/neo4j/actions/preload
v2/googlecloud-to-neo4j/.../neo4j/actions/function
 84:102 
80:98 
 view
17 x 2 it/jdbc/src/main/java/or...beam/it/jdbc/conditions
it/google-cloud-platform.../gcp/spanner/conditions
 49:69 
48:68 
 view
17 x 2 v1/src/main/java/com/goo...loud/teleport/templates
v1/src/main/java/com/google/cloud/teleport/util
 347:365 
38:56 
 view
16 x 2 v2/gcs-to-sourcedb/src/m...ort/v2/templates/common
v2/spanner-change-stream...ort/v2/templates/common
 44:61 
44:61 
 view
16 x 2 v2/gcs-to-sourcedb/src/m...ort/v2/templates/common
v2/spanner-change-stream...ort/v2/templates/common
 116:133 
116:133 
 view
15 x 2 v2/spanner-change-stream...port/v2/templates/utils
v2/spanner-to-sourcedb/s...port/v2/templates/utils
 34:51 
46:63 
 view
15 x 2 v2/common/src/main/java/...cloud/teleport/v2/utils
v1/src/main/java/com/google/cloud/teleport/util
 52:72 
52:72 
 view
14 x 2 v2/common/src/main/java/...cloud/teleport/v2/utils
v1/src/main/java/com/goo...loud/teleport/templates
 152:169 
416:433 
 view
14 x 2 v2/spanner-common/src/ma...teleport/v2/spanner/ddl
v1/src/main/java/com/goo...ud/teleport/spanner/ddl
 249:266 
467:484 
 view
14 x 2 v2/googlecloud-to-google...cloud/teleport/v2/utils
v1/src/main/java/com/goo...leport/templates/common
 433:450 
416:433 
 view
14 x 2 v2/mongodb-to-googleclou...rt/v2/mongodb/templates
v2/mongodb-to-googleclou.../teleport/v2/transforms
 250:268 
116:134 
 view
14 x 2 v1/src/main/java/com/google/cloud/teleport/spanner
v1/src/main/java/com/goo...leport/templates/common
 1000:1030 
846:876 
 view
14 x 2 v1/src/main/java/com/goo...loud/teleport/templates
v1/src/main/java/com/goo...loud/teleport/templates
 379:398 
374:393 
 view
13 x 2 v2/gcs-to-sourcedb/src/m...leport/v2/templates/dao
v2/spanner-change-stream...port/v2/templates/utils
 70:84 
65:80 
 view
13 x 2 v2/spanner-change-stream...port/v2/templates/utils
v2/spanner-common/src/ma...nner/migrations/spanner
 37:51 
42:56 
 view
13 x 2 v2/common/src/main/java/...loud/teleport/v2/common
v1/src/main/java/com/goo...leport/templates/common
 105:120 
106:121 
 view
12 x 2 v2/common/src/main/java/...cloud/teleport/v2/utils
v1/src/main/java/com/google/cloud/teleport/util
 124:140 
72:88 
 view