GoogleCloudPlatform / cloud-dataplex

Intro

For duplication, we look at places in code where there are 6 or more lines of code that are exactly the same.
Before duplication is calculated, the code is cleaned to remove empty lines, comments, and frequently duplicated constructs such as imports.
You should aim at having as little as possible (<5%) of duplicated code as high-level of duplication can lead to maintenance difficulties, poor factoring, and logical contradictions.

Learn more...

Duplication Overall

25% duplication:

2,908 cleaned lines of cleaned code (without empty lines, comments, and frequently duplicated constructs such as imports)
752 duplicated lines

67 duplicates

dependency graphs: 2D graph | 3D graph | 3D graph (with duplicates)...

Duplication per Extension

Duplication per Component (primary)

Duplication Between Components (50+ lines)

Download: SVG DOT (open online Graphviz editor)

From Component --> To Component	Duplicated Lines	File Pairs	Details
clouddq-migration (22%) --> datascan (5%)	180	5 file pairs	details...

Open 2D force graph... Open 3D force graph...

Show more details on duplication between components...

Longest Duplicates

The list of 50 longest duplicates.

See data for all 67 duplicates...

Size	#	Folders	Files	Lines	Code
35	x 2	datascan/airflow datascan/airflow	dq_create_runSync_evaluate_dag.py dq_runAsync_evaluate_dag.py	117:159 (29%) 63:104 (51%)	view
35	x 2	datascan/airflow datascan/airflow	dq_fail_scan_if_not_passed_dag.py dq_runAsync_evaluate_dag.py	54:95 (57%) 63:104 (51%)	view
35	x 2	datascan/airflow datascan/airflow	dq_create_runSync_evaluate_dag.py dq_fail_scan_if_not_passed_dag.py	117:159 (29%) 54:95 (57%)	view
27	x 2	datascan/airflow datascan/airflow	dataprofile_create_runSync_dag.py dp_create_runSync_evaluate_dag.py	53:83 (57%) 51:81 (60%)	view
22	x 2	clouddq-migration datascan/bulk-creation-scripts/dataquality	dataplex.py datascan.py	115:136 (15%) 71:92 (16%)	view
22	x 2	datascan/bulk-creation-scripts/dataquality datascan/bulk-creation-scripts	datascan.py datascan.py	17:52 (16%) 4:40 (36%)	view
15	x 2	datascan/airflow datascan/airflow	dp_create_runSync_evaluate_dag.py dp_delete_runSync_dag.py	26:47 (33%) 25:46 (39%)	view
15	x 2	clouddq-migration datascan/bulk-creation-scripts/dataquality	dataplex.py datascan.py	142:158 (10%) 21:39 (11%)	view
15	x 2	clouddq-migration datascan/bulk-creation-scripts	dataplex.py datascan.py	142:158 (10%) 8:26 (25%)	view
13	x 2	datascan/bulk-creation-scripts/dataquality datascan/bulk-creation-scripts	lib.py lib.py	48:74 (20%) 31:53 (23%)	view
12	x 2	datascan/bulk-creation-scripts/dataquality datascan/bulk-creation-scripts	main.py main.py	68:82 (11%) 54:68 (13%)	view
12	x 2	datascan/bulk-creation-scripts/dataquality datascan/bulk-creation-scripts	main.py main.py	55:66 (11%) 40:52 (13%)	view
11	x 2	datascan/bulk-creation-scripts/dataquality datascan/bulk-creation-scripts	main.py main.py	140:151 (10%) 113:124 (12%)	view
11	x 2	clouddq-migration clouddq-migration	main.py main.py	179:192 (6%) 225:237 (6%)	view
11	x 2	clouddq-migration clouddq-migration	main.py main.py	122:136 (6%) 222:236 (6%)	view
11	x 2	clouddq-migration datascan/bulk-creation-scripts/dataquality	lib.py lib.py	45:60 (25%) 31:46 (17%)	view
10	x 2	clouddq-migration clouddq-migration	main.py main.py	110:119 (5%) 164:173 (5%)	view
10	x 2	clouddq-migration clouddq-migration	main.py main.py	125:136 (5%) 179:190 (5%)	view
10	x 2	datascan/bulk-creation-scripts datascan/bulk-creation-scripts	main.py main.py	79:90 (10%) 110:121 (10%)	view
10	x 2	datascan/bulk-creation-scripts/dataquality datascan/bulk-creation-scripts	lib.py lib.py	84:97 (15%) 58:72 (18%)	view
10	x 2	datascan/bulk-creation-scripts/dataquality datascan/bulk-creation-scripts	main.py main.py	95:106 (9%) 82:91 (10%)	view
9	x 2	clouddq-migration clouddq-migration	main.py main.py	105:114 (5%) 211:220 (5%)	view
9	x 2	datascan/bulk-creation-scripts/dataquality datascan/bulk-creation-scripts	main.py main.py	95:104 (8%) 113:121 (9%)	view
9	x 2	clouddq-migration datascan/bulk-creation-scripts/dataquality	lib.py lib.py	30:43 (20%) 48:61 (14%)	view
9	x 2	datascan/bulk-creation-scripts/dataquality datascan/bulk-creation-scripts	main.py main.py	140:148 (8%) 82:90 (9%)	view
9	x 2	datascan/bulk-creation-scripts/dataquality datascan/bulk-creation-scripts/dataquality	main.py main.py	95:104 (8%) 140:148 (8%)	view
9	x 2	datascan/airflow datascan/airflow	dq_runAsync_evaluate_dag.py dq_runSync_evaluate_config_dag.py	72:80 (13%) 93:101 (10%)	view
9	x 2	clouddq-migration datascan/bulk-creation-scripts	lib.py lib.py	30:43 (20%) 31:39 (16%)	view
9	x 2	datascan/airflow datascan/airflow	dq_create_runSync_evaluate_dag.py dq_runSync_evaluate_config_dag.py	126:134 (7%) 93:101 (10%)	view
9	x 2	datascan/airflow datascan/airflow	dq_fail_scan_if_not_passed_dag.py dq_runSync_evaluate_config_dag.py	63:71 (14%) 93:101 (10%)	view
8	x 2	datascan/bulk-creation-scripts datascan/bulk-creation-scripts	main.py main.py	66:77 (8%) 96:107 (8%)	view
8	x 2	clouddq-migration datascan/bulk-creation-scripts/dataquality	sample_config.yaml sample_config.yaml	22:29 (29%) 19:26 (12%)	view
7	x 2	datascan/terraform/rules datascan/terraform/rules	parsed_rules_combined.yaml templates.yaml	61:67 (13%) 50:56 (17%)	view
7	x 2	datascan/bulk-creation-scripts/dataquality datascan/bulk-creation-scripts	main.py main.py	26:33 (6%) 11:18 (7%)	view
7	x 2	datascan/airflow datascan/airflow	dataprofile_create_runSync_dag.py dq_create_runSync_evaluate_dag.py	41:49 (14%) 84:92 (5%)	view
7	x 2	managed-connectivity/clo...byo-connector/terraform managed-connectivity/clo...byo-connector/terraform	byo-connector.tfvars byo-connector.tfvars	165:171 (3%) 203:209 (3%)	view
7	x 2	datascan/python-api-sample-scripts datascan/python-api-sample-scripts	dq_create_scan.py profile_create_scan.py	124:143 (7%) 60:76 (18%)	view
7	x 2	managed-connectivity/clo...byo-connector/templates managed-connectivity/clo...byo-connector/templates	byo-connector.yaml byo-connector.yaml	157:163 (3%) 195:201 (3%)	view
7	x 2	clouddq-migration datascan/bulk-creation-scripts/dataquality	sample_config.yaml sample_config.yaml	33:39 (25%) 72:78 (10%)	view
6	x 2	managed-connectivity/clo...byo-connector/templates managed-connectivity/clo...byo-connector/templates	byo-connector.yaml byo-connector.yaml	157:162 (2%) 220:225 (2%)	view
6	x 2	datascan/airflow datascan/airflow	dp_delete_runSync_dag.py dq_create_runSync_evaluate_dag.py	40:46 (15%) 86:92 (5%)	view
6	x 2	datascan/airflow datascan/airflow	dp_create_runSync_evaluate_dag.py dq_runAsync_evaluate_dag.py	41:47 (13%) 32:38 (8%)	view
6	x 2	datascan/airflow datascan/airflow	dp_create_runSync_evaluate_dag.py dq_create_runSync_evaluate_dag.py	51:56 (13%) 96:101 (5%)	view
6	x 2	datascan/airflow datascan/airflow	dp_delete_runSync_dag.py dq_runAsync_evaluate_dag.py	40:46 (15%) 32:38 (8%)	view
6	x 2	clouddq-migration datascan/bulk-creation-scripts	dataplex.py datascan.py	131:136 (4%) 79:84 (10%)	view
6	x 2	datascan/airflow datascan/airflow	dq_create_runSync_evaluate_dag.py dq_runAsync_evaluate_dag.py	86:92 (5%) 32:38 (8%)	view
6	x 2	datascan/airflow datascan/airflow	dp_create_runSync_evaluate_dag.py dq_create_runSync_evaluate_dag.py	59:64 (13%) 104:109 (5%)	view
6	x 2	clouddq-migration clouddq-migration	main.py main.py	96:105 (3%) 147:156 (3%)	view
6	x 2	datascan/airflow datascan/airflow	dataprofile_create_runSync_dag.py dq_create_runSync_evaluate_dag.py	69:74 (12%) 112:117 (5%)	view
6	x 2	datascan/airflow datascan/airflow	dp_create_runSync_evaluate_dag.py dq_create_runSync_evaluate_dag.py	67:72 (13%) 112:117 (5%)	view

Duplicated Units

The list of top 1 duplicated units.

See data for all 1 unit duplicate

Size	#	Folders	Files	Lines	Code
7	x 3	datascan/airflow datascan/airflow datascan/airflow	dq_fail_scan_if_not_passed_dag.py dq_create_runSync_evaluate_dag.py dq_runAsync_evaluate_dag.py	0:0 0:0 0:0	view