aws / modern-data-architecture-accelerator

Intro

For duplication, we look at places in code where there are 6 or more lines of code that are exactly the same.
Before duplication is calculated, the code is cleaned to remove empty lines, comments, and frequently duplicated constructs such as imports.
You should aim at having as little as possible (<5%) of duplicated code as high-level of duplication can lead to maintenance difficulties, poor factoring, and logical contradictions.

Learn more...

Duplication Overall

22% duplication:

42,226 cleaned lines of cleaned code (without empty lines, comments, and frequently duplicated constructs such as imports)
9,306 duplicated lines

2,419 duplicates

dependency graphs: 2D graph | 3D graph | 3D graph (with duplicates)...

Duplication per Extension

Duplication per Component (primary)

Duplication Between Components (50+ lines)

Download: SVG DOT (open online Graphviz editor)

From Component --> To Component	Duplicated Lines	File Pairs	Details
sample_blueprints (9%) --> sample_configs (3%)	167	10 file pairs	details...

Open 2D force graph... Open 3D force graph...

Show more details on duplication between components...

Longest Duplicates

The list of 50 longest duplicates.

See data for all 2,419 duplicates...

Size	#	Folders	Files	Lines	Code
268	x 2	packages/constructs/L3/a...n-sdk/python/genai_core packages/constructs/L3/a...ayers/python-sdk/python	workspaces.py workspaces.py	12:334 (85%) 12:334 (85%)	view
81	x 2	sample_configs/basic_datascience_platform sample_configs/lakeformation_datalake	roles.yaml roles.yaml	1:100 (81%) 1:100 (43%)	view
62	x 2	packages/constructs/L3/g...cy-statements/app-based packages/constructs/L3/g...tatements/service-based	utility-basic-policy.yaml dms-basic-policy.yaml	7:71 (56%) 1:65 (95%)	view
62	x 2	packages/constructs/L3/g...cy-statements/app-based packages/constructs/L3/g...tatements/service-based	datascience-basic-policy.yaml service-catalog-basic-policy.yaml	68:135 (50%) 2:69 (92%)	view
60	x 2	packages/constructs/L3/g...cy-statements/app-based packages/constructs/L3/g...tatements/service-based	datascience-basic-policy.yaml sagemaker-basic-policy.yaml	7:66 (48%) 1:60 (95%)	view
60	x 2	packages/constructs/L3/g...cy-statements/app-based packages/constructs/L3/g...statements/domain-based	datalake-basic-policy.yaml datalake-basic-policy.yaml	8:74 (95%) 8:74 (92%)	view
55	x 2	sample_configs/basic_datalake sample_configs/lakeformation_datalake	roles.yaml roles.yaml	2:64 (62%) 103:164 (29%)	view
47	x 2	packages/constructs/L3/a...engines/aurora-pgvector packages/constructs/L3/a...ngines/kendra-retrieval	create-aurora-workspace.ts create-kendra-workspace.ts	148:196 (23%) 28:76 (51%)	view
43	x 2	packages/constructs/L3/g...cy-statements/app-based packages/constructs/L3/g...tatements/service-based	dataops-basic-policy.yaml glue-basic-policy.yaml	8:50 (63%) 1:43 (93%)	view
37	x 2	packages/constructs/L3/a...ccount-l3-construct/lib packages/constructs/L3/a...roject-l3-construct/lib	quicksight-account-l3-construct.ts quicksight-project-l3-construct.ts	355:401 (8%) 429:475 (10%)	view
37	x 2	packages/constructs/L2/custom-constructs/lib packages/constructs/L3/a...roject-l3-construct/lib	custom.ts quicksight-project-l3-construct.ts	193:239 (17%) 429:475 (10%)	view
37	x 2	packages/constructs/L2/custom-constructs/lib packages/constructs/L3/a...ccount-l3-construct/lib	custom.ts quicksight-account-l3-construct.ts	193:239 (17%) 355:401 (8%)	view
37	x 2	sample_configs/basic_datalake sample_configs/lakeformation_datalake	mdaa.yaml mdaa.yaml	17:85 (90%) 17:82 (68%)	view
36	x 2	packages/constructs/L3/a...nstruct/lib/chatbot-api packages/constructs/L3/u...2m-api-l3-construct/lib	rest-api.ts m2m-api-l3-construct.ts	165:203 (6%) 521:560 (7%)	view
35	x 2	packages/constructs/L3/d...talake-l3-construct/lib packages/constructs/L3/g...atalog-l3-construct/lib	datalake-bucket-l3-construct.ts glue-catalog-l3-construct.ts	668:711 (6%) 428:471 (10%)	view
35	x 2	packages/constructs/L3/ai/sm-shared/lib packages/constructs/L3/d...ps-job-l3-construct/lib	sm-shared.ts dataops-job-l3-construct.ts	98:141 (34%) 217:260 (19%)	view
35	x 2	packages/constructs/L3/d...talake-l3-construct/lib packages/utilities/iam-role-helper/lib	datalake-bucket-l3-construct.ts rolehelper.ts	668:711 (6%) 256:299 (17%)	view
35	x 2	packages/constructs/L3/g...atalog-l3-construct/lib packages/utilities/iam-role-helper/lib	glue-catalog-l3-construct.ts rolehelper.ts	428:471 (10%) 256:299 (17%)	view
34	x 2	packages/constructs/L3/a...ccount-l3-construct/lib packages/constructs/L3/a...roject-l3-construct/lib	quicksight-account-l3-construct.ts quicksight-project-l3-construct.ts	277:319 (8%) 351:393 (9%)	view
34	x 2	packages/constructs/L3/d...talake-l3-construct/lib packages/constructs/L3/g...atalog-l3-construct/lib	datalake-bucket-l3-construct.ts glue-catalog-l3-construct.ts	588:630 (6%) 356:398 (9%)	view
34	x 2	packages/constructs/L3/d...-l3-construct/lib/cdk8s packages/constructs/L3/d...-l3-construct/lib/cdk8s	nifi-cluster-chart.ts nifi-cluster-chart.ts	461:499 (4%) 565:603 (4%)	view
34	x 2	packages/constructs/L3/ai/gaia-l3-construct/lib packages/constructs/L3/ai/sm-shared/lib	gaia-l3-construct.ts sm-shared.ts	162:204 (22%) 96:138 (33%)	view
33	x 2	packages/constructs/L2/eks-constructs/lib packages/constructs/L2/eks-constructs/lib	cluster.ts mdaa-kubectl-provider.ts	544:583 (5%) 142:181 (30%)	view
33	x 2	packages/constructs/L3/a...ccount-l3-construct/lib packages/constructs/L3/a...ccount-l3-construct/lib	quicksight-account-l3-construct.ts quicksight-account-l3-construct.ts	279:319 (7%) 361:401 (7%)	view
33	x 2	packages/constructs/L2/custom-constructs/lib packages/constructs/L3/a...roject-l3-construct/lib	custom.ts quicksight-project-l3-construct.ts	199:239 (15%) 353:393 (9%)	view
33	x 2	packages/constructs/L3/d...talake-l3-construct/lib packages/constructs/L3/d...talake-l3-construct/lib	datalake-bucket-l3-construct.ts datalake-bucket-l3-construct.ts	590:630 (6%) 671:711 (6%)	view
33	x 2	packages/constructs/L3/a...espace-l3-construct/lib packages/constructs/L3/a...espace-l3-construct/lib	quicksight-namespace-l3-construct.ts quicksight-namespace-l3-construct.ts	358:398 (5%) 437:477 (5%)	view
33	x 2	packages/constructs/L3/g...atalog-l3-construct/lib packages/utilities/iam-role-helper/lib	glue-catalog-l3-construct.ts rolehelper.ts	358:398 (9%) 259:299 (16%)	view
33	x 2	packages/constructs/L3/d...talake-l3-construct/lib packages/constructs/L3/g...atalog-l3-construct/lib	datalake-bucket-l3-construct.ts glue-catalog-l3-construct.ts	671:711 (6%) 358:398 (9%)	view
33	x 2	packages/constructs/L2/custom-constructs/lib packages/constructs/L3/a...ccount-l3-construct/lib	custom.ts quicksight-account-l3-construct.ts	199:239 (15%) 279:319 (7%)	view
33	x 2	packages/constructs/L3/a...roject-l3-construct/lib packages/constructs/L3/a...roject-l3-construct/lib	quicksight-project-l3-construct.ts quicksight-project-l3-construct.ts	353:393 (9%) 435:475 (9%)	view
33	x 2	packages/constructs/L3/g...atalog-l3-construct/lib packages/constructs/L3/g...atalog-l3-construct/lib	glue-catalog-l3-construct.ts glue-catalog-l3-construct.ts	358:398 (9%) 431:471 (9%)	view
33	x 2	packages/constructs/L3/a...ccount-l3-construct/lib packages/constructs/L3/a...roject-l3-construct/lib	quicksight-account-l3-construct.ts quicksight-project-l3-construct.ts	361:401 (7%) 353:393 (9%)	view
33	x 2	packages/constructs/L3/d...talake-l3-construct/lib packages/constructs/L3/g...atalog-l3-construct/lib	datalake-bucket-l3-construct.ts glue-catalog-l3-construct.ts	590:630 (6%) 431:471 (9%)	view
33	x 2	packages/constructs/L3/a...ccount-l3-construct/lib packages/constructs/L3/a...roject-l3-construct/lib	quicksight-account-l3-construct.ts quicksight-project-l3-construct.ts	279:319 (7%) 435:475 (9%)	view
33	x 2	packages/constructs/L3/d...talake-l3-construct/lib packages/utilities/iam-role-helper/lib	datalake-bucket-l3-construct.ts rolehelper.ts	590:630 (6%) 259:299 (16%)	view
32	x 2	sample_configs/lakeformation_datalake sample_configs/lakehouse	roles.yaml roles.yaml	139:180 (17%) 74:116 (19%)	view
32	x 2	packages/constructs/L3/ai/gaia-l3-construct/lib packages/constructs/L3/d...ps-job-l3-construct/lib	gaia-l3-construct.ts dataops-job-l3-construct.ts	164:204 (21%) 217:257 (17%)	view
31	x 2	packages/constructs/L3/g...statements/domain-based packages/constructs/L3/g...tatements/service-based	datalake-basic-policy.yaml lakeformation-basic-policy.yaml	40:74 (47%) 4:37 (93%)	view
31	x 2	packages/constructs/L3/g...cy-statements/app-based packages/constructs/L3/g...tatements/service-based	datalake-basic-policy.yaml lakeformation-basic-policy.yaml	40:74 (49%) 4:37 (93%)	view
31	x 2	packages/constructs/L3/a...ython/genai_core/aurora packages/constructs/L3/a...n/genai_core/opensearch	delete.py delete.py	33:71 (62%) 33:72 (62%)	view
31	x 2	packages/constructs/L3/a...ython/genai_core/kendra packages/constructs/L3/a...n/genai_core/opensearch	delete.py delete.py	20:59 (73%) 33:72 (62%)	view
31	x 2	packages/constructs/L3/a...ython/genai_core/aurora packages/constructs/L3/a...ython/genai_core/kendra	delete.py delete.py	33:71 (62%) 20:59 (73%)	view
30	x 2	packages/constructs/L2/opensearch-constructs/lib packages/utilities/iam-role-helper/lib	domain.ts rolehelper.ts	190:227 (17%) 180:217 (14%)	view
30	x 2	packages/constructs/L3/a...rag-engines/data-import packages/constructs/L3/a...rag-engines/data-import	file-import-workflow.ts website-crawling-workflow.ts	30:60 (24%) 164:194 (12%)	view
30	x 2	packages/constructs/L3/a...n-sdk/python/genai_core packages/constructs/L3/a...ayers/python-sdk/python	workspaces.py workspaces.py	352:390 (9%) 351:389 (9%)	view
30	x 2	packages/constructs/L3/g...cy-statements/app-based packages/constructs/L3/g...tatements/service-based	utility-basic-policy.yaml cloudwatch-basic-policy.yaml	74:107 (27%) 2:35 (88%)	view
28	x 2	packages/constructs/L3/a...ython/genai_core/aurora packages/constructs/L3/a...n/genai_core/opensearch	query.py query.py	148:181 (14%) 64:97 (17%)	view
28	x 2	packages/constructs/L3/g...cy-statements/app-based packages/constructs/L3/g...statements/domain-based	dataops-basic-policy.yaml dataops-basic-policy.yaml	45:73 (41%) 40:68 (29%)	view
27	x 2	packages/constructs/L3/a...engines/aurora-pgvector packages/constructs/L3/a...engines/aurora-pgvector	create-aurora-workspace.ts index.ts	83:115 (13%) 131:163 (19%)	view

Duplicated Units

The list of top 4 duplicated units.

See data for all 4 unit duplicates...

Size	#	Folders	Files	Lines	Code
24	x 2	packages/constructs/L3/a...ayers/python-sdk/python packages/constructs/L3/a...n-sdk/python/genai_core	workspaces.py workspaces.py	0:0 0:0	view
14	x 2	packages/constructs/L3/a...adapters/sagemaker/meta packages/constructs/L3/a...adapters/sagemaker/meta	llama2_chat.py llama2_base.py	0:0 0:0	view
8	x 3	packages/constructs/L3/d...-l3-construct/lib/cdk8s packages/constructs/L3/d...-l3-construct/lib/cdk8s packages/constructs/L3/d...-l3-construct/lib/cdk8s	nifi-cluster-chart.ts nifi-registry-chart.ts zookeeper-chart.ts	91:102 79:90 53:64	view
8	x 2	packages/constructs/L3/a...ndler/adapters/registry packages/constructs/L3/a...hon/genai_core/registry	index.py index.py	0:0 0:0	view