GoogleCloudPlatform / analytics-componentized-patterns

Intro

For duplication, we look at places in code where there are 6 or more lines of code that are exactly the same.
Before duplication is calculated, the code is cleaned to remove empty lines, comments, and frequently duplicated constructs such as imports.
You should aim at having as little as possible (<5%) of duplicated code as high-level of duplication can lead to maintenance difficulties, poor factoring, and logical contradictions.

Learn more...

Duplication Overall

38% duplication:

12,230 cleaned lines of cleaned code (without empty lines, comments, and frequently duplicated constructs such as imports)
4,726 duplicated lines

11,258 duplicates

dependency graphs: 2D graph | 3D graph | 3D graph (with duplicates)...

Duplication per Extension

Duplication per Component (primary)

Duplication Between Components (50+ lines)

Download: SVG DOT (open online Graphviz editor)

From Component --> To Component	Duplicated Lines	File Pairs	Details
gaming (35%) --> retail (5%)	1153	4 file pairs	details...

Open 2D force graph... Open 3D force graph...

Show more details on duplication between components...

Longest Duplicates

The list of 50 longest duplicates.

See data for all 11,258 duplicates...

Size	#	Folders	Files	Lines	Code
190	x 2	retail/recommendation-system/bqml-scann retail/recommendation-system/bqml-scann	ann01_create_index.ipynb ann02_run_pipeline.ipynb	319:508 (7%) 306:495 (12%)	view
55	x 2	retail/recommendation-system/bqml-scann retail/recommendation-system/bqml-scann	ann01_create_index.ipynb ann02_run_pipeline.ipynb	2464:2518 (2%) 1410:1464 (3%)	view
26	x 2	retail/recommendation-system/bqml-scann retail/recommendation-system/bqml-scann	ann02_run_pipeline.ipynb ann02_run_pipeline.ipynb	517:542 (1%) 596:621 (1%)	view
23	x 2	retail/recommendation-system/bqml-scann retail/recommendation-system/bqml-scann	ann02_run_pipeline.ipynb ann02_run_pipeline.ipynb	1192:1214 (1%) 1352:1374 (1%)	view
22	x 2	retail/propensity-model/bqml retail/propensity-model/bqml	bqml_kfp_retail_propensity_to_purchas... bqml_kfp_retail_propensity_to_purchas...	1341:1362 (<1%) 2272:2293 (<1%)	view
21	x 2	gaming/propensity-model/bqml retail/recommendation-system/bqml	bqml_ga4_gaming_propensity_to_churn.i... bqml_retail_recommendation_system.ipynb	8:28 (1%) 10:30 (1%)	view
18	x 2	retail/propensity-model/bqml retail/propensity-model/bqml	bqml_kfp_retail_propensity_to_purchas... bqml_kfp_retail_propensity_to_purchas...	1345:1362 (<1%) 2117:2134 (<1%)	view
18	x 2	retail/propensity-model/bqml retail/propensity-model/bqml	bqml_kfp_retail_propensity_to_purchas... bqml_kfp_retail_propensity_to_purchas...	684:701 (<1%) 1749:1766 (<1%)	view
18	x 2	retail/propensity-model/bqml retail/propensity-model/bqml	bqml_kfp_retail_propensity_to_purchas... bqml_kfp_retail_propensity_to_purchas...	2117:2134 (<1%) 2276:2293 (<1%)	view
16	x 2	gaming/propensity-model/bqml retail/recommendation-system/bqml-scann	bqml_ga4_gaming_propensity_to_churn.i... ann02_run_pipeline.ipynb	1619:1634 (<1%) 1443:1458 (1%)	view
16	x 2	retail/recommendation-system/bqml-mlops/part_3 retail/recommendation-system/bqml-scann	vertex_ai_pipeline.ipynb ann01_create_index.ipynb	745:760 (2%) 2497:2512 (<1%)	view
16	x 2	gaming/propensity-model/bqml retail/recommendation-system/bqml-mlops/part_3	bqml_ga4_gaming_propensity_to_churn.i... vertex_ai_pipeline.ipynb	1619:1634 (<1%) 745:760 (2%)	view
16	x 2	retail/recommendation-system/bqml-mlops/part_3 retail/recommendation-system/bqml-scann	vertex_ai_pipeline.ipynb ann02_run_pipeline.ipynb	745:760 (2%) 1443:1458 (1%)	view
16	x 2	retail/recommendation-sy...scann/embeddings_lookup retail/recommendation-sy...bqml-scann/tfx_pipeline	lookup_creator.py lookup_creator.py	57:73 (29%) 68:84 (25%)	view
16	x 2	gaming/propensity-model/bqml retail/recommendation-system/bqml-scann	bqml_ga4_gaming_propensity_to_churn.i... ann01_create_index.ipynb	1619:1634 (<1%) 2497:2512 (<1%)	view
15	x 2	retail/propensity-model/bqml retail/propensity-model/bqml	bqml_kfp_retail_propensity_to_purchas... bqml_kfp_retail_propensity_to_purchas...	1196:1210 (<1%) 1928:1942 (<1%)	view
13	x 2	retail/recommendation-system/bqml-scann retail/recommendation-system/bqml-scann	ann01_create_index.ipynb ann01_create_index.ipynb	1565:1577 (<1%) 1691:1703 (<1%)	view
13	x 2	retail/recommendation-system/bqml-scann retail/recommendation-system/bqml-scann	ann01_create_index.ipynb ann01_create_index.ipynb	1397:1409 (<1%) 1481:1493 (<1%)	view
13	x 2	retail/recommendation-system/bqml-scann retail/recommendation-system/bqml-scann	ann01_create_index.ipynb ann01_create_index.ipynb	1397:1409 (<1%) 1467:1479 (<1%)	view
13	x 2	retail/recommendation-system/bqml-scann retail/recommendation-system/bqml-scann	ann01_create_index.ipynb ann01_create_index.ipynb	1397:1409 (<1%) 1495:1507 (<1%)	view
13	x 2	retail/recommendation-system/bqml-scann retail/recommendation-system/bqml-scann	ann01_create_index.ipynb ann01_create_index.ipynb	1565:1577 (<1%) 1677:1689 (<1%)	view
13	x 2	retail/recommendation-system/bqml-scann retail/recommendation-system/bqml-scann	ann01_create_index.ipynb ann01_create_index.ipynb	879:891 (<1%) 1313:1325 (<1%)	view
13	x 2	retail/recommendation-system/bqml-scann retail/recommendation-system/bqml-scann	ann01_create_index.ipynb ann01_create_index.ipynb	1733:1745 (<1%) 1873:1885 (<1%)	view
13	x 2	retail/recommendation-system/bqml-scann retail/recommendation-system/bqml-scann	ann01_create_index.ipynb ann01_create_index.ipynb	879:891 (<1%) 1327:1339 (<1%)	view
13	x 2	retail/recommendation-system/bqml-scann retail/recommendation-system/bqml-scann	ann01_create_index.ipynb ann01_create_index.ipynb	1733:1745 (<1%) 1887:1899 (<1%)	view
13	x 2	retail/recommendation-system/bqml-scann retail/recommendation-system/bqml-scann	ann01_create_index.ipynb ann01_create_index.ipynb	1705:1717 (<1%) 1761:1773 (<1%)	view
13	x 2	retail/recommendation-system/bqml-scann retail/recommendation-system/bqml-scann	ann01_create_index.ipynb ann01_create_index.ipynb	879:891 (<1%) 1355:1367 (<1%)	view
13	x 2	retail/recommendation-system/bqml-scann retail/recommendation-system/bqml-scann	ann01_create_index.ipynb ann01_create_index.ipynb	1733:1745 (<1%) 1831:1843 (<1%)	view
13	x 2	retail/recommendation-system/bqml-scann retail/recommendation-system/bqml-scann	ann01_create_index.ipynb ann01_create_index.ipynb	879:891 (<1%) 1369:1381 (<1%)	view
13	x 2	retail/recommendation-system/bqml-scann retail/recommendation-system/bqml-scann	ann01_create_index.ipynb ann01_create_index.ipynb	1733:1745 (<1%) 1845:1857 (<1%)	view
13	x 2	retail/recommendation-system/bqml-scann retail/recommendation-system/bqml-scann	ann01_create_index.ipynb ann01_create_index.ipynb	1705:1717 (<1%) 1747:1759 (<1%)	view
13	x 2	retail/recommendation-system/bqml-scann retail/recommendation-system/bqml-scann	ann01_create_index.ipynb ann01_create_index.ipynb	1397:1409 (<1%) 1411:1423 (<1%)	view
13	x 2	retail/recommendation-system/bqml-scann retail/recommendation-system/bqml-scann	ann01_create_index.ipynb ann01_create_index.ipynb	1733:1745 (<1%) 1859:1871 (<1%)	view
13	x 2	retail/recommendation-system/bqml-scann retail/recommendation-system/bqml-scann	ann01_create_index.ipynb ann01_create_index.ipynb	1705:1717 (<1%) 1733:1745 (<1%)	view
13	x 2	retail/recommendation-system/bqml-scann retail/recommendation-system/bqml-scann	ann01_create_index.ipynb ann01_create_index.ipynb	879:891 (<1%) 1341:1353 (<1%)	view
13	x 2	retail/recommendation-system/bqml-scann retail/recommendation-system/bqml-scann	ann01_create_index.ipynb ann01_create_index.ipynb	879:891 (<1%) 1397:1409 (<1%)	view
13	x 2	retail/recommendation-system/bqml-scann retail/recommendation-system/bqml-scann	ann01_create_index.ipynb ann01_create_index.ipynb	1705:1717 (<1%) 1719:1731 (<1%)	view
13	x 2	retail/recommendation-system/bqml-scann retail/recommendation-system/bqml-scann	ann01_create_index.ipynb ann01_create_index.ipynb	1397:1409 (<1%) 1439:1451 (<1%)	view
13	x 2	retail/recommendation-system/bqml-scann retail/recommendation-system/bqml-scann	ann01_create_index.ipynb ann01_create_index.ipynb	1397:1409 (<1%) 1425:1437 (<1%)	view
13	x 2	retail/recommendation-system/bqml-scann retail/recommendation-system/bqml-scann	ann02_run_pipeline.ipynb ann02_run_pipeline.ipynb	596:608 (<1%) 752:764 (<1%)	view
13	x 2	retail/recommendation-system/bqml-scann retail/recommendation-system/bqml-scann	ann01_create_index.ipynb ann01_create_index.ipynb	1733:1745 (<1%) 1803:1815 (<1%)	view
13	x 2	retail/recommendation-system/bqml-scann retail/recommendation-system/bqml-scann	ann01_create_index.ipynb ann01_create_index.ipynb	1397:1409 (<1%) 1453:1465 (<1%)	view
13	x 2	retail/recommendation-system/bqml-scann retail/recommendation-system/bqml-scann	ann01_create_index.ipynb ann01_create_index.ipynb	1733:1745 (<1%) 1817:1829 (<1%)	view
13	x 2	retail/recommendation-system/bqml-scann retail/recommendation-system/bqml-scann	ann01_create_index.ipynb ann01_create_index.ipynb	879:891 (<1%) 1383:1395 (<1%)	view
13	x 2	retail/recommendation-system/bqml-scann retail/recommendation-system/bqml-scann	ann01_create_index.ipynb ann01_create_index.ipynb	1593:1605 (<1%) 1747:1759 (<1%)	view
13	x 2	retail/recommendation-system/bqml-scann retail/recommendation-system/bqml-scann	ann01_create_index.ipynb ann01_create_index.ipynb	1761:1773 (<1%) 1971:1983 (<1%)	view
13	x 2	retail/recommendation-system/bqml-scann retail/recommendation-system/bqml-scann	ann01_create_index.ipynb ann01_create_index.ipynb	1593:1605 (<1%) 1733:1745 (<1%)	view
13	x 2	retail/recommendation-system/bqml-scann retail/recommendation-system/bqml-scann	ann01_create_index.ipynb ann01_create_index.ipynb	1537:1549 (<1%) 1593:1605 (<1%)	view
13	x 2	retail/recommendation-system/bqml-scann retail/recommendation-system/bqml-scann	ann01_create_index.ipynb ann01_create_index.ipynb	1593:1605 (<1%) 1719:1731 (<1%)	view
13	x 2	retail/recommendation-system/bqml-scann retail/recommendation-system/bqml-scann	ann01_create_index.ipynb ann01_create_index.ipynb	1593:1605 (<1%) 1705:1717 (<1%)	view