apache / polaris - Duplication

Intro

For duplication, we look at places in code where there are 6 or more lines of code that are exactly the same.
Before duplication is calculated, the code is cleaned to remove empty lines, comments, and frequently duplicated constructs such as imports.
You should aim at having as little as possible (<5%) of duplicated code as high-level of duplication can lead to maintenance difficulties, poor factoring, and logical contradictions.

Learn more...

Duplication Overall

41% duplication:

62,363 cleaned lines of cleaned code (without empty lines, comments, and frequently duplicated constructs such as imports)
25,765 duplicated lines

57,777 duplicates

dependency graphs: 2D graph | 3D graph | 3D graph (with duplicates)...

Duplication per Extension

Duplication per Component (primary)

Duplication Between Components (50+ lines)

Download: SVG DOT (open online Graphviz editor)

From Component --> To Component	Duplicated Lines	File Pairs	Details
getting-started (47%) --> plugins (19%)	787	1 file pair	details...
extension (11%) --> polaris-core (2%)	741	16 file pairs	details...
helm (14%) --> site (20%)	412	160 file pairs	details...
server-templates (26%) --> site (14%)	294	80 file pairs	details...
helm (7%) --> server-templates (18%)	210	50 file pairs	details...
quarkus (3%) --> service (<1%)	92	2 file pairs	details...
gradle (22%) --> plugins (1%)	56	2 file pairs	details...
codestyle (100%) --> gradle (21%)	54	2 file pairs	details...
client (<1%) --> gradle (21%)	54	2 file pairs	details...

Open 2D force graph... Open 3D force graph...

Show more details on duplication between components...

Longest Duplicates

The list of 50 longest duplicates.

See data for all 57,777 duplicates...

Size	#	Folders	Files	Lines	Code
255	x 2	client/python/polaris/catalog client/python/polaris/management	api_client.py api_client.py	52:469 (54%) 52:469 (54%)	view
215	x 2	client/python/polaris/catalog client/python/polaris/management	api_client.py api_client.py	472:806 (45%) 472:806 (45%)	view
171	x 2	client/python/polaris/catalog client/python/polaris/management	rest.py rest.py	42:275 (100%) 42:275 (100%)	view
109	x 2	client/python/polaris/catalog client/python/polaris/management	configuration.py configuration.py	143:401 (46%) 143:401 (53%)	view
104	x 2	client/python/polaris/catalog client/python/polaris/management	exceptions.py exceptions.py	35:217 (100%) 35:217 (100%)	view
83	x 2	spec spec	iceberg-rest-catalog-open-api.yaml iceberg-rest-catalog-open-api.yaml	1022:1104 (1%) 1354:1436 (1%)	view
82	x 2	spec spec/polaris-catalog-apis	iceberg-rest-catalog-open-api.yaml oauth-tokens-api.yaml	3518:3616 (1%) 108:206 (38%)	view
72	x 2	extension/persistence/jp...ache/polaris/jpa/models extension/persistence/re.../relational/jdbc/models	ModelEntity.java ModelEntity.java	153:262 (44%) 185:294 (38%)	view
69	x 2	getting-started/spark/notebooks plugins/spark/v3.5/getting-started/notebooks	SparkPolaris.ipynb SparkPolaris.ipynb	179:247 (8%) 190:258 (8%)	view
57	x 2	spec spec/polaris-catalog-apis	iceberg-rest-catalog-open-api.yaml oauth-tokens-api.yaml	3768:3827 (1%) 207:266 (26%)	view
51	x 2	getting-started/spark/notebooks plugins/spark/v3.5/getting-started/notebooks	SparkPolaris.ipynb SparkPolaris.ipynb	83:133 (6%) 94:144 (5%)	view
48	x 2	extension/persistence/ec...stence/impl/eclipselink polaris-core/src/main/ja...rsistence/transactional	PolarisEclipseLinkMetaStoreSessionImp... TreeMapTransactionalPersistenceImpl.java	409:466 (10%) 291:348 (12%)	view
48	x 2	extension/persistence/re...istence/relational/jdbc polaris-core/src/main/ja...olaris/core/persistence	JdbcMetaStoreManagerFactory.java LocalPolarisMetaStoreManagerFactory.java	189:259 (27%) 165:235 (32%)	view
47	x 2	client/python/polaris/catalog/models client/python/polaris/catalog/models	data_file.py equality_delete_file.py	66:137 (78%) 59:130 (85%)	view
47	x 2	client/python/polaris/catalog/models client/python/polaris/catalog/models	data_file.py position_delete_file.py	66:137 (78%) 58:129 (87%)	view
47	x 2	client/python/polaris/catalog/models client/python/polaris/catalog/models	equality_delete_file.py position_delete_file.py	59:130 (85%) 58:129 (87%)	view
45	x 2	client/python/polaris/catalog/api client/python/polaris/catalog/api	iceberg_catalog_api.py iceberg_catalog_api.py	5111:5172 (<1%) 5426:5487 (<1%)	view
44	x 2	client/python/polaris/catalog/models client/python/polaris/catalog/models	commit_table_request.py commit_view_request.py	53:123 (84%) 53:123 (84%)	view
39	x 2	client/python/polaris/catalog/api client/python/polaris/catalog/api	iceberg_catalog_api.py iceberg_catalog_api.py	2896:2954 (<1%) 3226:3284 (<1%)	view
39	x 2	client/python/polaris/catalog/models client/python/polaris/catalog/models	literal_expression.py unary_expression.py	48:111 (97%) 48:111 (97%)	view
38	x 2	quarkus/service/src/main.../service/quarkus/config service/common/src/main/.../polaris/service/config	QuarkusBehaviorChangesConfiguration.java FeaturesConfiguration.java	40:87 (90%) 40:87 (88%)	view
38	x 2	extension/persistence/jp...ache/polaris/jpa/models extension/persistence/re.../relational/jdbc/models	ModelPolicyMappingRecord.java ModelPolicyMappingRecord.java	68:134 (42%) 46:112 (44%)	view
38	x 2	client/python/polaris/management/models client/python/polaris/management/models	external_catalog.py polaris_catalog.py	53:115 (84%) 52:114 (88%)	view
38	x 2	extension/persistence/jp...ache/polaris/jpa/models extension/persistence/re.../relational/jdbc/models	ModelGrantRecord.java ModelGrantRecord.java	86:143 (58%) 91:148 (51%)	view
37	x 2	getting-started/spark/notebooks plugins/spark/v3.5/getting-started/notebooks	SparkPolaris.ipynb SparkPolaris.ipynb	141:177 (4%) 152:188 (4%)	view
36	x 2	getting-started/spark/notebooks plugins/spark/v3.5/getting-started/notebooks	SparkPolaris.ipynb SparkPolaris.ipynb	15:50 (4%) 15:50 (4%)	view
36	x 2	spec spec/polaris-catalog-apis	iceberg-rest-catalog-open-api.yaml oauth-tokens-api.yaml	175:227 (<1%) 28:80 (16%)	view
36	x 2	client/python/polaris/management/models client/python/polaris/management/models	catalog_role.py principal_role.py	52:111 (85%) 52:111 (85%)	view
35	x 2	client/python/polaris/management/models client/python/polaris/management/models	update_principal_request.py update_principal_role_request.py	47:105 (97%) 47:105 (97%)	view
35	x 2	quarkus/admin quarkus/server	build.gradle.kts build.gradle.kts	92:148 (38%) 87:141 (39%)	view
35	x 2	client/python/polaris/management/models client/python/polaris/management/models	create_catalog_role_request.py grant_catalog_role_request.py	48:107 (97%) 48:107 (97%)	view
35	x 2	client/python/polaris/catalog/api client/python/polaris/catalog/api	iceberg_catalog_api.py iceberg_catalog_api.py	2241:2291 (<1%) 4183:4233 (<1%)	view
35	x 2	client/python/polaris/management/models client/python/polaris/management/models	update_catalog_role_request.py update_principal_request.py	47:105 (97%) 47:105 (97%)	view
35	x 2	client/python/polaris/catalog/api client/python/polaris/catalog/api	iceberg_catalog_api.py iceberg_catalog_api.py	2241:2291 (<1%) 7759:7809 (<1%)	view
35	x 2	client/python/polaris/management/models client/python/polaris/management/models	create_principal_role_request.py grant_principal_role_request.py	48:107 (97%) 48:107 (97%)	view
35	x 2	client/python/polaris/catalog/api client/python/polaris/catalog/api	iceberg_catalog_api.py iceberg_catalog_api.py	4183:4233 (<1%) 7759:7809 (<1%)	view
35	x 2	client/python/polaris/management/models client/python/polaris/management/models	catalog_roles.py principal_roles.py	49:108 (87%) 49:108 (87%)	view
35	x 2	client/python/polaris/catalog client/python/polaris/management	configuration.py configuration.py	466:519 (14%) 433:486 (17%)	view
35	x 2	client/python/polaris/management/models client/python/polaris/management/models	add_grant_request.py revoke_grant_request.py	48:107 (97%) 48:107 (97%)	view
35	x 2	client/python/polaris/management/models client/python/polaris/management/models	update_catalog_role_request.py update_principal_role_request.py	47:105 (97%) 47:105 (97%)	view
33	x 2	client/python/polaris/catalog/models client/python/polaris/catalog/models	create_namespace_request.py create_namespace_response.py	49:105 (91%) 49:105 (91%)	view
33	x 2	spec spec	iceberg-rest-catalog-open-api.yaml polaris-catalog-service.yaml	35:69 (<1%) 33:74 (31%)	view
33	x 2	client/python/polaris/catalog/models client/python/polaris/catalog/models	set_snapshot_ref_update.py snapshot_reference.py	64:121 (60%) 54:111 (70%)	view
32	x 2	client/python/polaris/catalog/models client/python/polaris/catalog/models	add_schema_update.py remove_statistics_update.py	59:114 (78%) 57:112 (80%)	view
32	x 2	client/python/polaris/catalog/models client/python/polaris/catalog/models	assert_current_schema_id.py assert_table_uuid.py	57:112 (80%) 57:112 (80%)	view
32	x 2	client/python/polaris/catalog/models client/python/polaris/catalog/models	add_snapshot_update.py remove_snapshots_update.py	58:113 (80%) 57:112 (80%)	view
32	x 2	client/python/polaris/catalog/models client/python/polaris/catalog/models	add_sort_order_update.py set_partition_statistics_update.py	58:113 (80%) 58:113 (80%)	view
32	x 2	client/python/polaris/catalog/models client/python/polaris/catalog/models	add_partition_spec_update.py add_schema_update.py	58:113 (80%) 59:114 (78%)	view
32	x 2	client/python/polaris/catalog/models client/python/polaris/catalog/models	add_schema_update.py remove_properties_update.py	59:114 (78%) 57:112 (80%)	view
32	x 2	client/python/polaris/catalog/models client/python/polaris/catalog/models	add_sort_order_update.py set_properties_update.py	58:113 (80%) 57:112 (80%)	view

Duplicated Units

The list of top 28 duplicated units.

See data for all 28 unit duplicates...

Size	#	Folders	Files	Lines	Code
40	x 2	client/python/polaris/management client/python/polaris/catalog	rest.py rest.py	0:0 0:0	view
21	x 2	polaris-core/src/main/ja...olaris/core/persistence extension/persistence/re...istence/relational/jdbc	LocalPolarisMetaStoreManagerFactory.java JdbcMetaStoreManagerFactory.java	250:274 285:309	view
31	x 2	client/python/polaris/catalog/models client/python/polaris/catalog/models	commit_table_request.py commit_view_request.py	0:0 0:0	view
19	x 2	quarkus/service/src/main.../service/quarkus/config service/common/src/main/.../polaris/service/config	QuarkusBehaviorChangesConfiguration.java FeaturesConfiguration.java	68:88 68:88	view
17	x 2	extension/persistence/re.../relational/jdbc/models extension/persistence/jp...ache/polaris/jpa/models	ModelEntity.java ModelEntity.java	272:290 240:258	view
13	x 2	quarkus/service/src/main.../service/quarkus/config service/common/src/main/.../polaris/service/config	QuarkusBehaviorChangesConfiguration.java FeaturesConfiguration.java	52:66 52:66	view
21	x 2	client/python/polaris/management/models client/python/polaris/management/models	principal_roles.py catalog_roles.py	0:0 0:0	view
13	x 2	service/common/src/main/.../service/catalog/policy service/common/src/main/...service/catalog/generic	PolicyCatalogAdapter.java GenericTableCatalogAdapter.java	60:76 55:71	view
20	x 2	client/python/polaris/management/models client/python/polaris/catalog/models	catalog_properties.py snapshot_summary.py	0:0 0:0	view
10	x 2	extension/persistence/re.../relational/jdbc/models extension/persistence/jp...ache/polaris/jpa/models	ModelPolicyMappingRecord.java ModelPolicyMappingRecord.java	116:128 147:159	view
17	x 2	client/python/polaris/management/models client/python/polaris/management/models	grant_catalog_role_request.py create_catalog_role_request.py	0:0 0:0	view
17	x 2	client/python/polaris/management/models client/python/polaris/management/models	add_grant_request.py revoke_grant_request.py	0:0 0:0	view
17	x 2	client/python/polaris/management/models client/python/polaris/management/models	create_principal_role_request.py grant_principal_role_request.py	0:0 0:0	view
17	x 3	client/python/polaris/catalog/models client/python/polaris/catalog/models client/python/polaris/catalog/models	literal_expression.py set_expression.py unary_expression.py	0:0 0:0 0:0	view
17	x 2	client/python/polaris/catalog/models client/python/polaris/catalog/models	load_view_result.py commit_table_response.py	0:0 0:0	view
9	x 2	service/common/src/main/.../service/catalog/policy service/common/src/main/...service/catalog/generic	PolicyCatalog.java GenericTableCatalog.java	77:87 53:63	view
8	x 2	service/common/src/main/.../service/catalog/policy service/common/src/main/...service/catalog/generic	PolicyCatalogHandler.java GenericTableCatalogHandler.java	62:71 44:53	view
8	x 2	extension/persistence/re.../relational/jdbc/models extension/persistence/jp...ache/polaris/jpa/models	ModelGrantRecord.java ModelGrantRecord.java	128:138 123:133	view
8	x 2	extension/persistence/re.../relational/jdbc/models extension/persistence/jp...ache/polaris/jpa/models	ModelPolicyMappingRecord.java ModelPolicyMappingRecord.java	130:140 161:171	view
7	x 2	polaris-core/src/main/ja...che/polaris/core/policy polaris-core/src/main/ja...che/polaris/core/policy	PolicyMappingPersistence.java TransactionalPolicyMappingPersistence...	95:103 65:73	view
7	x 2	polaris-core/src/main/ja...rsistence/transactional polaris-core/src/main/ja...olaris/core/persistence	TransactionalMetaStoreManagerImpl.java AtomicOperationMetaStoreManager.java	2059:2067 1629:1637	view
22	x 2	client/python/polaris/management client/python/polaris/catalog	exceptions.py exceptions.py	0:0 0:0	view
14	x 30	client/python/polaris/catalog/models client/python/polaris/catalog/models client/python/polaris/catalog/models client/python/polaris/catalog/models client/python/polaris/catalog/models client/python/polaris/catalog/models client/python/polaris/catalog/models client/python/polaris/catalog/models client/python/polaris/catalog/models client/python/polaris/catalog/models ...	add_schema_update.py remove_statistics_update.py view_requirement.py upgrade_format_version_update.py add_sort_order_update.py assert_last_assigned_partition_id.py set_statistics_update.py assert_create.py assert_default_sort_order_id.py set_default_spec_update.py ...	0:0 0:0 0:0 0:0 0:0 0:0 0:0 0:0 0:0 0:0 ...	view
7	x 2	extension/persistence/re.../relational/jdbc/models extension/persistence/jp...ache/polaris/jpa/models	ModelGrantRecord.java ModelGrantRecord.java	140:149 135:144	view
6	x 2	polaris-core/src/main/ja...olaris/core/persistence extension/persistence/re...istence/relational/jdbc	LocalPolarisMetaStoreManagerFactory.java JdbcMetaStoreManagerFactory.java	175:183 199:207	view
6	x 2	polaris-core/src/main/ja...olaris/core/persistence extension/persistence/re...istence/relational/jdbc	LocalPolarisMetaStoreManagerFactory.java JdbcMetaStoreManagerFactory.java	186:194 210:218	view
6	x 3	polaris-core/src/main/ja...rsistence/transactional extension/persistence/ec...stence/impl/eclipselink extension/persistence/re...istence/relational/jdbc	TreeMapTransactionalPersistenceImpl.java PolarisEclipseLinkMetaStoreSessionImp... JdbcBasePersistenceImpl.java	537:544 657:664 886:893	view
6	x 2	service/common/src/main/...aris/service/catalog/io service/common/src/testF...aris/service/catalog/io	WasbTranslatingFileIOFactory.java MeasuredFileIOFactory.java	44:51 56:63	view