apache / iceberg-rust

Intro

For duplication, we look at places in code where there are 6 or more lines of code that are exactly the same.
Before duplication is calculated, the code is cleaned to remove empty lines, comments, and frequently duplicated constructs such as imports.
You should aim at having as little as possible (<5%) of duplicated code as high-level of duplication can lead to maintenance difficulties, poor factoring, and logical contradictions.

Learn more...

Duplication Overall

34% duplication:

55,509 cleaned lines of cleaned code (without empty lines, comments, and frequently duplicated constructs such as imports)
19,223 duplicated lines

5,140 duplicates

dependency graphs: 2D graph | 3D graph | 3D graph (with duplicates)...

Duplication per Extension

Duplication per Component (primary)

Duplication Between Components (50+ lines)

Download: SVG DOT (open online Graphviz editor)

From Component --> To Component	Duplicated Lines	File Pairs	Details
crates (<1%) --> ROOT (63%)	256	48 file pairs	details...
bindings (83%) --> crates (<1%)	237	24 file pairs	details...
crates (<1%) --> website (371%)	208	12 file pairs	details...
ROOT (23%) --> bindings (33%)	96	8 file pairs	details...
ROOT (19%) --> website (142%)	80	4 file pairs	details...

Open 2D force graph... Open 3D force graph...

Show more details on duplication between components...

Longest Duplicates

The list of 50 longest duplicates.

See data for all 5,140 duplicates...

Size	#	Folders	Files	Lines	Code
145	x 2	crates/iceberg/src/arrow crates/iceberg/src/arrow	schema.rs schema.rs	1118:1293 (10%) 1411:1586 (10%)	view
106	x 2	crates/iceberg/src/scan crates/iceberg/src/scan	mod.rs mod.rs	848:981 (7%) 1066:1192 (7%)	view
98	x 2	crates/iceberg/src/expr/visitors crates/iceberg/src/expr/visitors	inclusive_projection.rs strict_projection.rs	109:237 (28%) 125:253 (3%)	view
81	x 2	crates/iceberg/src/expr/visitors crates/iceberg/src/expr/visitors	page_index_evaluator.rs row_group_metrics_evaluator.rs	1215:1315 (7%) 1839:1939 (5%)	view
75	x 2	crates/iceberg/src/spec/manifest crates/iceberg/src/spec/manifest	mod.rs mod.rs	131:206 (8%) 251:325 (8%)	view
73	x 2	crates/catalog/glue/src crates/catalog/hms/src	schema.rs schema.rs	228:315 (21%) 297:384 (21%)	view
61	x 2	crates/catalog/rest/src crates/catalog/rest/src	catalog.rs catalog.rs	1959:2020 (3%) 2135:2196 (3%)	view
53	x 2	crates/iceberg/src/spec/schema crates/iceberg/src/spec/schema	prune_columns.rs prune_columns.rs	617:669 (8%) 677:729 (8%)	view
51	x 2	crates/iceberg/src/spec crates/iceberg/src/spec	table_metadata.rs table_metadata.rs	1852:1917 (2%) 1943:2007 (2%)	view
50	x 2	crates/iceberg/src/expr/visitors crates/iceberg/src/expr/visitors	strict_projection.rs strict_projection.rs	1301:1354 (1%) 1529:1582 (1%)	view
50	x 2	crates/iceberg/src/expr/visitors crates/iceberg/src/expr/visitors	strict_projection.rs strict_projection.rs	691:744 (1%) 919:972 (1%)	view
50	x 2	crates/iceberg/src/expr/visitors crates/iceberg/src/expr/visitors	strict_projection.rs strict_projection.rs	691:744 (1%) 1147:1200 (1%)	view
50	x 2	crates/iceberg/src/expr/visitors crates/iceberg/src/expr/visitors	strict_projection.rs strict_projection.rs	265:317 (1%) 1147:1200 (1%)	view
50	x 2	crates/iceberg/src/expr/visitors crates/iceberg/src/expr/visitors	strict_projection.rs strict_projection.rs	265:317 (1%) 919:972 (1%)	view
50	x 2	crates/iceberg/src/expr/visitors crates/iceberg/src/expr/visitors	strict_projection.rs strict_projection.rs	919:972 (1%) 1147:1200 (1%)	view
50	x 2	crates/iceberg/src/expr/visitors crates/iceberg/src/expr/visitors	strict_projection.rs strict_projection.rs	265:317 (1%) 691:744 (1%)	view
50	x 2	crates/iceberg/src/expr/visitors crates/iceberg/src/expr/visitors	strict_projection.rs strict_projection.rs	1757:1810 (1%) 2189:2242 (1%)	view
50	x 2	crates/iceberg/src/expr/visitors crates/iceberg/src/expr/visitors	strict_projection.rs strict_projection.rs	265:317 (1%) 452:504 (1%)	view
50	x 2	crates/iceberg/src/expr/visitors crates/iceberg/src/expr/visitors	strict_projection.rs strict_projection.rs	1757:1810 (1%) 1985:2038 (1%)	view
50	x 2	crates/iceberg/src/expr/visitors crates/iceberg/src/expr/visitors	strict_projection.rs strict_projection.rs	452:504 (1%) 691:744 (1%)	view
50	x 2	crates/iceberg/src/expr/visitors crates/iceberg/src/expr/visitors	strict_projection.rs strict_projection.rs	1985:2038 (1%) 2189:2242 (1%)	view
50	x 2	crates/iceberg/src/expr/visitors crates/iceberg/src/expr/visitors	strict_projection.rs strict_projection.rs	452:504 (1%) 1147:1200 (1%)	view
50	x 2	crates/iceberg/src/expr/visitors crates/iceberg/src/expr/visitors	strict_projection.rs strict_projection.rs	452:504 (1%) 919:972 (1%)	view
49	x 2	crates/iceberg/src/transform crates/iceberg/src/transform	temporal.rs temporal.rs	1988:2047 (2%) 2191:2250 (2%)	view
42	x 2	crates/iceberg/src/spec crates/iceberg/src/spec	table_metadata.rs table_metadata.rs	1751:1805 (1%) 1857:1911 (1%)	view
42	x 2	crates/iceberg/src/spec crates/iceberg/src/spec	table_metadata.rs table_metadata.rs	1751:1805 (1%) 1948:2002 (1%)	view
41	x 2	crates/iceberg/src/spec crates/iceberg/src/spec	table_metadata.rs table_metadata.rs	1340:1392 (1%) 1746:1798 (1%)	view
40	x 2	crates/iceberg/src/spec crates/iceberg/src/spec	partition.rs partition.rs	905:946 (2%) 981:1022 (2%)	view
40	x 2	crates/iceberg/src/spec crates/iceberg/src/spec	view_metadata_builder.rs view_metadata_builder.rs	1010:1053 (3%) 1069:1112 (3%)	view
39	x 2	crates/iceberg/src/expr/visitors crates/iceberg/src/expr/visitors	strict_projection.rs strict_projection.rs	2709:2751 (1%) 2932:2974 (1%)	view
39	x 2	crates/iceberg/src/spec/schema crates/iceberg/src/spec/schema	prune_columns.rs prune_columns.rs	356:394 (6%) 410:448 (6%)	view
38	x 2	crates/iceberg/src/transform crates/iceberg/src/transform	temporal.rs temporal.rs	1716:1760 (1%) 2055:2099 (1%)	view
38	x 2	crates/iceberg/src/catalog crates/iceberg/src/catalog	mod.rs mod.rs	1304:1347 (2%) 1801:1844 (2%)	view
38	x 2	crates/iceberg/src/spec crates/iceberg/src/spec	table_metadata.rs table_metadata.rs	2190:2230 (1%) 2330:2370 (1%)	view
38	x 2	crates/iceberg/src/spec/schema crates/iceberg/src/spec/schema	prune_columns.rs prune_columns.rs	410:447 (5%) 456:493 (5%)	view
38	x 2	crates/iceberg/src/transform crates/iceberg/src/transform	temporal.rs temporal.rs	426:466 (1%) 478:518 (1%)	view
38	x 2	crates/iceberg/src/spec/schema crates/iceberg/src/spec/schema	prune_columns.rs prune_columns.rs	356:393 (5%) 456:493 (5%)	view
37	x 2	crates/iceberg/src/spec crates/iceberg/src/spec	table_metadata.rs table_metadata.rs	1345:1392 (1%) 1948:1995 (1%)	view
37	x 2	crates/iceberg/src/spec crates/iceberg/src/spec	table_metadata.rs table_metadata.rs	2126:2172 (1%) 2277:2323 (1%)	view
37	x 2	crates/iceberg/src/spec crates/iceberg/src/spec	table_metadata.rs table_metadata.rs	1345:1392 (1%) 1857:1904 (1%)	view
35	x 2	crates/iceberg/src/expr/visitors crates/iceberg/src/expr/visitors	strict_projection.rs strict_projection.rs	265:299 (1%) 1301:1336 (1%)	view
35	x 2	crates/iceberg/src/expr/visitors crates/iceberg/src/expr/visitors	strict_projection.rs strict_projection.rs	1529:1564 (1%) 1757:1792 (1%)	view
35	x 2	crates/iceberg/src/expr/visitors crates/iceberg/src/expr/visitors	strict_projection.rs strict_projection.rs	1529:1564 (1%) 1985:2020 (1%)	view
35	x 2	crates/iceberg/src/expr/visitors crates/iceberg/src/expr/visitors	strict_projection.rs strict_projection.rs	919:954 (1%) 1301:1336 (1%)	view
35	x 2	crates/iceberg/src/expr/visitors crates/iceberg/src/expr/visitors	strict_projection.rs strict_projection.rs	265:299 (1%) 1985:2020 (1%)	view
35	x 2	crates/iceberg/src/expr/visitors crates/iceberg/src/expr/visitors	strict_projection.rs strict_projection.rs	1301:1336 (1%) 1757:1792 (1%)	view
35	x 2	crates/iceberg/src/expr/visitors crates/iceberg/src/expr/visitors	strict_projection.rs strict_projection.rs	1147:1182 (1%) 1301:1336 (1%)	view
35	x 2	crates/iceberg/src/expr/visitors crates/iceberg/src/expr/visitors	strict_projection.rs strict_projection.rs	265:299 (1%) 2189:2224 (1%)	view
35	x 2	crates/iceberg/src/expr/visitors crates/iceberg/src/expr/visitors	strict_projection.rs strict_projection.rs	1529:1564 (1%) 2189:2224 (1%)	view
35	x 2	crates/iceberg/src/expr/visitors crates/iceberg/src/expr/visitors	strict_projection.rs strict_projection.rs	919:954 (1%) 1529:1564 (1%)	view

Duplicated Units

The list of top 17 duplicated units.

See data for all 17 unit duplicates...

Size	#	Folders	Files	Lines	Code
37	x 2	crates/iceberg/src/expr/visitors crates/iceberg/src/expr/visitors	row_group_metrics_evaluator.rs page_index_evaluator.rs	1895:1940 1271:1316	view
23	x 2	crates/iceberg/src/expr/visitors crates/iceberg/src/expr/visitors	row_group_metrics_evaluator.rs page_index_evaluator.rs	1866:1893 1242:1269	view
19	x 2	crates/iceberg/src/expr/visitors crates/iceberg/src/expr/visitors	row_group_metrics_evaluator.rs page_index_evaluator.rs	1842:1864 1218:1240	view
11	x 2	crates/catalog/hms/src crates/catalog/glue/src	utils.rs utils.rs	535:549 464:478	view
10	x 2	crates/iceberg/src/expr/visitors crates/iceberg/src/expr/visitors	strict_projection.rs inclusive_projection.rs	44:57 40:53	view
9	x 2	crates/catalog/sql/src crates/catalog/memory/src	catalog.rs catalog.rs	840:850 323:333	view
9	x 2	crates/catalog/hms/src crates/catalog/glue/src	utils.rs utils.rs	418:431 379:392	view
9	x 2	crates/catalog/glue/src crates/catalog/glue/src	schema.rs utils.rs	195:206 296:307	view
8	x 2	crates/iceberg/src/io crates/iceberg/src/scan	object_cache.rs mod.rs	249:258 762:771	view
6	x 2	crates/iceberg/src/expr/visitors crates/iceberg/src/expr/visitors	manifest_evaluator.rs inclusive_metrics_evaluator.rs	376:385 467:477	view
6	x 2	crates/iceberg/src/expr/visitors crates/iceberg/src/expr/visitors	strict_projection.rs inclusive_projection.rs	160:167 144:151	view
6	x 2	crates/iceberg/src/expr/visitors crates/iceberg/src/expr/visitors	strict_projection.rs inclusive_projection.rs	232:239 216:223	view
6	x 2	crates/iceberg/src/expr/visitors crates/iceberg/src/expr/visitors	row_group_metrics_evaluator.rs page_index_evaluator.rs	252:259 502:509	view
6	x 2	crates/iceberg/src/expr/visitors crates/iceberg/src/expr/visitors	row_group_metrics_evaluator.rs page_index_evaluator.rs	261:268 511:518	view
6	x 2	crates/iceberg/src/expr/visitors crates/iceberg/src/expr/visitors	row_group_metrics_evaluator.rs page_index_evaluator.rs	270:277 520:527	view
6	x 2	crates/iceberg/src/expr/visitors crates/iceberg/src/expr/visitors	row_group_metrics_evaluator.rs page_index_evaluator.rs	279:286 529:536	view
6	x 2	crates/catalog/hms/src crates/catalog/glue/src	schema.rs schema.rs	106:113 149:156	view