apache / orc - Duplication

Intro

For duplication, we look at places in code where there are 6 or more lines of code that are exactly the same.
Before duplication is calculated, the code is cleaned to remove empty lines, comments, and frequently duplicated constructs such as imports.
You should aim at having as little as possible (<5%) of duplicated code as high-level of duplication can lead to maintenance difficulties, poor factoring, and logical contradictions.

Learn more...

Duplication Overall

12% duplication:

55,972 cleaned lines of cleaned code (without empty lines, comments, and frequently duplicated constructs such as imports)
7,104 duplicated lines

2,595 duplicates

dependency graphs: 2D graph | 3D graph | 3D graph (with duplicates)...

Duplication per Extension

Duplication per Component (primary)

Duplication Between Components (50+ lines)

Download: SVG DOT (open online Graphviz editor)

From Component --> To Component	Duplicated Lines	File Pairs	Details
cmake_modules (6%) --> conan (51%)	192	11 file pairs	details...
Unclassified (<1%) --> java (<1%)	97	8 file pairs	details...

Open 2D force graph... Open 3D force graph...

Show more details on duplication between components...

Longest Duplicates

The list of 50 longest duplicates.

See data for all 2,595 duplicates...

Size	#	Folders	Files	Lines	Code
38	x 2	java/core/src/java/org/apache/orc/impl java/core/src/java/org/apache/orc/impl	TreeReaderFactory.java TreeReaderFactory.java	559:614 (1%) 626:681 (1%)	view
37	x 2	java/core/src/java/org/apache/orc/impl java/core/src/java/org/apache/orc/impl	TreeReaderFactory.java TreeReaderFactory.java	560:614 (1%) 695:749 (1%)	view
37	x 2	java/core/src/java/org/apache/orc/impl java/core/src/java/org/apache/orc/impl	TreeReaderFactory.java TreeReaderFactory.java	627:681 (1%) 695:749 (1%)	view
34	x 2	java/bench/core/src/java...bench/core/convert/json java/tools/src/java/org/apache/orc/tools	JsonWriter.java PrintData.java	128:162 (24%) 123:157 (17%)	view
34	x 2	java/bench/hive/src/java/org/apache/orc/bench/hive java/bench/hive/src/java/org/apache/orc/bench/hive	RowFilterProjectionBenchmark.java RowFilterProjectionBenchmark.java	111:151 (26%) 171:211 (26%)	view
33	x 2	c++/src c++/src	BpackingAvx512.cc BpackingAvx512.cc	979:1031 (1%) 1171:1223 (1%)	view
31	x 2	c++/src c++/src	BpackingAvx512.cc BpackingDefault.cc	2553:2584 (1%) 333:364 (12%)	view
30	x 2	c++/src c++/src	BpackingAvx512.cc BpackingAvx512.cc	1562:1609 (1%) 1962:2009 (1%)	view
30	x 2	c++/src c++/src	BpackingAvx512.cc BpackingAvx512.cc	1447:1494 (1%) 1846:1893 (1%)	view
30	x 2	c++/src c++/src	BpackingAvx512.cc BpackingAvx512.cc	1846:1893 (1%) 2125:2172 (1%)	view
30	x 2	c++/src c++/src	BpackingAvx512.cc BpackingAvx512.cc	1447:1494 (1%) 2125:2172 (1%)	view
30	x 2	c++/src c++/src	BpackingAvx512.cc BpackingAvx512.cc	1562:1609 (1%) 1731:1778 (1%)	view
30	x 2	c++/src c++/src	BpackingAvx512.cc BpackingAvx512.cc	1731:1778 (1%) 1962:2009 (1%)	view
27	x 2	java/core/src/java/org/apache/orc/impl java/core/src/java/org/apache/orc/impl	TreeReaderFactory.java TreeReaderFactory.java	703:738 (1%) 1412:1447 (1%)	view
27	x 2	java/core/src/java/org/apache/orc/impl java/core/src/java/org/apache/orc/impl	TreeReaderFactory.java TreeReaderFactory.java	635:670 (1%) 1412:1447 (1%)	view
27	x 2	c++/src c++/src	BpackingAvx512.cc BpackingAvx512.cc	1337:1379 (1%) 1967:2009 (1%)	view
27	x 2	c++/src c++/src	BpackingAvx512.cc BpackingAvx512.cc	1337:1379 (1%) 1567:1609 (1%)	view
27	x 2	c++/src c++/src	BpackingAvx512.cc BpackingAvx512.cc	1337:1379 (1%) 1736:1778 (1%)	view
27	x 2	java/core/src/java/org/apache/orc/impl java/core/src/java/org/apache/orc/impl	TreeReaderFactory.java TreeReaderFactory.java	568:603 (1%) 1412:1447 (1%)	view
25	x 2	c++/src c++/src	BpackingAvx512.cc BpackingAvx512.cc	799:836 (1%) 979:1016 (1%)	view
25	x 2	c++/src c++/src	BpackingAvx512.cc BpackingAvx512.cc	799:836 (1%) 1171:1208 (1%)	view
25	x 2	java/bench/hive/src/java...rc/bench/hive/rowfilter java/bench/hive/src/java...rc/bench/hive/rowfilter	DoubleRowFilterBenchmark.java TimestampRowFilterBenchmark.java	55:97 (59%) 57:100 (58%)	view
25	x 2	java/bench/hive/src/java...rc/bench/hive/rowfilter java/bench/hive/src/java...rc/bench/hive/rowfilter	DecimalRowFilterBenchmark.java DoubleRowFilterBenchmark.java	55:97 (59%) 55:97 (59%)	view
25	x 2	c++/src c++/src	BpackingAvx512.cc BpackingAvx512.cc	634:671 (1%) 994:1031 (1%)	view
25	x 2	c++/src c++/src	BpackingAvx512.cc BpackingAvx512.cc	634:671 (1%) 1186:1223 (1%)	view
25	x 2	java/bench/hive/src/java...rc/bench/hive/rowfilter java/bench/hive/src/java...rc/bench/hive/rowfilter	BooleanRowFilterBenchmark.java StringRowFilterBenchmark.java	54:96 (59%) 54:96 (59%)	view
25	x 2	java/bench/hive/src/java...rc/bench/hive/rowfilter java/bench/hive/src/java...rc/bench/hive/rowfilter	DecimalRowFilterBenchmark.java TimestampRowFilterBenchmark.java	55:97 (59%) 57:100 (58%)	view
24	x 2	java/core/src/java/org/apache/orc/impl/writer java/core/src/java/org/apache/orc/impl/writer	BooleanTreeWriter.java ByteTreeWriter.java	73:107 (46%) 83:117 (40%)	view
22	x 2	java/bench/core/src/java...bench/core/convert/avro java/bench/core/src/java...bench/core/convert/avro	AvroReader.java AvroWriter.java	268:289 (11%) 253:274 (9%)	view
21	x 2	c++/src c++/src	BpackingAvx512.cc BpackingAvx512.cc	760:783 (1%) 940:963 (1%)	view
20	x 2	c++/src c++/src	BpackingAvx512.cc BpackingAvx512.cc	296:327 (1%) 418:449 (1%)	view
20	x 2	java/core/src/java/org/apache/orc/impl/writer java/core/src/java/org/apache/orc/impl/writer	ByteTreeWriter.java IntegerTreeWriter.java	70:103 (33%) 87:120 (27%)	view
20	x 2	c++/src c++/src	BpackingAvx512.cc BpackingAvx512.cc	589:610 (1%) 1132:1153 (1%)	view
20	x 2	c++/src c++/src	BpackingAvx512.cc BpackingAvx512.cc	296:327 (1%) 552:583 (1%)	view
20	x 2	c++/src c++/src	BpackingAvx512.cc BpackingAvx512.cc	418:449 (1%) 552:583 (1%)	view
20	x 2	c++/src c++/src	Statistics.hh Statistics.hh	966:997 (1%) 1308:1339 (1%)	view
19	x 2	java/core/src/java/org/apache/orc/impl java/core/src/java/org/apache/orc/impl	TreeReaderFactory.java TreeReaderFactory.java	519:547 (<1%) 586:614 (<1%)	view
19	x 2	c++/src c++/src	BpackingAvx512.cc BpackingAvx512.cc	1297:1317 (1%) 1926:1947 (1%)	view
19	x 2	c++/src c++/src	BpackingAvx512.cc BpackingAvx512.cc	1412:1432 (1%) 1811:1831 (1%)	view
19	x 2	c++/src c++/src	BpackingAvx512.cc BpackingAvx512.cc	1297:1317 (1%) 1696:1716 (1%)	view
19	x 2	c++/src c++/src	BpackingAvx512.cc BpackingAvx512.cc	2090:2110 (1%) 2259:2279 (1%)	view
19	x 2	c++/src c++/src	BpackingAvx512.cc BpackingAvx512.cc	1811:1831 (1%) 2259:2279 (1%)	view
19	x 2	c++/src c++/src	BpackingAvx512.cc BpackingAvx512.cc	1412:1432 (1%) 2259:2279 (1%)	view
19	x 2	c++/src c++/src	BpackingAvx512.cc BpackingAvx512.cc	1412:1432 (1%) 2090:2110 (1%)	view
19	x 2	java/core/src/java/org/apache/orc/impl java/core/src/java/org/apache/orc/impl	TreeReaderFactory.java TreeReaderFactory.java	519:547 (<1%) 721:749 (<1%)	view
19	x 2	java/core/src/java/org/apache/orc/impl java/core/src/java/org/apache/orc/impl	TreeReaderFactory.java TreeReaderFactory.java	519:547 (<1%) 653:681 (<1%)	view
19	x 2	c++/src c++/src	BpackingAvx512.cc BpackingAvx512.cc	1297:1317 (1%) 1527:1547 (1%)	view
19	x 2	c++/src c++/src	BpackingAvx512.cc BpackingAvx512.cc	1811:1831 (1%) 2090:2110 (1%)	view
19	x 2	c++/src c++/src	BpackingAvx512.cc BpackingAvx512.cc	1527:1547 (1%) 1696:1716 (1%)	view
19	x 2	c++/src c++/src	BpackingAvx512.cc BpackingAvx512.cc	1527:1547 (1%) 1926:1947 (1%)	view

Duplicated Units

The list of top 21 duplicated units.

See data for all 21 unit duplicates...

Size	#	Folders	Files	Lines	Code
16	x 2	java/tools/src/java/org/apache/orc/tools java/bench/core/src/java...bench/core/convert/json	PrintData.java JsonWriter.java	59:76 62:79	view
15	x 2	java/tools/src/java/org/apache/orc/tools java/bench/core/src/java...bench/core/convert/json	PrintData.java JsonWriter.java	182:198 185:201	view
15	x 2	java/tools/src/java/org/apache/orc/tools/convert java/bench/core/src/java...bench/core/convert/json	JsonReader.java JsonReader.java	242:258 232:248	view
12	x 2	java/tools/src/java/org/apache/orc/tools/convert java/bench/core/src/java...bench/core/convert/json	JsonReader.java JsonReader.java	128:141 152:165	view
11	x 2	java/tools/src/java/org/apache/orc/tools java/bench/core/src/java...bench/core/convert/json	PrintData.java JsonWriter.java	99:111 102:114	view
11	x 2	java/tools/src/java/org/apache/orc/tools/convert java/bench/core/src/java...bench/core/convert/json	JsonReader.java JsonReader.java	219:231 210:222	view
10	x 2	java/tools/src/java/org/apache/orc/tools java/bench/core/src/java...bench/core/convert/json	PrintData.java JsonWriter.java	78:89 81:92	view
9	x 2	java/core/src/java/org/apache/orc/impl/writer java/core/src/java/org/apache/orc/impl/writer	FloatTreeWriter.java DoubleTreeWriter.java	38:48 38:48	view
9	x 2	c++/src c++/src	ByteRLE.cc RLE.cc	189:204 109:122	view
8	x 2	java/tools/src/java/org/apache/orc/tools/convert java/bench/core/src/java...bench/core/convert/json	JsonReader.java JsonReader.java	114:123 139:148	view
8	x 2	java/core/src/java/org/apache/orc/util c++/src	BloomFilter.java BloomFilter.hh	195:204 197:206	view
7	x 2	java/tools/src/java/org/apache/orc/tools/convert java/bench/core/src/java...bench/core/convert/json	JsonReader.java JsonReader.java	75:83 103:111	view
7	x 2	java/tools/src/java/org/apache/orc/tools/convert java/bench/core/src/java...bench/core/convert/json	JsonReader.java JsonReader.java	88:96 115:123	view
7	x 2	java/tools/src/java/org/apache/orc/tools/convert java/bench/core/src/java...bench/core/convert/json	JsonReader.java JsonReader.java	101:109 127:135	view
7	x 2	java/tools/src/java/org/apache/orc/tools/convert java/bench/core/src/java...bench/core/convert/json	JsonReader.java JsonReader.java	194:202 186:194	view
7	x 2	java/core/src/java/org/apache/orc/impl java/core/src/java/org/apache/orc/impl	DynamicByteArray.java DynamicIntArray.java	89:97 82:90	view
7	x 2	java/core/src/java/org/apache/orc/impl/writer java/core/src/java/org/apache/orc/impl/writer	Decimal64TreeWriter.java DecimalTreeWriter.java	115:123 157:165	view
7	x 2	java/core/src/java/org/apache/orc/impl/writer java/core/src/java/org/apache/orc/impl/writer	UnionTreeWriter.java StructTreeWriter.java	126:134 111:119	view
7	x 2	java/bench/hive/src/java/org/apache/orc/bench/hive java/bench/hive/src/java/org/apache/orc/bench/hive	ColumnProjectionBenchmark.java FullReadBenchmark.java	83:91 92:100	view
7	x 5	java/bench/hive/src/java...rc/bench/hive/rowfilter java/bench/hive/src/java...rc/bench/hive/rowfilter java/bench/hive/src/java...rc/bench/hive/rowfilter java/bench/hive/src/java...rc/bench/hive/rowfilter java/bench/hive/src/java...rc/bench/hive/rowfilter	DoubleRowFilterBenchmark.java StringRowFilterBenchmark.java DecimalRowFilterBenchmark.java TimestampRowFilterBenchmark.java BooleanRowFilterBenchmark.java	72:80 71:79 72:80 74:82 71:79	view
6	x 2	java/shims/src/java/org/apache/orc/impl java/shims/src/java/org/apache/orc/impl	SnappyDirectDecompressWrapper.java ZlibDirectDecompressWrapper.java	35:42 35:42	view