apache / datasketches-hiveDuplication

Intro

For duplication, we look at places in code where there are 6 or more lines of code that are exactly the same.
Before duplication is calculated, the code is cleaned to remove empty lines, comments, and frequently duplicated constructs such as imports.
You should aim at having as little as possible (<5%) of duplicated code as high-level of duplication can lead to maintenance difficulties, poor factoring, and logical contradictions.

Learn more...

Duplication Overall

32% duplication:

3,616 cleaned lines of cleaned code (without empty lines, comments, and frequently duplicated constructs such as imports)
1,190 duplicated lines

129 duplicates

dependency graphs: 2D graph | 3D graph | 3D graph (with duplicates)...

Duplication per Extension

Duplication per Component (primary)

Longest Duplicates

The list of 50 longest duplicates.

See data for all 129 duplicates...

Size	#	Folders	Files	Lines	Code
39	x 2	src/main/java/org/apache/datasketches/hive/cpc src/main/java/org/apache/datasketches/hive/hll	SketchState.java SketchState.java	41:83 (72%) 43:91 (70%)	view
26	x 2	src/main/java/org/apache/datasketches/hive/cpc src/main/java/org/apache/datasketches/hive/hll	ObjectInspectorValidator.java ObjectInspectorValidator.java	26:60 (100%) 26:60 (100%)	view
26	x 2	src/main/java/org/apache/datasketches/hive/theta src/main/java/org/apache/datasketches/hive/tuple	ObjectInspectorValidator.java ObjectInspectorValidator.java	26:60 (68%) 26:60 (100%)	view
26	x 2	src/main/java/org/apache/datasketches/hive/cpc src/main/java/org/apache/datasketches/hive/theta	ObjectInspectorValidator.java ObjectInspectorValidator.java	26:60 (100%) 26:60 (68%)	view
26	x 2	src/main/java/org/apache/datasketches/hive/cpc src/main/java/org/apache/datasketches/hive/tuple	ObjectInspectorValidator.java ObjectInspectorValidator.java	26:60 (100%) 26:60 (100%)	view
26	x 2	src/main/java/org/apache/datasketches/hive/hll src/main/java/org/apache/datasketches/hive/theta	ObjectInspectorValidator.java ObjectInspectorValidator.java	26:60 (100%) 26:60 (68%)	view
26	x 2	src/main/java/org/apache/datasketches/hive/hll src/main/java/org/apache/datasketches/hive/tuple	ObjectInspectorValidator.java ObjectInspectorValidator.java	26:60 (100%) 26:60 (100%)	view
19	x 2	src/main/java/org/apache/datasketches/hive/cpc src/main/java/org/apache/datasketches/hive/hll	DataToSketchUDAF.java DataToSketchUDAF.java	111:151 (25%) 111:154 (24%)	view
18	x 2	src/main/java/org/apache/datasketches/hive/tuple src/main/java/org/apache/datasketches/hive/tuple	DataToDoubleSummaryWithModeSketchUDAF... UnionDoubleSummaryWithModeSketchUDAF....	150:169 (21%) 138:157 (24%)	view
17	x 2	src/main/java/org/apache/datasketches/hive/theta src/main/java/org/apache/datasketches/hive/theta	DataToSketchUDAF.java UnionSketchUDAF.java	167:197 (20%) 134:163 (23%)	view
16	x 2	src/main/java/org/apache/datasketches/hive/cpc src/main/java/org/apache/datasketches/hive/hll	UnionSketchUDAF.java UnionSketchUDAF.java	58:96 (21%) 58:96 (21%)	view
13	x 2	src/main/java/org/apache/datasketches/hive/tuple src/main/java/org/apache/datasketches/hive/tuple	DataToSketchUDAF.java UnionSketchUDAF.java	133:149 (16%) 105:121 (23%)	view
13	x 2	src/main/java/org/apache/datasketches/hive/kll src/main/java/org/apache...sketches/hive/quantiles	ObjectInspectorValidator.java ObjectInspectorValidator.java	27:44 (100%) 27:44 (100%)	view
13	x 2	src/main/java/org/apache/datasketches/hive/cpc src/main/java/org/apache/datasketches/hive/hll	UnionSketchUDAF.java UnionSketchUDAF.java	153:178 (17%) 153:178 (17%)	view
13	x 2	src/main/java/org/apache/datasketches/hive/cpc src/main/java/org/apache/datasketches/hive/hll	DataToSketchUDAF.java DataToSketchUDAF.java	84:109 (17%) 84:109 (16%)	view
12	x 2	src/main/java/org/apache/datasketches/hive/kll src/main/java/org/apache...sketches/hive/quantiles	GetPmfUDF.java GetPmfFromDoublesSketchUDF.java	30:41 (60%) 30:41 (63%)	view
12	x 2	src/main/java/org/apache/datasketches/hive/kll src/main/java/org/apache...sketches/hive/quantiles	GetCdfUDF.java GetCdfFromDoublesSketchUDF.java	30:41 (60%) 30:41 (63%)	view
12	x 2	src/main/java/org/apache...etches/hive/frequencies src/main/java/org/apache...sketches/hive/quantiles	DataToStringsSketchUDAF.java DataToStringsSketchUDAF.java	48:59 (52%) 51:62 (50%)	view
11	x 2	src/main/java/org/apache...sketches/hive/quantiles src/main/java/org/apache...sketches/hive/quantiles	DoublesEvaluator.java ItemsEvaluator.java	38:57 (28%) 48:67 (23%)	view
11	x 2	src/main/java/org/apache/datasketches/hive/tuple src/main/java/org/apache/datasketches/hive/tuple	DataToDoubleSummaryWithModeSketchUDAF... UnionDoubleSummaryWithModeSketchUDAF....	122:135 (13%) 110:123 (14%)	view
11	x 2	src/main/java/org/apache/datasketches/hive/cpc src/main/java/org/apache/datasketches/hive/theta	DataToSketchUDAF.java UnionSketchUDAF.java	163:186 (14%) 138:160 (15%)	view
11	x 2	src/main/java/org/apache/datasketches/hive/hll src/main/java/org/apache/datasketches/hive/hll	DataToSketchUDAF.java UnionSketchUDAF.java	162:186 (14%) 146:169 (14%)	view
11	x 2	src/main/java/org/apache/datasketches/hive/cpc src/main/java/org/apache/datasketches/hive/theta	DataToSketchUDAF.java DataToSketchUDAF.java	163:186 (14%) 171:194 (13%)	view
10	x 2	src/main/java/org/apache/datasketches/hive/tuple src/main/java/org/apache/datasketches/hive/tuple	DataToArrayOfDoublesSketchUDAF.java UnionArrayOfDoublesSketchUDAF.java	138:151 (10%) 110:123 (13%)	view
10	x 2	src/main/java/org/apache...sketches/hive/quantiles src/main/java/org/apache...sketches/hive/quantiles	GetQuantilesFromDoublesSketchUDF.java GetQuantilesFromStringsSketchUDF.java	34:43 (40%) 37:46 (30%)	view
10	x 2	src/main/java/org/apache/datasketches/hive/cpc src/main/java/org/apache/datasketches/hive/hll	UnionSketchUDAF.java UnionSketchUDAF.java	98:131 (13%) 98:131 (13%)	view
10	x 2	src/main/java/org/apache/datasketches/hive/cpc src/main/java/org/apache/datasketches/hive/cpc	DataToSketchUDAF.java UnionSketchUDAF.java	161:183 (13%) 148:169 (13%)	view
10	x 2	src/main/java/org/apache...etches/hive/frequencies src/main/java/org/apache...sketches/hive/quantiles	ItemsEvaluator.java ItemsEvaluator.java	55:68 (24%) 70:83 (21%)	view
9	x 2	src/main/java/org/apache...etches/hive/frequencies src/main/java/org/apache/datasketches/hive/kll	ItemsEvaluator.java SketchEvaluator.java	55:67 (21%) 59:71 (23%)	view
9	x 2	src/main/java/org/apache/datasketches/hive/kll src/main/java/org/apache...sketches/hive/quantiles	SketchEvaluator.java ItemsEvaluator.java	59:71 (23%) 70:82 (19%)	view
9	x 2	src/main/java/org/apache/datasketches/hive/kll src/main/java/org/apache...sketches/hive/quantiles	SketchEvaluator.java DoublesEvaluator.java	59:71 (23%) 59:71 (23%)	view
9	x 2	src/main/java/org/apache...etches/hive/frequencies src/main/java/org/apache...sketches/hive/quantiles	ItemsEvaluator.java DoublesEvaluator.java	55:67 (21%) 59:71 (23%)	view
9	x 2	src/main/java/org/apache/datasketches/hive/tuple src/main/java/org/apache/datasketches/hive/tuple	UnionArrayOfDoublesSketchUDAF.java UnionSketchUDAF.java	114:126 (12%) 108:120 (16%)	view
9	x 2	src/main/java/org/apache/datasketches/hive/kll src/main/java/org/apache/datasketches/hive/kll	DataToSketchUDAF.java UnionSketchUDAF.java	59:67 (29%) 54:62 (36%)	view
9	x 2	src/main/java/org/apache...sketches/hive/quantiles src/main/java/org/apache...sketches/hive/quantiles	DoublesEvaluator.java ItemsEvaluator.java	59:71 (23%) 70:82 (19%)	view
9	x 2	src/main/java/org/apache/datasketches/hive/tuple src/main/java/org/apache/datasketches/hive/tuple	DataToSketchUDAF.java UnionArrayOfDoublesSketchUDAF.java	136:148 (11%) 114:126 (12%)	view
9	x 2	src/main/java/org/apache/datasketches/hive/kll src/main/java/org/apache...sketches/hive/quantiles	UnionSketchUDAF.java UnionDoublesSketchUDAF.java	39:49 (36%) 39:49 (36%)	view
8	x 2	src/main/java/org/apache/datasketches/hive/theta src/main/java/org/apache/datasketches/hive/theta	UnionSketchUDAF.java UnionSketchUDF.java	53:60 (11%) 40:47 (28%)	view
8	x 2	src/main/java/org/apache/datasketches/hive/cpc src/main/java/org/apache/datasketches/hive/theta	UnionSketchUDAF.java UnionSketchUDAF.java	150:169 (10%) 138:157 (11%)	view
8	x 2	src/main/java/org/apache...sketches/hive/quantiles src/main/java/org/apache...sketches/hive/quantiles	UnionDoublesSketchUDAF.java UnionItemsSketchUDAF.java	39:47 (32%) 42:50 (30%)	view
8	x 2	src/main/java/org/apache/datasketches/hive/hll src/main/java/org/apache/datasketches/hive/hll	DataToSketchUDAF.java UnionSketchUDAF.java	101:112 (10%) 88:99 (10%)	view
8	x 2	src/main/java/org/apache/datasketches/hive/cpc src/main/java/org/apache/datasketches/hive/hll	DataToSketchUDAF.java DataToSketchUDAF.java	166:186 (10%) 169:189 (10%)	view
8	x 2	src/main/java/org/apache/datasketches/hive/hll src/main/java/org/apache/datasketches/hive/theta	DataToSketchUDAF.java UnionSketchUDAF.java	169:189 (10%) 141:160 (11%)	view
8	x 2	src/main/java/org/apache/datasketches/hive/tuple src/main/java/org/apache/datasketches/hive/tuple	DataToDoubleSummarySketchUDAF.java DataToDoubleSummaryWithModeSketchUDAF...	55:64 (20%) 66:75 (9%)	view
8	x 2	src/main/java/org/apache/datasketches/hive/cpc src/main/java/org/apache/datasketches/hive/theta	UnionSketchUDAF.java DataToSketchUDAF.java	150:169 (10%) 171:191 (9%)	view
8	x 2	src/main/java/org/apache/datasketches/hive/hll src/main/java/org/apache/datasketches/hive/hll	DataToSketchUDAF.java UnionSketchUDAF.java	198:207 (10%) 183:192 (10%)	view
8	x 2	src/main/java/org/apache/datasketches/hive/kll src/main/java/org/apache...sketches/hive/quantiles	UnionSketchUDAF.java UnionItemsSketchUDAF.java	39:47 (32%) 42:50 (30%)	view
8	x 2	src/main/java/org/apache...etches/hive/frequencies src/main/java/org/apache/datasketches/hive/tuple	GetFrequentItemsFromStringsSketchUDTF... ArrayOfDoublesSketchToValuesUDTF.java	55:63 (14%) 57:65 (18%)	view
8	x 2	src/main/java/org/apache/datasketches/hive/hll src/main/java/org/apache/datasketches/hive/theta	DataToSketchUDAF.java DataToSketchUDAF.java	169:189 (10%) 174:194 (9%)	view
8	x 2	src/main/java/org/apache/datasketches/hive/theta src/main/java/org/apache/datasketches/hive/theta	ExcludeSketchUDF.java IntersectSketchUDF.java	60:70 (32%) 59:68 (36%)	view

Duplicated Units

The list of top 10 duplicated units.

See data for all 10 unit duplicates...

Size	#	Folders	Files	Lines	Code
38	x 2	src/main/java/org/apache/datasketches/hive/hll src/main/java/org/apache/datasketches/hive/cpc	SketchState.java SketchState.java	43:89 41:81	view
13	x 4	src/main/java/org/apache/datasketches/hive/hll src/main/java/org/apache/datasketches/hive/tuple src/main/java/org/apache/datasketches/hive/cpc src/main/java/org/apache/datasketches/hive/theta	ObjectInspectorValidator.java ObjectInspectorValidator.java ObjectInspectorValidator.java ObjectInspectorValidator.java	47:62 47:62 47:62 47:62	view
11	x 2	src/main/java/org/apache/datasketches/hive/tuple src/main/java/org/apache/datasketches/hive/tuple	DataToDoubleSummaryWithModeSketchUDAF... UnionDoubleSummaryWithModeSketchUDAF....	151:163 139:151	view
11	x 2	src/main/java/org/apache...etches/hive/frequencies src/main/java/org/apache...sketches/hive/quantiles	DataToStringsSketchUDAF.java DataToStringsSketchUDAF.java	49:61 52:64	view
10	x 2	src/main/java/org/apache/datasketches/hive/hll src/main/java/org/apache/datasketches/hive/hll	DataToSketchUDAF.java UnionSketchUDAF.java	197:208 182:193	view
9	x 2	src/main/java/org/apache/datasketches/hive/kll src/main/java/org/apache...sketches/hive/quantiles	UnionSketchUDAF.java UnionDoublesSketchUDAF.java	40:50 40:50	view
9	x 2	src/main/java/org/apache/datasketches/hive/cpc src/main/java/org/apache/datasketches/hive/cpc	DataToSketchUDAF.java UnionSketchUDAF.java	194:204 182:192	view
8	x 4	src/main/java/org/apache/datasketches/hive/hll src/main/java/org/apache/datasketches/hive/tuple src/main/java/org/apache/datasketches/hive/cpc src/main/java/org/apache/datasketches/hive/theta	ObjectInspectorValidator.java ObjectInspectorValidator.java ObjectInspectorValidator.java ObjectInspectorValidator.java	36:45 36:45 36:45 36:45	view
8	x 2	src/main/java/org/apache/datasketches/hive/kll src/main/java/org/apache...sketches/hive/quantiles	ObjectInspectorValidator.java ObjectInspectorValidator.java	37:46 37:46	view
8	x 2	src/main/java/org/apache...sketches/hive/quantiles src/main/java/org/apache...sketches/hive/quantiles	ItemsEvaluator.java DoublesEvaluator.java	49:63 39:53	view