apache / datasketches-pig

Intro

For duplication, we look at places in code where there are 6 or more lines of code that are exactly the same.
Before duplication is calculated, the code is cleaned to remove empty lines, comments, and frequently duplicated constructs such as imports.
You should aim at having as little as possible (<5%) of duplicated code as high-level of duplication can lead to maintenance difficulties, poor factoring, and logical contradictions.

Learn more...

Duplication Overall

39% duplication:

4,326 cleaned lines of cleaned code (without empty lines, comments, and frequently duplicated constructs such as imports)
1,693 duplicated lines

248 duplicates

dependency graphs: 2D graph | 3D graph | 3D graph (with duplicates)...

Duplication per Extension

Duplication per Component (primary)

Longest Duplicates

The list of 50 longest duplicates.

See data for all 248 duplicates...

Size	#	Folders	Files	Lines	Code
32	x 2	src/main/java/org/apache/datasketches/pig/hll src/main/java/org/apache/datasketches/pig/hll	DataToSketch.java UnionSketch.java	88:175 (32%) 89:161 (43%)	view
31	x 2	src/main/java/org/apache/datasketches/pig/kll src/main/java/org/apache/datasketches/pig/kll	DataToSketch.java UnionSketch.java	213:329 (32%) 227:342 (29%)	view
23	x 2	src/main/java/org/apache/datasketches/pig/hll src/main/java/org/apache/datasketches/pig/theta	DataToSketch.java DataToSketch.java	218:241 (23%) 340:363 (13%)	view
20	x 2	src/main/java/org/apache...asketches/pig/quantiles src/main/java/org/apache...asketches/pig/quantiles	DataToDoublesSketch.java UnionDoublesSketch.java	209:246 (17%) 188:225 (18%)	view
19	x 2	src/main/java/org/apache...asketches/pig/quantiles src/main/java/org/apache...asketches/pig/quantiles	DataToItemsSketch.java UnionItemsSketch.java	143:182 (15%) 129:168 (16%)	view
17	x 2	src/main/java/org/apache/datasketches/pig/kll src/main/java/org/apache/datasketches/pig/kll	DataToSketch.java UnionSketch.java	168:201 (18%) 157:190 (16%)	view
16	x 2	src/main/java/org/apache...asketches/pig/quantiles src/main/java/org/apache...asketches/pig/quantiles	UnionDoublesSketch.java UnionItemsSketch.java	145:176 (14%) 141:172 (13%)	view
16	x 2	src/main/java/org/apache/datasketches/pig/kll src/main/java/org/apache/datasketches/pig/kll	GetCdf.java GetPmf.java	40:60 (80%) 42:62 (64%)	view
15	x 2	src/main/java/org/apache/datasketches/pig/tuple src/main/java/org/apache/datasketches/pig/tuple	DataToArrayOfDoublesSketchAlgebraicIn... UnionArrayOfDoublesSketchAlgebraicInt...	67:88 (35%) 59:80 (41%)	view
14	x 2	src/main/java/org/apache/datasketches/pig/theta src/main/java/org/apache/datasketches/pig/theta	Intersect.java Union.java	143:177 (11%) 199:233 (9%)	view
14	x 2	src/main/java/org/apache/datasketches/pig/theta src/main/java/org/apache/datasketches/pig/theta	DataToSketch.java Union.java	220:252 (8%) 202:234 (9%)	view
14	x 2	src/main/java/org/apache...asketches/pig/quantiles src/main/java/org/apache...asketches/pig/quantiles	DataToDoublesSketch.java UnionDoublesSketch.java	258:319 (12%) 260:321 (12%)	view
13	x 2	src/main/java/org/apache...asketches/pig/quantiles src/main/java/org/apache...asketches/pig/quantiles	DataToDoublesSketch.java DataToItemsSketch.java	164:192 (11%) 158:186 (10%)	view
13	x 2	src/main/java/org/apache...asketches/pig/quantiles src/main/java/org/apache...asketches/pig/quantiles	DataToItemsSketch.java UnionItemsSketch.java	205:227 (10%) 186:208 (11%)	view
13	x 2	src/main/java/org/apache/datasketches/pig/tuple src/main/java/org/apache/datasketches/pig/tuple	DataToDoubleSummarySketch.java UnionDoubleSummarySketch.java	58:93 (38%) 62:97 (37%)	view
13	x 2	src/main/java/org/apache/datasketches/pig/theta src/main/java/org/apache/datasketches/pig/theta	DataToSketch.java Intersect.java	220:251 (7%) 146:177 (11%)	view
13	x 2	src/main/java/org/apache...asketches/pig/quantiles src/main/java/org/apache...asketches/pig/quantiles	DataToItemsSketch.java UnionItemsSketch.java	335:355 (10%) 315:335 (11%)	view
13	x 2	src/main/java/org/apache/datasketches/pig/cpc src/main/java/org/apache/datasketches/pig/hll	GetEstimateAndErrorBounds.java SketchToEstimateAndErrorBounds.java	97:114 (36%) 56:73 (54%)	view
13	x 2	src/main/java/org/apache/datasketches/pig/tuple src/main/java/org/apache/datasketches/pig/tuple	DataToArrayOfDoublesSketch.java UnionArrayOfDoublesSketch.java	68:103 (35%) 54:89 (46%)	view
12	x 2	src/main/java/org/apache/datasketches/pig/theta src/main/java/org/apache/datasketches/pig/theta	DataToSketch.java Union.java	281:301 (7%) 265:285 (8%)	view
12	x 2	src/main/java/org/apache/datasketches/pig/sampling src/main/java/org/apache/datasketches/pig/sampling	VarOptCommonImpl.java VarOptCommonImpl.java	149:165 (9%) 205:221 (9%)	view
12	x 2	src/main/java/org/apache/datasketches/pig/cpc src/main/java/org/apache/datasketches/pig/hll	UnionSketch.java UnionSketch.java	171:188 (16%) 176:193 (16%)	view
12	x 2	src/main/java/org/apache/datasketches/pig/cpc src/main/java/org/apache/datasketches/pig/cpc	DataToSketch.java UnionSketch.java	85:128 (12%) 86:114 (16%)	view
12	x 2	src/main/java/org/apache...asketches/pig/quantiles src/main/java/org/apache/datasketches/pig/theta	UnionDoublesSketch.java Union.java	206:225 (10%) 265:285 (8%)	view
12	x 2	src/main/java/org/apache/datasketches/pig/theta src/main/java/org/apache/datasketches/pig/theta	DataToSketch.java Union.java	124:206 (7%) 129:186 (8%)	view
12	x 2	src/main/java/org/apache/datasketches/pig/theta src/main/java/org/apache/datasketches/pig/theta	AexcludeB.java Union.java	138:155 (32%) 199:216 (8%)	view
12	x 2	src/main/java/org/apache/datasketches/pig/tuple src/main/java/org/apache/datasketches/pig/tuple	DataToSketchAlgebraicIntermediateFina... UnionSketchAlgebraicIntermediateFinal...	106:122 (26%) 77:93 (32%)	view
12	x 2	src/main/java/org/apache...asketches/pig/quantiles src/main/java/org/apache/datasketches/pig/theta	DataToDoublesSketch.java DataToSketch.java	227:246 (10%) 281:301 (7%)	view
12	x 2	src/main/java/org/apache...asketches/pig/quantiles src/main/java/org/apache/datasketches/pig/theta	UnionDoublesSketch.java DataToSketch.java	206:225 (10%) 281:301 (7%)	view
12	x 2	src/main/java/org/apache...asketches/pig/quantiles src/main/java/org/apache...asketches/pig/quantiles	DataToItemsSketch.java UnionDoublesSketch.java	155:182 (10%) 145:172 (10%)	view
12	x 2	src/main/java/org/apache...ketches/pig/frequencies src/main/java/org/apache...ketches/pig/frequencies	DataToFrequentStringsSketch.java UnionFrequentStringsSketch.java	36:55 (57%) 36:55 (57%)	view
12	x 2	src/main/java/org/apache...asketches/pig/quantiles src/main/java/org/apache/datasketches/pig/theta	DataToDoublesSketch.java Union.java	227:246 (10%) 265:285 (8%)	view
12	x 2	src/main/java/org/apache/datasketches/pig/theta src/main/java/org/apache/datasketches/pig/theta	AexcludeB.java Intersect.java	138:155 (32%) 143:160 (10%)	view
11	x 2	src/main/java/org/apache...asketches/pig/quantiles src/main/java/org/apache...asketches/pig/quantiles	DataToDoublesSketch.java UnionDoublesSketch.java	347:365 (9%) 342:360 (9%)	view
11	x 2	src/main/java/org/apache/datasketches/pig/theta src/main/java/org/apache/datasketches/pig/theta	Intersect.java Union.java	353:369 (9%) 477:493 (7%)	view
11	x 2	src/main/java/org/apache/datasketches/pig/tuple src/main/java/org/apache/datasketches/pig/tuple	UnionArrayOfDoublesSketch.java UnionDoubleSummarySketch.java	54:74 (39%) 62:82 (31%)	view
11	x 2	src/main/java/org/apache/datasketches/pig/theta src/main/java/org/apache/datasketches/pig/theta	AexcludeB.java DataToSketch.java	141:155 (29%) 220:234 (6%)	view
11	x 2	src/main/java/org/apache/datasketches/pig/tuple src/main/java/org/apache/datasketches/pig/tuple	DataToArrayOfDoublesSketch.java UnionDoubleSummarySketch.java	68:88 (29%) 62:82 (31%)	view
11	x 2	src/main/java/org/apache/datasketches/pig/tuple src/main/java/org/apache/datasketches/pig/tuple	DataToDoubleSummarySketch.java UnionArrayOfDoublesSketch.java	58:78 (32%) 54:74 (39%)	view
11	x 2	src/main/java/org/apache/datasketches/pig/kll src/main/java/org/apache...asketches/pig/quantiles	GetQuantiles.java GetQuantilesFromDoublesSketch.java	59:73 (35%) 58:72 (42%)	view
11	x 2	src/main/java/org/apache/datasketches/pig/tuple src/main/java/org/apache/datasketches/pig/tuple	DataToArrayOfDoublesSketch.java DataToDoubleSummarySketch.java	68:88 (29%) 58:78 (32%)	view
10	x 2	src/main/java/org/apache/datasketches/pig/tuple src/main/java/org/apache/datasketches/pig/tuple	DataToArrayOfDoublesSketchBase.java DataToSketch.java	94:108 (10%) 120:134 (11%)	view
10	x 2	src/main/java/org/apache/datasketches/pig/kll src/main/java/org/apache...asketches/pig/quantiles	UnionSketch.java UnionDoublesSketch.java	345:361 (9%) 330:346 (9%)	view
10	x 2	src/main/java/org/apache...asketches/pig/quantiles src/main/java/org/apache...asketches/pig/quantiles	DataToItemsSketch.java UnionItemsSketch.java	315:324 (8%) 300:309 (8%)	view
10	x 2	src/main/java/org/apache/datasketches/pig/hll src/main/java/org/apache/datasketches/pig/hll	AlgebraicFinal.java AlgebraicIntermediate.java	80:96 (25%) 82:98 (25%)	view
10	x 2	src/main/java/org/apache...asketches/pig/quantiles src/main/java/org/apache/datasketches/pig/sampling	DataToDoublesSketch.java ReservoirSampling.java	234:258 (8%) 166:181 (6%)	view
10	x 2	src/main/java/org/apache...asketches/pig/quantiles src/main/java/org/apache...asketches/pig/quantiles	DoublesSketchToString.java StringsSketchToString.java	35:48 (76%) 37:50 (66%)	view
10	x 2	src/main/java/org/apache...asketches/pig/quantiles src/main/java/org/apache...asketches/pig/quantiles	DataToDoublesSketch.java DataToItemsSketch.java	330:343 (8%) 318:331 (8%)	view
10	x 2	src/main/java/org/apache...ketches/pig/frequencies src/main/java/org/apache/datasketches/pig/tuple	DataToFrequentItemsSketch.java DataToSketch.java	65:79 (18%) 103:117 (11%)	view
10	x 2	src/main/java/org/apache/datasketches/pig/tuple src/main/java/org/apache/datasketches/pig/tuple	UnionArrayOfDoublesSketchBase.java UnionSketch.java	74:88 (17%) 90:104 (18%)	view

Duplicated Units

The list of top 11 duplicated units.

See data for all 11 unit duplicates...

Size	#	Folders	Files	Lines	Code
15	x 2	src/main/java/org/apache/datasketches/pig/hll src/main/java/org/apache/datasketches/pig/hll	DataToSketch.java UnionSketch.java	122:138 108:124	view
11	x 2	src/main/java/org/apache...ketches/pig/frequencies src/main/java/org/apache...ketches/pig/frequencies	UnionFrequentItemsSketch.java DataToFrequentItemsSketch.java	55:68 92:105	view
11	x 2	src/main/java/org/apache/datasketches/pig/cpc src/main/java/org/apache/datasketches/pig/hll	GetEstimateAndErrorBounds.java SketchToEstimateAndErrorBounds.java	104:116 63:75	view
11	x 2	src/main/java/org/apache/datasketches/pig/hll src/main/java/org/apache/datasketches/pig/hll	DataToSketch.java UnionSketch.java	152:164 138:150	view
11	x 2	src/main/java/org/apache/datasketches/pig/theta src/main/java/org/apache/datasketches/pig/theta	DataToSketch.java Union.java	123:136 128:141	view
11	x 4	src/main/java/org/apache/datasketches/pig/theta src/main/java/org/apache/datasketches/pig/theta src/main/java/org/apache/datasketches/pig/theta src/main/java/org/apache/datasketches/pig/theta	DataToSketch.java Union.java Intersect.java AexcludeB.java	222:235 204:217 148:161 143:156	view
10	x 2	src/main/java/org/apache...asketches/pig/quantiles src/main/java/org/apache...asketches/pig/quantiles	UnionItemsSketch.java DataToItemsSketch.java	187:199 206:218	view
10	x 3	src/main/java/org/apache/datasketches/pig/sampling src/main/java/org/apache/datasketches/pig/sampling src/main/java/org/apache/datasketches/pig/sampling	DataToVarOptSketch.java VarOptCommonImpl.java VarOptSampling.java	71:83 148:160 73:85	view
9	x 3	src/main/java/org/apache...asketches/pig/quantiles src/main/java/org/apache...asketches/pig/quantiles src/main/java/org/apache...asketches/pig/quantiles	UnionItemsSketch.java DataToItemsSketch.java UnionDoublesSketch.java	142:152 156:166 146:156	view
8	x 2	src/main/java/org/apache...asketches/pig/quantiles src/main/java/org/apache...asketches/pig/quantiles	UnionDoublesSketch.java DataToDoublesSketch.java	189:199 210:220	view
6	x 3	src/main/java/org/apache/datasketches/pig/sampling src/main/java/org/apache/datasketches/pig/sampling src/main/java/org/apache/datasketches/pig/sampling	DataToVarOptSketch.java VarOptCommonImpl.java VarOptSampling.java	56:64 133:141 58:66	view