apache / incubator-s2graphDuplication

Intro

For duplication, we look at places in code where there are 6 or more lines of code that are exactly the same.
Before duplication is calculated, the code is cleaned to remove empty lines, comments, and frequently duplicated constructs such as imports.
You should aim at having as little as possible (<5%) of duplicated code as high-level of duplication can lead to maintenance difficulties, poor factoring, and logical contradictions.

Learn more...

Duplication Overall

14% duplication:

22,211 cleaned lines of cleaned code (without empty lines, comments, and frequently duplicated constructs such as imports)
3,247 duplicated lines

192 duplicates

dependency graphs: 2D graph | 3D graph | 3D graph (with duplicates)...

Duplication per Extension

Duplication per Component (primary)

Duplication Between Components (50+ lines)

Download: SVG DOT (open online Graphviz editor)

From Component --> To Component	Duplicated Lines	File Pairs	Details
loader (55%) --> s2jobs (22%)	1202	11 file pairs	details...
dev_support (76%) --> s2core (1%)	300	1 file pair	details...
example (22%) --> s2core (<1%)	216	11 file pairs	details...

Open 3D force graph...

Show more details on duplication between components...

Longest Duplicates

The list of 50 longest duplicates.

See data for all 192 duplicates...

Size	#	Folders	Files	Lines	Code
114	x 2	loader/src/main/scala/or...he/s2graph/loader/spark s2jobs/src/main/scala/or...he/s2graph/s2jobs/spark	HBaseContext.scala HBaseContext.scala	235:497 (32%) 235:497 (33%)	view
99	x 2	loader/src/main/scala/or...he/s2graph/loader/spark s2jobs/src/main/scala/or...he/s2graph/s2jobs/spark	HBaseContext.scala HBaseContext.scala	501:707 (27%) 502:708 (29%)	view
67	x 2	loader/src/main/scala/or...he/s2graph/loader/spark s2jobs/src/main/scala/or...he/s2graph/s2jobs/spark	JavaHBaseContext.scala JavaHBaseContext.scala	109:341 (77%) 109:341 (77%)	view
67	x 2	s2counter_core/src/main/...s2graph/counter/core/v1 s2counter_core/src/main/...s2graph/counter/core/v1	ExactStorageAsyncHBase.scala ExactStorageHBase.scala	130:219 (32%) 115:204 (31%)	view
48	x 2	s2counter_core/src/main/...s2graph/counter/core/v1 s2counter_core/src/main/...s2graph/counter/core/v1	ExactStorageAsyncHBase.scala ExactStorageHBase.scala	263:334 (23%) 249:321 (22%)	view
43	x 2	loader/src/main/scala/or...he/s2graph/loader/spark s2jobs/src/main/scala/or...he/s2graph/s2jobs/spark	HBaseRDDFunctions.scala HBaseRDDFunctions.scala	35:205 (100%) 33:203 (100%)	view
37	x 2	loader/src/main/scala/or...he/s2graph/loader/spark s2jobs/src/main/scala/or...he/s2graph/s2jobs/spark	HBaseContext.scala HBaseContext.scala	98:230 (10%) 98:230 (10%)	view
31	x 2	s2core/src/main/scala/or...serde/snapshotedge/tall s2core/src/main/scala/or...serde/snapshotedge/wide	SnapshotEdgeDeserializable.scala SnapshotEdgeDeserializable.scala	72:111 (48%) 62:100 (53%)	view
30	x 2	loader/src/main/scala/or...he/s2graph/loader/spark s2jobs/src/main/scala/or...he/s2graph/s2jobs/spark	HBaseContext.scala HBaseContext.scala	784:848 (8%) 785:849 (8%)	view
30	x 2	dev_support/graph_mysql s2core/src/main/resource...che/s2graph/core/schema	schema.sql schema.sql	49:95 (15%) 38:84 (17%)	view
28	x 2	s2core/src/main/scala/or...ge/serde/indexedge/tall s2core/src/main/scala/or...ge/serde/indexedge/wide	IndexEdgeDeserializable.scala IndexEdgeDeserializable.scala	119:165 (29%) 94:139 (35%)	view
28	x 2	loader/src/main/scala/or...he/s2graph/loader/spark s2jobs/src/main/scala/or...he/s2graph/s2jobs/spark	HBaseDStreamFunctions.scala HBaseDStreamFunctions.scala	30:155 (100%) 30:155 (100%)	view
27	x 2	loader/src/main/scala/or...he/s2graph/loader/spark s2jobs/src/main/scala/or...he/s2graph/s2jobs/spark	HBaseContext.scala HBaseContext.scala	710:755 (7%) 711:756 (7%)	view
23	x 2	dev_support/graph_mysql s2core/src/main/resource...che/s2graph/core/schema	schema.sql schema.sql	246:274 (11%) 213:241 (13%)	view
23	x 2	dev_support/graph_mysql s2core/src/main/resource...che/s2graph/core/schema	schema.sql schema.sql	122:154 (11%) 111:143 (13%)	view
22	x 2	s2core/src/main/scala/or...ge/serde/indexedge/tall s2core/src/main/scala/or...ge/serde/indexedge/wide	IndexEdgeDeserializable.scala IndexEdgeDeserializable.scala	37:66 (23%) 33:61 (27%)	view
22	x 2	example/movielens/schema example/movielens/schema	vertex.tag.graphql vertex.user.graphql	1:22 (64%) 1:22 (64%)	view
22	x 2	example/movielens/schema example/movielens/schema	vertex.movie.graphql vertex.user.graphql	1:22 (45%) 1:22 (64%)	view
22	x 2	example/movielens/schema example/movielens/schema	vertex.movie.graphql vertex.tag.graphql	1:22 (45%) 1:22 (64%)	view
21	x 2	example/movielens/schema example/movielens/schema	edge.rated.graphql edge.related.graphql	1:22 (35%) 1:21 (36%)	view
21	x 2	example/movielens/schema example/movielens/schema	edge.rated.graphql edge.similar.movie.graphql	1:22 (35%) 1:22 (35%)	view
21	x 2	example/movielens/schema example/movielens/schema	edge.related.graphql edge.tagged.graphql	1:21 (36%) 1:21 (38%)	view
21	x 2	example/movielens/schema example/movielens/schema	edge.rated.graphql edge.tagged.graphql	1:22 (35%) 1:21 (38%)	view
21	x 2	example/movielens/schema example/movielens/schema	edge.similar.movie.graphql edge.tagged.graphql	1:22 (35%) 1:21 (38%)	view
21	x 2	example/movielens/schema example/movielens/schema	edge.related.graphql edge.similar.movie.graphql	1:21 (36%) 1:22 (35%)	view
20	x 2	example/movielens/schema example/movielens/schema	edge.tagged.graphql vertex.movie.graphql	1:20 (36%) 1:20 (41%)	view
20	x 2	loader s2jobs	loader.py loader.py	57:79 (27%) 71:93 (23%)	view
20	x 2	example/movielens/schema example/movielens/schema	edge.rated.graphql service.graphql	1:21 (33%) 1:20 (62%)	view
20	x 2	example/movielens/schema example/movielens/schema	edge.rated.graphql vertex.tag.graphql	1:21 (33%) 1:20 (58%)	view
20	x 2	example/movielens/schema example/movielens/schema	edge.similar.movie.graphql vertex.tag.graphql	1:21 (33%) 1:20 (58%)	view
20	x 2	example/movielens/schema example/movielens/schema	edge.tagged.graphql vertex.tag.graphql	1:20 (36%) 1:20 (58%)	view
20	x 2	s2graphql/src/main/scala...raph/graphql/repository s2graphql/src/main/scala...e/s2graph/graphql/types	GraphRepository.scala S2Type.scala	83:114 (10%) 294:325 (8%)	view
20	x 2	example/movielens/schema example/movielens/schema	edge.tagged.graphql vertex.user.graphql	1:20 (36%) 1:20 (58%)	view
20	x 2	example/movielens/schema example/movielens/schema	edge.similar.movie.graphql vertex.movie.graphql	1:21 (33%) 1:20 (41%)	view
20	x 2	example/movielens/schema example/movielens/schema	edge.tagged.graphql service.graphql	1:20 (36%) 1:20 (62%)	view
20	x 2	example/movielens/schema example/movielens/schema	edge.related.graphql vertex.user.graphql	1:20 (35%) 1:20 (58%)	view
20	x 2	example/movielens/schema example/movielens/schema	service.graphql vertex.user.graphql	1:20 (62%) 1:20 (58%)	view
20	x 2	example/movielens/query example/movielens/query	search_movie_by_genres_and_title_with... search_movie_by_title.graphql	1:21 (62%) 1:21 (57%)	view
20	x 2	example/movielens/schema example/movielens/schema	edge.similar.movie.graphql service.graphql	1:21 (33%) 1:20 (62%)	view
20	x 2	example/movielens/schema example/movielens/schema	edge.similar.movie.graphql vertex.user.graphql	1:21 (33%) 1:20 (58%)	view
20	x 2	example/movielens/schema example/movielens/schema	edge.related.graphql vertex.tag.graphql	1:20 (35%) 1:20 (58%)	view
20	x 2	example/movielens/schema example/movielens/schema	edge.related.graphql vertex.movie.graphql	1:20 (35%) 1:20 (41%)	view
20	x 2	example/movielens/schema example/movielens/schema	edge.rated.graphql vertex.user.graphql	1:21 (33%) 1:20 (58%)	view
20	x 2	example/movielens/query example/movielens/query	recommend_movie.graphql search_movie_by_genres_and_title_with...	1:21 (58%) 1:21 (62%)	view
20	x 2	example/movielens/query example/movielens/query	recommend_movie.graphql search_movie_by_title.graphql	1:21 (58%) 1:21 (57%)	view
20	x 2	example/movielens/schema example/movielens/schema	edge.rated.graphql vertex.movie.graphql	1:21 (33%) 1:20 (41%)	view
20	x 2	example/movielens/schema example/movielens/schema	service.graphql vertex.movie.graphql	1:20 (62%) 1:20 (41%)	view
20	x 2	example/movielens/schema example/movielens/schema	service.graphql vertex.tag.graphql	1:20 (62%) 1:20 (58%)	view
20	x 2	example/movielens/schema example/movielens/schema	edge.related.graphql service.graphql	1:20 (35%) 1:20 (62%)	view
19	x 2	example/movielens/schema example/movielens/schema	edge.rated.graphql edge.similar.movie.graphql	42:60 (32%) 42:60 (32%)	view

Duplicated Units

The list of top 30 duplicated units.

See data for all 30 unit duplicates...

Size	#	Folders	Files	Lines	Code
46	x 2	s2counter_core/src/main/...s2graph/counter/core/v1 s2counter_core/src/main/...s2graph/counter/core/v1	ExactStorageHBase.scala ExactStorageAsyncHBase.scala	132:185 147:200	view
22	x 2	loader/src/main/scala/or...he/s2graph/loader/spark s2jobs/src/main/scala/or...he/s2graph/s2jobs/spark	HBaseContext.scala HBaseContext.scala	315:339 315:339	view
20	x 2	s2counter_core/src/main/...s2graph/counter/core/v1 s2counter_core/src/main/...s2graph/counter/core/v1	ExactStorageHBase.scala ExactStorageAsyncHBase.scala	249:273 263:287	view
20	x 2	s2counter_core/src/main/...s2graph/counter/core/v1 s2counter_core/src/main/...s2graph/counter/core/v1	ExactStorageHBase.scala ExactStorageAsyncHBase.scala	275:297 289:311	view
18	x 2	loader/src/main/scala/or...he/s2graph/loader/spark s2jobs/src/main/scala/or...he/s2graph/s2jobs/spark	HBaseContext.scala HBaseContext.scala	536:558 537:559	view
17	x 2	loader/src/main/scala/or...graph/loader/subscriber s2jobs/src/main/scala/or...e/s2graph/s2jobs/loader	TransferToHFile.scala HFileGenerator.scala	296:315 69:88	view
15	x 2	loader/src/main/scala/or...he/s2graph/loader/spark s2jobs/src/main/scala/or...he/s2graph/s2jobs/spark	BulkLoadPartitioner.scala BulkLoadPartitioner.scala	37:55 37:55	view
14	x 2	s2counter_core/src/main/...s2graph/counter/core/v1 s2counter_core/src/main/...s2graph/counter/core/v1	ExactStorageHBase.scala ExactStorageAsyncHBase.scala	115:130 130:145	view
13	x 2	loader/src/main/scala/or...he/s2graph/loader/spark s2jobs/src/main/scala/or...he/s2graph/s2jobs/spark	HBaseContext.scala HBaseContext.scala	371:387 371:387	view
13	x 2	s2counter_core/src/main/...s2graph/counter/core/v1 s2counter_core/src/main/...s2graph/counter/core/v1	ExactStorageHBase.scala ExactStorageAsyncHBase.scala	187:201 202:216	view
12	x 2	loader/src/main/scala/or...he/s2graph/loader/spark s2jobs/src/main/scala/or...he/s2graph/s2jobs/spark	HBaseContext.scala HBaseContext.scala	406:421 406:421	view
12	x 2	loader/src/main/scala/or...he/s2graph/loader/spark s2jobs/src/main/scala/or...he/s2graph/s2jobs/spark	HBaseRDDFunctions.scala HBaseRDDFunctions.scala	193:206 191:204	view
11	x 2	loader/src/main/scala/or...he/s2graph/loader/spark s2jobs/src/main/scala/or...he/s2graph/s2jobs/spark	JavaHBaseContext.scala JavaHBaseContext.scala	246:260 246:260	view
11	x 2	loader/src/main/scala/or...he/s2graph/loader/spark s2jobs/src/main/scala/or...he/s2graph/s2jobs/spark	HBaseContext.scala HBaseContext.scala	215:228 215:228	view
11	x 2	loader/src/main/scala/or...he/s2graph/loader/spark s2jobs/src/main/scala/or...he/s2graph/s2jobs/spark	HBaseContext.scala HBaseContext.scala	508:523 509:524	view
11	x 2	s2counter_core/src/main/...s2graph/counter/core/v1 s2counter_core/src/main/...s2graph/counter/core/v2	BytesUtilV1.scala BytesUtilV2.scala	41:55 44:61	view
10	x 2	s2counter_core/src/main/...s2graph/counter/core/v1 s2counter_core/src/main/...s2graph/counter/core/v1	ExactStorageHBase.scala ExactStorageAsyncHBase.scala	299:312 313:325	view
9	x 2	loader/src/main/scala/or...he/s2graph/loader/spark s2jobs/src/main/scala/or...he/s2graph/s2jobs/spark	JavaHBaseContext.scala JavaHBaseContext.scala	280:290 280:290	view
9	x 2	loader/src/main/scala/or...he/s2graph/loader/spark s2jobs/src/main/scala/or...he/s2graph/s2jobs/spark	HBaseContext.scala HBaseContext.scala	433:446 433:446	view
9	x 2	loader/src/main/scala/or...he/s2graph/loader/spark s2jobs/src/main/scala/or...he/s2graph/s2jobs/spark	HBaseRDDFunctions.scala HBaseRDDFunctions.scala	102:112 100:110	view
8	x 2	loader/src/main/scala/or...he/s2graph/loader/spark s2jobs/src/main/scala/or...he/s2graph/s2jobs/spark	JavaHBaseContext.scala JavaHBaseContext.scala	302:311 302:311	view
8	x 2	loader/src/main/scala/or...he/s2graph/loader/spark s2jobs/src/main/scala/or...he/s2graph/s2jobs/spark	HBaseContext.scala HBaseContext.scala	469:481 469:481	view
7	x 2	loader/src/main/scala/or...he/s2graph/loader/spark s2jobs/src/main/scala/or...he/s2graph/s2jobs/spark	HBaseContext.scala HBaseContext.scala	346:354 346:354	view
6	x 2	s2core/src/main/scala/or...che/s2graph/core/schema s2rest_play/app/org/apac...h/rest/play/controllers	Schema.scala AdminController.scala	147:154 128:135	view
6	x 4	s2core/src/main/scala/org/apache/s2graph/core loader/src/main/scala/or...graph/loader/subscriber loader/src/main/scala/or...graph/loader/subscriber s2jobs/src/main/scala/or...e/s2graph/s2jobs/loader	GraphUtil.scala GraphSubscriber.scala TransferToHFile.scala GraphFileOptions.scala	175:182 145:152 122:129 93:100	view
6	x 2	loader/src/main/scala/or...he/s2graph/loader/spark s2jobs/src/main/scala/or...he/s2graph/s2jobs/spark	JavaHBaseContext.scala JavaHBaseContext.scala	221:228 221:228	view
6	x 2	loader/src/main/scala/or...he/s2graph/loader/spark s2jobs/src/main/scala/or...he/s2graph/s2jobs/spark	HBaseContext.scala HBaseContext.scala	194:201 194:201	view
6	x 2	loader/src/main/scala/or...he/s2graph/loader/spark s2jobs/src/main/scala/or...he/s2graph/s2jobs/spark	HBaseContext.scala HBaseContext.scala	261:268 261:268	view
6	x 2	loader/src/main/scala/or...he/s2graph/loader/spark s2jobs/src/main/scala/or...he/s2graph/s2jobs/spark	HBaseContext.scala HBaseContext.scala	833:840 834:841	view
6	x 2	loader/src/main/scala/or...he/s2graph/loader/spark s2jobs/src/main/scala/or...he/s2graph/s2jobs/spark	KeyFamilyQualifier.scala KeyFamilyQualifier.scala	35:42 35:42	view