Duplication · Netflix/edda

Duplication Overall

8% duplication:

5,729 cleaned lines of cleaned code (without empty lines, comments, and frequently duplicated constructs such as imports)
505 duplicated lines

50 duplicates

dependency graphs: 2D graph | 2D graph (with duplicates)... | 3D graph | 3D graph (with duplicates)...

Duplication per Extension

Duplication per Component (primary)

Longest Duplicates

The list of 50 longest duplicates.

See data for all 50 duplicates...

Size	#	Folders	Files	Lines	Code
14	x 2	src/main/scala/com/netflix/edda src/main/scala/com/netflix/edda	Collection.scala Crawler.scala	189:208 (3%) 83:102 (11%)	view
12	x 2	src/main/scala/com/netflix/edda/aws src/main/scala/com/netflix/edda/aws	AwsCrawlers.scala AwsCrawlers.scala	635:653 (1%) 1436:1454 (1%)	view
12	x 2	src/main/scala/com/netflix/edda/aws src/main/scala/com/netflix/edda/aws	AwsCrawlers.scala AwsCrawlers.scala	1013:1027 (1%) 1082:1096 (1%)	view
11	x 2	src/main/scala/com/netflix/edda/elasticsearch src/main/scala/com/netflix/edda/elasticsearch	ElasticSearchDatastore.scala ElasticSearchDatastore.scala	702:712 (1%) 790:800 (1%)	view
11	x 2	src/main/scala/com/netflix/edda/aws src/main/scala/com/netflix/edda/aws	AwsCrawlers.scala AwsCrawlers.scala	604:615 (1%) 1404:1415 (1%)	view
11	x 2	src/main/scala/com/netflix/edda/aws src/main/scala/com/netflix/edda/aws	DynamoDBElector.scala DynamoDBElector.scala	95:105 (8%) 130:140 (8%)	view
11	x 2	src/main/scala/com/netflix/edda/aws src/main/scala/com/netflix/edda/aws	DynamoDBElector.scala DynamoDBElector.scala	95:105 (8%) 166:176 (8%)	view
11	x 2	src/main/scala/com/netflix/edda/aws src/main/scala/com/netflix/edda/aws	DynamoDBElector.scala DynamoDBElector.scala	130:140 (8%) 166:176 (8%)	view
10	x 2	src/main/scala/com/netflix/edda/elasticsearch src/main/scala/com/netflix/edda/elasticsearch	ElasticSearchDatastore.scala ElasticSearchDatastore.scala	500:511 (1%) 579:590 (1%)	view
10	x 2	src/main/scala/com/netflix/edda/elasticsearch src/main/scala/com/netflix/edda/mongo	ElasticSearchDatastore.scala MongoDatastore.scala	607:620 (1%) 305:318 (3%)	view
9	x 2	src/main/scala/com/netflix/edda src/main/scala/com/netflix/edda	Observable.scala Observable.scala	62:71 (15%) 87:96 (15%)	view
9	x 2	src/main/scala/com/netflix/edda/elasticsearch src/main/scala/com/netflix/edda/elasticsearch	ElasticSearchDatastore.scala ElasticSearchDatastore.scala	477:485 (1%) 559:567 (1%)	view
9	x 2	src/main/scala/com/netflix/edda/aws src/main/scala/com/netflix/edda/aws	AwsCrawlers.scala AwsCrawlers.scala	842:850 (<1%) 1460:1468 (<1%)	view
8	x 2	src/main/scala/com/netflix/edda/aws src/main/scala/com/netflix/edda/aws	AwsCrawlers.scala AwsCrawlers.scala	604:612 (<1%) 1233:1241 (<1%)	view
8	x 2	src/main/scala/com/netflix/edda/aws src/main/scala/com/netflix/edda/aws	AwsCrawlers.scala AwsCrawlers.scala	1029:1042 (<1%) 1098:1111 (<1%)	view
8	x 2	src/main/scala/com/netflix/edda/aws src/main/scala/com/netflix/edda/aws	AwsCrawlers.scala AwsCrawlers.scala	1029:1042 (<1%) 1261:1274 (<1%)	view
8	x 2	src/main/scala/com/netflix/edda/aws src/main/scala/com/netflix/edda/aws	AwsCrawlers.scala AwsCrawlers.scala	1098:1111 (<1%) 1261:1274 (<1%)	view
8	x 2	src/main/scala/com/netflix/edda/aws src/main/scala/com/netflix/edda/aws	AwsCrawlers.scala AwsCrawlers.scala	1233:1241 (<1%) 1404:1412 (<1%)	view
8	x 2	src/main/scala/com/netflix/edda src/main/scala/com/netflix/edda	Collection.scala MergedCollection.scala	292:299 (1%) 105:112 (10%)	view
7	x 2	src/main/scala/com/netflix/edda/elasticsearch src/main/scala/com/netflix/edda/elasticsearch	ElasticSearchDatastore.scala ElasticSearchDatastore.scala	727:734 (1%) 810:817 (1%)	view
7	x 2	src/main/scala/com/netflix/edda/aws src/main/scala/com/netflix/edda/aws	AwsCrawlers.scala AwsCrawlers.scala	671:677 (<1%) 842:848 (<1%)	view
7	x 2	src/main/scala/com/netflix/edda/aws src/main/scala/com/netflix/edda/aws	AwsCrawlers.scala AwsCrawlers.scala	671:677 (<1%) 1460:1466 (<1%)	view
7	x 2	src/main/scala/com/netflix/edda/aws src/main/scala/com/netflix/edda/aws	AwsCrawlers.scala AwsCrawlers.scala	1021:1027 (<1%) 1633:1639 (<1%)	view
7	x 2	src/main/scala/com/netflix/edda/aws src/main/scala/com/netflix/edda/aws	AwsCrawlers.scala AwsCrawlers.scala	1090:1096 (<1%) 1633:1639 (<1%)	view
7	x 2	src/main/scala/com/netflix/edda/elasticsearch src/main/scala/com/netflix/edda/elasticsearch	ElasticSearchElector.scala ElasticSearchElector.scala	164:170 (4%) 202:208 (4%)	view
7	x 2	src/main/scala/com/netflix/edda src/main/scala/com/netflix/edda	CollectionProcessor.scala Elector.scala	180:186 (3%) 150:156 (7%)	view
7	x 2	src/main/scala/com/netflix/edda src/main/scala/com/netflix/edda	MergedCollection.scala Queryable.scala	105:111 (8%) 108:114 (10%)	view
7	x 2	src/main/scala/com/netflix/edda/aws src/main/scala/com/netflix/edda/elasticsearch	DynamoDBElector.scala ElasticSearchElector.scala	151:159 (5%) 175:183 (4%)	view
7	x 2	src/main/scala/com/netflix/edda src/main/scala/com/netflix/edda	GroupCollection.scala MergedCollection.scala	50:56 (6%) 106:112 (8%)	view
7	x 2	src/main/scala/com/netflix/edda src/main/scala/com/netflix/edda	Collection.scala Queryable.scala	292:298 (1%) 108:114 (10%)	view
7	x 2	src/main/scala/com/netflix/edda src/main/scala/com/netflix/edda	Collection.scala GroupCollection.scala	293:299 (1%) 50:56 (6%)	view
6	x 2	src/main/scala/com/netflix/edda/elasticsearch src/main/scala/com/netflix/edda/elasticsearch	ElasticSearchDatastore.scala ElasticSearchDatastore.scala	466:471 (1%) 536:541 (1%)	view
6	x 2	src/main/scala/com/netflix/edda/elasticsearch src/main/scala/com/netflix/edda/elasticsearch	ElasticSearchDatastore.scala ElasticSearchDatastore.scala	661:666 (1%) 685:690 (1%)	view
6	x 2	src/main/scala/com/netflix/edda/aws src/main/scala/com/netflix/edda/aws	AwsCrawlers.scala AwsCrawlers.scala	608:613 (<1%) 1021:1026 (<1%)	view
6	x 2	src/main/scala/com/netflix/edda/aws src/main/scala/com/netflix/edda/aws	AwsCrawlers.scala AwsCrawlers.scala	608:613 (<1%) 1090:1095 (<1%)	view
6	x 2	src/main/scala/com/netflix/edda/aws src/main/scala/com/netflix/edda/aws	AwsCrawlers.scala AwsCrawlers.scala	608:613 (<1%) 1633:1638 (<1%)	view
6	x 2	src/main/scala/com/netflix/edda/aws src/main/scala/com/netflix/edda/aws	AwsCrawlers.scala AwsCrawlers.scala	635:642 (<1%) 820:827 (<1%)	view
6	x 2	src/main/scala/com/netflix/edda/aws src/main/scala/com/netflix/edda/aws	AwsCrawlers.scala AwsCrawlers.scala	820:827 (<1%) 1436:1443 (<1%)	view
6	x 2	src/main/scala/com/netflix/edda/aws src/main/scala/com/netflix/edda/aws	AwsCrawlers.scala AwsCrawlers.scala	1021:1026 (<1%) 1408:1413 (<1%)	view
6	x 2	src/main/scala/com/netflix/edda/aws src/main/scala/com/netflix/edda/aws	AwsCrawlers.scala AwsCrawlers.scala	1090:1095 (<1%) 1408:1413 (<1%)	view
6	x 2	src/main/scala/com/netflix/edda/aws src/main/scala/com/netflix/edda/aws	AwsCrawlers.scala AwsCrawlers.scala	1408:1413 (<1%) 1633:1638 (<1%)	view
6	x 2	src/main/scala/com/netflix/edda/aws src/main/scala/com/netflix/edda/aws	DynamoDBElector.scala DynamoDBElector.scala	108:113 (4%) 144:149 (4%)	view
6	x 2	src/main/scala/com/netflix/edda/resources src/main/scala/com/netflix/edda/resources	CollectionResource.scala CollectionResource.scala	118:123 (2%) 141:146 (2%)	view
6	x 2	src/main/scala/com/netflix/edda src/main/scala/com/netflix/edda	Record.scala Record.scala	32:37 (8%) 47:52 (8%)	view
6	x 2	src/main/scala/com/netflix/edda src/main/scala/com/netflix/edda	GroupCollection.scala Queryable.scala	50:55 (5%) 109:114 (8%)	view
6	x 2	src/main/scala/com/netflix/edda/aws src/main/scala/com/netflix/edda/elasticsearch	DynamoDBElector.scala ElasticSearchElector.scala	115:122 (4%) 145:152 (3%)	view
6	x 2	src/main/scala/com/netflix/edda/aws src/main/scala/com/netflix/edda/elasticsearch	DynamoDBElector.scala ElasticSearchElector.scala	190:196 (4%) 224:230 (3%)	view
6	x 2	src/main/scala/com/netflix/edda src/main/scala/com/netflix/edda/aws	Crawler.scala AwsCrawlers.scala	162:167 (4%) 665:670 (<1%)	view
6	x 2	src/main/scala/com/netflix/edda src/main/scala/com/netflix/edda	Collection.scala Queryable.scala	173:178 (1%) 75:80 (8%)	view
6	x 2	src/main/scala/com/netflix/edda/aws src/main/scala/com/netflix/edda/resources	AwsCollections.scala CollectionResource.scala	791:796 (1%) 426:431 (2%)	view

Most Frequent Duplicates

The list of 20 most frequently found duplicates.

Size	#	Folders	Files	Lines	Code
6	x 5	src/main/scala/com/netflix/edda/aws src/main/scala/com/netflix/edda/aws src/main/scala/com/netflix/edda/aws src/main/scala/com/netflix/edda/aws src/main/scala/com/netflix/edda/aws	AwsCrawlers.scala AwsCrawlers.scala AwsCrawlers.scala AwsCrawlers.scala AwsCrawlers.scala	608:613 (<1%) 1021:1026 (<1%) 1090:1095 (<1%) 1408:1413 (<1%) 1633:1638 (<1%)	view
6	x 4	src/main/scala/com/netflix/edda src/main/scala/com/netflix/edda src/main/scala/com/netflix/edda src/main/scala/com/netflix/edda	Collection.scala MergedCollection.scala GroupCollection.scala Queryable.scala	293:298 (1%) 106:111 (7%) 50:55 (5%) 109:114 (8%)	view
6	x 3	src/main/scala/com/netflix/edda/aws src/main/scala/com/netflix/edda/aws src/main/scala/com/netflix/edda/aws	DynamoDBElector.scala DynamoDBElector.scala DynamoDBElector.scala	95:100 (4%) 130:135 (4%) 166:171 (4%)	view
6	x 3	src/main/scala/com/netflix/edda/aws src/main/scala/com/netflix/edda/aws src/main/scala/com/netflix/edda/aws	DynamoDBElector.scala DynamoDBElector.scala DynamoDBElector.scala	98:103 (4%) 133:138 (4%) 169:174 (4%)	view
6	x 3	src/main/scala/com/netflix/edda/aws src/main/scala/com/netflix/edda/aws src/main/scala/com/netflix/edda/aws	DynamoDBElector.scala DynamoDBElector.scala DynamoDBElector.scala	96:101 (4%) 131:136 (4%) 167:172 (4%)	view
6	x 3	src/main/scala/com/netflix/edda/aws src/main/scala/com/netflix/edda/aws src/main/scala/com/netflix/edda/aws	AwsCrawlers.scala AwsCrawlers.scala AwsCrawlers.scala	1030:1040 (<1%) 1099:1109 (<1%) 1262:1272 (<1%)	view
6	x 3	src/main/scala/com/netflix/edda/aws src/main/scala/com/netflix/edda/aws src/main/scala/com/netflix/edda/aws	AwsCrawlers.scala AwsCrawlers.scala AwsCrawlers.scala	672:677 (<1%) 843:848 (<1%) 1461:1466 (<1%)	view
6	x 3	src/main/scala/com/netflix/edda/aws src/main/scala/com/netflix/edda/aws src/main/scala/com/netflix/edda/aws	AwsCrawlers.scala AwsCrawlers.scala AwsCrawlers.scala	1034:1042 (<1%) 1103:1111 (<1%) 1266:1274 (<1%)	view
6	x 3	src/main/scala/com/netflix/edda/aws src/main/scala/com/netflix/edda/aws src/main/scala/com/netflix/edda/aws	AwsCrawlers.scala AwsCrawlers.scala AwsCrawlers.scala	606:611 (<1%) 1235:1240 (<1%) 1406:1411 (<1%)	view
6	x 3	src/main/scala/com/netflix/edda src/main/scala/com/netflix/edda src/main/scala/com/netflix/edda	Collection.scala MergedCollection.scala Queryable.scala	292:297 (1%) 105:110 (7%) 108:113 (8%)	view
6	x 3	src/main/scala/com/netflix/edda/aws src/main/scala/com/netflix/edda/aws src/main/scala/com/netflix/edda/aws	AwsCrawlers.scala AwsCrawlers.scala AwsCrawlers.scala	635:642 (<1%) 820:827 (<1%) 1436:1443 (<1%)	view
6	x 3	src/main/scala/com/netflix/edda/aws src/main/scala/com/netflix/edda/aws src/main/scala/com/netflix/edda/aws	AwsCrawlers.scala AwsCrawlers.scala AwsCrawlers.scala	1029:1039 (<1%) 1098:1108 (<1%) 1261:1271 (<1%)	view
6	x 3	src/main/scala/com/netflix/edda/aws src/main/scala/com/netflix/edda/aws src/main/scala/com/netflix/edda/aws	DynamoDBElector.scala DynamoDBElector.scala DynamoDBElector.scala	97:102 (4%) 132:137 (4%) 168:173 (4%)	view
6	x 3	src/main/scala/com/netflix/edda/aws src/main/scala/com/netflix/edda/aws src/main/scala/com/netflix/edda/aws	AwsCrawlers.scala AwsCrawlers.scala AwsCrawlers.scala	1022:1027 (<1%) 1091:1096 (<1%) 1634:1639 (<1%)	view
6	x 3	src/main/scala/com/netflix/edda/aws src/main/scala/com/netflix/edda/aws src/main/scala/com/netflix/edda/aws	DynamoDBElector.scala DynamoDBElector.scala DynamoDBElector.scala	100:105 (4%) 135:140 (4%) 171:176 (4%)	view
6	x 3	src/main/scala/com/netflix/edda/aws src/main/scala/com/netflix/edda/aws src/main/scala/com/netflix/edda/aws	DynamoDBElector.scala DynamoDBElector.scala DynamoDBElector.scala	99:104 (4%) 134:139 (4%) 170:175 (4%)	view
6	x 3	src/main/scala/com/netflix/edda/aws src/main/scala/com/netflix/edda/aws src/main/scala/com/netflix/edda/aws	AwsCrawlers.scala AwsCrawlers.scala AwsCrawlers.scala	607:612 (<1%) 1236:1241 (<1%) 1407:1412 (<1%)	view
6	x 3	src/main/scala/com/netflix/edda/aws src/main/scala/com/netflix/edda/aws src/main/scala/com/netflix/edda/aws	AwsCrawlers.scala AwsCrawlers.scala AwsCrawlers.scala	671:676 (<1%) 842:847 (<1%) 1460:1465 (<1%)	view
6	x 3	src/main/scala/com/netflix/edda/aws src/main/scala/com/netflix/edda/aws src/main/scala/com/netflix/edda/aws	AwsCrawlers.scala AwsCrawlers.scala AwsCrawlers.scala	604:610 (<1%) 1233:1239 (<1%) 1404:1410 (<1%)	view
6	x 3	src/main/scala/com/netflix/edda src/main/scala/com/netflix/edda src/main/scala/com/netflix/edda	Collection.scala MergedCollection.scala GroupCollection.scala	294:299 (1%) 107:112 (7%) 51:56 (5%)	view

Duplicated Units

The list of top 2 duplicated units.

See data for all 2 unit duplicates...

Size	#	Folders	Files	Lines	Code
8	x 2	src/main/scala/com/netflix/edda src/main/scala/com/netflix/edda	Collection.scala Crawler.scala	189:198 83:92	view
8	x 2	src/main/scala/com/netflix/edda src/main/scala/com/netflix/edda	Collection.scala Crawler.scala	201:210 95:104	view

About This Analysis

For duplication, we look at places in code where there are 6 or more lines of code that are exactly the same.
Before duplication is calculated, the code is cleaned to remove empty lines, comments, and frequently duplicated constructs such as imports.
You should aim at having as little as possible (<5%) of duplicated code as high-level of duplication can lead to maintenance difficulties, poor factoring, and logical contradictions.

Learn more...

To learn more about duplications and techniques for eliminating duplication, Sokrates recommends the following resources:

Avoid Repetition, MartinFlower.com (IEEE Software article)
Refactoring duplicated code, sourcemaking.com
Beck Design Rules, MartinFowler.com
DRY (Don't Repeat Yourself) Principle, Wikipedia
The Cut-and-Paste Programming Software Development Anti-Pattern, sourcemaking.com
Code Smells / Duplicated Code, Jeff Atwood, codinghorror.com:

"Duplicated code is the bane of software development. Stamp out duplication whenever possible. You should always be on the lookout for more subtle cases of near-duplication, too. Don't Repeat Yourself!"

Reducing Coupling, MartinFlower.com (IEEE Software article):

"There are several ways to describe coupling, but it boils down to this: If changing one module in a program requires changing another module, then coupling exists. ... Duplication always implies coupling, because changing one piece of duplicate code implies changing the other."