duplicated block id: 1 size: 21 cleaned lines of code in 2 files: - core/src/main/java/org/apache/stormcrawler/jsoup/LinkParseFilter.java (85:125) - core/src/main/java/org/apache/stormcrawler/parse/filter/LinkParseFilter.java (87:127) duplicated block id: 2 size: 13 cleaned lines of code in 2 files: - core/src/main/java/org/apache/stormcrawler/jsoup/XPathFilter.java (70:87) - core/src/main/java/org/apache/stormcrawler/parse/filter/XPathFilter.java (183:200) duplicated block id: 3 size: 13 cleaned lines of code in 2 files: - core/src/main/java/org/apache/stormcrawler/filtering/URLFilters.java (91:108) - core/src/main/java/org/apache/stormcrawler/parse/ParseFilters.java (90:107) duplicated block id: 4 size: 12 cleaned lines of code in 2 files: - core/src/main/java/org/apache/stormcrawler/parse/JSoupFilters.java (128:145) - core/src/main/java/org/apache/stormcrawler/parse/ParseFilters.java (161:178) duplicated block id: 5 size: 11 cleaned lines of code in 2 files: - core/src/main/java/org/apache/stormcrawler/bolt/FetcherBolt.java (546:559) - core/src/main/java/org/apache/stormcrawler/bolt/SimpleFetcherBolt.java (305:317) duplicated block id: 6 size: 11 cleaned lines of code in 2 files: - core/src/main/java/org/apache/stormcrawler/filtering/regex/FastURLFilter.java (94:109) - core/src/main/java/org/apache/stormcrawler/parse/filter/CollectionTagger.java (101:116) duplicated block id: 7 size: 11 cleaned lines of code in 2 files: - core/src/main/java/org/apache/stormcrawler/jsoup/LinkParseFilter.java (64:80) - core/src/main/java/org/apache/stormcrawler/parse/filter/LinkParseFilter.java (64:80) duplicated block id: 8 size: 11 cleaned lines of code in 2 files: - core/src/main/java/org/apache/stormcrawler/bolt/URLPartitionerBolt.java (140:154) - core/src/main/java/org/apache/stormcrawler/util/URLPartitioner.java (105:119) duplicated block id: 9 size: 11 cleaned lines of code in 2 files: - core/src/main/java/org/apache/stormcrawler/jsoup/XPathFilter.java (100:112) - core/src/main/java/org/apache/stormcrawler/parse/filter/XPathFilter.java (158:170) duplicated block id: 10 size: 11 cleaned lines of code in 2 files: - core/src/main/java/org/apache/stormcrawler/bolt/FetcherBolt.java (652:666) - core/src/main/java/org/apache/stormcrawler/bolt/SimpleFetcherBolt.java (425:438) duplicated block id: 11 size: 9 cleaned lines of code in 2 files: - core/src/main/java/org/apache/stormcrawler/jsoup/LDJsonParseFilter.java (74:88) - core/src/main/java/org/apache/stormcrawler/parse/filter/LDJsonParseFilter.java (104:118) duplicated block id: 12 size: 9 cleaned lines of code in 2 files: - core/src/main/java/org/apache/stormcrawler/indexing/DummyIndexer.java (33:44) - core/src/main/java/org/apache/stormcrawler/indexing/StdOutIndexer.java (36:47) duplicated block id: 13 size: 9 cleaned lines of code in 2 files: - core/src/main/java/org/apache/stormcrawler/bolt/SiteMapParserBolt.java (226:238) - core/src/main/java/org/apache/stormcrawler/bolt/SiteMapParserBolt.java (288:300) duplicated block id: 14 size: 8 cleaned lines of code in 2 files: - core/src/main/java/org/apache/stormcrawler/parse/JSoupFilters.java (95:106) - core/src/main/java/org/apache/stormcrawler/parse/ParseFilters.java (102:113) duplicated block id: 15 size: 8 cleaned lines of code in 2 files: - core/src/main/java/org/apache/stormcrawler/bolt/FeedParserBolt.java (144:154) - core/src/main/java/org/apache/stormcrawler/bolt/SiteMapParserBolt.java (162:172) duplicated block id: 16 size: 8 cleaned lines of code in 2 files: - core/src/main/java/org/apache/stormcrawler/jsoup/LDJsonParseFilter.java (101:113) - core/src/main/java/org/apache/stormcrawler/parse/filter/LDJsonParseFilter.java (64:76) duplicated block id: 17 size: 8 cleaned lines of code in 2 files: - core/src/main/java/org/apache/stormcrawler/spout/FileSpout.java (222:231) - core/src/main/java/org/apache/stormcrawler/spout/MemorySpout.java (146:155) duplicated block id: 18 size: 7 cleaned lines of code in 2 files: - core/src/main/java/org/apache/stormcrawler/bolt/FetcherBolt.java (526:532) - core/src/main/java/org/apache/stormcrawler/bolt/SimpleFetcherBolt.java (287:293) duplicated block id: 19 size: 7 cleaned lines of code in 2 files: - core/src/main/java/org/apache/stormcrawler/parse/JSoupFilters.java (84:93) - core/src/main/java/org/apache/stormcrawler/parse/ParseFilters.java (90:99) duplicated block id: 20 size: 7 cleaned lines of code in 2 files: - core/src/main/java/org/apache/stormcrawler/util/URLUtil.java (163:170) - core/src/main/java/org/apache/stormcrawler/util/URLUtil.java (182:189) duplicated block id: 21 size: 7 cleaned lines of code in 2 files: - core/src/main/java/org/apache/stormcrawler/bolt/FetcherBolt.java (976:982) - core/src/main/java/org/apache/stormcrawler/bolt/SimpleFetcherBolt.java (269:275) duplicated block id: 22 size: 7 cleaned lines of code in 2 files: - core/src/main/java/org/apache/stormcrawler/bolt/FetcherBolt.java (276:284) - core/src/main/java/org/apache/stormcrawler/bolt/SimpleFetcherBolt.java (204:212) duplicated block id: 23 size: 7 cleaned lines of code in 2 files: - core/src/main/java/org/apache/stormcrawler/bolt/FetcherBolt.java (856:863) - core/src/main/java/org/apache/stormcrawler/bolt/SimpleFetcherBolt.java (170:177) duplicated block id: 24 size: 7 cleaned lines of code in 2 files: - core/src/main/java/org/apache/stormcrawler/filtering/URLFilters.java (91:100) - core/src/main/java/org/apache/stormcrawler/parse/JSoupFilters.java (84:93) duplicated block id: 25 size: 7 cleaned lines of code in 2 files: - core/src/main/java/org/apache/stormcrawler/jsoup/LDJsonParseFilter.java (62:68) - core/src/main/java/org/apache/stormcrawler/parse/filter/LDJsonParseFilter.java (92:98) duplicated block id: 26 size: 7 cleaned lines of code in 2 files: - core/src/main/java/org/apache/stormcrawler/bolt/FeedParserBolt.java (131:142) - core/src/main/java/org/apache/stormcrawler/bolt/SiteMapParserBolt.java (149:160) duplicated block id: 27 size: 6 cleaned lines of code in 2 files: - core/src/main/java/org/apache/stormcrawler/jsoup/LDJsonParseFilter.java (88:93) - core/src/main/java/org/apache/stormcrawler/parse/JSoupFilters.java (113:118) duplicated block id: 28 size: 6 cleaned lines of code in 2 files: - core/src/main/java/org/apache/stormcrawler/bolt/SimpleFetcherBolt.java (306:311) - core/src/main/java/org/apache/stormcrawler/filtering/sitemap/SitemapFilter.java (75:80) duplicated block id: 29 size: 6 cleaned lines of code in 2 files: - core/src/main/java/org/apache/stormcrawler/bolt/FetcherBolt.java (977:982) - core/src/main/java/org/apache/stormcrawler/bolt/SimpleFetcherBolt.java (343:348) duplicated block id: 30 size: 6 cleaned lines of code in 2 files: - core/src/main/java/org/apache/stormcrawler/bolt/FetcherBolt.java (977:982) - core/src/main/java/org/apache/stormcrawler/bolt/SimpleFetcherBolt.java (401:406) duplicated block id: 31 size: 6 cleaned lines of code in 2 files: - core/src/main/java/org/apache/stormcrawler/jsoup/XPathFilter.java (104:109) - core/src/main/java/org/apache/stormcrawler/parse/filter/LinkParseFilter.java (84:89) duplicated block id: 32 size: 6 cleaned lines of code in 2 files: - core/src/main/java/org/apache/stormcrawler/parse/filter/LinkParseFilter.java (84:89) - core/src/main/java/org/apache/stormcrawler/parse/filter/XPathFilter.java (162:167) duplicated block id: 33 size: 6 cleaned lines of code in 2 files: - core/src/main/java/org/apache/stormcrawler/util/URLUtil.java (156:161) - core/src/main/java/org/apache/stormcrawler/util/URLUtil.java (175:180) duplicated block id: 34 size: 6 cleaned lines of code in 2 files: - core/src/main/java/org/apache/stormcrawler/bolt/FetcherBolt.java (582:589) - core/src/main/java/org/apache/stormcrawler/bolt/FetcherBolt.java (610:617) duplicated block id: 35 size: 6 cleaned lines of code in 2 files: - core/src/main/java/org/apache/stormcrawler/bolt/FeedParserBolt.java (238:245) - core/src/main/java/org/apache/stormcrawler/bolt/FetcherBolt.java (911:918) duplicated block id: 36 size: 6 cleaned lines of code in 2 files: - core/src/main/java/org/apache/stormcrawler/bolt/URLPartitionerBolt.java (106:111) - core/src/main/java/org/apache/stormcrawler/util/URLPartitioner.java (78:83) duplicated block id: 37 size: 6 cleaned lines of code in 2 files: - archetype/src/main/resources/archetype-resources/crawler-conf.yaml (48:53) - core/src/main/resources/crawler-default.yaml (75:80) duplicated block id: 38 size: 6 cleaned lines of code in 2 files: - core/src/main/java/org/apache/stormcrawler/bolt/FetcherBolt.java (548:553) - core/src/main/java/org/apache/stormcrawler/filtering/sitemap/SitemapFilter.java (75:80) duplicated block id: 39 size: 6 cleaned lines of code in 2 files: - core/src/main/java/org/apache/stormcrawler/bolt/URLPartitionerBolt.java (77:82) - core/src/main/java/org/apache/stormcrawler/util/URLPartitioner.java (57:62) duplicated block id: 40 size: 6 cleaned lines of code in 2 files: - core/src/main/java/org/apache/stormcrawler/util/Configurable.java (113:118) - core/src/main/java/org/apache/stormcrawler/util/ConfigurableHelper.java (76:81) duplicated block id: 41 size: 6 cleaned lines of code in 2 files: - core/src/main/java/org/apache/stormcrawler/parse/filter/LDJsonParseFilter.java (90:95) - core/src/main/java/org/apache/stormcrawler/parse/filter/XPathFilter.java (182:187) duplicated block id: 42 size: 6 cleaned lines of code in 2 files: - core/src/main/java/org/apache/stormcrawler/bolt/SimpleFetcherBolt.java (343:348) - core/src/main/java/org/apache/stormcrawler/bolt/SimpleFetcherBolt.java (401:406) duplicated block id: 43 size: 6 cleaned lines of code in 2 files: - core/src/main/java/org/apache/stormcrawler/bolt/SimpleFetcherBolt.java (270:275) - core/src/main/java/org/apache/stormcrawler/bolt/SimpleFetcherBolt.java (343:348) duplicated block id: 44 size: 6 cleaned lines of code in 2 files: - core/src/main/java/org/apache/stormcrawler/bolt/SimpleFetcherBolt.java (270:275) - core/src/main/java/org/apache/stormcrawler/bolt/SimpleFetcherBolt.java (401:406) duplicated block id: 45 size: 6 cleaned lines of code in 2 files: - core/src/main/java/org/apache/stormcrawler/bolt/FetcherBolt.java (561:574) - core/src/main/java/org/apache/stormcrawler/bolt/SimpleFetcherBolt.java (319:332)