G

Intro

For duplication, we look at places in code where there are 6 or more lines of code that are exactly the same.
Before duplication is calculated, the code is cleaned to remove empty lines, comments, and frequently duplicated constructs such as imports.
You should aim at having as little as possible (<5%) of duplicated code as high-level of duplication can lead to maintenance difficulties, poor factoring, and logical contradictions.

Learn more...

Duplication Overall

21% duplication:

63,922 cleaned lines of cleaned code (without empty lines, comments, and frequently duplicated constructs such as imports)
13,747 duplicated lines

11,179 duplicates

Duplication per Extension

Duplication per Component (primary)

Duplication Between Components (50+ lines)

Download: SVG DOT (open online Graphviz editor)

From Component --> To Component	Duplicated Lines	File Pairs	Details
benchs/distributed_ondisk (14%) --> contrib (10%)	306	6 file pairs	details...
faiss (1%) --> faiss/impl (1%)	286	18 file pairs	details...
faiss (<1%) --> faiss/utils (1%)	157	9 file pairs	details...
faiss/impl (<1%) --> faiss/utils (<1%)	92	8 file pairs	details...
benchs (3%) --> benchs/link_and_code (8%)	90	3 file pairs	details...
benchs/distributed_ondisk (4%) --> benchs/link_and_code (8%)	86	6 file pairs	details...
c_api (2%) --> c_api/gpu (9%)	82	1 file pair	details...
conda/faiss-gpu (50%) --> conda/faiss (53%)	78	1 file pair	details...
benchs/bench_all_ivf (2%) --> benchs/link_and_code (7%)	76	2 file pairs	details...
c_api (2%) --> faiss (<1%)	70	9 file pairs	details...
contrib (2%) --> faiss/python (3%)	64	1 file pair	details...
benchs (2%) --> benchs/distributed_ondisk (2%)	62	3 file pairs	details...
benchs/link_and_code (4%) --> contrib (1%)	50	1 file pair	details...

Open 3D force graph...

Show more details on duplication between components...

Longest Duplicates

The list of 20 longest duplicates.

See data for all 11,179 duplicates...

Size	#	Folders	Files	Lines	Code
70	x 2	benchs/distributed_ondisk contrib	rpc.py rpc.py	19:146 (48%) 20:145 (47%)	view
62	x 2	faiss/gpu/impl faiss/gpu/impl	cu IVFFlatScan.cu PQScanMultiPassPrecomputed.cu	405:476 (13%) 613:684 (9%)	view
62	x 2	faiss/gpu/impl faiss/gpu/impl	cuh PQScanMultiPassNoPrecomputed-inl.cuh PQScanMultiPassPrecomputed.cu	423:486 (9%) 439:502 (9%)	view
57	x 2	faiss/gpu/impl faiss/gpu/impl	cu Distance.cu GeneralDistance.cuh	185:252 (9%) 294:361 (15%)	view
56	x 2	faiss/gpu/utils faiss/gpu/utils	cuh Tensor.cuh Tensor.cuh	421:486 (9%) 557:622 (9%)	view
54	x 2	faiss/gpu/perf faiss/gpu/perf	cu PerfIVFFlat.cu PerfIVFPQ.cu	87:156 (43%) 97:166 (41%)	view
50	x 2	faiss/gpu/impl faiss/gpu/impl	cu IVFUtilsSelect1.cu IVFUtilsSelect2.cu	113:168 (33%) 179:234 (24%)	view
38	x 2	faiss/gpu/impl faiss/gpu/impl	cu IVFFlatScan.cu PQScanMultiPassNoPrecomputed-inl.cuh	433:476 (8%) 650:693 (5%)	view
38	x 2	faiss/gpu/impl faiss/gpu/impl	cuh PQScanMultiPassNoPrecomputed-inl.cuh PQScanMultiPassPrecomputed.cu	650:693 (5%) 641:684 (5%)	view
35	x 2	faiss/gpu/impl faiss/gpu/impl	cu IVFFlatScan.cu PQScanMultiPassNoPrecomputed-inl.cuh	391:431 (7%) 582:622 (5%)	view
31	x 2	faiss/gpu/impl faiss/gpu/impl	cu IVFFlatScan.cu PQScanMultiPassPrecomputed.cu	365:403 (6%) 573:611 (4%)	view
26	x 2	faiss/gpu/impl faiss/gpu/impl	cuh GpuScalarQuantizer.cuh GpuScalarQuantizer.cuh	301:332 (3%) 439:470 (3%)	view
25	x 2	benchs/link_and_code contrib	datasets.py vecs_io.py	29:65 (16%) 14:50 (100%)	view
25	x 2	faiss/gpu/utils faiss/gpu/utils	cuh Select.cuh Select.cuh	190:219 (4%) 467:496 (4%)	view
25	x 2	faiss/gpu/utils faiss/gpu/utils	cuh MergeNetworkWarp.cuh MergeNetworkWarp.cuh	272:300 (5%) 359:387 (5%)	view
24	x 2	faiss/utils faiss/utils	distances.cpp distances.cpp	177:204 (6%) 246:273 (6%)	view
24	x 2	faiss/gpu/impl faiss/gpu/impl	cuh PQScanMultiPassNoPrecomputed-inl.cuh PQScanMultiPassNoPrecomputed.cuh	523:546 (3%) 21:44 (55%)	view
24	x 2	faiss/gpu/impl faiss/gpu/impl	cuh PQScanMultiPassNoPrecomputed-inl.cuh PQScanMultiPassPrecomputed.cu	596:622 (3%) 613:639 (3%)	view
23	x 2	faiss/gpu/impl faiss/gpu/impl	cu BroadcastSum.cu BroadcastSum.cu	21:48 (7%) 130:157 (7%)	view
23	x 2	faiss/utils faiss/utils	partitioning.cpp partitioning.cpp	951:973 (2%) 959:981 (2%)	view

Duplicated Units

The list of top 20 duplicated units.

See data for all 20 unit duplicates...

Size	#	Folders	Files	Lines	Code
14	x 2	benchs/link_and_code benchs/bench_all_ivf	datasets.py datasets.py	0:0 0:0	view
12	x 2	faiss faiss	IndexPQFastScan.cpp IndexIVFPQFastScan.cpp	175:188 655:668	view
11	x 3	benchs benchs/distributed_ondisk benchs/link_and_code	bench_for_interrupt.py search_server.py datasets.py	0:0 0:0 0:0	view
12	x 2	contrib benchs/distributed_ondisk	rpc.py rpc.py	0:0 0:0	view
9	x 2	contrib benchs/distributed_ondisk	rpc.py rpc.py	0:0 0:0	view
9	x 2	faiss/impl faiss/impl	simd_result_handlers.h ResultHandler.h	380:389 152:161	view
8	x 2	faiss/impl faiss/utils	simd_result_handlers.h partitioning.cpp	347:355 389:397	view
8	x 2	faiss/utils faiss/utils	Heap.h Heap.h	220:228 231:239	view
8	x 2	faiss/utils faiss/utils	Heap.h Heap.h	272:280 283:291	view
7	x 2	faiss/utils faiss/utils	distances_simd.cpp distances_simd.cpp	641:648 690:697	view
7	x 2	faiss/utils faiss/utils	distances_simd.cpp distances_simd.cpp	658:665 699:706	view
7	x 2	faiss/utils faiss/utils	Heap.h Heap.h	143:150 168:175	view
7	x 2	faiss/utils faiss/utils	Heap.h Heap.h	153:160 178:185	view
7	x 2	faiss/invlists faiss/invlists	BlockInvertedLists.cpp InvertedLists.cpp	83:96 194:201	view
7	x 2	benchs/link_and_code benchs/bench_all_ivf	datasets.py datasets.py	0:0 0:0	view
8	x 2	contrib benchs/distributed_ondisk	rpc.py rpc.py	0:0 0:0	view
6	x 2	faiss faiss	IndexHNSW.cpp IndexNSG.cpp	273:280 67:74	view
6	x 2	faiss faiss	IndexIVF.cpp IndexBinaryIVF.cpp	285:291 111:117	view
6	x 3	benchs benchs/distributed_ondisk benchs/link_and_code	bench_for_interrupt.py search_server.py datasets.py	0:0 0:0 0:0	view
8	x 2	contrib benchs/distributed_ondisk	rpc.py rpc.py	0:0 0:0	view