alibaba / graphlearn-for-pytorch

Intro

For duplication, we look at places in code where there are 6 or more lines of code that are exactly the same.
Before duplication is calculated, the code is cleaned to remove empty lines, comments, and frequently duplicated constructs such as imports.
You should aim at having as little as possible (<5%) of duplicated code as high-level of duplication can lead to maintenance difficulties, poor factoring, and logical contradictions.

Learn more...

Duplication Overall

13% duplication:

11,400 cleaned lines of cleaned code (without empty lines, comments, and frequently duplicated constructs such as imports)
1,559 duplicated lines

221 duplicates

dependency graphs: 2D graph | 3D graph | 3D graph (with duplicates)...

Duplication per Extension

Duplication per Component (primary)

Longest Duplicates

The list of 50 longest duplicates.

See data for all 221 duplicates...

Size	#	Folders	Files	Lines	Code
21	x 2	graphlearn_torch/csrc/cuda graphlearn_torch/csrc/cuda	random_sampler.cu random_sampler.cu	278:302 (6%) 321:345 (6%)	view
20	x 2	graphlearn_torch/csrc/cuda graphlearn_torch/csrc/cuda	random_sampler.cu random_sampler.cu	85:104 (5%) 140:159 (5%)	view
17	x 2	graphlearn_torch/python/distributed graphlearn_torch/python/distributed	dist_random_partitioner.py dist_table_dataset.py	173:189 (4%) 83:99 (7%)	view
16	x 2	graphlearn_torch/python/distributed graphlearn_torch/python/distributed	dist_dataset.py dist_table_dataset.py	154:171 (7%) 328:345 (7%)	view
16	x 2	graphlearn_torch/csrc/cuda graphlearn_torch/csrc/cuda	random_sampler.cu random_sampler.cu	68:84 (4%) 122:138 (4%)	view
16	x 2	graphlearn_torch/python/distributed graphlearn_torch/python/sampler	dist_neighbor_sampler.py neighbor_sampler.py	451:468 (2%) 370:387 (2%)	view
15	x 2	graphlearn_torch/python/data graphlearn_torch/python/distributed	table_dataset.py dist_table_dataset.py	83:97 (13%) 225:239 (6%)	view
15	x 2	graphlearn_torch/python/partition graphlearn_torch/python/partition	base.py random_partitioner.py	193:208 (2%) 29:58 (35%)	view
14	x 2	graphlearn_torch/python/distributed graphlearn_torch/python/distributed	dist_neighbor_loader.py dist_subgraph_loader.py	78:92 (31%) 69:82 (48%)	view
14	x 2	graphlearn_torch/csrc/cpu graphlearn_torch/csrc/cpu	random_sampler.cc weighted_sampler.cc	76:91 (11%) 82:97 (10%)	view
14	x 2	graphlearn_torch/python/data graphlearn_torch/python/distributed	table_dataset.py dist_table_dataset.py	112:125 (12%) 259:272 (6%)	view
14	x 2	benchmarks/api benchmarks/api	bench_dist_neighbor_loader.py run_dist_bench.py	37:50 (10%) 26:39 (21%)	view
13	x 2	graphlearn_torch/python/distributed graphlearn_torch/python/sampler	dist_neighbor_sampler.py neighbor_sampler.py	489:501 (2%) 410:422 (2%)	view
13	x 2	graphlearn_torch/python/partition graphlearn_torch/python/partition	base.py base.py	140:153 (2%) 159:172 (2%)	view
12	x 2	graphlearn_torch/python/distributed graphlearn_torch/python/sampler	dist_neighbor_sampler.py neighbor_sampler.py	335:347 (1%) 277:288 (2%)	view
12	x 2	graphlearn_torch/python/loader graphlearn_torch/python/loader	neighbor_loader.py subgraph_loader.py	89:101 (22%) 79:91 (26%)	view
12	x 2	graphlearn_torch/csrc/cuda graphlearn_torch/csrc/cuda	random_negative_sampler.cu random_sampler.cu	1:16 (7%) 1:16 (3%)	view
11	x 2	graphlearn_torch/python/distributed graphlearn_torch/python/loader	dist_link_neighbor_loader.py link_loader.py	130:142 (22%) 120:132 (10%)	view
11	x 2	graphlearn_torch/csrc/cuda graphlearn_torch/csrc/cuda	inducer.cu random_negative_sampler.cu	1:14 (3%) 1:14 (6%)	view
11	x 2	graphlearn_torch/csrc/cuda graphlearn_torch/csrc/cuda	random_sampler.cu stitch_sample_results.cu	1:14 (3%) 1:14 (11%)	view
11	x 2	graphlearn_torch/csrc/cuda graphlearn_torch/include	random_negative_sampler.cu unified_tensor.cuh	1:14 (6%) 1:14 (16%)	view
11	x 2	graphlearn_torch/csrc/cuda graphlearn_torch/include	weighted_sampler.cuh hash_table.cuh	1:14 (36%) 1:14 (8%)	view
11	x 2	graphlearn_torch/csrc/cuda graphlearn_torch/include	random_sampler.cuh common.cuh	1:14 (37%) 1:14 (16%)	view
11	x 2	graphlearn_torch/csrc/cuda graphlearn_torch/include	graph.cu unified_tensor.cuh	1:14 (8%) 1:14 (16%)	view
11	x 2	graphlearn_torch/csrc/cuda graphlearn_torch/csrc/cuda	inducer.cu unified_tensor.cu	1:14 (3%) 1:14 (3%)	view
11	x 2	graphlearn_torch/csrc/cuda graphlearn_torch/csrc/cuda	random_sampler.cuh stitch_sample_results.cu	1:14 (37%) 1:14 (11%)	view
11	x 2	graphlearn_torch/csrc/cuda graphlearn_torch/csrc/cuda	hash_table.cu random_negative_sampler.cu	1:14 (11%) 1:14 (6%)	view
11	x 2	graphlearn_torch/csrc/cuda graphlearn_torch/include	random_sampler.cuh hash_table.cuh	1:14 (37%) 1:14 (8%)	view
11	x 2	graphlearn_torch/csrc/cuda graphlearn_torch/include	random_negative_sampler.cu hash_table.cuh	1:14 (6%) 1:14 (8%)	view
11	x 2	graphlearn_torch/csrc/cuda graphlearn_torch/csrc/cuda	random_negative_sampler.cu subgraph_op.cu	1:14 (6%) 1:14 (4%)	view
11	x 2	graphlearn_torch/csrc/cuda graphlearn_torch/csrc/cuda	random_negative_sampler.cuh random_sampler.cu	1:14 (45%) 1:14 (3%)	view
11	x 2	graphlearn_torch/csrc/cuda graphlearn_torch/include	subgraph_op.cu unified_tensor.cuh	1:14 (4%) 1:14 (16%)	view
11	x 2	graphlearn_torch/csrc/cuda graphlearn_torch/include	subgraph_op.cu hash_table.cuh	1:14 (4%) 1:14 (8%)	view
11	x 2	graphlearn_torch/csrc/cuda graphlearn_torch/csrc/cuda	graph.cu random_sampler.cu	1:14 (8%) 1:14 (3%)	view
11	x 2	graphlearn_torch/csrc/cuda graphlearn_torch/csrc/cuda	graph.cu inducer.cuh	1:14 (8%) 1:14 (14%)	view
11	x 2	graphlearn_torch/csrc/cuda graphlearn_torch/csrc/cuda	graph.cu random_negative_sampler.cuh	1:14 (8%) 1:14 (45%)	view
11	x 2	graphlearn_torch/csrc/cuda graphlearn_torch/include	subgraph_op.cuh common.cuh	1:14 (22%) 1:14 (16%)	view
11	x 2	graphlearn_torch/csrc/cuda graphlearn_torch/include	random_negative_sampler.cuh unified_tensor.cuh	1:14 (45%) 1:14 (16%)	view
11	x 2	graphlearn_torch/csrc/cuda graphlearn_torch/csrc/cuda	inducer.cu subgraph_op.cu	1:14 (3%) 1:14 (4%)	view
11	x 2	graphlearn_torch/csrc/cuda graphlearn_torch/csrc/cuda	hash_table.cu unified_tensor.cu	1:14 (11%) 1:14 (3%)	view
11	x 2	graphlearn_torch/csrc/cuda graphlearn_torch/csrc/cuda	hash_table.cu random_sampler.cu	1:14 (11%) 1:14 (3%)	view
11	x 2	graphlearn_torch/csrc/cuda graphlearn_torch/csrc/cuda	graph.cu subgraph_op.cu	1:14 (8%) 1:14 (4%)	view
11	x 2	graphlearn_torch/csrc/cuda graphlearn_torch/csrc/cuda	stitch_sample_results.cu weighted_sampler.cuh	1:14 (11%) 1:14 (36%)	view
11	x 2	graphlearn_torch/csrc/cuda graphlearn_torch/csrc/cuda	graph.cu random_sampler.cuh	1:14 (8%) 1:14 (37%)	view
11	x 2	graphlearn_torch/csrc/cuda graphlearn_torch/csrc/cuda	random_sampler.cuh subgraph_op.cu	1:14 (37%) 1:14 (4%)	view
11	x 2	graphlearn_torch/csrc/cuda graphlearn_torch/csrc/cuda	subgraph_op.cuh unified_tensor.cu	1:14 (22%) 1:14 (3%)	view
11	x 2	graphlearn_torch/csrc/cuda graphlearn_torch/csrc/cuda	unified_tensor.cu weighted_sampler.cuh	1:14 (3%) 1:14 (36%)	view
11	x 2	graphlearn_torch/python/distributed graphlearn_torch/python/sampler	dist_neighbor_sampler.py neighbor_sampler.py	470:482 (1%) 389:400 (1%)	view
11	x 2	graphlearn_torch/csrc/cuda graphlearn_torch/include	random_sampler.cu common.cuh	1:14 (3%) 1:14 (16%)	view
11	x 2	graphlearn_torch/csrc/cuda graphlearn_torch/csrc/cuda	random_sampler.cu random_sampler.cuh	1:14 (3%) 1:14 (37%)	view

Duplicated Units

The list of top 1 duplicated units.

See data for all 1 unit duplicate

Size	#	Folders	Files	Lines	Code
16	x 2	graphlearn_torch/csrc/cpu graphlearn_torch/csrc/cpu	random_sampler.cc weighted_sampler.cc	75:92 81:98	view