facebookresearch / torchrec
Duplication

Places in code with 6 or more lines that are exactly the same.

Intro
  • For duplication, we look at places in code where there are 6 or more lines of code that are exactly the same.
  • Before duplication is calculated, the code is cleaned to remove empty lines, comments, and frequently duplicated constructs such as imports.
  • You should aim at having as little as possible (<5%) of duplicated code as high-level of duplication can lead to maintenance difficulties, poor factoring, and logical contradictions.
Learn more...
Duplication Overall
  • 21% duplication:
    • 12,334 cleaned lines of cleaned code (without empty lines, comments, and frequently duplicated constructs such as imports)
    • 2,650 duplicated lines
  • 284 duplicates
system21% (2,650 lines)
Duplication per Extension
py21% (2,630 lines)
h16% (10 lines)
cpp3% (10 lines)
Duplication per Component (primary)
torchrec/distributed25% (2,168 lines)
torchrec/modules23% (186 lines)
torchrec/sparse12% (140 lines)
torchrec/models36% (89 lines)
torchrec/quant20% (37 lines)
torchrec/datasets3% (30 lines)
torchrec/linter0% (0 lines)
torchrec/fx0% (0 lines)
torchrec0% (0 lines)
torchrec/optim0% (0 lines)
ROOT0% (0 lines)

Duplication Between Components (50+ lines)

G torchrec/distributed torchrec/distributed torchrec/quant torchrec/quant torchrec/distributed--torchrec/quant 156

Download: SVG DOT (open online Graphviz editor)

Open 3D force graph...

Show more details on duplication between components...
Longest Duplicates
The list of 20 longest duplicates.
See data for all 284 duplicates...
Size#FoldersFilesLinesCode
46 x 2 torchrec/distributed
torchrec/distributed
batched_embedding_kernel.py
batched_embedding_kernel.py
 390:442 (7%)
644:696 (7%)
 view
36 x 2 torchrec/distributed
torchrec/distributed
batched_embedding_kernel.py
batched_embedding_kernel.py
 351:388 (6%)
605:642 (6%)
 view
34 x 2 torchrec/distributed
torchrec/distributed
embedding_kernel.py
embedding_kernel.py
 162:198 (12%)
308:344 (12%)
 view
31 x 2 torchrec/distributed
torchrec/distributed
batched_embedding_kernel.py
batched_embedding_kernel.py
 312:347 (5%)
566:601 (5%)
 view
31 x 2 torchrec/distributed
torchrec/distributed
dp_sharding.py
rw_sharding.py
 237:271 (16%)
314:348 (9%)
 view
26 x 2 torchrec/distributed/planner
torchrec/distributed/planner
shard_estimators.py
shard_estimators.py
 223:248 (4%)
252:277 (4%)
 view
25 x 2 torchrec/distributed
torchrec/distributed
embeddingbag.py
quant_embeddingbag.py
 307:333 (3%)
107:133 (17%)
 view
25 x 2 torchrec/distributed
torchrec/distributed
rw_sharding.py
tw_sharding.py
 277:303 (7%)
243:269 (7%)
 view
23 x 2 torchrec/distributed
torchrec/distributed
train_pipeline.py
utils.py
 319:343 (6%)
49:73 (31%)
 view
20 x 2 torchrec/distributed
torchrec/distributed
batched_embedding_kernel.py
batched_embedding_kernel.py
 459:481 (3%)
713:735 (3%)
 view
19 x 2 torchrec/distributed/planner
torchrec/distributed/planner
shard_estimators.py
shard_estimators.py
 694:714 (2%)
733:753 (2%)
 view
16 x 2 torchrec/distributed
torchrec/distributed
rw_sharding.py
twrw_sharding.py
 73:88 (5%)
134:149 (3%)
 view
14 x 2 torchrec/models
torchrec/models
deepfm.py
dlrm.py
 282:295 (12%)
314:327 (11%)
 view
14 x 2 torchrec/distributed
torchrec/distributed
rw_sharding.py
twrw_sharding.py
 172:190 (4%)
288:305 (3%)
 view
14 x 2 torchrec/modules
torchrec/modules
crossnet.py
crossnet.py
 74:89 (6%)
160:175 (6%)
 view
14 x 2 torchrec/modules
torchrec/modules
crossnet.py
crossnet.py
 74:89 (6%)
235:250 (6%)
 view
14 x 2 torchrec/distributed
torchrec/distributed
batched_embedding_kernel.py
batched_embedding_kernel.py
 292:306 (2%)
537:551 (2%)
 view
14 x 2 torchrec/modules
torchrec/modules
crossnet.py
crossnet.py
 160:175 (6%)
235:250 (6%)
 view
13 x 2 torchrec/distributed
torchrec/distributed
dp_sharding.py
rw_sharding.py
 130:142 (7%)
197:209 (4%)
 view
13 x 2 torchrec/distributed
torchrec/distributed
embeddingbag.py
embeddingbag.py
 480:493 (1%)
767:781 (1%)
 view
Duplicated Units
The list of top 15 duplicated units.
See data for all 15 unit duplicates...
Size#FoldersFilesLinesCode
14 x 2 torchrec/distributed
torchrec/distributed
batched_embedding_kernel.py
batched_embedding_kernel.py
 0:0 
0:0 
 view
10 x 2 torchrec/quant
torchrec/distributed
embedding_modules.py
quant_embedding_kernel.py
 0:0 
0:0 
 view
10 x 2 torchrec/sparse
torchrec/sparse
jagged_tensor.py
jagged_tensor.py
 0:0 
0:0 
 view
8 x 2 torchrec/distributed
torchrec/distributed
batched_embedding_kernel.py
batched_embedding_kernel.py
 0:0 
0:0 
 view
7 x 2 torchrec/distributed
torchrec/distributed
twrw_sharding.py
rw_sharding.py
 0:0 
0:0 
 view
7 x 2 torchrec/distributed/planner
torchrec/distributed/planner
types.py
stats.py
 0:0 
0:0 
 view
7 x 2 torchrec/distributed/planner
torchrec/distributed/planner
shard_estimators.py
shard_estimators.py
 0:0 
0:0 
 view
6 x 2 torchrec/distributed
torchrec/distributed
quant_embeddingbag.py
embeddingbag.py
 0:0 
0:0 
 view
6 x 2 torchrec/distributed/planner
torchrec/distributed/planner
partitioners.py
partitioners.py
 0:0 
0:0 
 view
6 x 2 torchrec/distributed/planner
torchrec/distributed/planner
shard_estimators.py
shard_estimators.py
 0:0 
0:0 
 view
6 x 2 torchrec/distributed
torchrec/modules
embedding_kernel.py
embedding_modules.py
 0:0 
0:0 
 view
6 x 2 torchrec/distributed
torchrec/distributed
dp_sharding.py
rw_sharding.py
 0:0 
0:0 
 view
6 x 2 torchrec/distributed
torchrec/distributed
dp_sharding.py
rw_sharding.py
 0:0 
0:0 
 view
6 x 2 torchrec/distributed
torchrec/distributed
dp_sharding.py
rw_sharding.py
 0:0 
0:0 
 view
7 x 2 torchrec/datasets
torchrec/datasets
movielens.py
movielens.py
 0:0 
0:0 
 view