facebookresearch / fairscale
Duplication

Places in code with 6 or more lines that are exactly the same.

Intro
  • For duplication, we look at places in code where there are 6 or more lines of code that are exactly the same.
  • Before duplication is calculated, the code is cleaned to remove empty lines, comments, and frequently duplicated constructs such as imports.
  • You should aim at having as little as possible (<5%) of duplicated code as high-level of duplication can lead to maintenance difficulties, poor factoring, and logical contradictions.
Learn more...
Duplication Overall
  • 5% duplication:
    • 16,979 cleaned lines of cleaned code (without empty lines, comments, and frequently duplicated constructs such as imports)
    • 869 duplicated lines
  • 59 duplicates
system5% (869 lines)
Duplication per Extension
py6% (803 lines)
cu15% (38 lines)
pyi<1% (28 lines)
Duplication per Component (primary)
fairscale/nn5% (301 lines)
fairscale/experimental5% (193 lines)
benchmarks/experimental13% (141 lines)
benchmarks12% (83 lines)
benchmarks/golden_configs35% (39 lines)
fairscale/clib10% (38 lines)
benchmarks/models27% (30 lines)
stubs/torch<1% (28 lines)
fairscale/optim1% (16 lines)
ROOT0% (0 lines)
fairscale0% (0 lines)
fairscale/utils0% (0 lines)
benchmarks/datasets0% (0 lines)

Duplication Between Components (50+ lines)

G benchmarks/experimental benchmarks/experimental benchmarks benchmarks benchmarks/experimental--benchmarks 146 benchmarks/models benchmarks/models benchmarks/experimental--benchmarks/models 60 fairscale/experimental fairscale/experimental fairscale/nn fairscale/nn fairscale/experimental--fairscale/nn 50

Download: SVG DOT (open online Graphviz editor)

Open 3D force graph...

Show more details on duplication between components...
Longest Duplicates
The list of 20 longest duplicates.
See data for all 59 duplicates...
Size#FoldersFilesLinesCode
20 x 2 fairscale/nn/checkpoint
fairscale/nn/pipe
checkpoint_activations.py
checkpoint.py
 45:95 (12%)
128:181 (12%)
 view
14 x 2 benchmarks/experimental
benchmarks/models
experimental_async_approaches.py
transformer_lm.py
 63:78 (2%)
28:45 (12%)
 view
13 x 2 fairscale/experimental/nn/data_parallel/gossip
fairscale/experimental/nn/data_parallel/gossip
graph_manager.py
graph_manager.py
 202:219 (6%)
252:269 (6%)
 view
13 x 2 benchmarks/golden_configs
benchmarks/golden_configs
lm_wikitext2.py
lm_wikitext2.py
 12:24 (12%)
70:82 (12%)
 view
12 x 2 fairscale/experimental/nn/data_parallel/gossip
fairscale/experimental/nn/data_parallel/gossip
distributed.py
distributed.py
 690:708 (1%)
775:794 (1%)
 view
12 x 2 benchmarks/experimental
benchmarks
experimental_async_approaches.py
pipe.py
 695:706 (2%)
248:259 (5%)
 view
11 x 2 benchmarks/experimental
benchmarks
offload.py
pipe.py
 171:183 (3%)
85:97 (4%)
 view
11 x 2 fairscale/clib/fused_adam_cuda
fairscale/clib/fused_adam_cuda
cu
fused_adam_cuda_kernel.cu
fused_adam_cuda_kernel.cu
 227:237 (4%)
246:256 (4%)
 view
11 x 2 fairscale/nn/pipe
fairscale/nn/pipe
async_pipe.py
pipe.py
 338:353 (5%)
328:343 (6%)
 view
11 x 2 fairscale/nn/data_parallel
fairscale/nn/data_parallel
sharded_ddp.py
sharded_ddp.py
 479:489 (3%)
637:647 (3%)
 view
10 x 2 benchmarks/experimental
benchmarks
experimental_async_approaches.py
pipe.py
 637:648 (1%)
174:185 (4%)
 view
10 x 2 fairscale/experimental/nn
fairscale/nn/misc
ssd_offload.py
flatten_params_wrapper.py
 313:327 (5%)
95:109 (3%)
 view
10 x 2 benchmarks/experimental
benchmarks/models
experimental_async_approaches.py
transformer_lm.py
 98:110 (1%)
178:192 (9%)
 view
10 x 2 fairscale/nn/model_parallel
fairscale/nn/model_parallel
layers.py
layers.py
 95:104 (4%)
163:172 (4%)
 view
9 x 2 fairscale/nn/pipe
fairscale/nn/pipe
copy.py
copy.py
 46:55 (14%)
107:116 (14%)
 view
9 x 2 benchmarks/experimental
benchmarks/experimental
experimental_async_approaches.py
experimental_async_approaches.py
 192:201 (1%)
327:336 (1%)
 view
9 x 2 fairscale/nn/pipe
fairscale/nn/pipe
checkpoint.py
checkpoint.py
 247:256 (5%)
293:302 (5%)
 view
9 x 2 fairscale/experimental/nn/data_parallel/gossip
fairscale/experimental/nn/data_parallel/gossip
graph_manager.py
graph_manager.py
 154:166 (4%)
230:242 (4%)
 view
8 x 2 fairscale/optim
fairscale/optim
adam.py
grad_scaler.py
 32:40 (4%)
32:40 (3%)
 view
8 x 2 fairscale/experimental/nn/data_parallel/gossip
fairscale/experimental/n...a_parallel/gossip/utils
distributed.py
__init__.py
 31:38 (1%)
7:14 (100%)
 view
Duplicated Units
The list of top 7 duplicated units.
See data for all 7 unit duplicates...
Size#FoldersFilesLinesCode
12 x 2 benchmarks/golden_configs
benchmarks/golden_configs
lm_wikitext2.py
lm_wikitext2.py
 0:0 
0:0 
 view
11 x 2 benchmarks/experimental
benchmarks
experimental_async_approaches.py
pipe.py
 0:0 
0:0 
 view
9 x 2 benchmarks/experimental
benchmarks/models
experimental_async_approaches.py
transformer_lm.py
 0:0 
0:0 
 view
7 x 2 fairscale/nn/pipe
fairscale/nn/pipe
checkpoint.py
checkpoint.py
 0:0 
0:0 
 view
7 x 2 fairscale/nn/pipe
fairscale/nn/checkpoint
checkpoint.py
checkpoint_activations.py
 0:0 
0:0 
 view
7 x 2 fairscale/nn/pipe
fairscale/nn/checkpoint
checkpoint.py
checkpoint_activations.py
 0:0 
0:0 
 view
6 x 2 fairscale/nn/pipe/skip
fairscale/nn/pipe/skip
tracker.py
tracker.py
 0:0 
0:0 
 view