facebookresearch / mmf
Duplication

Places in code with 6 or more lines that are exactly the same.

Intro
  • For duplication, we look at places in code where there are 6 or more lines of code that are exactly the same.
  • Before duplication is calculated, the code is cleaned to remove empty lines, comments, and frequently duplicated constructs such as imports.
  • You should aim at having as little as possible (<5%) of duplicated code as high-level of duplication can lead to maintenance difficulties, poor factoring, and logical contradictions.
Learn more...
Duplication Overall
  • 42% duplication:
    • 52,475 cleaned lines of cleaned code (without empty lines, comments, and frequently duplicated constructs such as imports)
    • 22,108 duplicated lines
  • 12,139 duplicates
system42% (22,108 lines)
Duplication per Extension
yaml84% (14,755 lines)
py21% (7,353 lines)
Duplication per Component (primary)
mmf/projects98% (7,814 lines)
mmf/configs44% (1,989 lines)
projects/krisp100% (1,817 lines)
mmf/models16% (1,674 lines)
mmf/datasets19% (1,259 lines)
mmf/modules14% (675 lines)
projects/visual_bert96% (601 lines)
projects/unit100% (583 lines)
projects/m4c99% (456 lines)
tools/scripts13% (424 lines)
projects/vilbert96% (420 lines)
projects/others97% (407 lines)
projects/m4c_captioner98% (397 lines)
mmf/utils8% (389 lines)
projects/hateful_memes95% (381 lines)
projects/lxmert98% (365 lines)
projects/mmf_transformer100% (357 lines)
projects/pretrain_vl_right96% (344 lines)
projects/vilt100% (333 lines)
projects/pythia100% (321 lines)
projects/mmbt97% (274 lines)
projects/lorra100% (227 lines)
projects/butd100% (206 lines)
projects/movie_mcan100% (133 lines)
projects/uniter100% (119 lines)
projects/vinvl100% (46 lines)
mmf/trainers2% (32 lines)
projects/ban100% (27 lines)
mmf/common1% (14 lines)
tools/sweeps2% (12 lines)
mmf_cli4% (12 lines)
tools0% (0 lines)
ROOT0% (0 lines)
website0% (0 lines)
website/src0% (0 lines)
mmf0% (0 lines)

Duplication Between Components (50+ lines)

G mmf/projects mmf/projects projects/krisp projects/krisp mmf/projects--projects/krisp 4259 projects/visual_bert projects/visual_bert mmf/projects--projects/visual_bert 2362 projects/others projects/others mmf/projects--projects/others 2116 projects/vilbert projects/vilbert mmf/projects--projects/vilbert 2114 projects/mmf_transformer projects/mmf_transformer mmf/projects--projects/mmf_transformer 1989 projects/hateful_memes projects/hateful_memes mmf/projects--projects/hateful_memes 1972 projects/unit projects/unit mmf/projects--projects/unit 1535 projects/mmbt projects/mmbt mmf/projects--projects/mmbt 1365 projects/m4c projects/m4c mmf/projects--projects/m4c 1299 projects/lxmert projects/lxmert mmf/projects--projects/lxmert 1295 projects/pretrain_vl_right projects/pretrain_vl_right mmf/projects--projects/pretrain_vl_right 1174 projects/m4c_captioner projects/m4c_captioner mmf/projects--projects/m4c_captioner 1117 projects/pythia projects/pythia mmf/projects--projects/pythia 884 projects/vilt projects/vilt mmf/projects--projects/vilt 750 projects/lorra projects/lorra mmf/projects--projects/lorra 692 projects/uniter projects/uniter mmf/projects--projects/uniter 617 projects/butd projects/butd mmf/projects--projects/butd 562 projects/movie_mcan projects/movie_mcan mmf/projects--projects/movie_mcan 279 projects/krisp--projects/visual_bert 320 projects/krisp--projects/vilbert 182 projects/others--projects/visual_bert 433 projects/others--projects/vilbert 309 projects/vilbert--projects/visual_bert 771 projects/mmf_transformer--projects/visual_bert 381 projects/mmf_transformer--projects/others 339 projects/mmf_transformer--projects/vilbert 293 projects/hateful_memes--projects/visual_bert 386 projects/hateful_memes--projects/others 540 projects/hateful_memes--projects/vilbert 355 projects/hateful_memes--projects/mmf_transformer 320 projects/hateful_memes--projects/mmbt 329 mmf/configs mmf/configs mmf/configs--mmf/projects 1847 mmf/configs--projects/krisp 221 mmf/configs--projects/visual_bert 277 mmf/configs--projects/vilbert 216 mmf/configs--projects/mmf_transformer 357 mmf/configs--projects/mmbt 156 mmf/configs--projects/m4c 332 mmf/configs--projects/lxmert 505 projects/mmbt--projects/visual_bert 180 projects/mmbt--projects/others 278 projects/mmbt--projects/vilbert 152 projects/mmbt--projects/mmf_transformer 284 projects/lxmert--projects/visual_bert 227 projects/lxmert--projects/vilbert 247 projects/pretrain_vl_right--projects/visual_bert 335 projects/pretrain_vl_right--projects/vilbert 296 projects/lorra--projects/pythia 225 projects/butd--projects/m4c_captioner 268 mmf/datasets mmf/datasets mmf/utils mmf/utils mmf/datasets--mmf/utils 432

Download: SVG DOT (open online Graphviz editor)

Open 3D force graph...

Show more details on duplication between components...
Longest Duplicates
The list of 20 longest duplicates.
See data for all 12,139 duplicates...
Size#FoldersFilesLinesCode
1211 x 2 mmf/projects/krisp
projects/krisp
graphnetwork_module.py
graphnetwork_module.py
 25:1774 (100%)
25:1774 (100%)
 view
199 x 2 mmf/datasets/processors
mmf/utils
processors.py
m4c_evaluators.py
 1181:1385 (19%)
12:216 (70%)
 view
145 x 2 mmf/projects/m4c/scripts
projects/m4c/scripts
extract_ocr_frcn_feature.py
extract_ocr_frcn_feature.py
 21:196 (100%)
21:196 (100%)
 view
107 x 2 mmf/projects/krisp/configs/krisp/vqa2
projects/krisp/configs/krisp/vqa2
krisp_pretrain.yaml
krisp_pretrain.yaml
 1:112 (100%)
1:112 (100%)
 view
105 x 2 mmf/projects/vilt/configs/masked_coco
projects/vilt/configs/masked_coco
pretrain.yaml
pretrain.yaml
 1:111 (100%)
1:111 (100%)
 view
105 x 2 mmf/projects/krisp/configs/krisp/okvqa
projects/krisp/configs/krisp/okvqa
defaults.yaml
defaults.yaml
 1:109 (100%)
1:109 (100%)
 view
103 x 2 mmf/projects/mmbt/configs/mmimdb
projects/mmbt/configs/mmimdb
paper_ablations_reducedlabel.yaml
paper_ablations_reducedlabel.yaml
 1:110 (100%)
1:110 (100%)
 view
95 x 2 mmf/projects/unit/configs/all_8_datasets
projects/unit/configs/all_8_datasets
shared_dec.yaml
shared_dec.yaml
 1:105 (100%)
1:105 (100%)
 view
91 x 2 mmf/projects/m4c/configs/stvqa
projects/m4c/configs/stvqa
defaults.yaml
defaults.yaml
 1:95 (100%)
1:95 (100%)
 view
90 x 2 mmf/projects/m4c/configs/textvqa
projects/m4c/configs/textvqa
defaults.yaml
defaults.yaml
 1:94 (100%)
1:94 (100%)
 view
90 x 2 mmf/projects/m4c/configs/ocrvqa
projects/m4c/configs/ocrvqa
defaults.yaml
defaults.yaml
 1:94 (100%)
1:94 (100%)
 view
88 x 2 mmf/projects/vilt/configs/masked_coco
projects/vilt/configs/masked_coco
defaults.yaml
defaults.yaml
 1:98 (100%)
1:98 (100%)
 view
86 x 2 mmf/projects/vilt/configs/vqa2
projects/vilt/configs/vqa2
defaults.yaml
defaults.yaml
 1:97 (100%)
1:97 (100%)
 view
75 x 2 mmf/projects/butd/configs/textcaps
projects/m4c_captioner/configs/butd/textcaps
defaults.yaml
defaults.yaml
 1:80 (100%)
1:80 (100%)
 view
75 x 2 mmf/projects/butd/configs/textcaps
projects/butd/configs/textcaps
defaults.yaml
defaults.yaml
 1:80 (100%)
1:80 (100%)
 view
75 x 2 mmf/projects/butd/configs/textcaps
mmf/projects/m4c_captioner/configs/butd/textcaps
defaults.yaml
defaults.yaml
 1:80 (100%)
1:80 (100%)
 view
75 x 2 projects/butd/configs/textcaps
projects/m4c_captioner/configs/butd/textcaps
defaults.yaml
defaults.yaml
 1:80 (100%)
1:80 (100%)
 view
75 x 2 mmf/projects/m4c_captioner/configs/butd/textcaps
projects/butd/configs/textcaps
defaults.yaml
defaults.yaml
 1:80 (100%)
1:80 (100%)
 view
75 x 2 mmf/projects/m4c_captioner/configs/butd/textcaps
projects/m4c_captioner/configs/butd/textcaps
defaults.yaml
defaults.yaml
 1:80 (100%)
1:80 (100%)
 view
70 x 2 mmf/projects/lorra/configs/vqa2
projects/lorra/configs/vqa2
defaults.yaml
defaults.yaml
 2:76 (100%)
2:76 (100%)
 view
Duplicated Units
The list of top 20 duplicated units.
See data for all 57 unit duplicates...
Size#FoldersFilesLinesCode
346 x 2 projects/krisp
mmf/projects/krisp
graphnetwork_module.py
graphnetwork_module.py
 0:0 
0:0 
 view
184 x 2 projects/krisp
mmf/projects/krisp
graphnetwork_module.py
graphnetwork_module.py
 0:0 
0:0 
 view
188 x 2 projects/krisp
mmf/projects/krisp
graphnetwork_module.py
graphnetwork_module.py
 0:0 
0:0 
 view
106 x 2 projects/krisp
mmf/projects/krisp
graphnetwork_module.py
graphnetwork_module.py
 0:0 
0:0 
 view
71 x 2 projects/m4c/scripts
mmf/projects/m4c/scripts
extract_ocr_frcn_feature.py
extract_ocr_frcn_feature.py
 0:0 
0:0 
 view
98 x 2 projects/krisp
mmf/projects/krisp
graphnetwork_module.py
graphnetwork_module.py
 0:0 
0:0 
 view
22 x 2 projects/m4c/scripts
mmf/projects/m4c/scripts
extract_ocr_frcn_feature.py
extract_ocr_frcn_feature.py
 0:0 
0:0 
 view
21 x 2 projects/krisp
mmf/projects/krisp
graphnetwork_module.py
graphnetwork_module.py
 0:0 
0:0 
 view
21 x 2 projects/m4c/scripts
mmf/projects/m4c/scripts
extract_ocr_frcn_feature.py
extract_ocr_frcn_feature.py
 0:0 
0:0 
 view
19 x 2 mmf/datasets/builders/mmimdb
mmf/datasets/builders/gqa
masked_dataset.py
masked_dataset.py
 0:0 
0:0 
 view
18 x 2 projects/m4c/scripts
mmf/projects/m4c/scripts
extract_ocr_frcn_feature.py
extract_ocr_frcn_feature.py
 0:0 
0:0 
 view
17 x 2 tools/scripts/features/frcnn
mmf/utils/features
frcnn_utils.py
visualizing_image.py
 0:0 
0:0 
 view
24 x 2 projects/krisp
mmf/projects/krisp
graphnetwork_module.py
graphnetwork_module.py
 0:0 
0:0 
 view
14 x 4 projects/m4c_captioner/scripts
projects/m4c_captioner/scripts
mmf/projects/m4c_captioner/scripts
mmf/projects/m4c_captioner/scripts
coco_eval.py
textcaps_eval.py
coco_eval.py
textcaps_eval.py
 0:0 
0:0 
0:0 
0:0 
 view
20 x 2 mmf/datasets/processors
mmf/datasets/processors
processors.py
processors.py
 0:0 
0:0 
 view
13 x 2 tools/scripts/features
tools/scripts/features
extract_features_vmb.py
extract_features_vinvl.py
 0:0 
0:0 
 view
13 x 2 mmf/datasets/processors
mmf/utils
processors.py
m4c_evaluators.py
 0:0 
0:0 
 view
11 x 2 tools/scripts/features/frcnn
mmf/datasets/processors
processing_image.py
frcnn_processor.py
 0:0 
0:0 
 view
13 x 2 projects/krisp
mmf/projects/krisp
graphnetwork_module.py
graphnetwork_module.py
 0:0 
0:0 
 view
10 x 2 mmf/datasets/builders/gqa
mmf/datasets/builders/vqa2
builder.py
builder.py
 0:0 
0:0 
 view