pytorch / data
Duplication

Places in code with 6 or more lines that are exactly the same.

Intro
  • For duplication, we look at places in code where there are 6 or more lines of code that are exactly the same.
  • Before duplication is calculated, the code is cleaned to remove empty lines, comments, and frequently duplicated constructs such as imports.
  • You should aim at having as little as possible (<5%) of duplicated code as high-level of duplication can lead to maintenance difficulties, poor factoring, and logical contradictions.
Learn more...
Duplication Overall
  • 9% duplication:
    • 1,321 cleaned lines of cleaned code (without empty lines, comments, and frequently duplicated constructs such as imports)
    • 126 duplicated lines
  • 14 duplicates
system9% (126 lines)
Duplication per Extension
py9% (126 lines)
Duplication per Component (primary)
torchdata/datapipes/iter/util9% (81 lines)
torchdata/datapipes/iter/load18% (45 lines)
ROOT0% (0 lines)
torchdata0% (0 lines)
torchdata/datapipes/iter0% (0 lines)
torchdata/datapipes/iter/transform0% (0 lines)
torchdata/datapipes0% (0 lines)
torchdata/datapipes/utils0% (0 lines)
torchdata/datapipes/map0% (0 lines)
torchdata/datapipes/map/load0% (0 lines)
torchdata/datapipes/map/transform0% (0 lines)
Longest Duplicates
The list of 14 longest duplicates.
See data for all 14 duplicates...
Size#FoldersFilesLinesCode
14 x 2 torchdata/datapipes/iter/util
torchdata/datapipes/iter/util
plain_text_reader.py
plain_text_reader.py
147:175 (9%)
188:218 (9%)
view
10 x 2 torchdata/datapipes/iter/util
torchdata/datapipes/iter/util
xzfilereader.py
ziparchivereader.py
16:39 (45%)
18:43 (34%)
view
7 x 2 torchdata/datapipes/iter/load
torchdata/datapipes/iter/load
online.py
online.py
83:97 (9%)
105:120 (9%)
view
7 x 2 torchdata/datapipes/iter/util
torchdata/datapipes/iter/util
plain_text_reader.py
plain_text_reader.py
177:183 (4%)
220:226 (4%)
view
7 x 2 torchdata/datapipes/iter/load
torchdata/datapipes/iter/util
fsspec.py
saver.py
107:123 (9%)
14:29 (28%)
view
6 x 2 torchdata/datapipes/iter/util
torchdata/datapipes/iter/util
plain_text_reader.py
plain_text_reader.py
69:88 (3%)
188:210 (3%)
view
6 x 2 torchdata/datapipes/iter/load
torchdata/datapipes/iter/load
iopath.py
iopath.py
60:65 (6%)
150:155 (6%)
view
6 x 2 torchdata/datapipes/iter/load
torchdata/datapipes/iter/load
iopath.py
iopath.py
101:106 (6%)
150:155 (6%)
view
6 x 2 torchdata/datapipes/iter/load
torchdata/datapipes/iter/util
iopath.py
saver.py
126:146 (6%)
14:28 (24%)
view
6 x 2 torchdata/datapipes/iter/util
torchdata/datapipes/iter/util
tararchivereader.py
xzfilereader.py
38:44 (20%)
33:39 (27%)
view
6 x 2 torchdata/datapipes/iter/load
torchdata/datapipes/iter/load
iopath.py
iopath.py
60:65 (6%)
101:106 (6%)
view
6 x 2 torchdata/datapipes/iter/util
torchdata/datapipes/iter/util
tararchivereader.py
ziparchivereader.py
38:44 (20%)
37:43 (20%)
view
6 x 2 torchdata/datapipes/iter/load
torchdata/datapipes/iter/load
fsspec.py
iopath.py
107:122 (7%)
126:146 (6%)
view
6 x 2 torchdata/datapipes/iter/util
torchdata/datapipes/iter/util
plain_text_reader.py
plain_text_reader.py
69:88 (3%)
147:167 (3%)
view
Duplicated Units
The list of top 1 duplicated units.
See data for all 1 unit duplicate
Size#FoldersFilesLinesCode
9 x 2 torchdata/datapipes/iter/util
torchdata/datapipes/iter/util
plain_text_reader.py
plain_text_reader.py
0:0 
0:0 
view