tensorflow / io
Duplication

Places in code with 6 or more lines that are exactly the same.

Intro
  • For duplication, we look at places in code where there are 6 or more lines of code that are exactly the same.
  • Before duplication is calculated, the code is cleaned to remove empty lines, comments, and frequently duplicated constructs such as imports.
  • You should aim at having as little as possible (<5%) of duplicated code as high-level of duplication can lead to maintenance difficulties, poor factoring, and logical contradictions.
Learn more...
Duplication Overall
  • 14% duplication:
    • 39,741 cleaned lines of cleaned code (without empty lines, comments, and frequently duplicated constructs such as imports)
    • 5,943 duplicated lines
  • 1,266 duplicates
system14% (5,943 lines)
Duplication per Extension
cc17% (4,569 lines)
py11% (1,196 lines)
h3% (82 lines)
Dockerfile40% (36 lines)
tpl17% (34 lines)
R2% (14 lines)
go14% (12 lines)
Duplication per Component (primary)
tensorflow_io/core16% (4,523 lines)
tensorflow_io/python12% (1,184 lines)
tensorflow_io_gcs_filesystem/core10% (152 lines)
tools/docker40% (36 lines)
tools/lint12% (34 lines)
R-package/R2% (14 lines)
R-package/scripts0% (0 lines)
R-package0% (0 lines)
R-package/pkgdown0% (0 lines)
ROOT0% (0 lines)
tools/build0% (0 lines)
tensorflow_io0% (0 lines)
tensorflow_io_gcs_filesystem0% (0 lines)

Duplication Between Components (50+ lines)

G tensorflow_io/core tensorflow_io/core tensorflow_io_gcs_filesystem/core tensorflow_io_gcs_filesystem/core tensorflow_io/core--tensorflow_io_gcs_filesystem/core 115

Download: SVG DOT (open online Graphviz editor)

Open 3D force graph...

Show more details on duplication between components...
Longest Duplicates
The list of 20 longest duplicates.
See data for all 1,266 duplicates...
Size#FoldersFilesLinesCode
42 x 2 tensorflow_io/core/kernels
tensorflow_io/core/kernels
kafka_kernels.cc
kafka_kernels_deprecated.cc
 576:630 (5%)
665:721 (4%)
 view
42 x 2 tensorflow_io/core/kernels/arrow
tensorflow_io/core/kernels/arrow
arrow_kernels.cc
arrow_kernels.cc
 447:490 (7%)
589:632 (7%)
 view
34 x 2 tensorflow_io/core/kernels
tensorflow_io/core/kernels
hdf5_kernels.cc
parquet_kernels.cc
 663:709 (6%)
366:411 (10%)
 view
34 x 2 tensorflow_io/core/kernels
tensorflow_io/core/kernels
avro_kernels.cc
json_kernels.cc
 418:462 (7%)
135:179 (17%)
 view
34 x 2 tensorflow_io/core/kernels
tensorflow_io/core/kernels
csv_kernels.cc
json_kernels.cc
 210:243 (15%)
214:247 (17%)
 view
34 x 2 tensorflow_io/python/experimental
tensorflow_io/python/experimental
kafka_batch_io_dataset_ops.py
kafka_group_io_dataset_ops.py
 108:146 (53%)
152:190 (59%)
 view
31 x 2 tensorflow_io/core/kernels
tensorflow_io/core/kernels
hdf5_kernels.cc
parquet_kernels.cc
 604:644 (5%)
312:352 (9%)
 view
30 x 2 tensorflow_io/python/ops
tensorflow_io/python/ops
avro_dataset_ops.py
json_dataset_ops.py
 75:108 (41%)
75:108 (41%)
 view
30 x 2 tensorflow_io/core/kernels
tensorflow_io/core/kernels
audio_video_ogg_kernels.cc
audio_video_wav_kernels.cc
 202:237 (10%)
314:349 (8%)
 view
30 x 2 tensorflow_io/core/kernels
tensorflow_io/core/kernels
audio_video_mp3_kernels.cc
audio_video_mp4_kernels.cc
 209:244 (10%)
482:517 (4%)
 view
30 x 2 tensorflow_io/core/kernels
tensorflow_io/core/kernels
audio_video_mp3_kernels.cc
audio_video_ogg_kernels.cc
 209:244 (10%)
202:237 (10%)
 view
30 x 2 tensorflow_io/core/kernels
tensorflow_io/core/kernels
audio_video_mp4_kernels.cc
audio_video_ogg_kernels.cc
 482:517 (4%)
202:237 (10%)
 view
30 x 2 tensorflow_io/python/ops
tensorflow_io/python/ops
json_dataset_ops.py
orc_dataset_ops.py
 75:108 (41%)
69:102 (42%)
 view
30 x 2 tensorflow_io/core/kernels
tensorflow_io/core/kernels
audio_video_flac_kernels.cc
audio_video_ogg_kernels.cc
 356:391 (7%)
202:237 (10%)
 view
30 x 2 tensorflow_io/core/kernels
tensorflow_io/core/kernels
audio_video_flac_kernels.cc
audio_video_wav_kernels.cc
 356:391 (7%)
314:349 (8%)
 view
30 x 2 tensorflow_io/python/ops
tensorflow_io/python/ops
hdf5_io_tensor_ops.py
parquet_io_tensor_ops.py
 134:168 (31%)
133:167 (30%)
 view
30 x 2 tensorflow_io/python/ops
tensorflow_io/python/ops
avro_dataset_ops.py
orc_dataset_ops.py
 75:108 (41%)
69:102 (42%)
 view
30 x 2 tensorflow_io/core/kernels
tensorflow_io/core/kernels
audio_video_flac_kernels.cc
audio_video_mp4_kernels.cc
 356:391 (7%)
482:517 (4%)
 view
30 x 2 tensorflow_io/core/kernels
tensorflow_io/core/kernels
audio_video_mp3_kernels.cc
audio_video_wav_kernels.cc
 209:244 (10%)
314:349 (8%)
 view
30 x 2 tensorflow_io/core/kernels
tensorflow_io/core/kernels
audio_video_flac_kernels.cc
audio_video_mp3_kernels.cc
 356:391 (7%)
209:244 (10%)
 view
Duplicated Units
The list of top 16 duplicated units.
See data for all 16 unit duplicates...
Size#FoldersFilesLinesCode
25 x 2 tensorflow_io/core/kernels
tensorflow_io/core/kernels
ffmpeg_kernels.cc
ffmpeg_kernels.cc
 406:434 
680:708 
 view
20 x 2 tensorflow_io/core/kernels
tensorflow_io/core/kernels
ffmpeg_kernels_deprecated.cc
ffmpeg_kernels.cc
 570:590 
435:455 
 view
12 x 2 tensorflow_io/core/kernels
tensorflow_io/core/kernels
pubsub_kernels.cc
kinesis_kernels.cc
 164:179 
415:430 
 view
12 x 2 tensorflow_io/core/kernels
tensorflow_io/core/kernels
mongodb_kernels.cc
mongodb_kernels.cc
 158:174 
336:352 
 view
9 x 2 tensorflow_io/core/filesystems/az
tensorflow_io/core/filesystems/az
az_filesystem.cc
az_filesystem.cc
 552:562 
564:574 
 view
9 x 5 tensorflow_io/core/kernels/arrow
tensorflow_io/core/kernels
tensorflow_io/core/kernels
tensorflow_io/core/kernels/orc
tensorflow_io/core/kernels
arrow_kernels.cc
json_kernels.cc
ffmpeg_kernels_deprecated.cc
orc_kernels.cc
avro_kernels.cc
 649:658 
144:153 
745:754 
211:220 
428:437 
 view
9 x 2 tensorflow_io/core/kernels
tensorflow_io/core/kernels
kafka_kernels_deprecated.cc
kafka_kernels.cc
 630:639 
656:665 
 view
8 x 2 tensorflow_io/core/kernels
tensorflow_io/core/kernels
ffmpeg_kernels.cc
ffmpeg_kernels.cc
 527:538 
829:840 
 view
8 x 4 tensorflow_io/python/ops
tensorflow_io/python/ops
tensorflow_io/python/ops
tensorflow_io/python/ops
json_dataset_ops.py
orc_dataset_ops.py
ffmpeg_dataset_ops.py
avro_dataset_ops.py
 0:0 
0:0 
0:0 
0:0 
 view
6 x 3 tensorflow_io/core/filesystems/hdfs
tensorflow_io/core/filesystems/http
tensorflow_io/core/filesystems/az
hadoop_filesystem.cc
http_filesystem.cc
az_filesystem.cc
 666:678 
678:684 
576:582 
 view
6 x 6 tensorflow_io/core/kernels/arrow
tensorflow_io/core/kernels
tensorflow_io/core/kernels
tensorflow_io/core/kernels/orc
tensorflow_io/core/kernels
tensorflow_io/core/kernels
arrow_kernels.cc
json_kernels.cc
ffmpeg_kernels_deprecated.cc
orc_kernels.cc
csv_kernels.cc
avro_kernels.cc
 642:648 
137:143 
738:744 
203:209 
133:139 
421:427 
 view
6 x 5 tensorflow_io/core/kernels
tensorflow_io/core/kernels
tensorflow_io/core/kernels
tensorflow_io/core/kernels
tensorflow_io/core/kernels
audio_video_ogg_kernels.cc
audio_video_wav_kernels.cc
audio_video_mp3_kernels.cc
audio_video_mp4_kernels.cc
audio_video_flac_kernels.cc
 118:124 
177:183 
140:146 
330:336 
289:295 
 view
6 x 2 tensorflow_io/core/kernels
tensorflow_io/core/kernels
kafka_kernels_deprecated.cc
kafka_kernels.cc
 509:515 
63:69 
 view
6 x 2 tensorflow_io/core/kernels
tensorflow_io/core/kernels
video_kernels.cc
audio_video_mp4_kernels.cc
 133:141 
845:853 
 view
6 x 2 tensorflow_io/python/ops
tensorflow_io/python/ops
io_tensor_ops.py
csv_io_tensor_ops.py
 0:0 
0:0 
 view
11 x 2 tensorflow_io/python/ops
tensorflow_io/python/ops
io_tensor_ops.py
io_tensor_ops.py
 0:0 
0:0 
 view