tensorflow / tfx
Duplication

Places in code with 6 or more lines that are exactly the same.

Intro
  • For duplication, we look at places in code where there are 6 or more lines of code that are exactly the same.
  • Before duplication is calculated, the code is cleaned to remove empty lines, comments, and frequently duplicated constructs such as imports.
  • You should aim at having as little as possible (<5%) of duplicated code as high-level of duplication can lead to maintenance difficulties, poor factoring, and logical contradictions.
Learn more...
Duplication Overall
  • 15% duplication:
    • 37,395 cleaned lines of cleaned code (without empty lines, comments, and frequently duplicated constructs such as imports)
    • 5,631 duplicated lines
  • 4,253 duplicates
system15% (5,631 lines)
Duplication per Extension
pb47% (2,798 lines)
py8% (2,460 lines)
proto17% (331 lines)
bzl24% (26 lines)
yaml8% (16 lines)
Duplication per Component (primary)
tfx/benchmarks42% (2,988 lines)
tfx/orchestration8% (846 lines)
tfx/tools25% (413 lines)
tfx/experimental20% (372 lines)
tfx/dsl7% (292 lines)
tfx/proto16% (289 lines)
tfx/components3% (194 lines)
tfx/extensions8% (146 lines)
tfx/types3% (58 lines)
tfx11% (26 lines)
tfx/scripts3% (7 lines)
package_build/ml-pipelines-sdk0% (0 lines)
package_build/tfx0% (0 lines)
ROOT0% (0 lines)
tfx/utils0% (0 lines)
tfx/v10% (0 lines)

Duplication Between Components (50+ lines)

G tfx/extensions tfx/extensions tfx/proto tfx/proto tfx/extensions--tfx/proto 291 tfx/orchestration tfx/orchestration tfx/orchestration--tfx/proto 275

Download: SVG DOT (open online Graphviz editor)

Open 3D force graph...

Show more details on duplication between components...
Longest Duplicates
The list of 20 longest duplicates.
See data for all 4,253 duplicates...
Size#FoldersFilesLinesCode
63 x 2 tfx/orchestration/launcher
tfx/orchestration/portable
kubernetes_component_launcher.py
kubernetes_executor_operator.py
 88:161 (48%)
85:158 (49%)
 view
29 x 2 tfx/tools/cli/commands
tfx/tools/cli/commands
run.py
run.py
 41:69 (11%)
210:238 (11%)
 view
29 x 2 tfx/tools/cli/commands
tfx/tools/cli/commands
pipeline.py
pipeline.py
 92:120 (10%)
170:198 (10%)
 view
27 x 2 tfx/experimental/templates/penguin
tfx/experimental/templates/taxi
kubeflow_runner.py
kubeflow_runner.py
 50:93 (84%)
50:100 (84%)
 view
26 x 2 tfx/tools/cli/commands
tfx/tools/cli/commands
run.py
run.py
 32:57 (10%)
156:181 (10%)
 view
24 x 2 tfx/experimental/templates/penguin
tfx/experimental/templates/taxi
local_runner.py
local_runner.py
 28:80 (100%)
28:80 (100%)
 view
23 x 2 tfx/tools/cli/handler
tfx/tools/cli/handler
airflow_handler.py
beam_handler.py
 54:88 (22%)
43:77 (34%)
 view
23 x 2 tfx/tools/cli/commands
tfx/tools/cli/commands
run.py
run.py
 204:226 (9%)
259:281 (9%)
 view
22 x 2 tfx/orchestration/experimental/core
tfx/orchestration/experimental/core
async_pipeline_task_gen.py
sync_pipeline_task_gen.py
 267:292 (11%)
347:372 (7%)
 view
17 x 2 tfx/tools/cli/commands
tfx/tools/cli/commands
run.py
run.py
 41:57 (6%)
265:281 (6%)
 view
17 x 2 tfx/tools/cli/commands
tfx/tools/cli/commands
run.py
run.py
 41:57 (6%)
126:142 (6%)
 view
17 x 2 tfx/tools/cli/commands
tfx/tools/cli/commands
run.py
run.py
 126:142 (6%)
265:281 (6%)
 view
17 x 2 tfx/tools/cli/commands
tfx/tools/cli/commands
pipeline.py
pipeline.py
 176:192 (6%)
228:244 (6%)
 view
17 x 2 tfx/tools/cli/commands
tfx/tools/cli/commands
pipeline.py
pipeline.py
 176:192 (6%)
261:277 (6%)
 view
17 x 2 tfx/tools/cli/commands
tfx/tools/cli/commands
run.py
run.py
 126:142 (6%)
165:181 (6%)
 view
17 x 2 tfx/tools/cli/commands
tfx/tools/cli/commands
run.py
run.py
 126:142 (6%)
210:226 (6%)
 view
17 x 2 tfx/tools/cli/commands
tfx/tools/cli/commands
pipeline.py
pipeline.py
 98:114 (6%)
228:244 (6%)
 view
17 x 2 tfx/tools/cli/commands
tfx/tools/cli/commands
pipeline.py
pipeline.py
 98:114 (6%)
261:277 (6%)
 view
17 x 2 tfx/tools/cli/commands
tfx/tools/cli/commands
pipeline.py
pipeline.py
 228:244 (6%)
261:277 (6%)
 view
17 x 2 tfx/tools/cli/commands
tfx/tools/cli/commands
run.py
run.py
 165:181 (6%)
265:281 (6%)
 view
Duplicated Units
The list of top 9 duplicated units.
See data for all 9 unit duplicates...
Size#FoldersFilesLinesCode
7 x 3 tfx/dsl/components/common
tfx/dsl/components/common
tfx/dsl/components/base
importer.py
resolver.py
base_driver.py
 0:0 
0:0 
0:0 
 view
7 x 2 tfx/dsl/components/base
tfx/components/example_gen
base_driver.py
driver.py
 0:0 
0:0 
 view
9 x 2 tfx/tools/cli/handler
tfx/tools/cli/handler
beam_handler.py
airflow_handler.py
 0:0 
0:0 
 view
6 x 2 tfx/components/trainer/rewriting
tfx/components/trainer/rewriting
tfjs_rewriter.py
tflite_rewriter.py
 0:0 
0:0 
 view
7 x 2 tfx/orchestration/experimental/core
tfx/orchestration/experimental/core
sync_pipeline_task_gen.py
async_pipeline_task_gen.py
 0:0 
0:0 
 view
7 x 2 tfx/orchestration/experimental/core
tfx/orchestration/experimental/core
sync_pipeline_task_gen.py
async_pipeline_task_gen.py
 0:0 
0:0 
 view
6 x 2 tfx/orchestration/beam
tfx/orchestration/beam
beam_dag_runner.py
beam_dag_runner.py
 0:0 
0:0 
 view
7 x 2 tfx/benchmarks
tfx/benchmarks
tfma_benchmark_base.py
tfma_v2_benchmark_base.py
 0:0 
0:0 
 view
9 x 2 tfx/components/trainer/rewriting
tfx/components/trainer/rewriting
tfjs_rewriter.py
tflite_rewriter.py
 0:0 
0:0 
 view