tensorflow / text
Unit Size

The distribution of size of units (measured in lines of code).

Intro
  • Unit size measurements show the distribution of size of units of code (methods, functions...).
  • Units are classified in four categories based on their size (lines of code): 1-20 (small units), 20-50 (medium size units), 51-100 (long units), 101+ (very long units).
  • You should aim at keeping units small (< 20 lines). Long units may become "bloaters", code that have increased to such gargantuan proportions that they are hard to work with.
Learn more...
Unit Size Overall
  • There are 569 units with 8,951 lines of code in units (65.4% of code).
    • 4 very long units (591 lines of code)
    • 25 long units (1,684 lines of code)
    • 117 medium size units (3,624 lines of code)
    • 114 small units (1,715 lines of code)
    • 309 very small units (1,337 lines of code)
6% | 18% | 40% | 19% | 14%
Legend:
101+
51-100
21-50
11-20
1-10
Unit Size per Extension
101+
51-100
21-50
11-20
1-10
cc12% | 25% | 38% | 14% | 9%
py0% | 9% | 46% | 25% | 19%
h0% | 20% | 25% | 22% | 31%
Unit Size per Logical Component
primary logical decomposition
101+
51-100
21-50
11-20
1-10
tensorflow_text/core/kernels10% | 25% | 36% | 15% | 11%
tensorflow_text/python/ops0% | 11% | 53% | 21% | 13%
tensorflow_text/tools/wordpiece_vocab0% | 12% | 37% | 27% | 21%
tensorflow_text/python/benchmarks0% | 0% | 28% | 34% | 37%
tensorflow_text/python/numpy0% | 0% | 100% | 0% | 0%
tensorflow_text/core/ops0% | 0% | 42% | 32% | 25%
tensorflow_text/python/keras0% | 0% | 17% | 31% | 50%
oss_scripts0% | 0% | 0% | 92% | 7%
tensorflow_text/python/metrics0% | 0% | 0% | 100% | 0%
oss_scripts/pip_package0% | 0% | 0% | 0% | 100%
Alternative Visuals
Longest Units
Top 20 longest units
Unit# linesMcCabe index# params
void ViterbiAnalysis()
in tensorflow_text/core/kernels/constrained_sequence.cc
 213 47 7
bool IsEmoticon()
in tensorflow_text/core/kernels/sentence_fragmenter_v2.cc
 160 3 2
absl::Status FastWordpieceBuilder::BuildFailureStructure()
in tensorflow_text/core/kernels/fast_wordpiece_tokenizer_model_builder.cc
 113 21 1
void Compute()
in tensorflow_text/core/kernels/sentencepiece_kernels.cc
 105 16 1
void Compute()
in tensorflow_text/core/kernels/sentencepiece_kernels.cc
 97 16 1
void Compute()
in tensorflow_text/core/kernels/constrained_sequence_kernel.cc
 88 12 1
void Compute()
in tensorflow_text/core/kernels/sentence_breaking_kernels.cc
 87 9 1
def generate_vocab()
in tensorflow_text/tools/wordpiece_vocab/generate_vocab.py
 85 6 6
void Compute()
in tensorflow_text/core/kernels/unicode_script_tokenize_kernel.cc
 84 14 1
absl::Status Invoke()
in tensorflow_text/core/kernels/ngrams_kernel_template.h
 80 10 1
void Compute()
in tensorflow_text/core/kernels/tokenizer_from_logits_kernel.cc
 77 6 1
void FastWordpieceTokenizer::TokenizeTextImpl()
in tensorflow_text/core/kernels/fast_wordpiece_tokenizer.cc
 74 14 5
void Compute()
in tensorflow_text/core/kernels/wordpiece_kernel.cc
 71 9 1
void Compute()
in tensorflow_text/core/kernels/split_merge_tokenize_kernel.cc
 66 6 1
void Compute()
in tensorflow_text/core/kernels/whitespace_tokenize_kernel.cc
 65 7 1
void Compute()
in tensorflow_text/core/kernels/normalize_kernels.cc
 63 6 1
absl::Status FastWordpieceTokenizeWithOffsetsOp::Invoke()
in tensorflow_text/core/kernels/fast_wordpiece_tokenizer_kernel_template.h
 63 7 1
def span_overlaps()
in tensorflow_text/python/ops/pointer_ops.py
 63 13 8
void Compute()
in tensorflow_text/core/kernels/regex_split_kernels.cc
 60 7 1
absl::StatusOr FastWordpieceBuilder::ExportToFlatBuffer()
in tensorflow_text/core/kernels/fast_wordpiece_tokenizer_model_builder.cc
 60 8 0