openai / tiktoken
File Size

The distribution of size of files (measured in lines of code).

Intro
  • File size measurements show the distribution of size of files.
  • Files are classified in four categories based on their size (lines of code): 1-100 (very small files), 101-200 (small files), 201-500 (medium size files), 501-1000 (long files), 1001+(very long files).
  • It is a good practice to keep files small. Long files may become "bloaters", code that have increased to such gargantuan proportions that they are hard to work with.
Learn more...
File Size Overall
  • There are 13 files with 1,626 lines of code.
    • 0 very long files (0 lines of code)
    • 1 long files (521 lines of code)
    • 2 medium size files (421 lines of code)
    • 3 small files (394 lines of code)
    • 7 very small files (290 lines of code)
0% | 32% | 25% | 24% | 17%
Legend:
1001+
501-1000
201-500
101-200
1-100


explore: grouped by folders | grouped by size | sunburst | 3D view
File Size per Extension
1001+
501-1000
201-500
101-200
1-100
rs0% | 71% | 28% | 0% | 0%
py0% | 0% | 23% | 44% | 31%
in0% | 0% | 0% | 0% | 100%
File Size per Logical Decomposition
primary
1001+
501-1000
201-500
101-200
1-100
src0% | 71% | 28% | 0% | 0%
tiktoken0% | 0% | 33% | 40% | 25%
tiktoken_ext0% | 0% | 0% | 100% | 0%
scripts0% | 0% | 0% | 0% | 100%
ROOT0% | 0% | 0% | 0% | 100%
Longest Files (Top 13)
File# lines# units
lib.rs
in src
 521 16
core.py
in tiktoken
 211 28
py.rs
in src
 210 12
openai_public.py
in tiktoken_ext
 138 7
_educational.py
in tiktoken
 134 11
load.py
in tiktoken
 122 7
model.py
in tiktoken
 85 2
registry.py
in tiktoken
 70 4
redact.py
in scripts
 52 3
41 1
benchmark.py
in scripts
 28 1
in
 8 -
__init__.py
in tiktoken
 6 -
Files With Most Units (Top 11)
File# lines# units
core.py
in tiktoken
 211 28
lib.rs
in src
 521 16
py.rs
in src
 210 12
_educational.py
in tiktoken
 134 11
load.py
in tiktoken
 122 7
openai_public.py
in tiktoken_ext
 138 7
registry.py
in tiktoken
 70 4
redact.py
in scripts
 52 3
model.py
in tiktoken
 85 2
41 1
benchmark.py
in scripts
 28 1
Files With Long Lines (Top 1)

There is only one file with lines longer than 120 characters. In total, there is only one long line.

File# lines# units# long lines
lib.rs
in src
 521 16 1
Correlations

File Size vs. Commits (all time): 12 points

tiktoken/__init__.py x: 8 commits (all time) y: 6 lines of code tiktoken/core.py x: 15 commits (all time) y: 211 lines of code src/lib.rs x: 16 commits (all time) y: 521 lines of code tiktoken/load.py x: 14 commits (all time) y: 122 lines of code src/py.rs x: 6 commits (all time) y: 210 lines of code tiktoken/model.py x: 16 commits (all time) y: 85 lines of code scripts/wheel_download.py x: 1 commits (all time) y: 41 lines of code tiktoken_ext/openai_public.py x: 12 commits (all time) y: 138 lines of code tiktoken/_educational.py x: 4 commits (all time) y: 134 lines of code tiktoken/registry.py x: 3 commits (all time) y: 70 lines of code scripts/redact.py x: 2 commits (all time) y: 52 lines of code MANIFEST.in x: 2 commits (all time) y: 8 lines of code
521.0
lines of code
  min: 6.0
  average: 133.17
  25th percentile: 43.75
  median: 103.5
  75th percentile: 192.0
  max: 521.0
0 16.0
commits (all time)
min: 1.0 | average: 8.25 | 25th percentile: 2.25 | median: 7.0 | 75th percentile: 14.75 | max: 16.0

File Size vs. Contributors (all time): 12 points

tiktoken/__init__.py x: 2 contributors (all time) y: 6 lines of code tiktoken/core.py x: 5 contributors (all time) y: 211 lines of code src/lib.rs x: 6 contributors (all time) y: 521 lines of code tiktoken/load.py x: 3 contributors (all time) y: 122 lines of code src/py.rs x: 3 contributors (all time) y: 210 lines of code tiktoken/model.py x: 6 contributors (all time) y: 85 lines of code scripts/wheel_download.py x: 1 contributors (all time) y: 41 lines of code tiktoken_ext/openai_public.py x: 6 contributors (all time) y: 138 lines of code tiktoken/_educational.py x: 2 contributors (all time) y: 134 lines of code tiktoken/registry.py x: 1 contributors (all time) y: 70 lines of code scripts/redact.py x: 1 contributors (all time) y: 52 lines of code MANIFEST.in x: 1 contributors (all time) y: 8 lines of code
521.0
lines of code
  min: 6.0
  average: 133.17
  25th percentile: 43.75
  median: 103.5
  75th percentile: 192.0
  max: 521.0
0 6.0
contributors (all time)
min: 1.0 | average: 3.08 | 25th percentile: 1.0 | median: 2.5 | 75th percentile: 5.75 | max: 6.0

File Size vs. Commits (30 days): 0 points

No data for "commits (30d)" vs. "lines of code".

File Size vs. Contributors (30 days): 0 points

No data for "contributors (30d)" vs. "lines of code".

File Size vs. Commits (90 days): 2 points

tiktoken/__init__.py x: 1 commits (90d) y: 6 lines of code tiktoken/core.py x: 1 commits (90d) y: 211 lines of code
211.0
lines of code
  min: 6.0
  average: 108.5
  25th percentile: 6.0
  median: 108.5
  75th percentile: 211.0
  max: 211.0
0 1.0
commits (90d)
min: 1.0 | average: 1.0 | 25th percentile: 1.0 | median: 1.0 | 75th percentile: 1.0 | max: 1.0

File Size vs. Contributors (90 days): 2 points

tiktoken/__init__.py x: 1 contributors (90d) y: 6 lines of code tiktoken/core.py x: 1 contributors (90d) y: 211 lines of code
211.0
lines of code
  min: 6.0
  average: 108.5
  25th percentile: 6.0
  median: 108.5
  75th percentile: 211.0
  max: 211.0
0 1.0
contributors (90d)
min: 1.0 | average: 1.0 | 25th percentile: 1.0 | median: 1.0 | 75th percentile: 1.0 | max: 1.0