File Size Overall
  • There are 13 files with 1,622 lines of code.
    • 0 very long files (0 lines of code)
    • 1 long files (521 lines of code)
    • 2 medium size files (421 lines of code)
    • 3 small files (394 lines of code)
    • 7 very small files (286 lines of code)
0% | 32% | 25% | 24% | 17%
Legend:
1001+
501-1000
201-500
101-200
1-100


explore: grouped by folders | grouped by size | sunburst | 3D view
File Size per Extension
1001+
501-1000
201-500
101-200
1-100
rs0% | 71% | 28% | 0% | 0%
py0% | 0% | 23% | 44% | 31%
in0% | 0% | 0% | 0% | 100%
File Size per Logical Decomposition
primary
1001+
501-1000
201-500
101-200
1-100
src0% | 71% | 28% | 0% | 0%
tiktoken0% | 0% | 33% | 41% | 25%
tiktoken_ext0% | 0% | 0% | 100% | 0%
scripts0% | 0% | 0% | 0% | 100%
ROOT0% | 0% | 0% | 0% | 100%
Longest Files (Top 13)
File# lines# units
lib.rs
in src
521 16
core.py
in tiktoken
211 28
py.rs
in src
210 12
openai_public.py
in tiktoken_ext
138 7
_educational.py
in tiktoken
134 11
load.py
in tiktoken
122 7
model.py
in tiktoken
81 2
registry.py
in tiktoken
70 4
redact.py
in scripts
52 3
41 1
benchmark.py
in scripts
28 1
in
8 -
__init__.py
in tiktoken
6 -
Large Files That Change Often (Top 5)
To change a file, an AI coding agent (or a person) has to read it: these files are read the most for the changes in the past year (lines × changes).
File# lines≈ tokens
to read
# changes
(1y)
≈ lines edited
per change
lines read per
line changed
lib.rs
in src
521 ~3k–10k 1 ~70 ~7
core.py
in tiktoken
211 ~1k–3k 1 ~15 ~15
load.py
in tiktoken
122 ~700–2k 1 ~15 ~10
model.py
in tiktoken
81 ~500–1.5k 1 ~10 ~7
__init__.py
in tiktoken
6 under 1k 2 ~2 ~3

Tokens are lines × 7 to 14 (analysis.fileReadsForChanges). Lines edited per change are the file's lines added and deleted divided by its commits over the whole history. Agents read about 2,000 lines at a time: when a 17,000-line file was split into modules, the tokens an agent read for the same change fell by 83% (The Economic Benefit of Refactoring, 2026).

Files With Most Units (Top 11)
File# lines# units
core.py
in tiktoken
211 28
lib.rs
in src
521 16
py.rs
in src
210 12
_educational.py
in tiktoken
134 11
load.py
in tiktoken
122 7
openai_public.py
in tiktoken_ext
138 7
registry.py
in tiktoken
70 4
redact.py
in scripts
52 3
model.py
in tiktoken
81 2
41 1
benchmark.py
in scripts
28 1
Files With Long Lines (Top 1)

There is only one file with lines longer than 120 characters. In total, there is only one long line.

File# lines# units# long lines
lib.rs
in src
521 16 1
Correlations

File Size vs. Commits (all time): 12 points

tiktoken/__init__.py x: 9 commits (all time) y: 6 lines of code tiktoken/model.py x: 17 commits (all time) y: 81 lines of code tiktoken/core.py x: 15 commits (all time) y: 211 lines of code src/lib.rs x: 16 commits (all time) y: 521 lines of code tiktoken/load.py x: 14 commits (all time) y: 122 lines of code src/py.rs x: 6 commits (all time) y: 210 lines of code scripts/wheel_download.py x: 1 commits (all time) y: 41 lines of code tiktoken_ext/openai_public.py x: 12 commits (all time) y: 138 lines of code tiktoken/_educational.py x: 4 commits (all time) y: 134 lines of code tiktoken/registry.py x: 3 commits (all time) y: 70 lines of code scripts/redact.py x: 2 commits (all time) y: 52 lines of code MANIFEST.in x: 2 commits (all time) y: 8 lines of code
521.0
lines of code
  min: 6.0
  average: 132.83
  25th percentile: 43.75
  median: 101.5
  75th percentile: 192.0
  max: 521.0
0 17.0
commits (all time)
min: 1.0 | average: 8.42 | 25th percentile: 2.25 | median: 7.5 | 75th percentile: 14.75 | max: 17.0

File Size vs. Contributors (all time): 12 points

tiktoken/__init__.py x: 2 contributors (all time) y: 6 lines of code tiktoken/model.py x: 6 contributors (all time) y: 81 lines of code tiktoken/core.py x: 5 contributors (all time) y: 211 lines of code src/lib.rs x: 6 contributors (all time) y: 521 lines of code tiktoken/load.py x: 3 contributors (all time) y: 122 lines of code src/py.rs x: 3 contributors (all time) y: 210 lines of code scripts/wheel_download.py x: 1 contributors (all time) y: 41 lines of code tiktoken_ext/openai_public.py x: 6 contributors (all time) y: 138 lines of code tiktoken/_educational.py x: 2 contributors (all time) y: 134 lines of code tiktoken/registry.py x: 1 contributors (all time) y: 70 lines of code scripts/redact.py x: 1 contributors (all time) y: 52 lines of code MANIFEST.in x: 1 contributors (all time) y: 8 lines of code
521.0
lines of code
  min: 6.0
  average: 132.83
  25th percentile: 43.75
  median: 101.5
  75th percentile: 192.0
  max: 521.0
0 6.0
contributors (all time)
min: 1.0 | average: 3.08 | 25th percentile: 1.0 | median: 2.5 | 75th percentile: 5.75 | max: 6.0

File Size vs. Commits (30 days): 0 points

No data for "commits (30d)" vs. "lines of code".

File Size vs. Contributors (30 days): 0 points

No data for "contributors (30d)" vs. "lines of code".


File Size vs. Commits (90 days): 2 points

tiktoken/__init__.py x: 1 commits (90d) y: 6 lines of code tiktoken/model.py x: 1 commits (90d) y: 81 lines of code
81.0
lines of code
  min: 6.0
  average: 43.5
  25th percentile: 6.0
  median: 43.5
  75th percentile: 81.0
  max: 81.0
0 1.0
commits (90d)
min: 1.0 | average: 1.0 | 25th percentile: 1.0 | median: 1.0 | 75th percentile: 1.0 | max: 1.0

File Size vs. Contributors (90 days): 2 points

tiktoken/__init__.py x: 1 contributors (90d) y: 6 lines of code tiktoken/model.py x: 1 contributors (90d) y: 81 lines of code
81.0
lines of code
  min: 6.0
  average: 43.5
  25th percentile: 6.0
  median: 43.5
  75th percentile: 81.0
  max: 81.0
0 1.0
contributors (90d)
min: 1.0 | average: 1.0 | 25th percentile: 1.0 | median: 1.0 | 75th percentile: 1.0 | max: 1.0
About This Analysis
  • File size measurements show the distribution of size of files.
  • Files are classified in four categories based on their size (lines of code): 1-100 (very small files), 101-200 (small files), 201-500 (medium size files), 501-1000 (long files), 1001+(very long files).
  • It is a good practice to keep files small. Long files may become "bloaters", code that have increased to such gargantuan proportions that they are hard to work with.
Learn more...