openai/tiktoken
tiktoken is a fast BPE tokeniser for use with OpenAI's models.
GitHub 
1.6K
lines of main code
13 files
0.3K
lines of test code
7 files
0.8K
lines of other code
8 files
4y
age
1,393 days
59%
main code touched
1 year (969 LOC)
ABCDE
ease of change
for people* (8/10)
ABCDE
ease of change
for AI* (7/9)
0.9K
py
0.7K
rs
IN
0.01K
in
30 days 2026 2025 2024 2023 2022
+0
-0
+144
-18
+618
-326
+462
-257
+617
-110
+33
-3
0

6

28

23

39

5

0

4

11

12

19

3

0

1

4

8

7

3

2026 2025 2024 2023 2022
Rule-of-thumb estimates

Rebuild value

to rebuild
rebuild value

Maintenance effort

per year to maintain
maintenance cost per year
Rules of thumb, not measurements: one man-year rebuilds about 10,000 lines of main code (design, testing and coordination included), and keeping a code base alive takes about 15% of its rebuild value per year (20 man-years of rebuild value need about 3 man-years of maintenance every year). Your changes are remembered in this browser for every report.