openai / tiktoken
tiktoken is a fast BPE tokeniser for use with OpenAI's models.
GitHub Repo 
1.6K
lines of main code
13 files
0.3K
lines of test code
7 files
0.7K
lines of other code
7 files
4y
age
1,288 days
83%
main code touched
1 year (1.4K LOC)
2%
new main code
1 year (69 LOC)
0.9K
py
0.7K
rs
IN
0.01K
in
30 days 90 days all time 2026 2025 2024 2023 2022
+0
-0
+4
-1
+1.9K
-704
+138
-8
+618
-326
+462
-257
+617
-110
+33
-3
0 2 99

4

28

23

39

5

0 1 47

2

11

12

19

3

0 1 18

1

4

8

7

3

2026 2025 2024 2023 2022

generated by sokrates.dev (configuration) on 2026-06-24