huggingface / tokenizers
Unit Size

The distribution of size of units (measured in lines of code).

Intro
Learn more...
Unit Size Overall
9% | 10% | 30% | 19% | 29%
Legend:
101+
51-100
21-50
11-20
1-10
Unit Size per Extension
101+
51-100
21-50
11-20
1-10
rs11% | 10% | 31% | 19% | 26%
py0% | 10% | 31% | 19% | 37%
js0% | 0% | 0% | 100% | 0%
pyi0% | 0% | 0% | 2% | 97%
Unit Size per Logical Component
primary logical decomposition
101+
51-100
21-50
11-20
1-10
tokenizers16% | 15% | 36% | 16% | 15%
bindings0% | 2% | 20% | 24% | 52%
Alternative Visuals
Longest Units
Top 50 longest units
Unit# linesMcCabe index# params
409 1 0
327 1 0
172 1 0
151 5 3
136 1 1
128 3 3
118 1 0
116 22 4
89 1 0
89 1 0
84 16 2
82 1 0
82 1 0
80 1 0
79 1 0
79 1 0
79 1 0
77 10 2
76 16 2
66 4 0
65 11 3
63 11 3
63 1 0
62 1 0
59 6 2
58 2 0
55 1 0
53 8 2
53 1 0
53 7 1
52 6 2
51 9 3
50 2 0
50 1 0
50 2 0
50 4 3
49 1 0
48 2 0
48 2 0
48 19 5
47 3 2
47 8 4
46 1 0
46 5 1
43 1 0
43 13 5
43 8 1
42 2 4
42 14 3
42 2 4