tokenicer
A (nicer) tokenizer you want to use for model `inference` and `training`: with all known peventable `gotchas` normalized or auto-fixed.
tokenicer has been downloaded 299,698 times in total on PyPI, including 21,240 in the last 30 days. The latest version is 0.0.14, released Jul 22, 2026.
Version0.0.14
Downloads
299.70k
License—
AuthorModelCloud
UpdatedJul 22, 2026
Downloads
Weekly, last 90d.
Includes CI traffic.
VersionsTotal0.*
Range
View
Granularity
Group by
CI traffic
Stack: OffCI: Included2 / 16 series
Selected total125.78k
299.7kAll-time
21.2kLast 30 days
1.5kLast 24 h
<0.01/sPer second
Sponsored
Sponsorships keep pepy free to read
Version distribution
Share of downloads by released version. Computed over the last quarter.
- 0161.1%
0.0.14
38.1k downloadsDownloads38.1k61.1% - 0214.2%
0.0.4
8.8k downloadsDownloads8.8k14.2% - 0313.7%
0.0.13
8.5k downloadsDownloads8.5k13.7% - 046.1%
0.0.12
3.8k downloadsDownloads3.8k6.1% - 053.0%
0.0.11
1.9k downloadsDownloads1.9k3.0% - 060.4%
0.0.5
262 downloadsDownloads2620.4% - 070.3%
0.0.6
175 downloadsDownloads1750.3% - 080.2%
0.0.8
128 downloadsDownloads1280.2% - 091.0%
Other
651 downloadsDownloads6511.0%
Guess the next day
Thirteen recent days of tokenicer downloads. Drag the green handle on the right to guess where day fourteen lands.