pepy.tech

tokenicer

A (nicer) tokenizer you want to use for model `inference` and `training`: with all known peventable `gotchas` normalized or auto-fixed.

tokenicer has been downloaded 285,185 times in total on PyPI, including 17,840 in the last 30 days. The latest version is 0.0.5, released Sep 4, 2025.

Version0.0.5
Downloads
285.19k
License—
AuthorModelCloud
UpdatedSep 4, 2025
PyPI →

Downloads

Weekly, last 90d.
Includes CI traffic.

VersionsTotal0.*
Range
View
Granularity
Group by
CI traffic
Stack: OffCI: Included2 / 16 series
Selected total121.05k
285.2kAll-time
17.8kLast 30 days
781Last 24 h
<0.01/sPer second

Version distribution

Share of downloads by released version. Computed over the last quarter.

  • 01

    0.0.14

    27.8k downloads
    46.5%
  • 02

    0.0.13

    17.7k downloads
    29.6%
  • 03

    0.0.4

    8.3k downloads
    13.9%
  • 04

    0.0.12

    2.8k downloads
    4.7%
  • 05

    0.0.11

    1.9k downloads
    3.1%
  • 06

    0.0.5

    249 downloads
    0.4%
  • 07

    0.0.6

    184 downloads
    0.3%
  • 08

    0.0.8

    147 downloads
    0.2%
  • 09

    Other

    745 downloads
    1.2%

Guess the next day

Thirteen recent days of tokenicer downloads. Drag the green handle on the right to guess where day fourteen lands.

TRUTH781640FRISATSUNMONTUEWEDTHUFRISATSUNMONTUEWED