A BPE (Byte Pair Encoding) tokenizer written in Rust with Python bindings
No known vulnerabilities in the latest version
[](/packages/rustbpe)
<a href="/packages/rustbpe"><img src="/api/badges/rustbpe?period=month" alt="PyPI Stats"></a>