JRRToken is a Ruby gem that wraps the tiktoken Rust library, enabling fast and efficient tokenization for OpenAI models. It supports multiple models including o200k_base, cl100k_base, p50k_base, p50k_edit, and r50k_base.
Required Ruby Version
>= 3.0, < 4.1.dev
Required Rubygems Version
>= 3.0.0, >= 3.3.22
Authors
Logan Bresnahan
Versions
- 1.2.1 May 13, 2026 x86_64-linux (24.5 MB)
- 1.2.1 May 13, 2026 arm64-darwin (23.8 MB)
- 1.2.1 May 13, 2026 x86_64-darwin (23.8 MB)
- 1.2.1 May 13, 2026 x86_64-linux-musl (24.5 MB)
- 1.2.1 May 13, 2026 aarch64-linux (24.2 MB)
- 1.2.1 May 13, 2026 aarch64-linux-musl (24.2 MB)
