riktoken 0.0.1

A pure Ruby implementation of OpenAI's tiktoken library for BPE tokenization

Gemfile:
=

install:
=

Versions:

0.0.1 June 04, 2025 (14.5 KB)

Runtime Dependencies (1):

base64 >= 0

Show all transitive dependencies

Owners:

Pushed by:

Authors:

moznion

SHA 256 checksum:

=