riktoken 0.0.1

A pure Ruby implementation of OpenAI's tiktoken library for BPE tokenization

Gemfile:
=

installation:
=

Versions:

0.0.1 June 04, 2025 (14,5 ko)

Dépendances de Runtime (1):

base64 >= 0

Show all transitive dependencies

Propriétaires:

Pushed by:

Auteurs:

moznion

Total de contrôle SHA 256:

=