riktoken 0.0.1

A pure Ruby implementation of OpenAI's tiktoken library for BPE tokenization

Gemfile:
=

Installeer:
=

Versies:

0.0.1 June 04, 2025 (14,5 KB)

Runtime afhankelijkheden (1):

base64 >= 0

Show all transitive dependencies

Eigenaren:

Pushed by:

Authors:

moznion

SHA 256 checksum:

=