riktoken 0.0.1
A pure Ruby implementation of OpenAI's tiktoken library for BPE tokenization
Gemfile:
=
instalar:
=
Versiones:
- 0.0.1 June 04, 2025 (14,5 KB)
dependencias de Runtime (1):
base64
>= 0
Propietarios:
Subida por:
Autores:
-
moznion
SHA 256 checksum:
=