⬢ RubyGems Navigation menu

Search Gems…

Releases Blog Gems Guides Sign in Sign up

ruby_llm-tokenizer 0.1.1

Click on the arrow icon to expand.

Runtime :

sentencepiece 0.2.1 ~> 0.2

tiktoken_ruby 0.0.16 ~> 0.0.9

tokenizers 0.7.0 ~> 0.5

Status Uptime Code Data Stats Contribute About Help API Policies Support Us Security

RubyGems.org is the Ruby community’s gem hosting service. Instantly publish your gems and then install them. Use the API to find out more about available gems. Become a contributor and improve the site yourself.

The RubyGems.org website and service are maintained and operated by Ruby Central’s Open Source Program and the RubyGems team. It is funded by the greater Ruby community through support from sponsors, members, and infrastructure donations. If you build with Ruby and believe in our mission, you can join us in keeping RubyGems.org, RubyGems, and Bundler secure and sustainable for years to come by contributing here.

Operated by Ruby Central Designed by DockYard Hosted by AWS Resolved with DNSimple Monitored by Datadog Gems served by Fastly Monitored by Honeybadger Secured by Mend.io