tokenizers 0.5.0
Fast state-of-the-art tokenizers for Ruby
Gemfile:
=
install:
=
Versions:
- 0.5.3 September 17, 2024 x86_64-linux (5.91 MB)
- 0.5.3 September 17, 2024 x86_64-linux-musl (5.89 MB)
- 0.5.3 September 17, 2024 x86_64-darwin (6.17 MB)
- 0.5.3 September 17, 2024 x64-mingw-ucrt (8.09 MB)
- 0.5.3 September 17, 2024 arm64-darwin (5.87 MB)
- 0.5.0 May 21, 2024 (38.5 KB)
Runtime Dependencies (1):
rb_sys
>= 0