7 Repositories
Rust tokenization Libraries
๐ฅ Vaporetto is a fast and lightweight pointwise prediction based tokenizer. This is a Python wrapper for Vaporetto.
๐ python-vaporetto ๐ฅ Vaporetto is a fast and lightweight pointwise prediction based tokenizer. This is a Python wrapper for Vaporetto. Installation
17 Dec 22, 2022
CodeChain's official implementation in Rust.
CodeChain CodeChain is a programmable open source blockchain technology optimal for developing and customizing multi-asset management systems. Build D
259 Dec 21, 2022
An official Sudachi clone in Rust ๐ฆ
sudachi.rs - English README 2021-12-09 UPDATE: 0.6.2 Release Try it: pip install --update 'sudachipy=0.6.2' sudachi.rs is a Rust implementation of Su
175 Dec 30, 2022
An official Sudachi clone in Rust (incomplete) ๐ฆ
2021-07-07 UPDATE: The official Sudachi team will take over this project (cf. ๆฅๆฌ่ชๅฝขๆ ็ด ่งฃๆๅจ SudachiPy ใฎ ็พ็ถใจไปๅพใซใคใใฆ - Speaker Deck) sudachi.rs An official S
173 Dec 29, 2022
Rust wrapper for the BlingFire tokenization library
BlingFire in Rust blingfire is a thin Rust wrapper for the BlingFire tokenization library. Add the library to Cargo.toml to get started cargo add blin
14 Sep 5, 2022
Simple NLP in Rust with Python bindings
vtext NLP in Rust with Python bindings This package aims to provide a high performance toolkit for ingesting textual data for machine learning applica
133 Jan 3, 2023
JSON parser which picks up values directly without performing tokenization in Rust
Pikkr JSON parser which picks up values directly without performing tokenization in Rust Abstract Pikkr is a JSON parser which picks up values directl
615 Dec 29, 2022