huggingface/tokenizers
💥 Fast State-of-the-Art Tokenizers optimized for Research and Production
10768 stars
Rust
3 awesome lists
openai/tiktoken
tiktoken is a fast BPE tokeniser for use with OpenAI's models.
- Stars
- 18,385
- Forks
- 1,494
- Commits
- 72
- Language
- Python
- Awesome lists
- 1
Similar repositories
dqbd/tiktokenizer
Online playground for OpenAPI tokenizers
1607 stars
TypeScript
2 awesome lists
botisan-ai/gpt3-tokenizer
Isomorphic JavaScript/TypeScript Tokenizer for GPT-3 and Codex Models by OpenAI.
171 stars
TypeScript
1 awesome list
coqui-ai/TTS
🐸💬 - a deep learning toolkit for Text-to-Speech, battle-tested in research and production
45475 stars
Python
1 awesome list
sashabaranov/go-openai
OpenAI ChatGPT, GPT-5, GPT-Image-1, Whisper API clients for Go
10667 stars
Go
4 awesome lists
huggingface/sentence-transformers
State-of-the-Art Embeddings, Retrieval, and Reranking
18723 stars
Python
1 awesome list
Tracked growth
2 captures since 2026-06-02
Latest capture 2026-06-03 04:23
Stars history
Total starsCommits history
Default branch commitsDetected stack
Frameworks and tools
- No framework dependencies detected.
Cargo
PEP 517
pip
Dependency files
-
Cargo.toml· rust · 5 dependencies -
pyproject.toml· python · 6 dependencies -
setup.py· python · 0 dependencies
Metadata
- Created: 2022-12-01
- First commit: 2022-12-12
- Last pushed: 2026-05-24
- Archived: no
- Stack detected: 2026-06-03 04:23
- License: MIT
AI development signals
No AI development config files detected.