huggingface/tokenizers
💥 Fast State-of-the-Art Tokenizers optimized for Research and Production
10768 stars
Rust
3 awesome lists
dqbd/tiktokenizer
Online playground for OpenAPI tokenizers
- Stars
- 1,607
- Forks
- 176
- Commits
- 59
- Language
- TypeScript
- Awesome lists
- 2
Similar repositories
botisan-ai/gpt3-tokenizer
Isomorphic JavaScript/TypeScript Tokenizer for GPT-3 and Codex Models by OpenAI.
171 stars
TypeScript
1 awesome list
openai/tiktoken
tiktoken is a fast BPE tokeniser for use with OpenAI's models.
18377 stars
Python
1 awesome list
sashabaranov/go-openai
OpenAI ChatGPT, GPT-5, GPT-Image-1, Whisper API clients for Go
10667 stars
Go
4 awesome lists
briansunter/logseq-plugin-gpt3-openai
A plugin for GPT-3 AI assisted note taking in Logseq
745 stars
TypeScript
1 awesome list
hegelai/prompttools
Open-source tools for prompt testing and experimentation, with support for both LLMs (e.g. OpenAI, LLaMA) and vector databases (e.g. Chroma, Weaviate, LanceDB).
3035 stars
Python
2 awesome lists
Tracked growth
2 captures since 2026-05-27
Latest capture 2026-05-27 12:41
Stars history
Total starsCommits history
Default branch commitsMetadata
- Created: 2023-03-02
- First commit: 2023-03-02
- Last pushed: 2025-04-24
- Website: https://tiktokenizer.vercel.app
- Archived: no
- Stack detected: —
- License: MIT
AI development signals
No AI development config files detected.