Sign in
← Back to search
github Active

Repository profile

open-compass/VLMEvalKit

Open-source evaluation toolkit of large multi-modality models (LMMs), support 220+ LMMs, 80+ benchmarks

Python Apache-2.0 main Stack scanned README.md
Sign in to like open-compass/VLMEvalKit
Open website Open GitHub
Stars
4,282
Forks
730
Watchers
12
Issues
262
Commits
1,772
Awesome lists
2

Repository updates

Follow repository updates

Get generated open-compass/VLMEvalKit development summaries by email, or follow the weekly and monthly RSS feeds.

Weekly RSS Monthly RSS

Sign in to subscribe by email. RSS feeds are public.

Sign in to subscribe

Activity and growth

Tracked growth, recent movement, and commit velocity from stored repository snapshots.

Latest capture 2026-07-15 03:15

Star growth, last 7 days
0 0.0%
Commit velocity, last 7 days
0 0.0%
Stars since baseline
+131
Snapshot coverage
5

Tracked growth

5 captures since 2026-05-25

Stars from baseline +131

Time horizon

All tracked data

Stars history

Total stars

Commits history

Default branch commits

Detected stack

Frameworks, package managers, ecosystems, and dependency manifests found during catalog scans.

Scanned 2026-07-15 03:15

Stack signals
0
Package managers
1
Manifest files
8
Dependencies
0

Frameworks and tools

  • No framework dependencies detected.
pip python

Dependency files

8 manifests
  • requirements.txt python ecosystem, 0 dependencies
  • setup.cfg python ecosystem, 0 dependencies
  • setup.py python ecosystem, 0 dependencies
  • requirements/docs.txt python ecosystem, 0 dependencies
  • vlmeval/dataset/CGAVCounting/requirements.txt python ecosystem, 0 dependencies
  • vlmeval/dataset/OmniDocBench/requirements.txt python ecosystem, 0 dependencies
  • vlmeval/dataset/utils/megabench/requirements.txt python ecosystem, 0 dependencies
  • vlmeval/dataset/utils/Ocrbench_v2/requirements.txt python ecosystem, 0 dependencies

Classification

Searchable topics, generated tags, and stack labels that explain where this repository fits.

Topics
19
Tags
0
Stacks
0

Topics

#chatgpt #claude #clip #computer-vision #evaluation #gemini #gpt #gpt-4v #gpt4 #large-language-models #llava #llm #multi-modal #openai #openai-api #pytorch #qwen #vit #vqa

Generated tags

No generated tags yet.

Stack labels

No stack labels yet.

AI development signals

Agent instructions and tool configuration paths found in the repository tree.

0 paths
No AI development config files detected.

Similar repositories

Nearest indexed repositories by embedding similarity.

open-compass/opencompass

OpenCompass is an LLM evaluation platform, supporting a wide range of models (Llama3, Mistral, InternLM2,GPT-4,LLaMa2, Qwen,GLM, Claude, etc) over 100+ datasets.

7,191 stars
Python 3 awesome lists

EvolvingLMMs-Lab/lmms-eval

One-for-All Multimodal Evaluation Toolkit Across Text, Image, Video, and Audio Tasks

4,310 stars
Python 1 awesome list

huggingface/lighteval

Lighteval is your all-in-one toolkit for evaluating LLMs across multiple backends

2,476 stars
Python 2 awesome lists

evalplus/evalplus

Rigourous evaluation of LLM-synthesized code - NeurIPS 2023 & COLM 2024

1,770 stars
Python 2 awesome lists

ludwig-ai/ludwig

Low-code framework for building custom LLMs, neural networks, and other AI models

11,737 stars
Python 2 awesome lists

NVlabs/VILA

VILA is a family of state-of-the-art vision language models (VLMs) for diverse multimodal AI tasks across the edge, data center, and cloud.

3,832 stars
Python 1 awesome list

Metadata

Language
Python
License
Apache-2.0
Default branch
main
Created
2023-12-01
First commit
2023-11-25
Last pushed
2026-07-14
GitHub updated
2026-07-14
Last synced
2026-07-15 03:15
Stack detected
2026-07-15 03:15
Archived
no

Links and files

GitHub Website

https://huggingface.co/spaces/opencompass/open_vlm_leaderboard

README

403 Forbidden | https://api.github.com/repos/open-compass/VLMEvalKit/readme | message=API rate limit exceeded for user ID 260990068. If you reach out to GitHub Support for help, please include the request ID 84DA:7D562:211E53E:1FF95F3:6A56FB3D and timestamp 2026-07-15 03:15:09 UTC. For more on scraping GitHub and how it may affect your rights, please review our Terms of Service (http | rate_limit_remaining=0 | rate_limit_reset=1784088007

Appears in

2