A versatile Ruby web spidering library that can spider a site, multiple domains, certain links or infinitely. Spidr is designed to be fast and easy to use.
Appears on
Quick read
Latest capture 2026-08-20 03:04
0 paths
Agent instructions and tool configuration found in this repository.
No config files detected.
5 observed captures since 2026-06-09. Observed captures are shown by default.
Stars from first capture -1
Observed captures only
All tracked data
Observed snapshots
Observed snapshots
Nearest indexed repositories by embedding similarity.
Get web data for AI agents and LLMs - fast, efficient, and reliable with Rust
Interact with REST services in an ActiveRecord-like manner
The unified web layer for AI agents. Search (8 engines), stealth browse, auth, and act on 24 platforms. One npm install, self-hosted.
A personal search engine: Create a searchable library from your personal documents, interests, and more!
Ruby gem for web scraping purposes. It scrapes a given URL, and returns you its title, meta description, meta keywords, links, images...
Log and Analyze Outgoing HTTP Requests
