Sign in
← Back to search
github Active

Repository profile

danielmiessler/RobotsDisallowed

A curated list of the most common and most interesting robots.txt disallowed directories.

Shell master Stack scanned README.md
Sign in to like danielmiessler/RobotsDisallowed
Open GitHub
Stars
1,489
Forks
304
Watchers
66
Issues
5
Commits
44
Awesome lists
1

Repository updates

Follow repository updates

Get generated danielmiessler/RobotsDisallowed development summaries by email, or follow the weekly and monthly RSS feeds.

Weekly RSS Monthly RSS

Sign in to subscribe by email. RSS feeds are public.

Sign in to subscribe

Activity and growth

Tracked growth, recent movement, and commit velocity from stored repository snapshots.

Latest capture 2026-06-24 12:56

Star growth, last 7 days
0 0.0%
Commit velocity, last 7 days
0 0.0%
Stars since baseline
0
Snapshot coverage
1

Tracked growth

1 capture since 2026-06-24

Stars from baseline 0

Time horizon

All tracked data

Stars history

Total stars

Commits history

Default branch commits

Detected stack

Frameworks, package managers, ecosystems, and dependency manifests found during catalog scans.

Scanned 2026-06-24 12:56

Stack signals
0
Package managers
0
Manifest files
0
Dependencies
0

Frameworks and tools

  • No framework dependencies detected.

Dependency files

  • No dependency manifests detected.

Classification

Searchable topics, generated tags, and stack labels that explain where this repository fits.

Topics
0
Tags
0
Stacks
0

Topics

No topics indexed.

Generated tags

No generated tags yet.

Stack labels

No stack labels yet.

AI development signals

Agent instructions and tool configuration paths found in the repository tree.

0 paths
No AI development config files detected.

Similar repositories

Nearest indexed repositories by embedding similarity.

danielmiessler/SecLists

SecLists is the security tester's companion. It's a collection of multiple types of lists used during security assessments, collected in one place. List types include usernames, passwords, URLs, sensitive data patterns, fuzzing payloads, web shells, and many more.

71,734 stars
PHP 1 awesome list

JasonLovesDoggo/caddy-defender

Caddy module to block or manipulate requests originating from AIs or cloud services trying to train on your websites

560 stars
Go 1 awesome list

berzerk0/Probable-Wordlists

Version 2 is live! Wordlists sorted by probability originally created for password generation and testing - make sure your passwords aren't popular!

9,298 stars
1 awesome list

protectai/ai-exploits

A collection of real world AI/ML exploits for responsibly disclosed vulnerabilities

1,731 stars
Python 1 awesome list

Metadata

Language
Shell
License
n/a
Default branch
master
Created
2015-09-21
First commit
2015-09-21
Last pushed
2022-08-22
GitHub updated
2026-06-21
Last synced
2026-06-24 12:56
Stack detected
2026-06-24 12:56
Archived
no

Links and files

GitHub README

Appears in

1