Official MCP RegistryListed
AI Crawler Permission Check
Check a site's robots.txt rules for AI crawlers (GPTBot, ClaudeBot, PerplexityBot and more).
First seen 21 Nov 2026. Evidence as of 22 Nov 2026.
2
Tools
From an anonymous probe
1
Source listings
Each with its own history
1
Recorded changes
Since first seen
Tools
| Tool | Description | Behaviour |
|---|---|---|
| check_ai_crawlers | For a domain (and optional path), whether each known AI crawler/agent user-agent is allowed by robots.txt, with the deciding rule. | Read-only |
| check_user_agent | Whether a given crawler user-agent token may fetch a path on a domain according to robots.txt. | Read-only |
Change history
- Listed (registry)
| Source | Listing | First seen | Last seen | Versions |
|---|---|---|---|---|
| Official MCP Registry | ai.opencurrent/robots-ai-check | 21 Nov 2026 | 22 Nov 2026 | 1 |