Skip to content
Official MCP RegistryListed

AI Crawler Permission Check

Part ofAI Crawler Permission CheckMCP server

Check a site's robots.txt rules for AI crawlers (GPTBot, ClaudeBot, PerplexityBot and more).

First seen 21 Nov 2026. Evidence as of 22 Nov 2026.

2
Tools
From an anonymous probe
1
Source listings
Each with its own history
1
Recorded changes
Since first seen

Tools

ToolDescriptionBehaviour
check_ai_crawlersFor a domain (and optional path), whether each known AI crawler/agent user-agent is allowed by robots.txt, with the deciding rule.Read-only
check_user_agentWhether a given crawler user-agent token may fetch a path on a domain according to robots.txt.Read-only

Change history

  1. Listed (registry)
Source listings
SourceListingFirst seenLast seenVersions
Official MCP Registryai.opencurrent/robots-ai-check21 Nov 202622 Nov 20261