Skip to content
MCP server

AI Crawler Permission Check

By opencurrentAll Opencurrent servers

Check a site's robots.txt rules for AI crawlers (GPTBot, ClaudeBot, PerplexityBot and more).

First seen 21 Nov 2026. One server, whatever directories list it: each directory listing keeps its own page and history.

1
Directories
Collected by InvokeRank
2
Tools
From an anonymous probe
-
ToolBench grade
Not graded by Arcade
-
GitHub stars
No repository data

Tools

ToolDescriptionBehaviour
check_ai_crawlersFor a domain (and optional path), whether each known AI crawler/agent user-agent is allowed by robots.txt, with the deciding rule.Read-only
check_user_agentWhether a given crawler user-agent token may fetch a path on a domain according to robots.txt.Read-only

Directory listings

DirectoryListingTierFirst seen
Official MCP RegistryAI Crawler Permission Check-21 Nov 2026
AI Crawler Permission Check MCP server | InvokeRank