Skip to content
Migrate your GPTCheck my InvokeRank
Official MCP RegistryListed

Octocrawl

Part ofOctocrawllisted on 2 directories

Web pages as clean Markdown and JSON for AI agents, in fewer tokens, with an Evidence Record.

First seen 9 Oct 2026. Evidence as of 28 Oct 2026.

2
Tools
From an anonymous probe
1
Source listings
Each with its own history
24
Recorded changes
Since first seen

Tools

ToolDescriptionBehaviour
mapList a site's URLs without reading each page: the start page's links (read over HTTP alone) and the entries of the sitemaps the site declares, within one deadline. URLs stay in scope (the start host and its www twin, the start path's subtree, assets left out, similar URLs folded) and robots.txt applies unless ignoreRobotsTxt is set; what was left out is counted. No page title is fetched. At the deadline the answer is what was found (partial, stoppedBy timeout). Compact by default; debug adds each link's evidence and the sources read. Not for: reading pages, as one page body is read at most.Read-only
scrapeRead one URL as Markdown with its Evidence Record (final URL, fetch time, HTTP status, robots.txt decision, hashes), over HTTP first and in the browser when the page needs it. Not for: many URLs or a whole site section. Compact by default; debug adds the full audit. warnings name what the content cannot vouch for (client_rendered_suspected, robots_overridden), and agentHints the next step.Read-only

Change history

  1. scrape: input schema changed
  2. scrape: input schema changed
  3. scrape: input schema changed
  4. scrape: description changed (+"Read" +"as Markdown with its Evidence Record (final URL, fetch time, HTTP status, robots.txt decision, hashes), over HTTP first and in the browser when" +"page needs it. Not for: many URLs or a whole site section.")
  5. map: input schema changed
  6. map: description changed (+"reading" +"page's" +"over HTTP alone)")
  7. Description changed (registry)
  8. scrape: title changed
  9. scrape: input schema changed (-access, -actions, -handoff, -lane, -lockdown, -maxAge, -minAge, -robotsOverride, -skipTlsVerification, -storeInCache, -waitFor)
  10. scrape: annotations changed
  11. map: title changed
  12. map: input schema changed (-ignoreRobotsTxt)
  13. map: annotations changed
  14. server instructions changed (+"Octocrawl reads web pages as Markdown and attaches an Evidence Record to each (final URL, fetch time, HTTP status, robots.txt decision, content hashes). A blocked, empty or wrong page comes back as a result whose status says so, with a failureReason or blockReason: it is evidence about the page, not an error, and not an empty success. Start with: one known URL: scrape; the URLs of a site without reading each page: map. Read a result in three parts: status is the fetch; verification.status is the task contract sent in verify (not_requested when none was); readiness says whether the part of the page that was needed had loaded. agentHints, when present, name the next step. An error answer is about the call, not about the page. This connection has no key: pages are read over HTTP alone, within a daily allowance per address, so a page that fills itself in with scripts comes back as a shell (client_rendered_suspected). Signing in or a key adds the browser lane and spends the account's credits. Crawls, batches, Monitors and saved logins run on a server on the person's computer: npx octocrawl serve, with the @octocrawl/mcp server.")
  15. Name changed from "io.github.77777R7/octocrawl" to "Octocrawl" (registry)
  16. scrape: input schema changed
  17. scrape: description changed (-"metadata.scrapeId names the call's record for get_scrape.")
  18. Website changed (registry)
  19. scrape: input schema changed
  20. scrape_product: tool removed
  21. scrape: input schema changed (+verify)
  22. scrape: annotations changed
  23. scrape: input schema changed (+access, +lane)
  24. Listed (registry)
Source listings
SourceListingFirst seenLast seenVersions
Official MCP Registryio.github.77777R7/octocrawl9 Oct 202628 Oct 20264