AI

AI Crawler Checker — Robots.txt Policy for Six Bots MCP

Check declared robots.txt permissions for six AI crawlers at URL paths

not scanned Remote · Streamable HTTP OAuth 2.1 tools not listed v1.0.2
No reviews yet by kriton-dev The publisher’s own endpoint — this marketplace is not in the path
—
trust score
—
calls last 7d
remote
at its publisher
—
p95 latency

What it does

Check declared robots.txt permissions for six AI crawlers at URL paths

Quickstart

# 1 — add it to Claude Code claude mcp add --scope user --transport http ai-crawler-checker-robots-txt-policy-for-six-mcp 'https://mcp.apify.com/?tools=hermeskd/ai-crawler-access-check'

AI Crawler Checker — Robots.txt Policy for Six Bots costs nothing on mcprush: there is no plan to choose here, no cap to set and nothing mcprush can bill you for.

Publishernot claimed
KD
kriton-dev29 servers
12 categories
All 29 servers

Collected from a public index. Nobody has claimed this account, so nothing here was written by its author — claim it if it is yours.

More in this category

See also

See all