How do I install Robots.txt & AI Crawler Checker?

Pick a client above and copy the generated config. Cursor and VS Code also have a deep link when the server can run locally or remotely.

Is Robots.txt & AI Crawler Checker official?

No. Registry-listed is not the same as vendor-official. Read the methodology for the difference.

This listing scores 52 (Fair). The formula is published on the methodology page.

Robots.txt & AI Crawler Checker MCP Server | MCPpedia

robots.txt tester (RFC 9309), AI crawler access (GPTBot, ClaudeBot...) and sitemap check.

RegistryRemote

52Fair

~/Library/Application Support/Claude/claude_desktop_config.json

{
  "mcpServers": {
    "robots-check-robots-check": {
      "url": "https://robots-check.quietworklabs.workers.dev/mcp"
    }
  }
}

This server does not declare environment variables.

83Excellent

Parse Korean documents (HWP, HWPX, PDF, XLSX, DOCX, PPTX) to Markdown; fill forms, diff, render.

Registrychrisryugj2.5K stars31.7K/wk

77Good

MCP server for Firecrawl — web search, scraping, and biomedical/arXiv paper search.