Skip to content
MCPpedia
Submit
Use case

Scrape and extract web data with an AI assistant

Scraping servers turn web pages into something a model can use. They fetch a URL, render JavaScript when needed, strip navigation and ads, and return clean Markdown or structured fields, and many can crawl a whole site or follow a sitemap. That covers jobs like building a product price list, collecting documentation for a project, or monitoring competitor pages. Hosted services handle proxies and blocking, while local tools are free but slower. Respect robots.txt and each site's terms, keep request rates polite, and avoid collecting personal data you do not need.

751 matching serversRanked by trust score
All use cases
Try it

Example prompts

Paste one into Claude, Cursor, or any MCP client once a matching server is installed.

  • Scrape the pricing page at this URL and return each plan with its price and limits.

  • Crawl the docs site and give me a list of every page title and URL.

  • Extract the product name, price, and rating from these five product pages into a table.

Top matches

MCP servers for web scraping

The 12 strongest of 751 matching servers. Servers that name the topic directly come first.

More in Web scraping
  • Trust score 77 of 100, Good

    Firecrawl, the context API for AI agents: search, scrape, interact with the web, plus paper search.

    RegistryRemote
    7.7K stars—/wk weekly downloads— tools
  • Trust score 74 of 100, Good

    apify-mcp-server

    by apify · TypeScript

    Extract data from any website with thousands of scrapers, crawlers, and automations on Apify Store ⚡

    RegistryRemote
    11.5K stars—/wk weekly downloads— tools
  • Xquik MCP Server logo
    Trust score 74 of 100, Good

    Xquik MCP Server

    by xquik-dev · JavaScript

    Xquik is the world's fastest and cheapest X Twitter data API with the most complete X data.

    RegistryRemote
    214 stars367/wk weekly downloads— tools
  • Trust score 72 of 100, Good

    Open-source web scraper for AI agents with scrape, crawl, and map tools

    Registry
    1.1K stars—/wk weekly downloads— tools
  • Trust score 71 of 100, Good

    Free proxies that actually work: checked HTTP/SOCKS proxies, and pages fetched through them

    Registry
    55 stars—/wk weekly downloads— tools
  • Trust score 69 of 100, Good

    Crawl documentation sites into local corpora agents can search, read, and diff fully offline.

    Registry
    107 stars—/wk weekly downloads— tools
  • Trust score 68 of 100, Good

    Apify

    by pipeworx-io · TypeScript

    Apify MCP — run web-scraping Actors and fetch their dataset results.

    RegistryRemote
    1 stars28/wk weekly downloads— tools
  • Trust score 65 of 100, Good

    Comment Scraper

    by supertj · TypeScript

    Read your Comment Scraper exports of Reddit and Hacker News threads. Local and read-only.

    Registry
    0 stars—/wk weekly downloads— tools
  • Trust score 65 of 100, Good

    Check what AI crawlers may read and what a page permits, generate llms.txt and schema.

    Registry
    0 stars—/wk weekly downloads— tools
  • Trust score 65 of 100, Good

    Google SERP + AI Overviews, AI Mode, SEA news mentions, Maps, Jobs, Trends and 13 more data tools.

    Registry
    0 stars—/wk weekly downloads— tools
  • Jobs Data logo
    Trust score 62 of 100, Good

    YouTube, TikTok, Instagram and Twitch growth history and 7/30/90-day gains from SocialBlade.

    RegistryRemote
    0 stars—/wk weekly downloads— tools
  • Atlas Web Compiler & Scraper logo
    Trust score 59 of 100, Fair

    Deterministic Zero-LLM Web Compiler, Scraper, Crawler, and Site Mapper for AI Agents.

    RegistryRemote
    — stars582/wk weekly downloads— tools
Setup

How to set it up

  1. Pick a server. Firecrawl MCP Server is the strongest match. Check its tools and the API keys it needs on its page.
  2. Copy the config for your client. Open the install guide for Claude Desktop, Claude Code, Cursor, or VS Code, and paste the generated block into the client's config file. The client guides show where each config file lives.
  3. Restart and try a prompt. Restart the client so it loads the server, then paste one of the example prompts above.
FAQ

Questions about web scraping

What is the best MCP server for web scraping?

Firecrawl MCP Server ranks first here with a trust score of 77/100, followed by apify-mcp-server (74) and Xquik MCP Server (74). Servers that name the topic directly come first, each group ordered by the public trust score, which nobody can pay for. See the methodology for how it is calculated.

How do I set up Firecrawl MCP Server in Claude or Cursor?

Open the install guide for Claude Desktop or Cursor, copy the generated config into the client's config file, add any API keys the server asks for, then restart the client. The client guides cover VS Code, Claude Code, Windsurf, and more.

Can I use a hosted server instead of installing one locally?

6 of the 12 servers listed here can connect remotely over HTTP, so there is nothing to run on your machine: Firecrawl MCP Server, apify-mcp-server, Xquik MCP Server, and others. The rest run locally over stdio.

How many MCP servers can do this?

MCPpedia lists 751 published servers that match this use case. This page shows the 12 strongest matches; thin and duplicate listings are left out.

Keep exploring
All use cases