Skip to content
AgentAtlas

Web Scraping MCP Servers Compared (2026): Bright Data, Firecrawl, Apify, Tavily and Browserbase

Which web scraping MCP server should my AI agent use, and how do I set it up?

For pages that block bots, Bright Data's MCP server is the only one of the five whose docs say it handles anti-bot measures, CAPTCHAs and proxy rotation on every request, and it has the largest free tier (5,000 requests a month). Firecrawl is best for turning pages into clean Markdown or JSON. Apify runs ready-made scrapers from its store. Tavily is built for search. Browserbase drives a cloud browser step by step. All five have a hosted endpoint and an npx package.

Compare: the best MCP servers for scraping, search, browsers, databases, hosting, monitoring, payments, auth and design →

Quick comparison

  • Bright Data: anti-bot, CAPTCHA solving and proxy rotation stated in its MCP docs; search, Markdown scraping, 13 browser tools and structured web_data_* extractors. Free: 5,000 requests/month, no card. Paid: pay as you go from $1.50 per 1,000 requests.
  • Firecrawl: scrape to Markdown or schema JSON, crawl, map, search, a research agent, and firecrawl_interact for browser actions. Free: 1,000 credits/month. Paid: Hobby, $19/month for 5,000 credits.
  • Apify: the agent searches the Apify Store and runs ready-made scrapers (Actors) with search-actors and call-actor, plus a RAG web browser. Free: $5 of usage/month. Paid: Starter, $19/month.
  • Tavily: web search and page extraction built for agents (tavily-search, tavily-extract), plus map and crawl. Free: 1,000 credits/month. Paid: $30 for 4,000 credits, or $0.008 per credit pay as you go.
  • Browserbase: a cloud browser the agent drives with navigate, act, observe and extract. Free: 1 browser hour and 3 concurrent sessions, without CAPTCHA solving or proxies. Paid: Developer plan from $20.
  • Prices and free tiers were checked on each vendor's pricing or docs page on 2026-10-20 and change often.

Bright Data MCP

Try Bright Data free →Partner link

Bright Data's MCP server routes every request through its Web Unlocker, which its docs say handles anti-bot detection, CAPTCHAs and proxy rotation automatically. Use it when a site blocks plain fetches. The hosted endpoint takes your API token in the URL; the local server reads API_TOKEN (the GitHub README's name; one docs page says API_KEY).

Claude Code, hosted endpoint

claude mcp add --transport http brightdata "https://mcp.brightdata.com/mcp?token=YOUR_API_TOKEN"

Any agent, local (mcpServers JSON)

{
  "mcpServers": {
    "brightdata": {
      "command": "npx",
      "args": ["@brightdata/mcp"],
      "env": { "API_TOKEN": "YOUR_API_TOKEN" }
    }
  }
}
  • Main tools: search_engine, scrape_as_markdown, scrape_batch, discover, scraping_browser_* (browser control) and web_data_* (structured data, such as product or profile pages).
  • Optional env vars: WEB_UNLOCKER_ZONE, BROWSER_ZONE, RATE_LIMIT, GROUPS and TOOLS to limit which tools load.

Firecrawl MCP

Try Firecrawl free →Partner link

Firecrawl's server turns pages into Markdown or into JSON that follows a schema you give it, and can crawl or map a whole site. Its docs say to send the API key as a Bearer header, never in the URL, or to use the OAuth endpoint.

Local (mcpServers JSON)

{
  "mcpServers": {
    "firecrawl": {
      "command": "npx",
      "args": ["-y", "firecrawl-mcp"],
      "env": { "FIRECRAWL_API_KEY": "fc-YOUR_KEY" }
    }
  }
}
  • Hosted: https://mcp.firecrawl.dev/v2/mcp with Authorization: Bearer, or https://mcp.firecrawl.dev/v2/mcp-oauth to sign in.
  • Main tools: firecrawl_scrape, firecrawl_search, firecrawl_map, firecrawl_crawl, firecrawl_interact, firecrawl_agent.
  • Self-hosted Firecrawl: set FIRECRAWL_API_URL.

Apify MCP

Apify's server lets the agent find a ready-made scraper for a site (Google Maps, Instagram, Amazon and thousands more in the Apify Store) and run it. The hosted endpoint supports OAuth sign-in, which is the simplest setup.

Claude Code, hosted endpoint with OAuth

claude mcp add --transport http apify https://mcp.apify.com
  • Local: npx -y @apify/actors-mcp-server with APIFY_TOKEN.
  • Main tools: search-actors, fetch-actor-details, call-actor, apify/rag-web-browser, search-apify-docs.

Tavily and Browserbase

  • Tavily: hosted at https://mcp.tavily.com/mcp/ (API key as tavilyApiKey, a Bearer header, or OAuth); local npx -y tavily-mcp@latest with TAVILY_API_KEY. Pick it when the agent mostly needs search results, not full pages.
  • Browserbase: hosted at https://mcp.browserbase.com/mcp with browserbaseApiKey; local npx @browserbasehq/mcp with BROWSERBASE_API_KEY. Pick it for multi-step flows such as logging in and clicking through pages. CAPTCHA solving and proxies are paid-plan features.

Which one to pick

  • Sites that block bots, or structured data from big sites: Bright Data.
  • Clean Markdown or JSON from docs and blogs, or a whole-site crawl: Firecrawl.
  • A scraper someone already built for a specific site: Apify.
  • Search answers with sources: Tavily.
  • Clicking through a site like a person: Browserbase.
  • Keep API keys out of committed files: use environment variables or the OAuth endpoints where the vendor offers them.

Sources

More guides

Reviewed Oct 20, 2026. Settings change often; the linked vendor docs are the source of truth.

Built an MCP server? Get it listed everywhere →

Shipping an Android app? Google Play needs 12 testers for 14 days first. Get them with Tester Exchange →