Firecrawl

Firecrawl is an AI-powered web scraping API that turns any website into LLM-ready markdown, JSON, or structured data with a single call. It handles JavaScript rendering, proxies, anti-bot protection, and rate limiting natively, so Vibe Builders can pipe live web data into their agents without writing scrapers, and developers can replace fragile BeautifulSoup or Playwright stacks with one managed endpoint. Used by Zapier, Replit, and LangChain as the default scraping layer for AI pipelines.

Key Features:

• •/scrape endpoint Single-call conversion of any URL into Markdown, JSON, HTML, links, or screenshots. Supports schema-based structured extraction via JSON Schema or Pydantic models passed as the extraction contract.
• •/crawl endpoint Discovers and scrapes entire sites respecting robots.txt, returning clean Markdown for every page. Webhook delivery or polling, pagination handled automatically, up to 10,000 pages per job on paid tiers.
• •/map endpoint Returns every URL on a website in seconds, giving AI agents a sitemap without running a full crawl. Useful for RAG ingestion planning and coverage checks before committing credit spend.
• •/extract endpoint Schema-driven structured extraction: pass a JSON Schema, get back typed data. Paid add-on starting at $89/mo on top of the base credit plan, powered by an LLM that fills your schema from raw page content.
• •Official MCP server Native Model Context Protocol server lets Claude Desktop, Cursor, Windsurf, and other MCP-compatible clients scrape the web directly from inside the chat window, with no custom integration required.
• •CLI + typed SDKs First-party TypeScript (`npm install @mendable/firecrawl-js`) and Python (`pip install firecrawl-py`) clients, plus a CLI for terminal work. First-party integrations with LangChain, LlamaIndex, Dify, Flowise, n8n, and Zapier ship in the docs.
• •Open source under AGPL-3.0 The full Firecrawl stack is released on GitHub (firecrawl/firecrawl) and can be self-hosted. The cloud service is the paid convenience layer; teams with compliance constraints can run the whole pipeline on their own infrastructure.
• •Managed anti-bot handling Proxy rotation, headless Chromium rendering, and stealth techniques for Cloudflare and basic bot protection are built in, removing the need to buy a separate proxy pool or run Playwright in your own infrastructure.

Limitations:

• •Free tier runs out in seconds on real tasks The free plan caps at 10 scrape requests per minute and 1 crawl per minute. Any meaningful research task, whether auditing a competitor site, pulling product data, or gathering leads, exhausts the free allowance almost immediately.
• •Struggles with heavily protected sites In independent 2026 testing, Firecrawl failed to extract data from 5 out of 6 sites using serious bot protection. Social platforms (Instagram, YouTube, TikTok) return an explicit "no longer supported" error. Cloudflare-protected sites produce inconsistent results even with enhanced proxy enabled.
• •AI extraction is a separate, additional subscription The Extract/AI feature that pulls structured data from pages runs on a separate token-based plan starting at $89/month, charged on top of your existing credit plan. The all-in cost is substantially higher than the headline price suggests.
• •No interactive browser automation Firecrawl cannot scroll infinite-scroll pages, click through pagination, or fill forms. Content that requires user interaction to load is out of reach, and you will need Browserbase or Playwright for those cases.
• •Agent endpoint burns credits unpredictably The AI Agent feature can consume 1,500+ credits on a single complex query with no upfront cost estimate shown. It is easy to drain a monthly credit budget on just a handful of research tasks.

Pricing:

• •Free 500 lifetime credits, 10 scrapes/min, 1 crawl/min. Enough to test the API but not to run production workloads.
• •Hobby ($16/mo) 3,000 credits/mo, API keys for 2 team members, email support. The entry point for most solo Vibe Builders using Firecrawl inside n8n or Zapier.
• •Standard ($83/mo) 100,000 credits/mo, 100 concurrent requests, stealth proxy mode, priority support. The default choice for an AI agent in production with moderate scraping volume.
• •Growth ($333/mo) 500,000 credits/mo, 500 concurrent requests. Built for teams running RAG ingestion pipelines or research agents at scale.
• •Scale (custom) Dedicated rate limits, enterprise SLAs, VPC deployment, and compliance contracts. Quote-based pricing for high-volume customers.
• •Extract/AI add-on ($89/mo) Token-based structured extraction, billed on top of any credit plan. Required for schema-driven output from the /extract endpoint.