Vibe Coding Discoverfirecrawl
View on GitHubThe web data API to search, scrape, and interact at scale. 🔥
Firecrawl is an open-source web data API that searches, scrapes, crawls, and interacts with sites, returning clean markdown or structured JSON for LLMs. Ships SDKs, a CLI, an MCP server, and agent skills, and can also run an AI agent that gathers data from a prompt.
Use Cases
Convert any web page to LLM-ready markdownScrape a full site or crawl all URLs with one requestFeed clean web content into RAG pipelines and vector storesGive AI agents and MCP clients real-time web searchExtract structured JSON from pages via schemaAgentic data gathering from a natural-language promptBatch scrape thousands of URLs asynchronouslyInteract with pages (click, scroll, type) to retrieve gated contentParse web-hosted PDFs and DOCX into textMap/discover all URLs on a domainCompetitor and pricing research automationScrape JS-heavy pages without proxy setup
Built With
- Language
- TypeScript
- Frameworks
- MCP · Node.js · Python SDK
Tags
web-scraping · web-crawling · markdown · llm · data-extraction · mcp · web-search · structured-data · browser-automation · api · crawler · html-to-markdown · ai-agents · pdf-parsing · cli · typescript