Submit a resource

Navigation

Back to directory

Firecrawl

A web data API that searches, scrapes and crawls websites and returns clean Markdown or structured JSON for AI agents, available as open source or a hosted service.

Visit website (opens in a new tab)firecrawl.dev
Firecrawl

Firecrawl turns web pages into data for AI applications. Its core endpoints search the web and return full page content, scrape a URL into Markdown, HTML, screenshots or structured JSON, and interact with a page before extracting from it.

Further features crawl every URL of a site, map a site's URLs, run batch scrapes asynchronously and gather data from a description. It handles JavaScript-heavy pages and parses PDFs and DOCX files, and it can be connected to AI agents and MCP clients.

It has Python and Node.js SDKs, a CLI and a REST API. It is written in TypeScript and licensed under AGPL-3.0.

More in RAG

See all 540

  • ai·rete·rag

    ai-rete-rag.com

    A platform that pairs a Rete rule engine with retrieval-augmented generation, so deterministic rules decide an outcome and grounded retrieval explains the decision in plain language.

  • Xtriever

    github.com

    A hybrid retrieval engine for retrieval-augmented generation, written in Rust to run on-device on a phone or a laptop, combining lexical BM25 search, dense embedding search, reciprocal-rank fusion and cross-encoder re-ranking with no server.

  • FileFlux

    github.com

    A .NET library that turns PDF, DOCX, HWP and other documents into optimized chunks for RAG systems, through a five-stage pipeline built on high-performance Rust FFI readers.

Cookies

We use necessary cookies to run the site. Measurement and marketing are optional; neither is used today. Privacy policy