crawl4ai: Official Crawl4AI Cloud: a hosted web scraping, extraction, and search API for AI agents

by unclecode · MCP Server · ★ 87.6k

Last updated: · Indexed by AgentSkillsHub · Auto-synced every 8h

How to install crawl4ai

pip install -U crawl4ai
git clone https://github.com/unclecode/crawl4ai.git

Taken from the project's README on GitHub. Check the security audit before running it.

Security audit verdict: SAFE · quality 64/100

None of the 44 rules matched — no credential harvesting, no data exfiltration, no curl-pipe-shell installer. Scanned against 44 published rules built on the SlowMist agent-security taxonomy, refreshed every 8 hours. Full audit →

The quality score combines nine heuristic checks of the repo and its README: completeness, clarity, specificity, examples, structure, agent readiness, step-by-step content, instruction quality and security notes. It rates how well the project explains itself to an agent, not the code.

aiai-agentscrawl4ai-cloudcrawlerdata-extractionllmmarkdownmcpopen-sourceplaywright

Quick Facts

Stars87,564
Forks9,130
LanguagePython
CategoryMCP Server
LicenseApache-2.0
Quality Score64/100
Open Issues348
Last Updated2026-11-19
Created2024-05-09
Platformsbrowser, mcp, python
Est. Tokens~20k

About crawl4ai

🚀🤖 Crawl4AI: the open-source web crawler for LLMs and AI agents Latest: v0.9.4 (23 Sep 2026) · all releases → <img alt="Crawl4AI Cloud is live. Soft launch: your first $10 is on us until 31 December 2026, no card. Get your key." src="https://raw.githubusercontent.com/unclecode/crawl4ai/

Compatible Skills

These tools work well together with crawl4ai for enhanced workflows:

  • scrapai-cli — semantic(0.36)+complementary+rare_topics+same_lang+shared_platform (62%)
  • Scrapling — semantic(0.45)+rare_topics+same_lang+similar_pop+shared_platform (60%)

crawl4ai alternative? Top 6 similar tools

Looking for a crawl4ai alternative? If you're comparing crawl4ai with other mcp server tools, these 6 projects are the closest alternatives on Agent Skills Hub — ranked by topic overlap, star count, and community traction.

  • Scrapling by D4Vinci · ⭐ 103.9k

    🕷️ The BEST Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't

  • browser-use by browser-use · ⭐ 121.5k

    Open-source Python library for AI browser agents, plus Browser Use Cloud: stealth browsers and a hosted agent

  • stagehand by browserbase · ⭐ 26.1k

    The SDK to extract data and interact with any site on the web. Get started with Claude Code, Codex, Eve, Mastr

  • moli by lexmount · ⭐ 24.1k

    Best headless browser for AI agents. Lite, Fast, High-Compatibility. Built in Rust

  • open-webui by open-webui · ⭐ 158.9k

    User-friendly AI Interface (Supports Ollama, OpenAI API, ...)

  • ragflow by infiniflow · ⭐ 94.0k

    RAGFlow is a leading open-source Retrieval-Augmented Generation (RAG) engine that fuses cutting-edge RAG with

More MCP Server Tools

Explore other popular mcp server tools:

View all MCP Server tools →

Popular Python Agent Tools

How this security grade is produced

Grades come from a rule-based scan built on the SlowMist agent-security taxonomy: 44 published rules, covering credential harvesting, data exfiltration, curl | sh installers and more. It is a first-layer scan, not a manual audit — we say so rather than overstate it.

The scale of the problem is documented independently: Liu et al. (2026), in a study of 31,132 agent skills, report that 26.1% contain security vulnerabilities. Our own full-catalog census is published as a citable open dataset.

Sources & who's responsible:

View on GitHub → Browse MCP Server tools