crawlforge: Open source alternative to Screaming Frog SEO Spider — LLM-native, MCP server included, 269 rules

by mario-hernandez · MCP Server · ★ 8

Last updated: · Indexed by AgentSkillsHub · Auto-synced every 8h

Security audit verdict: SAFE · quality 67/100

None of the 42 rules matched — no credential harvesting, no data exfiltration, no curl-pipe-shell installer. Scanned against 42 published rules built on the SlowMist agent-security taxonomy, refreshed every 8 hours. Full audit →

The quality score combines nine heuristic checks of the repo and its README: completeness, clarity, specificity, examples, structure, agent readiness, step-by-step content, instruction quality and security notes. It rates how well the project explains itself to an agent, not the code.

claudeclaude-codecrawlerduckdbmcpmcp-serveropen-sourcepythonscreaming-frogscreaming-frog-alternative

Quick Facts

Stars8
Forks0
LanguagePython
CategoryMCP Server
LicenseMIT
Quality Score67/100
Last Updated2026-05-06
Created2026-05-06
Platformsclaude-code, mcp, python
Est. Tokens~15k

About crawlforge

Open source alternative to Screaming Frog SEO Spider — LLM-native, MCP server included, 269 rules, DuckDB+Parquet storage, MIT licensed.

crawlforge alternative? Top 6 similar tools

Looking for a crawlforge alternative? If you're comparing crawlforge with other mcp server tools, these 6 projects are the closest alternatives on Agent Skills Hub — ranked by topic overlap, star count, and community traction.

  • screaming-frog-mcp by bzsasson · ⭐ 93

    Headless MCP server for Screaming Frog SEO Spider – run crawls, export and analyze crawl data via the CLI. Sma

  • brain-mcp by mordechaipotash · ⭐ 78

    The recorder for your AI conversations — local-first MCP memory that keeps byte-exact copies of your Claude Co

  • youtube-connector-mcp by ShellyDeng08 · ⭐ 71

    MCP server for YouTube — search videos, get transcripts, channels, and playlists. Works with Claude, Cursor &

  • intellegix-code-agent-toolkit by intellegix · ⭐ 57

    Automated loop driver, slash commands, council automation, MCP browser bridge, and portfolio governance for Cl

  • skills by squirrelscan · ⭐ 99

    Agent skills for squirrelscan, the website QA tool for your coding agent (300+ audit rules).

  • claudekit by duthaho · ⭐ 97

    A verification-first engineering toolkit for Claude Code. Built for senior ICs and tech leads who already know

More MCP Server Tools

Explore other popular mcp server tools:

View all MCP Server tools →

Popular Python Agent Tools

How this security grade is produced

Grades come from a rule-based scan built on the SlowMist agent-security taxonomy: 42 published rules, covering credential harvesting, data exfiltration, curl | sh installers and more. It is a first-layer scan, not a manual audit — we say so rather than overstate it.

The scale of the problem is documented independently: Liu et al. (2026), in a study of 31,132 agent skills, report that 26.1% contain security vulnerabilities. Our own full-catalog census is published as a citable open dataset.

Sources & who's responsible:

View on GitHub → Browse MCP Server tools