by spronta · MCP Server · ★ 114
Last updated: · Indexed by AgentSkillsHub · Auto-synced every 8h
npm i -g crawlie
git clone https://github.com/spronta/crawlieTaken from the project's README on GitHub. Check the security audit before running it.
None of the 42 rules matched — no credential harvesting, no data exfiltration, no curl-pipe-shell installer. Scanned against 42 published rules built on the SlowMist agent-security taxonomy, refreshed every 8 hours. Full audit →
The quality score combines nine heuristic checks of the repo and its README: completeness, clarity, specificity, examples, structure, agent readiness, step-by-step content, instruction quality and security notes. It rates how well the project explains itself to an agent, not the code.
| Stars | 114 |
| Forks | 16 |
| Language | Rust |
| Category | MCP Server |
| Quality Score | 72/100 |
| Open Issues | 2 |
| Last Updated | 2026-07-18 |
| Created | 2026-06-18 |
| Platforms | mcp, rust |
| Est. Tokens | ~25k |
crawlie The fast, free, open-source technical SEO + GEO crawler — built for humans and agents. Crawl any site for broken links, redirects, missing metadata, and 40+ SEO & Generative-Engine checks — with plain-English guidance on every fix. Runs locally, ships a CLI and an MCP server, and costs nothing.
These tools work well together with crawlie for enhanced workflows:
Looking for a crawlie alternative? If you're comparing crawlie with other mcp server tools, these 6 projects are the closest alternatives on Agent Skills Hub — ranked by topic overlap, star count, and community traction.
A curated list of indie-built AI startups — bootstrapped, pre-seed, and angel-funded products only.
"ULTRASHIP" Claude Code plugin — 39 skills, 33 tools, 11 agents for ship-ready workflows: planning, review, pe
Claude Code plugin that tells you why your site won't get cited by AI — and fixes the discoverability files au
The website QA tool for your coding agent. 300+ audit rules across SEO, performance, security, accessibility a
SEOBuild Onpage - The first AI agent that writes pages Google ranks AND LLMs cite. One command in, ranking pag
On Page Agent -- The first AI agent that writes pages Google ranks AND LLMs cite. One command in, ranking page
Explore other popular mcp server tools:
Grades come from a rule-based scan built on the SlowMist agent-security taxonomy: 42 published rules, covering credential harvesting, data exfiltration, curl | sh installers and more. It is a first-layer scan, not a manual audit — we say so rather than overstate it.
The scale of the problem is documented independently: Liu et al. (2026), in a study of 31,132 agent skills, report that 26.1% contain security vulnerabilities. Our own full-catalog census is published as a citable open dataset.
Sources & who's responsible: