spider: Foundational low latency web data collecting in Rust

by spider-rs · Agent Tool · ★ 2.8k

Last updated: · Indexed by AgentSkillsHub · Auto-synced every 8h

Security audit verdict: SAFE · quality 69/100

None of the 42 rules matched — no credential harvesting, no data exfiltration, no curl-pipe-shell installer. Scanned against 42 published rules built on the SlowMist agent-security taxonomy, refreshed every 8 hours. Full audit →

The quality score combines nine heuristic checks of the repo and its README: completeness, clarity, specificity, examples, structure, agent readiness, step-by-step content, instruction quality and security notes. It rates how well the project explains itself to an agent, not the code.

ai-agentautomationcrawlerheadless-chromerustscrapingspiderweb-crawlerweb-dataweb-scraping

Quick Facts

Stars2,835
Forks323
LanguageRust
CategoryAgent Tool
LicenseMIT
Quality Score69/100
Open Issues2
Last Updated2026-11-06
Created2018-01-07
Platformsbrowser, rust
Est. Tokens~23k

About spider

Spider The fastest web crawler and scraper for Rust. spider.cloud · Guides · Docs · Examples · Discord Spider is a concurrency-first crawling engine written in Rust. It str

Compatible Skills

These tools work well together with spider for enhanced workflows:

  • webclaw — semantic(0.54)+complementary+rare_topics+same_lang+similar_pop+shared_platform (78%)
  • brightdata-mcp — semantic(0.46)+complementary+rare_topics+similar_pop+shared_platform (65%)
  • graphlit-mcp-server — semantic(0.43)+complementary+rare_topics+similar_pop+shared_platform (59%)

spider alternative? Top 6 similar tools

Looking for a spider alternative? If you're comparing spider with other agent tool tools, these 6 projects are the closest alternatives on Agent Skills Hub — ranked by topic overlap, star count, and community traction.

  • wigolo by KnockOutEZ · ⭐ 5.5k

    The go-to web for your AI coding agent — local-first search, fetch, crawl & research over MCP. No API keys, no

  • brightdata-mcp by brightdata · ⭐ 2.7k

    A powerful Model Context Protocol (MCP) server that provides an all-in-one solution for public web access.

  • webclaw by 0xMassi · ⭐ 2.4k

    Fast, local-first web content extraction for LLMs. Scrape, crawl, extract structured data — all from Rust. CLI

  • catbus by cv-cat · ⭐ 578

    一个命令操作小红书、抖音、TikTok、B 站、快手、微博、闲鱼、淘宝、京东、X:统一的命令与 JSON 输出,给人和 AI Agent 用

  • nanobrowser by nanobrowser · ⭐ 14.2k

    Open-source AI agent for web automation. Use your real, logged-in browser to automate daily tasks.

  • firecrawl-mcp-server by firecrawl · ⭐ 7.8k

    Official Firecrawl MCP server, the context API for AI agents: search, scrape, and interact with the web, plus

More Agent Tool Tools

Explore other popular agent tool tools:

View all Agent Tool tools →

Popular Rust Agent Tools

How this security grade is produced

Grades come from a rule-based scan built on the SlowMist agent-security taxonomy: 42 published rules, covering credential harvesting, data exfiltration, curl | sh installers and more. It is a first-layer scan, not a manual audit — we say so rather than overstate it.

The scale of the problem is documented independently: Liu et al. (2026), in a study of 31,132 agent skills, report that 26.1% contain security vulnerabilities. Our own full-catalog census is published as a citable open dataset.

Sources & who's responsible:

View on GitHub → Browse Agent Tool tools