caliper: Run your real agent with and without your skills, MCPs, and rules

by edonadei · MCP Server · ★ 222

Last updated: · Indexed by AgentSkillsHub · Auto-synced every 8h

How to install caliper

npx skills@latest add edonadei/caliper
pipx install caliper-eval   # requires Python 3.10+

Taken from the project's README on GitHub. Check the security audit before running it.

Security audit verdict: SAFE · quality 67/100

None of the 42 rules matched — no credential harvesting, no data exfiltration, no curl-pipe-shell installer. Scanned against 42 published rules built on the SlowMist agent-security taxonomy, refreshed every 8 hours. Full audit →

The quality score combines nine heuristic checks of the repo and its README: completeness, clarity, specificity, examples, structure, agent readiness, step-by-step content, instruction quality and security notes. It rates how well the project explains itself to an agent, not the code.

ai-agentsclaude-codeclicodexevalsevaluationhermeshermes-agenthermes-skillpi

Quick Facts

Stars222
Forks24
LanguagePython
CategoryMCP Server
LicenseMIT
Quality Score67/100
Open Issues14
Last Updated2026-10-25
Created2026-05-13
Platformsclaude-code, cli, codex, mcp, python
Est. Tokens~17k

About caliper

Caliper Reliability testing for agent skills. Define what success looks like, run your skill k times, and get a pass@k score you can track and compare. Agent skills are hard to test. A skill that works on your machine, on this prompt, today, might fail tomorrow after a model update or a one-line prompt edit.

caliper alternative? Top 6 similar tools

Looking for a caliper alternative? If you're comparing caliper with other mcp server tools, these 6 projects are the closest alternatives on Agent Skills Hub — ranked by topic overlap, star count, and community traction.

  • maestro by ReinaMacCredy · ⭐ 234

    Local-first coordination for human and agent work: durable work, decisions, dispatches, evidence, and prompt-f

  • kindly-web-search-mcp-server by Shelpuk-AI-Technology-Consulting · ⭐ 398

    Kindly Web Search MCP Server: Web search + robust content retrieval for AI coding tools (Claude Code, Codex, C

  • vibepod-cli by VibePod · ⭐ 180

    Unified CLI for running AI coding agents in isolated containers. Includes built-in local metrics collection, H

  • skills by fabricioctelles · ⭐ 121

    A collection of skills for AI agents (Kiro, Cursor, Windsurf, Claude Code, and others). Each skill is a reusab

  • untether by littlebearapps · ⭐ 73

    Run your coding agents from Telegram. A bridge for Claude Code, Codex, OpenCode and Pi: send tasks by text or

  • Skills-Manager by jiweiyeah · ⭐ 991

    Free, open-source desktop manager for AI Agent Skills. Write a skill once, sync it to 32 AI coding tools (Clau

More MCP Server Tools

Explore other popular mcp server tools:

View all MCP Server tools →

Popular Python Agent Tools

How this security grade is produced

Grades come from a rule-based scan built on the SlowMist agent-security taxonomy: 42 published rules, covering credential harvesting, data exfiltration, curl | sh installers and more. It is a first-layer scan, not a manual audit — we say so rather than overstate it.

The scale of the problem is documented independently: Liu et al. (2026), in a study of 31,132 agent skills, report that 26.1% contain security vulnerabilities. Our own full-catalog census is published as a citable open dataset.

Sources & who's responsible:

View on GitHub → Browse MCP Server tools