Head-to-head comparison of coding agents (Claude Code, Aider, Codex, etc.) on custom tasks with pass rate, cost, time, and consistency metrics. Use when choosing between coding agents, or when a change to an agent setup needs measured pass rate, cost, and time rather than an impression.
Security grade of this SKILL.md: SAFE · how we grade
How to install agent-eval
npx skills add affaan-m/ECC --skill agent-eval -a claude-code
The skills CLI copies this one skill into Claude Code (use -a for another agent). Or read its SKILL.md on GitHub.
What the SKILL.md says
A lightweight CLI tool for comparing coding agents head-to-head on reproducible tasks. Every "which coding agent is best?" comparison runs on vibes — this tool systematizes it. Comparing coding agents (Claude Code, Aider, Codex, etc.) on your own codebase. Measuring agent performance before adopting a new tool or model. Running regression checks when an agent updates its model or tooling. Producing data-backed agent selection decisions for a team. Note: Install agent-eval from its repository after reviewing the source. Define tasks declaratively. Each task specifies what to do, which files to touch, and how to judge success: Each agent run gets its own git worktree — no Docker required. This provides reproducibility isolation so agents cannot interfere with each other or corrupt the base repo. Metric What It Measures Pass rate Did the agent produce code that passes the judge?
Other skills in affaan-m/ECC
accessibilityDesign, implement, and audit accessible UI to WCAG 2.2 Level AA across Web, iOS, and Android — semantic ARIA roles and labels, accessibilit…
agent-architecture-auditFull-stack diagnostic for agent and LLM applications. Audits the 12-layer agent stack for wrapper regression, memory pollution, tool discip…
agent-harness-constructionDesign and optimize AI agent action spaces, tool definitions, and observation formatting for higher completion rates. Use when defining or…
agent-introspection-debuggingStructured self-debugging workflow for AI agent failures using capture, diagnosis, contained recovery, and introspection reports. Use when…
agent-payment-x402Add x402 payment execution to AI agents with per-task budgets, spending controls, and non-custodial wallets. Supports Base through agentwal…
agent-runtime-gateway-smoke-testVerify a local agent API, temporary gateway tunnel, and remote sandbox callback with a tool-free task, then restore the original app connec…
agent-security-hardeningSecurity hardening guidance for AI agent frameworks that process untrusted content, invoke tools, write workspace files, manage runtime ide…
agent-self-evaluationUse after completing any non-trivial task. The agent self-rates its output on 5 axes — accuracy, completeness, clarity, actionability, conc…
agent-sortBuild an evidence-backed ECC install plan for a specific repo by sorting skills, commands, rules, hooks, and extras into DAILY vs LIBRARY b…
agentic-engineeringOperate as an agentic engineer using eval-first execution, decomposition, and cost-aware model routing. Use when planning or executing engi…
agentic-osBuild persistent multi-agent operating systems on Claude Code. Covers kernel architecture, specialist agents, slash commands, file-based me…
ai-first-engineeringEngineering operating model for teams where AI agents generate a large share of implementation output. Use when setting team process, revie…
ai-regression-testingRegression testing strategies for AI-assisted development. Sandbox-mode API testing without database dependencies, automated bug-check work…
android-clean-architectureClean Architecture patterns for Android and Kotlin Multiplatform projects — module structure, dependency rules, UseCases, Repositories, and…
angular-developerGenerates Angular code and provides architectural guidance. Trigger when creating projects, components, or services, or for best practices…
api-connector-builderBuild a new API connector or provider by matching the target repo's existing integration pattern exactly. Use when adding one more integrat…
api-designREST API design patterns including resource naming, status codes, pagination, filtering, error responses, versioning, and rate limiting for…
architecture-decision-recordsCapture architectural decisions as numbered ADR markdown files in docs/adr/ with context, alternatives considered, consequences, and an ind…
article-writingWrite articles, guides, blog posts, tutorials, newsletter issues, and other long-form content in a distinctive voice derived from supplied…
automation-audit-opsEvidence-first automation inventory and overlap audit workflow for ECC. Use when the user wants to know which jobs, hooks, connectors, MCP…