SkillEvaluator: Multi-tier framework for evaluating AI agent skills and plugins with quality gates, security scanning, harness

by NVIDIA · Codex Skill · ★ 4.0k

Last updated: · Indexed by AgentSkillsHub · Auto-synced every 8h

How to install SkillEvaluator

uv tool install --python 3.13 "skillevaluator[all] @ git+https://github.com/NVIDIA/SkillEvaluator.git"

Taken from the project's README on GitHub. Check the security audit before running it.

Security audit verdict: SAFE · quality 69/100

None of the 44 rules matched — no credential harvesting, no data exfiltration, no curl-pipe-shell installer. Scanned against 44 published rules built on the SlowMist agent-security taxonomy, refreshed every 8 hours. Full audit →

The quality score combines nine heuristic checks of the repo and its README: completeness, clarity, specificity, examples, structure, agent readiness, step-by-step content, instruction quality and security notes. It rates how well the project explains itself to an agent, not the code.

agent-evaluationagent-pluginsagent-securityagent-skillsagentic-aibenchmarkci-templatesclaude-codecodexevaluate

Quick Facts

Stars4,046
Forks535
LanguagePython
CategoryCodex Skill
LicenseApache-2.0
Quality Score69/100
Open Issues59
Last Updated2026-11-19
Created2026-06-24
Platformsclaude-code, codex, python
Est. Tokens~15k

About SkillEvaluator

SkillEvaluator SkillEvaluator is an open-source, multi-tier framework for evaluating AI agent artifacts, starting with agent skills: deterministic quality gates, semantic overlap detection, synthetic eval dataset generation, and live agent evaluation. Agent skills are folders of instructions and supporting files that extend AI agents, as defined by the Agent Skills specification.

Compatible Skills

These tools work well together with SkillEvaluator for enhanced workflows:

  • qiaomu-meta-skill — semantic(0.33)+complementary+rare_topics+same_lang+similar_pop+shared_platform (66%)
  • OpenJudge — semantic(0.45)+complementary+same_lang+similar_pop+shared_platform (66%)
  • coder_eval — semantic(0.32)+complementary+rare_topics+same_lang+similar_pop+shared_platform (66%)

SkillEvaluator alternative? Top 6 similar tools

Looking for a SkillEvaluator alternative? If you're comparing SkillEvaluator with other codex skill tools, these 6 projects are the closest alternatives on Agent Skills Hub — ranked by topic overlap, star count, and community traction.

  • skillshare by runkids · ⭐ 3.2k

    📚 Sync skills, agents, MCP, plugins to all AI CLI tools with one command and simplify team sharing.

  • claude-workflow-v2 by CloudAI-X · ⭐ 1.4k

    Universal Claude Code workflow plugin with agents, skills, hooks, and commands

  • modsearch by liustack · ⭐ 989

    🥇 The strongest free web search plugin for DeepSeek Harness, and the search bridge for every model without na

  • mcp-use by mcp-use · ⭐ 11.0k

    The fullstack MCP framework to develop MCP Apps for ChatGPT / Claude & MCP Servers for AI Agents.

  • mobile-mcp by mobile-next · ⭐ 10.7k

    Mobile MCP: let AI agents drive your iOS and Android apps. MCP server for simulators, emulators and real devic

  • refly by refly-ai · ⭐ 7.6k

    The first open-source agent skills builder. Define skills by vibe workflow, run on Claude Code, Cursor, Codex

More Codex Skill Tools

Explore other popular codex skill tools:

View all Codex Skill tools →

Popular Python Agent Tools

How this security grade is produced

Grades come from a rule-based scan built on the SlowMist agent-security taxonomy: 44 published rules, covering credential harvesting, data exfiltration, curl | sh installers and more. It is a first-layer scan, not a manual audit — we say so rather than overstate it.

The scale of the problem is documented independently: Liu et al. (2026), in a study of 31,132 agent skills, report that 26.1% contain security vulnerabilities. Our own full-catalog census is published as a citable open dataset.

Sources & who's responsible:

View on GitHub → Browse Codex Skill tools