uaef: Universal Agent Evaluation Framework (UAEF) is a framework-agnostic evaluation system for AI agents

by awslabs · Agent Tool · ★ 91

Last updated: · Indexed by AgentSkillsHub · Auto-synced every 8h

How to install uaef

git clone https://github.com/awslabs/uaef.git
uv pip install ./packages/uaef-0.2.0-py3-none-any.whl

Taken from the project's README on GitHub.

Security audit verdict: SAFE · quality 66/100

None of the 44 rules matched — no credential harvesting, no data exfiltration, no curl-pipe-shell installer. Scanned against 44 published rules built on the SlowMist agent-security taxonomy, refreshed every 8 hours.

The quality score combines nine heuristic checks of the repo and its README: completeness, clarity, specificity, examples, structure, agent readiness, step-by-step content, instruction quality and security notes. It rates how well the project explains itself to an agent, not the code.

agentagentcoreagentic-aiagentsawsdeepevalevalevalsevaluationevaluation-framework

Quick Facts

Stars91
Forks7
LanguagePython
CategoryAgent Tool
LicenseApache-2.0
Quality Score66/100
Open Issues11
Last Updated2026-11-13
Created2026-09-15
Platformsaws, python
Est. Tokens~17k

About uaef

Universal Agent Evaluation Framework (UAEF) A framework-agnostic evaluation system for AI agents. Plug it into any agentic platform to measure agent quality across 36 built-in metrics, persist results to AWS, and track performance over time. UAEF works with LangGraph, LangChain, AWS Bedrock, Langfuse, Strands, AgentCore, or any custom framework via adapters.

uaef alternative? Top 6 similar tools

Looking for a uaef alternative? If you're comparing uaef with other agent tool tools, these 6 projects are the closest alternatives on Agent Skills Hub — ranked by topic overlap, star count, and community traction.

  • eval by frontier-harness-eval · ⭐ 337

    Public results and task definitions for FrontierHarness Eval

  • oreilly-ai-agents by sinanuozdemir · ⭐ 326

    An introduction to the world of AI Agents

  • Agentic-AI-Pipeline by hoangsonww · ⭐ 89

    🦾 A production‑ready research outreach AI agent that plans, discovers, reasons, uses tools, auto‑builds cited

  • skill-optimizer by fastxyz · ⭐ 85

    Benchmark, evaluate, and optimize skills to ensure reliable performance across all LLMs

  • awesome-agent-observability by anhermon · ⭐ 64

    Curated list of LLM and AI agent observability and evaluation tools: tracing, evals, LLM-as-judge, guardrails,

  • arag by Ayanami0730 · ⭐ 365

    A-RAG: Agentic Retrieval-Augmented Generation via Hierarchical Retrieval Interfaces. State-of-the-art RAG fram

More Agent Tool Tools

Explore other popular agent tool tools:

View all Agent Tool tools →

Popular Python Agent Tools

How this security grade is produced

Grades come from a rule-based scan built on the SlowMist agent-security taxonomy: 44 published rules, covering credential harvesting, data exfiltration, curl | sh installers and more. It is a first-layer scan, not a manual audit — we say so rather than overstate it.

The scale of the problem is documented independently: Liu et al. (2026), in a study of 31,132 agent skills, report that 26.1% contain security vulnerabilities. Our own full-catalog census is published as a citable open dataset.

Sources & who's responsible:

View on GitHub → Browse Agent Tool tools