jevals: Agent evals and guardrails as Jev decisions: one request per trace, a fraction of a cent, fast enough

by openlayer-ai · LLM Plugin · ★ 120

Last updated: · Indexed by AgentSkillsHub · Auto-synced every 8h

How to install jevals

pip install jevals
pip install "jevals[pii]"           # Presidio, for PII / PHI entity detection

Taken from the project's README on GitHub.

Security audit verdict: SAFE · quality 70/100

None of the 44 rules matched — no credential harvesting, no data exfiltration, no curl-pipe-shell installer. Scanned against 44 published rules built on the SlowMist agent-security taxonomy, refreshed every 8 hours.

The quality score combines nine heuristic checks of the repo and its README: completeness, clarity, specificity, examples, structure, agent readiness, step-by-step content, instruction quality and security notes. It rates how well the project explains itself to an agent, not the code.

agentsevalsguardrailsjevllmllm-evaluationragastypesafe

Quick Facts

Stars120
Forks14
LanguagePython
CategoryLLM Plugin
LicenseMIT
Quality Score70/100
Open Issues4
Last Updated2026-11-07
Created2026-09-20
Platformspython
Est. Tokens~14k

About jevals

jevals Evals and guardrails for agents, using Jev-style decision models instead of an LLM judge. All the evals for a trace go out as one request that costs a few thousandths of a cent and comes back in a few hundred milliseconds, so you can run them on every trace and inside the agent loop.

jevals alternative? Top 6 similar tools

Looking for a jevals alternative? If you're comparing jevals with other llm plugin tools, these 6 projects are the closest alternatives on Agent Skills Hub — ranked by topic overlap, star count, and community traction.

  • awesome-jev by OmniJev · ⭐ 65

    🔥🔥 Papers, open reproductions and independent evaluations behind System One models and Jev.

  • awesome-typesafe by AbdelStark · ⭐ 400

    A curated list of official resources and community projects for TypeSafe, System One models, and Jev.

  • jev-codex-router by 0xNatoshi · ⭐ 275

    Per-turn model & reasoning routing for Codex, driven by Jev (TypeSafe System One): picks the model, thinking d

  • gate22 by aipotheosis-labs · ⭐ 181

    Open-source MCP gateway and control plane for teams to govern which tools agents can use, what they can do, an

  • neurolink by juspay · ⭐ 167

    The pipe layer of an AI nervous system — one interface connecting provider neurons to your application, across

  • pi-jev by y0usaf · ⭐ 166

    TypeSafe Jev as a decision layer for the Pi coding agent: a measured tool-call gate plus jev_ask for typed, ca

More LLM Plugin Tools

Explore other popular llm plugin tools:

View all LLM Plugin tools →

Popular Python Agent Tools

How this security grade is produced

Grades come from a rule-based scan built on the SlowMist agent-security taxonomy: 44 published rules, covering credential harvesting, data exfiltration, curl | sh installers and more. It is a first-layer scan, not a manual audit — we say so rather than overstate it.

The scale of the problem is documented independently: Liu et al. (2026), in a study of 31,132 agent skills, report that 26.1% contain security vulnerabilities. Our own full-catalog census is published as a citable open dataset.

Sources & who's responsible:

View on GitHub → Browse LLM Plugin tools