pytest-jev: Pytest plugin for semantic assertions on LLM output using Jev's calibrated probabilities

by allebee · LLM Plugin · ★ 8

Last updated: · Indexed by AgentSkillsHub · Auto-synced every 8h

Security audit verdict: SAFE · quality 68/100

None of the 42 rules matched — no credential harvesting, no data exfiltration, no curl-pipe-shell installer. Scanned against 42 published rules built on the SlowMist agent-security taxonomy, refreshed every 8 hours. Full audit →

The quality score combines nine heuristic checks of the repo and its README: completeness, clarity, specificity, examples, structure, agent readiness, step-by-step content, instruction quality and security notes. It rates how well the project explains itself to an agent, not the code.

jevllmllm-evaluationpytestpytest-pluginpythontestingtypesafe

Quick Facts

Stars8
Forks1
LanguagePython
CategoryLLM Plugin
LicenseMIT
Quality Score68/100
Open Issues1
Last Updated2026-09-21
Created2026-09-21
Platformspython
Est. Tokens~16k

About pytest-jev

Pytest plugin for semantic assertions on LLM output using Jev's calibrated probabilities.

pytest-jev alternative? Top 6 similar tools

Looking for a pytest-jev alternative? If you're comparing pytest-jev with other llm plugin tools, these 6 projects are the closest alternatives on Agent Skills Hub — ranked by topic overlap, star count, and community traction.

  • HA-Jev by AboveColin · ⭐ 78

    Ask your house a question, get a number back. Home Assistant integration for TypeSafe Jev: typed answers as se

  • awesome-jev by OmniJev · ⭐ 65

    🔥🔥 Papers, open reproductions and independent evaluations behind System One models and Jev.

  • just-eval by Re-Align · ⭐ 90

    A simple GPT-based evaluation tool for multi-aspect, interpretable assessment of LLMs.

  • Agentic-AI-Pipeline by hoangsonww · ⭐ 85

    🦾 A production‑ready research outreach AI agent that plans, discovers, reasons, uses tools, auto‑builds cited

  • AI-Agents-Orchestrator by hoangsonww · ⭐ 83

    🪈 Intelligent orchestration system that coordinates multiple AI coding assistants (Claude, Codex, Gemini CLI,

  • Awesome-LLMs-ICLR-24 by azminewasi · ⭐ 73

    It is a comprehensive resource hub compiling all LLM papers accepted at the International Conference on Learni

More LLM Plugin Tools

Explore other popular llm plugin tools:

View all LLM Plugin tools →

Popular Python Agent Tools

How this security grade is produced

Grades come from a rule-based scan built on the SlowMist agent-security taxonomy: 42 published rules, covering credential harvesting, data exfiltration, curl | sh installers and more. It is a first-layer scan, not a manual audit — we say so rather than overstate it.

The scale of the problem is documented independently: Liu et al. (2026), in a study of 31,132 agent skills, report that 26.1% contain security vulnerabilities. Our own full-catalog census is published as a citable open dataset.

Sources & who's responsible:

View on GitHub → Browse LLM Plugin tools