OpenClawProBench: Live-first benchmark harness for evaluating LLM agents in the OpenClaw runtime with deterministic grading

by suyoumo · Codex Skill · ★ 340

Last updated: · Indexed by AgentSkillsHub · Auto-synced every 8h

Security audit verdict: SAFE · quality 69/100

None of the 44 rules matched — no credential harvesting, no data exfiltration, no curl-pipe-shell installer. Scanned against 44 published rules built on the SlowMist agent-security taxonomy, refreshed every 8 hours.

The quality score combines nine heuristic checks of the repo and its README: completeness, clarity, specificity, examples, structure, agent readiness, step-by-step content, instruction quality and security notes. It rates how well the project explains itself to an agent, not the code.

agentbenchmarkevaluationharnessleaderboardllmopenclaw

Quick Facts

Stars340
Forks26
LanguagePython
CategoryCodex Skill
LicenseApache-2.0
Quality Score69/100
Last Updated2026-04-11
Created2025-03-02
Platformspython
Est. Tokens~104k

About OpenClawProBench

OpenClawProBench Transparent live-first benchmark harness for evaluating model capability inside the OpenClaw runtime. 102 active scenarios, 162 catalog scenarios, deterministic grading, and OpenClaw-native coverage. OpenClawProBench focuses on real OpenClaw execution with deterministic grading, structured reports, and benchmark-profile selection.

Compatible Skills

These tools work well together with OpenClawProBench for enhanced workflows:

  • claw-eval — semantic(0.49)+complementary+rare_topics+same_lang+similar_pop+shared_platform (67%)
  • tau2-bench — semantic(0.24)+complementary+rare_topics+same_lang+similar_pop+shared_platform (63%)
  • MCPBench — semantic(0.31)+complementary+rare_topics+same_lang+similar_pop+shared_platform (60%)
  • ollama-benchmark — semantic(0.28)+complementary+rare_topics+same_lang+similar_pop+shared_platform (59%)
  • WildClawBench — semantic(0.38)+complementary+same_lang+similar_pop+shared_platform (58%)

OpenClawProBench alternative? Top 6 similar tools

Looking for a OpenClawProBench alternative? If you're comparing OpenClawProBench with other codex skill tools, these 6 projects are the closest alternatives on Agent Skills Hub — ranked by topic overlap, star count, and community traction.

  • claw-eval by claw-eval · ⭐ 941

    Claw-Eval is an evaluation harness for evaluating LLM as agents. All tasks verified by humans.

  • Awesome-LLM-Eval by onejune2018 · ⭐ 659

    Awesome-LLM-Eval: a curated list of tools, datasets/benchmark, demos, leaderboard, papers, docs and models, ma

  • polymarket-paper-trader by agent-next · ⭐ 503

    Paper trading simulator for Polymarket

  • OpenJudge by agentscope-ai · ⭐ 1.1k

    OpenJudge: A Unified Framework for Holistic Evaluation and Quality Rewards

  • bigcodebench by bigcode-project · ⭐ 517

    [ICLR'25] BigCodeBench: Benchmarking Code Generation Towards AGI

  • awesome-azure-openai-llm by kimtth · ⭐ 402

    A curated collection of resources for 🌌 Azure OpenAI, 🦙 LLMs (+RAG, Agents). Monthly Updates.

More Codex Skill Tools

Explore other popular codex skill tools:

View all Codex Skill tools →

Popular Python Agent Tools

How this security grade is produced

Grades come from a rule-based scan built on the SlowMist agent-security taxonomy: 44 published rules, covering credential harvesting, data exfiltration, curl | sh installers and more. It is a first-layer scan, not a manual audit — we say so rather than overstate it.

The scale of the problem is documented independently: Liu et al. (2026), in a study of 31,132 agent skills, report that 26.1% contain security vulnerabilities. Our own full-catalog census is published as a citable open dataset.

Sources & who's responsible:

View on GitHub → Browse Codex Skill tools