by hiyouga · Agent Tool · ★ 89
Last updated: · Indexed by AgentSkillsHub · Auto-synced every 8h
git clone https://github.com/hiyouga/MathRuler.gitTaken from the project's README on GitHub.
None of the 42 rules matched — no credential harvesting, no data exfiltration, no curl-pipe-shell installer. Scanned against 42 published rules built on the SlowMist agent-security taxonomy, refreshed every 8 hours.
The quality score combines nine heuristic checks of the repo and its README: completeness, clarity, specificity, examples, structure, agent readiness, step-by-step content, instruction quality and security notes. It rates how well the project explains itself to an agent, not the code.
| Stars | 89 |
| Forks | 12 |
| Language | Python |
| Category | Agent Tool |
| License | Apache-2.0 |
| Quality Score | 65/100 |
| Open Issues | 1 |
| Last Updated | 2025-06-19 |
| Created | 2024-12-31 |
| Platforms | python |
| Est. Tokens | ~313k |
MathRuler A light-weight tool for evaluating LLMs in rule-based ways. Installation We use vLLM to accelerate the generation. Datasets MATH: 500 problems. GSM8K: 1319 problems. AIME24: 30 problems. AIME25: 30 problems. Generate Example output: Processed prompts: 100% 500/500 [00:36<00:00, 13.75it/s, est.
Explore other popular agent tool tools:
Grades come from a rule-based scan built on the SlowMist agent-security taxonomy: 42 published rules, covering credential harvesting, data exfiltration, curl | sh installers and more. It is a first-layer scan, not a manual audit — we say so rather than overstate it.
The scale of the problem is documented independently: Liu et al. (2026), in a study of 31,132 agent skills, report that 26.1% contain security vulnerabilities. Our own full-catalog census is published as a citable open dataset.
Sources & who's responsible: