by hiyouga · Agent Tool · ★ 85
Last updated: · Indexed by AgentSkillsHub · Auto-synced every 8h
MathRuler A light-weight tool for evaluating LLMs in rule-based ways. Installation We use vLLM to accelerate the generation. Datasets MATH: 500 problems. GSM8K: 1319 problems. AIME24: 30 problems. AIME25: 30 problems. Generate Example output: Processed prompts: 100% 500/500 [00:36<00:00, 13.75it/s, est. speed input: 15765.84 toks/s, output: 5299.80 toks/s] Optional Arguments jsonpath (str): path to the eval file, defaults to savepath (str): path to the predicted file, defaults to nshot (int): number of few-shot examples, defaults to demosplit (str): split to build few-shot examples, defaults to system (str): system message for generation, defaults to temperature (float): decode temperature value, defaults to topp (float): decode top p value, defaults to maxtokens (int): maximum number of generated tokens, defaults to sam
| Stars | 85 |
| Forks | 9 |
| Language | Python |
| Category | Agent Tool |
| License | Apache-2.0 |
| Quality Score | 65.3882738358056/100 |
| Open Issues | 1 |
| Last Updated | 2025-06-19 |
| Created | 2024-12-31 |
| Platforms | python |
| Est. Tokens | ~313k |
Looking for a MathRuler alternative? If you're comparing MathRuler with other agent tool tools, these 3 projects are the closest alternatives on Agent Skills Hub — ranked by topic overlap, star count, and community traction.
A Claude Code skill that turns PDFs, docs, and codebases into Obsidian study vaults
86 product management skills from Lenny's Podcast for Claude Code and AI agents. Hiring, user research, strate
Power rename/refactor tool (now with agent skill support!)
Explore other popular agent tool tools:
MathRuler is A light-weight tool for evaluating LLMs in rule-based ways.. It is categorized as a Agent Tool with 85 GitHub stars.
MathRuler is primarily written in Python.
You can find installation instructions and usage details in the MathRuler GitHub repository at github.com/hiyouga/MathRuler. The project has 85 stars and 9 forks, indicating an active community.
MathRuler is released under the Apache-2.0 license, making it free to use and modify according to the license terms.
The top alternatives to MathRuler on Agent Skills Hub include tutor-skills, lenny-skills, repren. Each offers a different approach to the same problem space — compare them side-by-side by stars, quality score, and community activity.