No red flags found in any of the 11 categories — no credential harvesting, no data exfiltration, no curl-pipe-shell installer. Scanned against the SlowMist agent-security taxonomy, refreshed every 8 hours. Full audit →
by eigent-ai · MCP Server · ★ 143
Last updated: · Indexed by AgentSkillsHub · Auto-synced every 8h
🔒 Is toolathlon_gym safe to install? View the security audit →
Toolathlon-GYM — 503 multi-tool tasks backed by a local PostgreSQL database, no external APIs required Training and evaluating LLM agents on real-world tool use is hard. Most existing datasets are either too narrow in tool coverage, too small in scale, or depend on live external APIs that change over time. We introduce Toolathlon-GYM, a large-scale, self-contained environment with 503 tasks, 25 MCP servers, and a rich mock database. It runs entirely locally, with no external API calls required at running time. Toolathlon-GYM is built on and extends the infrastructure from Toolathlon by HKUST-NLP. The task format, evaluation framework, MCP server interfaces, and database schema design all originate from the Toolathlon project. This dataset applies the same format at larger scale, producing a substantially bigger and more diverse task pool for training and evaluation. Each task asks an agent to complete an end-to-end goal such as pulling data from a mock enterprise database, producing a spreadsheet report, scheduling a calendar event, and sending a summary email, using a fixed set of MCP (Model Context Protocol) servers as tools.
| Stars | 143 |
| Forks | 13 |
| Language | Python |
| Category | MCP Server |
| License | Apache-2.0 |
| Quality Score | 68.940039433352/100 |
| Open Issues | 3 |
| Last Updated | 2026-07-22 |
| Created | 2026-03-07 |
| Platforms | mcp, python |
| Est. Tokens | ~18k |
These tools work well together with toolathlon_gym for enhanced workflows:
Explore other popular mcp server tools:
toolathlon_gym is Toolathlon-Gym for testing AI agents real-world tool-use capabilities across diverse MCP servers.. It is categorized as a MCP Server with 143 GitHub stars.
toolathlon_gym is primarily written in Python.
You can find installation instructions and usage details in the toolathlon_gym GitHub repository at github.com/eigent-ai/toolathlon_gym. The project has 143 stars and 13 forks, indicating an active community.
toolathlon_gym is released under the Apache-2.0 license, making it free to use and modify according to the license terms.
Grades come from a rule-based scan built on the SlowMist agent-security taxonomy, covering 11 red-flag categories including credential harvesting, data exfiltration, and curl | sh installers. It is a first-layer scan, not a manual audit — we say so rather than overstate it.
The scale of the problem is documented independently: Liu et al. (2026), in a study of 31,132 agent skills, report that 26.1% contain security vulnerabilities. Our own full-catalog census is published as a citable open dataset.
Sources & who's responsible: