Toolathlon: [ICLR 2026] The Tool Decathlon: Benchmarking Language Agents for Diverse, Realistic, and Long-Horizon Task

by hkust-nlp · Agent Tool · ★ 486

Last updated: · Indexed by AgentSkillsHub · Auto-synced every 8h

Security audit verdict: CAUTION · quality 69/100

Flagged: sudo usage. Scanned against 42 published rules built on the SlowMist agent-security taxonomy, refreshed every 8 hours. Full audit →

The quality score combines nine heuristic checks of the repo and its README: completeness, clarity, specificity, examples, structure, agent readiness, step-by-step content, instruction quality and security notes. It rates how well the project explains itself to an agent, not the code.

Quick Facts

Stars486
Forks66
LanguagePython
CategoryAgent Tool
Quality Score69/100
Open Issues25
Last Updated2026-08-18
Created2025-10-24
Platformspython
Est. Tokens~17k

About Toolathlon

The Tool Decathlon: Benchmarking Language Agents for Diverse, Realistic, and Long-Horizon Task Execution Introduction Toolathlon is a benchmark to assess language agents' general tool use in realistic environments. It features 600+ diverse tools based on real-world software environments. Each task requires long-horizon tool calls to complete.

Toolathlon alternative? Top 6 similar tools

Looking for a Toolathlon alternative? If you're comparing Toolathlon with other agent tool tools, these 6 projects are the closest alternatives on Agent Skills Hub — ranked by topic overlap, star count, and community traction.

  • skills by GuDaStudio · ⭐ 2.0k

    This repository contains a collection of Agent Skills developed by GudaStudio, enabling seamless collaboration

  • awesome-android-agent-skills by new-silvermoon · ⭐ 992

    A collection of standardized Agent Skills to teach GitHub Copilot, Claude, Gemini and Cursor about modern Andr

  • agent-skills by hashicorp · ⭐ 911

    A collection of Agent skills and Claude Code plugins for HashiCorp products.

  • synalinks-skills by SynaLinks · ⭐ 907

    Coding Agents skills for Synalinks OSS

  • claude-code-skill-factory by alirezarezvani · ⭐ 869

    Claude Code Skill Factory — A powerful open-source toolkit for building and deploying production-ready Claude

  • claude-plugins by Kamalnrf · ⭐ 564

    Lightweight registry to discover, install, and manage all public Claude plugins and agent skills for your favo

More Agent Tool Tools

Explore other popular agent tool tools:

View all Agent Tool tools →

Popular Python Agent Tools

How this security grade is produced

Grades come from a rule-based scan built on the SlowMist agent-security taxonomy: 42 published rules, covering credential harvesting, data exfiltration, curl | sh installers and more. It is a first-layer scan, not a manual audit — we say so rather than overstate it.

The scale of the problem is documented independently: Liu et al. (2026), in a study of 31,132 agent skills, report that 26.1% contain security vulnerabilities. Our own full-catalog census is published as a citable open dataset.

Sources & who's responsible:

View on GitHub → Browse Agent Tool tools