GTA: [NeurIPS 2024 D&B] GTA: A Benchmark for General Tool Agents & [arXiv 2026] GTA-2

by open-compass · Agent Tool · ★ 153

Last updated: · Indexed by AgentSkillsHub · Auto-synced every 8h

How to install GTA

git clone https://github.com/open-compass/GTA.git

Taken from the project's README on GitHub. Check the security audit before running it.

Security audit verdict: SAFE · quality 65/100

None of the 42 rules matched — no credential harvesting, no data exfiltration, no curl-pipe-shell installer. Scanned against 42 published rules built on the SlowMist agent-security taxonomy, refreshed every 8 hours. Full audit →

The quality score combines nine heuristic checks of the repo and its README: completeness, clarity, specificity, examples, structure, agent readiness, step-by-step content, instruction quality and security notes. It rates how well the project explains itself to an agent, not the code.

llm-agentllm-evaluation

Quick Facts

Stars153
Forks12
LanguagePython
CategoryAgent Tool
LicenseApache-2.0
Quality Score65/100
Last Updated2026-10-24
Created2024-06-06
Platformspython
Est. Tokens~16k

About GTA

GTA: General Tool Agent Benchmark and Evaluation Framework [[NeurIPS 2024 D&B] GTA: A Benchmark for General Tool Agents](https://proceedings.neurips.cc/paperfiles/paper/2024/file/8a75ee6d4b2eb0b777f549a32a5a5c28-Paper-DatasetsandBenchmarksTrack.pdf) [[arXiv 2026] GTA-2: Benchmarking General Tool Agents from Atomic Tool-Use to Open-Ended Workflows](https://arxiv.org/pdf/2604.15715) ⬇️ Download...

GTA alternative? Top 6 similar tools

Looking for a GTA alternative? If you're comparing GTA with other agent tool tools, these 6 projects are the closest alternatives on Agent Skills Hub — ranked by topic overlap, star count, and community traction.

  • Awesome-LLM-in-Social-Science by ValueByte-AI · ⭐ 648

    Awesome papers involving LLMs in Social Science.

  • palico-ai by palico-ai · ⭐ 342

    Build, Improve Performance, and Productionize your LLM Application with an Integrated Framework

  • WebCanvas by iMeanAI · ⭐ 281

    All-in-one Web Agent framework for post-training. Start building with a few clicks!

  • CivAgent by fuxiAIlab · ⭐ 166

    CivAgent is an LLM-based Human-like Agent acting as a Digital Player within the Strategy Game Unciv.

  • Awesome-LLMs-ICLR-24 by azminewasi · ⭐ 73

    It is a comprehensive resource hub compiling all LLM papers accepted at the International Conference on Learni

  • wcgw by rusiaaman · ⭐ 674

    Shell and coding agent on mcp clients

More Agent Tool Tools

Explore other popular agent tool tools:

View all Agent Tool tools →

Popular Python Agent Tools

How this security grade is produced

Grades come from a rule-based scan built on the SlowMist agent-security taxonomy: 42 published rules, covering credential harvesting, data exfiltration, curl | sh installers and more. It is a first-layer scan, not a manual audit — we say so rather than overstate it.

The scale of the problem is documented independently: Liu et al. (2026), in a study of 31,132 agent skills, report that 26.1% contain security vulnerabilities. Our own full-catalog census is published as a citable open dataset.

Sources & who's responsible:

View on GitHub → Browse Agent Tool tools