ToolEmu: [ICLR'24 Spotlight] A language model-based emulation framework for identifying the risks of LM agents

by ryoungj · Agent Tool · ★ 223

Last updated: · Indexed by AgentSkillsHub · Auto-synced every 8h

How to install ToolEmu

git clone https://github.com/ryoungj/ToolEmu.git

Taken from the project's README on GitHub.

Security audit verdict: SAFE · quality 76/100

None of the 42 rules matched — no credential harvesting, no data exfiltration, no curl-pipe-shell installer. Scanned against 42 published rules built on the SlowMist agent-security taxonomy, refreshed every 8 hours.

The quality score combines nine heuristic checks of the repo and its README: completeness, clarity, specificity, examples, structure, agent readiness, step-by-step content, instruction quality and security notes. It rates how well the project explains itself to an agent, not the code.

agentai-safetylanguage-agentlanguage-modellarge-language-modelsprompt-engineering

Quick Facts

Stars223
Forks23
LanguagePython
CategoryAgent Tool
LicenseApache-2.0
Quality Score76/100
Open Issues2
Last Updated2024-03-22
Created2023-09-26
Platformspython
Est. Tokens~275k

About ToolEmu

ToolEmu: Identifying the Risks of LM Agents with an LM-Emulated Sandbox [📃 Paper]     [🌐 Website]     [🎮 Demo]     [🐦 Twitter] Recent advances in Language Model (LM) agents and tool use, exemplified by applications like ChatGPT Plugins, enable a rich set of capabilities but also amplify potential risks—such as leaking private data or causing...

Compatible Skills

These tools work well together with ToolEmu for enhanced workflows:

  • agent-studio — semantic(0.27)+complementary+rare_topics+same_lang+similar_pop+shared_platform (59%)
  • mxcp — semantic(0.31)+complementary+same_lang+similar_pop+shared_platform (56%)
  • clientai — semantic(0.16)+complementary+rare_topics+same_lang+similar_pop+shared_platform (55%)
  • MLLM-Tool — semantic(0.22)+complementary+same_lang+similar_pop+shared_platform (53%)
  • SimplerLLM — semantic(0.21)+complementary+same_lang+similar_pop+shared_platform (53%)

ToolEmu alternative? Top 6 similar tools

Looking for a ToolEmu alternative? If you're comparing ToolEmu with other agent tool tools, these 6 projects are the closest alternatives on Agent Skills Hub — ranked by topic overlap, star count, and community traction.

  • anchoring-ai by AnchoringAI · ⭐ 155

    An open-source no-code tool for teams to collaborate on building, evaluating, and hosting applications leverag

  • langkit by whylabs · ⭐ 998

    🔍 LangKit: An open-source toolkit for monitoring Large Language Models (LLMs). 📚 Extracts signals from promp

  • chatgpt-cli by kardolus · ⭐ 961

    ChatGPT CLI is a powerful, multi-provider command-line interface for working with modern LLMs. It supports Ope

  • ToolOrchestra by NVlabs · ⭐ 766

    ToolOrchestra is an end-to-end RL training framework for orchestrating tools and agentic workflows.

  • gateway by adaline · ⭐ 607

    The only fully local production-grade Super SDK that provides a simple, unified, and powerful interface for ca

  • LLM-Tool-Survey by quchangle1 · ⭐ 489

    This is the repository for the Tool Learning survey.

More Agent Tool Tools

Explore other popular agent tool tools:

View all Agent Tool tools →

Popular Python Agent Tools

How this security grade is produced

Grades come from a rule-based scan built on the SlowMist agent-security taxonomy: 42 published rules, covering credential harvesting, data exfiltration, curl | sh installers and more. It is a first-layer scan, not a manual audit — we say so rather than overstate it.

The scale of the problem is documented independently: Liu et al. (2026), in a study of 31,132 agent skills, report that 26.1% contain security vulnerabilities. Our own full-catalog census is published as a citable open dataset.

Sources & who's responsible:

View on GitHub → Browse Agent Tool tools