promptfoo — LLM Plugin by promptfoo

by promptfoo · LLM Plugin · ★ 21.1k

Last updated: · Indexed by AgentSkillsHub · Auto-synced every 8h

About promptfoo

Promptfoo: LLM evals & red teaming promptfoo is a CLI and library for evaluating and red-teaming LLM apps. Stop the trial-and-error approach - start shipping secure, reliable AI apps. Website · Getting Started · Red Teaming · Documentation · Discord Update (March 16, 2026): Promptfoo has joined OpenAI. Promptfoo remains open source and MIT licensed. Read the [company update](https://www.promptfoo.dev/

cici-cdcicdevaluationevaluation-frameworkllmllm-evalllm-evaluationllm-evaluation-frameworkllmops

Quick Facts

Stars21,105
Forks1,829
LanguageTypeScript
CategoryLLM Plugin
LicenseMIT
Quality Score50.102/100
Open Issues268
Last Updated2026-05-10
Created2023-04-28
Platformsclaude-code, gemini, node
Est. Tokens~40471k

Compatible Skills

These tools work well together with promptfoo for enhanced workflows:

  • lmnr — semantic(0.18)+complementary+rare_topics+same_lang+similar_pop+shared_platform (65%)
  • agentuse — semantic(0.22)+complementary+shared_fw(anthropic,openai)+same_lang+shared_platform (64%)
  • eval-view — semantic(0.22)+complementary+shared_fw(anthropic,openai)+rare_topics (53%)

promptfoo alternative? Top 6 similar tools

Looking for a promptfoo alternative? If you're comparing promptfoo with other llm plugin tools, these 6 projects are the closest alternatives on Agent Skills Hub — ranked by topic overlap, star count, and community traction.

  • phoenix by Arize-ai · ⭐ 9.6k

    AI Observability & Evaluation

  • prompt-optimizer by linshenkx · ⭐ 28.5k

    An AI prompt optimizer for writing better prompts and getting better AI results.

  • ChatGPT-Shortcut by rockbenben · ⭐ 8.5k

    🚀💪Maximize your efficiency and productivity. The ultimate hub to manage, customize, and share prompts. (Engl

  • Kiln by Kiln-AI · ⭐ 4.8k

    Build, Evaluate, and Optimize AI Systems. Includes evals, RAG, agents, fine-tuning, synthetic data generation,

  • ragflow by infiniflow · ⭐ 79.0k

    RAGFlow is a leading open-source Retrieval-Augmented Generation (RAG) engine that fuses cutting-edge RAG with

  • caveman by JuliusBrussee · ⭐ 57.4k

    🪨 why use many token when few token do trick — Claude Code skill that cuts 65% of tokens by talking like cave

More LLM Plugin Tools

Explore other popular llm plugin tools:

View all LLM Plugin tools →

Popular TypeScript Agent Tools

  • openclaw ⭐ 370.6k · Codex Skill
  • n8n ⭐ 187.4k · MCP Server
  • dify ⭐ 140.8k · MCP Server
  • gemini-cli ⭐ 103.6k · MCP Server
  • gstack ⭐ 93.0k · Agent Tool

Frequently Asked Questions

What is promptfoo?

promptfoo is Test your prompts, agents, and RAGs. Red teaming/pentesting/vulnerability scanning for AI. Compare performance of GPT, Claude, Gemini, Llama, and more. Simple declarative configs with command line and. It is categorized as a LLM Plugin with 21.1k GitHub stars.

What programming language is promptfoo written in?

promptfoo is primarily written in TypeScript. It covers topics such as ci, ci-cd, cicd.

How do I install or use promptfoo?

You can find installation instructions and usage details in the promptfoo GitHub repository at github.com/promptfoo/promptfoo. The project has 21.1k stars and 1829 forks, indicating an active community.

What license does promptfoo use?

promptfoo is released under the MIT license, making it free to use and modify according to the license terms.

What are the best alternatives to promptfoo?

The top alternatives to promptfoo on Agent Skills Hub include phoenix, prompt-optimizer, ChatGPT-Shortcut. Each offers a different approach to the same problem space — compare them side-by-side by stars, quality score, and community activity.

View on GitHub → Browse LLM Plugin tools