llm_context_benchmarks: A comprehensive benchmarking tool for testing LLMs with varying context sizes using Ollama

by ivanfioravanti · LLM Plugin · ★ 102

Last updated: · Indexed by AgentSkillsHub · Auto-synced every 8h

Security audit verdict: SAFE · quality 72/100

None of the 42 rules matched — no credential harvesting, no data exfiltration, no curl-pipe-shell installer. Scanned against 42 published rules built on the SlowMist agent-security taxonomy, refreshed every 8 hours.

The quality score combines nine heuristic checks of the repo and its README: completeness, clarity, specificity, examples, structure, agent readiness, step-by-step content, instruction quality and security notes. It rates how well the project explains itself to an agent, not the code.

aibenchmarkingllms

Quick Facts

Stars102
Forks12
LanguagePython
CategoryLLM Plugin
LicenseApache-2.0
Quality Score72/100
Last Updated2026-10-15
Created2025-08-06
Platformscli, python
Est. Tokens~15k

About llm_context_benchmarks

LLM Context Benchmarks Benchmark prompt-processing and generation throughput across context sizes (0.5k–128k tokens) for many inference engines: Ollama (API & CLI), MLX, MLX Distributed, MLX-VLM, llama.cpp, LM Studio, Exo, Apple Foundation Models Serve, vMLX, oMLX, Paroquant, and any OpenAI-compatible endpoint. Optimized for Apple Silicon but works anywhere Python runs.

llm_context_benchmarks alternative? Top 6 similar tools

Looking for a llm_context_benchmarks alternative? If you're comparing llm_context_benchmarks with other llm plugin tools, these 6 projects are the closest alternatives on Agent Skills Hub — ranked by topic overlap, star count, and community traction.

  • nexus by Nexus-Router · ⭐ 436

    Govern & Secure your AI

  • deeppowers by deeppowers · ⭐ 252

    DEEPPOWERS is a Fully Homomorphic Encryption (FHE) framework built for MCP (Model Context Protocol), aiming to

  • weam by weam-ai · ⭐ 222

    Web app for teams of 20+ members. In-built connections to major LLMs via API. Share chats, prompts, and agents

  • manim-generator by makefinks · ⭐ 117

    Automatic LLM-based video generation using the manim library. Usage of a code-writer and code-reviewer feedbac

  • log10 by log10-io · ⭐ 97

    Python client library for improving your LLM app accuracy

  • swarms-examples by The-Swarm-Corporation · ⭐ 83

    A vast array of examples for the enterprise-grade and production-ready swarms framework.

More LLM Plugin Tools

Explore other popular llm plugin tools:

View all LLM Plugin tools →

Popular Python Agent Tools

How this security grade is produced

Grades come from a rule-based scan built on the SlowMist agent-security taxonomy: 42 published rules, covering credential harvesting, data exfiltration, curl | sh installers and more. It is a first-layer scan, not a manual audit — we say so rather than overstate it.

The scale of the problem is documented independently: Liu et al. (2026), in a study of 31,132 agent skills, report that 26.1% contain security vulnerabilities. Our own full-catalog census is published as a citable open dataset.

Sources & who's responsible:

View on GitHub → Browse LLM Plugin tools