thruwire/foreman: first run, 2026-10-11
Installed from its README in a throwaway sandbox by Claude Code (Claude Opus 5.5), with a Jev API key in the environment and nothing else: no GPU, no desktop, no other paid keys, no accounts. The README's first example was run once. Repo commit 381ca2de90af; 71 seconds. All results · Task
Quick start: did not run. Jev calls made: 4.
| Install | python3 -m venv .venv && .venv/bin/python -m pip install foreman-core (README: python -m pip install foreman-core; got foreman-core 0.5.0, typesafe-sdk 0.7.4) |
|---|---|
| Quick start, as run | TYPESAFE_API_KEY exported; foreman run --repo ~/work/my-project --job "Add rate limiting to the API and make sure it is properly tested." (README 'Running Foreman'; `foreman demo` and the README `foreman hook` SessionStart example were skipped because they make no Jev calls) |
| What came back | Jev routing + 3 assessments came back fine (e.g. Worker stuck 88%, Needs human 87%, Implementation 1%), but every coding worker died with "FileNotFoundError: [Errno 2] No such file or directory: 'codex'" (termination_reason app_server_error). Last lines: 'Decision ESCALATE - semantic assessment requires human input' / 'Factory escalated: semantic assessment requires human input' / 'Run ID: f852e835f24c'. Foreman has no preflight check for the codex binary: it started and failed 3 workers and spent Jev calls judging them, never saying codex is missing. |
| What stopped it | The default worker backend needs the Codex CLI on PATH plus Codex authentication (an OpenAI/ChatGPT account). This machine has neither. The other backends (opencode, hermes, dcode) also need their own CLIs and model-provider credentials. Jev itself worked. |
Steps the README does not mention
- Created a virtualenv for the published-package install (the README's bare `python -m pip install` gives no venv step; Debian system Python is externally managed)
- Created a scratch git repo at ~/work/my-project because the README example points at ./my-project but never says to create it
- Exported TYPESAFE_API_KEY in the shell; the README only shows putting it in .env for the development checkout
The session's closing message
The install worked and Jev answered, but the quick start didn't produce a real result. No coding agent could start, because this machine doesn't have one. - **Install:** `pip install foreman-core` (version 0.5.0) went cleanly into a fresh virtualenv, and the `foreman` command runs. - **Picking the quick start:** `foreman demo` makes no Jev calls (the README says so), and the README's `foreman hook` SessionStart example doesn't call Jev either. So I ran the README's `foreman run` example against a scratch git repo, with `TYPESAFE_API_KEY` exported. - **Jev:** about 4 calls (one routing call and three assessments), and all came back with scores. - **Blocked:** by default Foreman hands the coding to the Codex CLI. That needs the `codex` command installed plus an OpenAI/ChatGPT login, and this machine has neither. All three coding-agent attempts died with `FileNotFoundError: ... 'codex'`. Foreman then stopped and asked for a human: "Factory escalated: semantic assessment requires human input". The other options (OpenCode, Hermes, Deep Agents) each need their own CLI and model-provider credentials too. - **Finding:** Foreman doesn't check for `codex` before it starts. It launched the coding agent three times and spent Jev calls judging failures, and it never said plainly that `codex` was missing. - **Steps the README didn't mention:** creating a virtualenv for the package install, creating the `./my-project` repo the example points at, and exporting the key in the shell (the README only shows putting it in `.env` for a development checkout). I didn't patch any code, and the API key wasn't printed or written anywhere; I checked the run files and it isn't in them. DELIVERABLE: ~/work/output/report.json