anombyte93/prd-taskmaster: vibe coding test
Installed from its README in a throwaway sandbox, then used by Claude Code (Claude Opus 5.5) to build two apps from requests written the way someone who cannot code would write them. Whenever the build stopped to ask a question or for approval, it got the same reply. A separate tester session, with no tool loaded, opened each app in a browser and ran the checks; we went through every screenshot. All results · The task · The tester's checks
Install: SETUP: done prd-taskmaster Claude Code plugin + MCP server deps (`claude plugin marketplace add ~/src`; `claude plugin install prd@atlas-prd-taskmaster`; `pip install --user --break-system-packages -r ~/src/mcp-server/requirements.txt`; `pip install --user --break-system-packages 'mcp>=1.0.0,<2'`) (repo commit 3a9756ad2f0b)
One sentence: an expense page
The request, word for word: 帮我做一个记账小网页：每次花钱记一笔（花了多少、买了什么），页面上能看到一共花了多少，关掉浏览器再打开，记录还在。
1 of 4 checks passed. Opens: double-click. Built in 30.0 minutes (limit 30); it stopped to ask 2 time(s); how it ended: out of time. Usage at API prices: $5.3 (31.2 times the run with no tool). Packages: none (plain HTML, CSS and JavaScript).
Our note: The install session registered its Claude Code plugin but not task-master-ai, which the README also lists, so the Auto-Execute mode was not on offer. The tool recommended its Verified Loop, wrote a PRD of 12 tasks and 24 subtasks for the page, and was still working at the 30-minute limit. The page it had by then sent the form as a page reload and recorded nothing (screenshots agree with the tester).
| Check | Verdict | What the tester saw | Screenshot |
|---|---|---|---|
| opens | pass | file:// 打开后页面标题“记账本”，显示：'记账本\n一共花了 ¥0.00\n记一笔\n\n还没有记录，记下第一笔吧' |  |
| records an expense | fail | 填入 25 和 午饭 后点「记一笔」：页面整页刷新，地址变成 http://127.0.0.1:8700/index.html?amount=25&item=%E5%8D%88%E9%A5%AD，两个输入框被清空；列表仍显示：['还没有记录，记下第一笔吧']；一共花了 ¥0.00 |  |
| total is right | fail | 总计显示“一共花了 ¥0.00”；列表：['还没有记录，记下第一笔吧'] |  |
| still there after closing and reopening | fail | 重新打开后列表：['还没有记录，记下第一笔吧']；总计“一共花了 ¥0.00” |  |
Where it stopped, and got the reply 「我不懂这些，你按你推荐的来，继续做。」
I'm in the Discovery phase now. This run has a user present, so the tool uses its Interactive Mode: it asks adaptive questions one at a time, then asks you to approve a summary. Its normal question helper (`superpowers:brainstorming`) isn't installed here, so I'll ask the questions directly. ``` DISCOVERY CHECKLIST: - [x] Mode detected: Interactive (user present) - [x] Goal captured: 记账小网页：记一笔（金额 + 买了什么），显示总共花了多少，关掉浏览器再打开记录还在 - [ ] Adaptive questions completed (one at a time) - [ ] Constraints extracted and listed - [ ] Scale classified - [ ] Discovery summary captured - [ ] User approved ``` **问题 1：** 除了"花了多少"和"买了什么"，每一笔还需要别的信息吗？ - **A.** 不用，就这两样，越简单越好（日期由网页自动记上，只用来显示） - **B.** 还要选一个分类（比如 吃饭 / 交通 / 购物 / 其他） - **C.** 还要能自己填日期（比如补记昨天的账） - **D.** B 和 C 都要 直接回复字母就行，也可以用你自己的话说。
I've finished every handoff step that doesn't need you: capabilities checked (free tier), task workflow written to `CLAUDE.md`, summary shown. The tool now requires **you** to choose an execution mode, and it explicitly forbids me from choosing. Its usual multiple-choice picker isn't available in this session, so here are the options in text: ``` 规格文档: .taskmaster/docs/prd.md — 评分 EXCELLENT（57/57），0 处占位 任务: 12 个任务 / 24 个子任务，依赖关系已排好 内容: 一个文件 output/index.html，双击打开；记金额 + 买了什么；顶部显示总共花了多少； 记录存在浏览器里，关掉再开还在；可删除记错的一笔 ``` **问题 2：接下来用哪种方式把它做出来？** - **A. 验证循环（Verified Loop）**：我按顺序把 12 个任务自动做完。每个任务都要真的打开浏览器跑测试，测试通过才算完成，中途不再问你。工具的状态面板把这个标为推荐。**我建议选这个，因为你不需要自己写代码。** - **B. 计划并手动推进（Plan & Drive）**：只生成一份实施计划，由你一步步推进实现。这台机器没有装额外的执行插件，所以工具的能力检测默认推荐这个。 - 自动执行（Auto-Execute）：这台机器没装 TaskMaster，**这里用不了**。 - 🔒 Atlas Fleet：付费的私测功能，**不能选**。 回复 **A** 或 **B** 就行。
One paragraph: a family ledger
The request, word for word: 做一个家庭记账本网页：能记收入和支出（金额、类别、日期、备注），能按月看收入、支出和结余，能看每个类别花了多少（最好有图），可以设每月预算、超了要提醒，还能把记录导出成 Excel 表格。数据存在我电脑上就行。
3 of 8 checks passed. Opens: double-click. Built in 45.0 minutes (limit 45); it stopped to ask 2 time(s); how it ended: out of time. Usage at API prices: $7.89 (7.7 times the run with no tool). Packages: none (plain HTML, CSS and JavaScript).
Our note: Two questions, then its Verified Loop. At the 45-minute limit the page recorded entries and exported them, but its month overview read ¥0.00, the category panel held three empty placeholders and the budget panel said 等待实现 (still to be built). The Mac slept for about 90 minutes during this run; the build's limit counts working time only, so it is shown as 45 minutes, not the 137 the clock recorded.
| Check | Verdict | What the tester saw | Screenshot |
|---|---|---|---|
| opens | pass | file:// page shows title 家庭记账本 with 记一笔 form, 本月概览, 类别统计, 记录列表, 导出与备份 panels |  |
| records income and expenses | pass | All four appear in 记录列表 (共 4 条): 2026-10-01 收入 工资 十月工资 +¥5,000.00; 2026-10-05 支出 餐饮 午饭 −¥120.00; 2026-10-08 支出 交通 地铁 −¥30.00; 2026-10-12 支出 餐饮 晚饭 −¥80.00. Income uses the app's fixed list (工资/奖金/理财/兼职/其他), so 工资 was available. |  |
| month's income, spending and balance | fail | 本月概览 shows 本月收入 ¥0.00, 本月支出 ¥0.00, 本月结余 ¥0.00 with 4 October records in the list (expected 5000 / 230 / 4770). The month label shows only the placeholder text '当前月份', not a month. |  |
| spending by category | fail | 类别统计 panel contains only three empty placeholder boxes labelled [饼图], [表格], [趋势]. No category totals or chart; 餐饮 200 / 交通 30 shown nowhere. |  |
| switches between months | fail | Saved expense 999 购物 2026-09-15. The record list then switched by itself to show only that record (共 1 条, 2026-09-15 支出 购物 −¥999.00), but 本月概览 still showed income ¥0.00 / expense ¥0.00 / balance ¥0.00 (not 999), and there is no month picker or prev/next control ('当前月份' is an empty placeholder div). I could not choose September or switch back to October on purpose; October's totals were never shown as 5000/230/4770 anyway. |  |
| warns over budget | fail | The 预算 panel only shows the placeholder text '等待实现...' ('waiting to be implemented'). It has no inputs, so no budget (monthly or per-category) could be set, and no warning appeared. |  |
| exports a spreadsheet | pass | Clicked 导出全部 Excel; downloaded 记账本_全部.xlsx (a real .xlsx). Sheet 记账记录 with columns 日期/类型/类别/金额/备注 and all five rows: 2026-09-15 支出 购物 999; 2026-10-01 收入 工资 5000 十月工资; 2026-10-05 支出 餐饮 120 午饭; 2026-10-08 支出 交通 30 地铁; 2026-10-12 支出 餐饮 80 晚饭; then 合计收入 5000, 合计支出 1229, 结余 3771. (There is also an 导出本月 Excel button.) |  |
| still there after closing and reopening | fail | After closing and reopening: the list shows the 4 October records (5,000 / 120 / 30 / 80, 共 4 条), so the data was saved. The September 999 record can't be seen on the page because there is no way to switch months (it is still in the export). The overview still shows 本月收入 ¥0.00, 本月支出 ¥0.00, 本月结余 ¥0.00, not 5000/230/4770. |  |
Where it stopped, and got the reply 「我不懂这些，你按你推荐的来，继续做。」
I'm in the Discovery phase now. The tool runs an interview, one question at a time, before it writes the spec. ``` DISCOVERY CHECKLIST: - [x] Mode detected (Interactive) - [x] Goal captured - [ ] Adaptive questions completed (one at a time) - [ ] Constraints extracted and listed - [ ] Scale classified - [ ] Discovery summary captured - [ ] User approved ``` **问题 1：** 这个记账本由谁来记？ - **A.** 只有我一个人，在这台电脑上记 - **B.** 家里几个人都会在这台电脑上记，每条记录要标明是谁记的或谁花的（比如"爸爸""妈妈""孩子"） - **C.** 其他（请说明） 请回复 A、B 或你自己的说明。
bilities detected (tier: free; superpowers, ralph-loop, TaskMaster, Playwright all missing) - [x] Recommended mode: Plan & Drive (reason: universal fallback, no execution tooling detected) - [x] Summary displayed - [x] CLAUDE.md task workflow appended (created) - [x] Mode picker surfaced (prose fallback, below) - [ ] User choice dispatched ``` > `[AI]` AskUserQuestion isn't available in this session, so the tool's mode picker is shown as a plain list. You still make the choice. The spec and task plan are ready: - **Spec:** `.taskmaster/docs/prd.md`, graded EXCELLENT (57 of 57 points) - **Plan:** 10 tasks broken into 27 smaller checkable steps Before building, the tool requires you to choose **how the work gets done**: | Option | What it means for you | |---|---| | **1. Plan & Drive** (the tool's default here) | The tool writes a step-by-step build plan. I then build each task from that plan. | | **2. Verified Loop** | I build the tasks one by one in a loop. Each task needs test results as proof before it counts as done, and the run ends with an automated final check. This takes longer but is more thorough. | | ~~Auto-Execute~~ | Not available: it needs TaskMaster, which isn't installed. | | 🔒 Atlas Fleet | Not available: it's a paid Pro feature in private pilot. | | Show more detail | I explain more before you decide. | **Reply 1 or 2.** I recommend **2 (Verified Loop)**, since you can't check the code yourself and this option tests every feature before it's marked done.