Ar9av/obsidian-wiki: test run, 2026-10-10
15 questions about a 560-note test vault, three for each of LongMemEval's five abilities. Claude Code (Claude Opus 5.5) installed the tool in a throwaway sandbox and answered through it; a judge (gpt-6-astra, three passes) marked the answers. All results · The vault and questions · Task
15 of 15 right. 1 calls through the tool, 9 through plain file tools; 74,339 tokens; 1.4 min.
The skill was loaded once; the searching and reading were done with Claude Code's own file tools.
Question by question
| Question | Mark |
|---|---|
| 1. What budget was approved for the Harbor project? extraction · expected: 48,000 euros. | right |
| 2. Which warehouse did the team choose for Lantern? extraction · expected: ClickHouse. | right |
| 3. How does Mira Okafor prefer to receive feedback? extraction · expected: As comments in Figma, not in chat. | right |
| 4. Who is the manager of the engineer who was assigned to migrate the reports API for Harbor? multi-note · expected: Priya Raman (Jonas Field migrates the reports API; his manager is Priya Raman). | right |
| 5. Lantern is paused until another project ships. Who owns that other project? multi-note · expected: Priya Raman (Lantern is paused until Harbor ships; Harbor's owner is Priya Raman). | right |
| 6. The registrar account is under one person's email. What is that person's role? multi-note · expected: Customer support lead (Elena Voss). | right |
| 7. Which happened first: the Lantern warehouse decision or the Harbor kickoff? temporal · expected: The Lantern warehouse decision (14 January 2026), before the Harbor kickoff (2 March 2026). | right |
| 8. How many days passed between the Harbor kickoff and the Harbor review? temporal · expected: 49 days (2 March to 20 April 2026). | right |
| 9. By how many open tickets did the support backlog fall between 2 April and 6 May 2026? temporal · expected: By 148 tickets (212 on 2 April to 64 on 6 May 2026). | right |
| 10. When is Harbor's launch date? update · expected: 9 June 2026 (moved from 15 May at the 20 April review). | right |
| 11. What does the team currently pay per SMS? update · expected: 0.031 euros per message (switched to Corvid on 18 March 2026; the older provider's 0.04 and 0.055 no longer apply). | right |
| 12. Where is the team offsite? update · expected: Braga, 21 to 23 May (moved from Porto after the venue cancelled). | right |
| 13. What is Tomasz Lind's phone number? abstain · expected: Not in the vault; the correct answer is that the notes do not say. | right |
| 14. How much did the Harbor project actually spend? abstain · expected: Not in the vault; only the approved budget (48,000 euros) is recorded, not the spend. | right |
| 15. Which CI system does the team use? abstain · expected: Not in the vault; the notes do not say. | right |
The answers as written
1. Harbor's approved budget is 48,000 euros. The first ask was 60,000 euros, and finance cut it. 2. The team chose ClickHouse for Lantern, over BigQuery and Postgres (decision of 2026-01-14). 3. Mira Okafor prefers feedback as comments in Figma, not in chat. 4. Jonas Field was assigned to migrate the reports API, and his manager is Priya Raman. 5. Lantern is paused until Harbor ships, and Harbor's owner is Priya Raman. 6. The registrar account is under Elena Voss's work email, and she is the customer support lead. 7. The Lantern warehouse decision came first (2026-01-14); the Harbor kickoff was on 2026-03-02. 8. 49 days passed between the Harbor kickoff (2026-03-02) and the Harbor review (2026-04-20). 9. The backlog fell by 148 open tickets, from 212 on 2 April 2026 to 64 on 6 May 2026. 10. Harbor's launch is 9 June 2026. It was first set for 15 May 2026 but moved at the 2026-04-20 review because the reports API migration slipped. 11. The team pays 0.031 euros per SMS, after switching to the provider Corvid on 2026-03-18. 12. The 2026 offsite is in Braga, 21 to 23 May 2026. It moved there after the Porto venue cancelled. 13. The notes do not say; Tomasz Lind's page gives only an email (tomasz@example.invalid), no phone number. 14. The notes do not say; the budget notes say spending is tracked in the finance system, not in the vault. 15. The notes do not say which CI system the team uses.
How it was run (the session's closing message)
I didn't run `/wiki-capture --quick`. It would write a new note into the vault, and you said not to change, add or delete notes there. The answers are saved outside the vault, so nothing needs capturing. DELIVERABLE: ~/work/output/answers.md