eugeniughelbur/obsidian-second-brain: test run, 2026-10-10
15 questions about a 560-note test vault, three for each of LongMemEval's five abilities. Claude Code (Claude Opus 5.5) installed the tool in a throwaway sandbox and answered through it; a judge (gpt-6-astra, three passes) marked the answers. All results · The vault and questions · Task
15 of 15 right. 0 calls through the tool, 8 through plain file tools; 60,051 tokens; 1.2 min.
Answered with Claude Code's own file tools; the skill is for building and maintaining a vault.
Question by question
| Question | Mark |
|---|---|
| 1. What budget was approved for the Harbor project? extraction · expected: 48,000 euros. | right |
| 2. Which warehouse did the team choose for Lantern? extraction · expected: ClickHouse. | right |
| 3. How does Mira Okafor prefer to receive feedback? extraction · expected: As comments in Figma, not in chat. | right |
| 4. Who is the manager of the engineer who was assigned to migrate the reports API for Harbor? multi-note · expected: Priya Raman (Jonas Field migrates the reports API; his manager is Priya Raman). | right |
| 5. Lantern is paused until another project ships. Who owns that other project? multi-note · expected: Priya Raman (Lantern is paused until Harbor ships; Harbor's owner is Priya Raman). | right |
| 6. The registrar account is under one person's email. What is that person's role? multi-note · expected: Customer support lead (Elena Voss). | right |
| 7. Which happened first: the Lantern warehouse decision or the Harbor kickoff? temporal · expected: The Lantern warehouse decision (14 January 2026), before the Harbor kickoff (2 March 2026). | right |
| 8. How many days passed between the Harbor kickoff and the Harbor review? temporal · expected: 49 days (2 March to 20 April 2026). | right |
| 9. By how many open tickets did the support backlog fall between 2 April and 6 May 2026? temporal · expected: By 148 tickets (212 on 2 April to 64 on 6 May 2026). | right |
| 10. When is Harbor's launch date? update · expected: 9 June 2026 (moved from 15 May at the 20 April review). | right |
| 11. What does the team currently pay per SMS? update · expected: 0.031 euros per message (switched to Corvid on 18 March 2026; the older provider's 0.04 and 0.055 no longer apply). | right |
| 12. Where is the team offsite? update · expected: Braga, 21 to 23 May (moved from Porto after the venue cancelled). | right |
| 13. What is Tomasz Lind's phone number? abstain · expected: Not in the vault; the correct answer is that the notes do not say. | right |
| 14. How much did the Harbor project actually spend? abstain · expected: Not in the vault; only the approved budget (48,000 euros) is recorded, not the spend. | right |
| 15. Which CI system does the team use? abstain · expected: Not in the vault; the notes do not say. | right |
The answers as written
1. The approved budget for Harbor is 48,000 euros. The first ask was 60,000 euros, and finance cut it. 2. The team chose ClickHouse for Lantern over BigQuery and Postgres (decision of 2026-01-14). 3. Mira Okafor prefers feedback as comments in Figma, not in chat. 4. Jonas Field was assigned the reports API migration, and his manager is Priya Raman. 5. Lantern is paused until Harbor ships, and Harbor is owned by Priya Raman. 6. The registrar account is under Elena Voss's work email, and she is the customer support lead. 7. The Lantern warehouse decision (2026-01-14) came before the Harbor kickoff (2026-03-02). 8. 49 days passed between the Harbor kickoff (2026-03-02) and the Harbor review (2026-04-20). 9. The support backlog fell by 148 open tickets, from 212 on 2 April to 64 on 6 May 2026. 10. Harbor's launch date is 9 June 2026. The 2026-04-20 review moved it from the original 15 May. 11. The team pays 0.031 euros per SMS since switching to the provider Corvid on 2026-03-18. 12. The 2026 offsite is in Braga, 21 to 23 May. It moved there after the Porto venue cancelled. 13. The notes do not say; Tomasz Lind's note lists only an email (tomasz@example.invalid) and no phone number. 14. The notes do not say; the Harbor budget notes state that spend is tracked in the finance system, not in the vault. 15. The notes do not say which CI system the team uses.
How it was run (the session's closing message)
I've written answers to all 15 questions in `~/work/output/answers.md`. For 12 of them the notes give a clear answer. For the other three (Tomasz Lind's phone number, what Harbor actually spent, and which CI system the team uses), the file says the notes do not say. **How I searched:** The tool's MCP server is set up but could not start, because `uv` is not installed on this machine. So I followed the skill's own search workflow instead. I ran its bundled search and read functions (`vault_ops.search` and `read_note`) directly with Python. That search ignores very short words like "CI", so for that question and the phone number I fell back to a plain text search (grep) of the vault, which is the skill's own fallback when no search tool is available. Nothing in the vault was changed. **Where the notes changed over time,** I used the latest note: - **Harbor launch (Q10):** 9 June 2026. The 20 April review moved it from 15 May. - **SMS price (Q11):** 0.031 euros per message, after the switch to a new provider, Corvid, on 18 March. - **Offsite (Q12):** Braga, 21 to 23 May. It moved there after the Porto venue cancelled. **Two answers I worked out from dates and numbers in the notes:** - **Q8:** 49 days between the Harbor kickoff (2 March) and the review (20 April). - **Q9:** the support backlog fell by 148 tickets, from 212 to 64. DELIVERABLE: ~/work/output/answers.md