StevenStavrakis/obsidian-mcp: test run, 2026-10-10
15 questions about a 560-note test vault, three for each of LongMemEval's five abilities. Claude Code (Claude Opus 5.5) installed the tool in a throwaway sandbox and answered through it; a judge (gpt-6-astra, three passes) marked the answers. All results · The vault and questions · Task
15 of 15 right. 36 calls through the tool, 3 through plain file tools; 194,602 tokens; 1.4 min.
Reads the vault folder directly; its search returns long results, which is where the tokens went.
Question by question
| Question | Mark |
|---|---|
| 1. What budget was approved for the Harbor project? extraction · expected: 48,000 euros. | right |
| 2. Which warehouse did the team choose for Lantern? extraction · expected: ClickHouse. | right |
| 3. How does Mira Okafor prefer to receive feedback? extraction · expected: As comments in Figma, not in chat. | right |
| 4. Who is the manager of the engineer who was assigned to migrate the reports API for Harbor? multi-note · expected: Priya Raman (Jonas Field migrates the reports API; his manager is Priya Raman). | right |
| 5. Lantern is paused until another project ships. Who owns that other project? multi-note · expected: Priya Raman (Lantern is paused until Harbor ships; Harbor's owner is Priya Raman). | right |
| 6. The registrar account is under one person's email. What is that person's role? multi-note · expected: Customer support lead (Elena Voss). | right |
| 7. Which happened first: the Lantern warehouse decision or the Harbor kickoff? temporal · expected: The Lantern warehouse decision (14 January 2026), before the Harbor kickoff (2 March 2026). | right |
| 8. How many days passed between the Harbor kickoff and the Harbor review? temporal · expected: 49 days (2 March to 20 April 2026). | right |
| 9. By how many open tickets did the support backlog fall between 2 April and 6 May 2026? temporal · expected: By 148 tickets (212 on 2 April to 64 on 6 May 2026). | right |
| 10. When is Harbor's launch date? update · expected: 9 June 2026 (moved from 15 May at the 20 April review). | right |
| 11. What does the team currently pay per SMS? update · expected: 0.031 euros per message (switched to Corvid on 18 March 2026; the older provider's 0.04 and 0.055 no longer apply). | right |
| 12. Where is the team offsite? update · expected: Braga, 21 to 23 May (moved from Porto after the venue cancelled). | right |
| 13. What is Tomasz Lind's phone number? abstain · expected: Not in the vault; the correct answer is that the notes do not say. | right |
| 14. How much did the Harbor project actually spend? abstain · expected: Not in the vault; only the approved budget (48,000 euros) is recorded, not the spend. | right |
| 15. Which CI system does the team use? abstain · expected: Not in the vault; the notes do not say. | right |
The answers as written
1. The approved Harbor budget is 48,000 euros. The first ask was 60,000 euros, but finance cut it. 2. The team chose ClickHouse for Lantern, over BigQuery and Postgres (decided 2026-01-14). 3. Mira Okafor prefers feedback as comments in Figma, not in chat. 4. Jonas Field was assigned to migrate the reports API, and his manager is Priya Raman. 5. Lantern is paused until Harbor ships, and Harbor's owner is Priya Raman. 6. The registrar account is under Elena Voss's work email, and Elena Voss is the customer support lead. 7. The Lantern warehouse decision (2026-01-14) happened before the Harbor kickoff (2026-03-02). 8. 49 days passed between the Harbor kickoff (2 March 2026) and the Harbor review (20 April 2026). 9. The support backlog fell by 148 open tickets, from 212 on 2 April to 64 on 6 May 2026. 10. Harbor's launch date is 9 June 2026. It was moved from 15 May at the 20 April review. 11. The team pays 0.031 euros per SMS, after switching to the provider Corvid on 2026-03-18. 12. The offsite is in Braga, 21 to 23 May 2026. It moved there from Porto after the Porto venue cancelled. 13. The notes do not say; Tomasz Lind's note lists only an email (tomasz@example.invalid), not a phone number. 14. The notes do not say; the budget notes say Harbor spend is tracked in the finance system, not in the vault. 15. The notes do not say which CI system the team uses.
How it was run (the session's closing message)
I answered all 15 questions using only the Obsidian MCP server's search and read tools. I used the shell only to write the answers file, and nothing in the vault was changed. - **12 questions are answered from the notes.** Where a figure later changed, I used the newest one: - Harbor's launch moved from 15 May to 9 June 2026. - The SMS price is now 0.031 euros per message, after the switch to the provider Corvid. - The offsite moved from Porto to Braga, on the same dates. - **2 answers are my own arithmetic from dates and figures in the notes:** - 49 days between the Harbor kickoff and review. - The support backlog fell by 148 tickets, from 212 to 64. I left out a separate engineering backlog count of 97 issues, because it is a different queue. - **3 answers say the notes do not say:** - Tomasz Lind's phone number: his note lists only an email. - Harbor's actual spend: the notes say it is tracked in the finance system, not in the vault. - Which CI system the team uses. DELIVERABLE: ~/work/output/answers.md