Compare
What packset measured
packset is the standing-knowledge store in the seat. It writes only on Remember: and Prefer:. The crate explanation holds the rows. Neighbouring systems are named by protocol.
What we measured
LongMemEval_S (doi:10.48550/arXiv.2410.10813), 470 answerable questions, session-level retrieval. The pack writes claims, not harvested chat. Reproduction: scripts/longmemeval_qa.py and docs/orgmode/results/longmemeval-s-qa.org in packset.
| Arm | hit@1 | recall@5 | recall@10 |
|---|---|---|---|
| packset BM25+ | 0.855 | 0.914 | 0.952 |
| packset fused panel | 0.889 | 0.949 | 0.981 |
Answer accuracy on the same 470 uses a local 7 B reader and judge (Qwen2.5-7B-Instruct Q5_K_M, LongMemEval prompts, top five sessions). CombMNZ fusion is 0.549. Lexical alone is 0.532. Labelled sessions, the ceiling for any retriever, are 0.634.
MemoryAgentBench (doi:10.48550/arXiv.2507.05257), same 7 B reader. Conflict-resolution split 0.480. Published retrieval baselines on that split range from 0.155 to 0.295 with a stronger reader. Accurate-retrieval split 0.675 of 2000, against published 0.605 (BM25) and 0.651 (HippoRAG-v2).
Neighbouring published numbers
Two public repositories share the name agentmemory. They report different protocols. The first is session retrieval (R@5). The second is question answering with a hosted generator and a hosted judge. packset publishes both a retrieval row and a local-reader QA row so those protocols stay distinct.
| System | Protocol | Published figure | Write policy |
|---|---|---|---|
| rohitg00/agentmemory | LongMemEval-S retrieval, R@5, all-MiniLM-L6-v2 | 95.2% (their COMPARISON.md) | Lifecycle hooks capture transcript text |
| JordanMcCann/agentmemory | LongMemEval QA, hosted generator and judge | 96.20% on 481/500 (their README) | Chat over a MemoryStore |
| packset | LongMemEval_S session retrieval; QA with a local 7 B reader | recall@5 0.949 fused, 0.914 lexical; QA 0.549 against a 0.634 labelled ceiling | Remember: and Prefer: only |
The 0.949 fused recall@5 is the neighbour of the 95.2% R@5 retrieval figure. The 96.20% QA figure uses a different generator and a different judge. InjecMEM and MINJA describe the risk of writing from a transcript. packset writes only on Remember: and Prefer:. Privacy-of-write.
What the other habitats answer
Retrieval is one store. The seat also cites frozen product, the tracker, this-session claims, argv law, and weighted agreement. Those questions are not retrieval scores.
| Question | Store |
|---|---|
| What did the human freeze? | cards |
| What does this seat know, standing? | packset |
| What did this unit produce? | deedar |
| What does this node stand on, and who agrees? | vissue |
| What is this session handing out? | claimdag |
| May this argv run? | ljos-policyd |
| Who is right, under trust? | ljos-consensus |
policyd is argv law
ljos policy -- argv prints the line and any pack rule that matches it. ljos rule writes that law into the pack. ljos-policyd is the TCB when it is on PATH or POLICYD_BIN: it prints allow or deny. Reloading a pack is not a check. ljos-policyd docs · Argv field note.
Source: packset explanation · packset