LoCoMo evidence retrieval
Full public run. Scores show whether the required source evidence appears in the retrieved memories.
Public benchmark
Engram benchmark methodology
See the public datasets, retrieval metrics, answer checks, baselines, limitations, and claim rules behind Engram's published results.
Updated 4 min read
LoCoMo evidence retrieval93.29%
LoCoMo answer-level judge89.65%
How It Works
LoCoMo answer-level judge
Same full set of 1,536 evidence-bearing tasks and top-50 cutoff for every row. Codex generated the answers and a separate blind judge scored them; this is an Engram-run diagnostic, not an official LoCoMo leaderboard.
How to read the two tables
Both published evaluations use all 1,536 evidence-bearing tasks. Evidence retrieval requires source IDs, and answer accuracy uses the same answer and judge models for every included system. Systems that do not satisfy the same evaluation contract are documented separately rather than mixed into the ranking.
1,536
scored evidence-retrieval questions
93.29%
Engram R@50
89.65%
answer accuracy from top-50 context
We keep retrieval and answer accuracy separate. Both use public LoCoMo, but they measure different parts of the memory loop.
AgentMemory returns at most 100 results, so R@200 is shown as N/A. Tenure does not preserve LoCoMo source evidence IDs and is not included in the public ranking; its experimental run remains documented in the methodology.
Engram is measured two ways: first, whether it retrieves the right evidence; second, whether an LLM can answer correctly from that retrieved context.
We keep retrieval and answer accuracy separate. Both use public LoCoMo, but they measure different parts of the memory loop.