Skip to main content

Performance and cost

We measure Memlake two ways: head to head with Postgres through the full Hindsight API, which is what an agent experiences, and engine-level, straight against Memlake, to reach sizes the API benchmark can't. Unless noted, recall numbers are without the cross-encoder reranker, which would otherwise dominate recall time on both sides and is limited by the reranker's own GPU capacity.

The numbers below come with caveats that matter; they are listed at the end of the page.

Head to head with Postgres​

The same workloads, run on the same development cluster once on Postgres and once on Memlake, with the same data and benchmark driver (no LLM), measured 2026-09-15:

Recall throughput: Memlake 207, 106, 113 and 113 recalls per second against Postgres 19, 0.45, 4 and 2 across four workloads. One recall at p99: Memlake 106 and 113 ms against Postgres 513 ms and 5.3 sRecall throughput: Memlake 207, 106, 113 and 113 recalls per second against Postgres 19, 0.45, 4 and 2 across four workloads. One recall at p99: Memlake 106 and 113 ms against Postgres 513 ms and 5.3 s

Throughput is at 16 concurrent requests for single banks, and the highest steady rate for the multi-bank workloads.

  • Small bank (4k memories). Memlake answers in a third to a fifth of the time and serves ten times the traffic. Postgres runs every recall on its one primary; Memlake answers warm queries from memory with a single request to object storage.
  • Large bank (~150–210k memories). The gap is two orders of magnitude, because Memlake's per-query work barely moves as the bank grows. Most of Postgres's time here is keyword search.
  • Many banks. One organization with 1,000 small banks held about 2 recalls per second on Postgres, because every query is planned over every bank's indexes. On Memlake each bank is its own small index. Memlake's 113 recalls per second is a ceiling of the test setup, not of Memlake.

At scale​

Engine-level, 100 million memories across 20 banks ran on 4 query nodes of 6 vCPU each, about a year of agents for 1,500 employees. Latency stayed flat from 1 to 200 recalls per second, and one 3M-memory bank answered in 71 ms at p50 and 90 ms at p99, one request at a time:

Recall latency: one 3M-memory bank, one request at a time, p50 71 ms and p99 90 ms; 100M memories on 4 servers, p50 40–43 ms and p99 73–80 ms from 1 to 200 recalls per secondRecall latency: one 3M-memory bank, one request at a time, p50 71 ms and p99 90 ms; 100M memories on 4 servers, p50 40–43 ms and p99 73–80 ms from 1 to 200 recalls per second
  • Capacity grows linearly with query nodes: each handles roughly 8–14 warm recalls per second per vCPU.
  • Bank size costs CPU slowly: growing one bank 30×, from 100k to 3M memories, made a recall about 6× slower under heavy load, close to the square root of the growth.
  • Largest single bank run so far: 10 million memories, about two years of an autonomous agent working nonstop.

Writes​

A single bank sustains 1,200–1,500 memories per second at the storage layer, flat as the bank grows, with peaks near 2,300 when writes arrive in large batches: 15–27× what Postgres sustained. A write commits in about a quarter of a second. In Hindsight, every retain goes through an LLM first, so in practice how fast memories land is bound by the LLM.

Cold starts​

The first time a query node opens a bank it loads the bank's index metadata: 8.6 s at p95 for a 10M-memory bank, once per query node. Until data blocks are cached, a query can take around a second at p99. Left to warm from traffic alone, a 1M-memory bank went from 1.7 s p99 to 74 ms once fully cached, which is why query nodes now warm banks in the background. See Architecture.

Retrieval quality​

Changing the storage engine didn't cost retrieval quality. On standard RAG datasets from the BEIR benchmark, with the same embeddings (bge-small-en-v1.5), Memlake's vector, keyword and hybrid search came out within 0.3% of an established vector database on every comparison, and ahead on three of six:

BEIR nDCG@10: on SciFact and NFCorpus, Memlake's vector, BM25 and hybrid search score within 0.3% of an established vector database; adding graph search lifts NFCorpus hybrid to 0.3645BEIR nDCG@10: on SciFact and NFCorpus, Memlake's vector, BM25 and hybrid search score within 0.3% of an established vector database; adding graph search lifts NFCorpus hybrid to 0.3645

BEIR has no entities or timestamps, so it tests that the storage engine doesn't cost quality, not what graph and time search add on agent memory.

Storage​

Each memory takes about 4–5 KB in the bucket for its indexes and links, measured on real Hindsight banks, plus the stored documents (about twice their input size). A year of one employee's agent, about 68k memories, is roughly 350 MB of indexes: under a cent a month on S3.

Cost​

We price a whole Hindsight cluster on each store with our internal sizing model: the memory store, the API, workers, disks and buckets, at GCP list prices after 12 months of growth. We leave out the LLM, and the GPUs for embeddings and reranking, which cost the same either way.

How a cluster is sized. The model gives each setup the smallest cluster that holds a year of memories and answers recall within 1 second at p99 at peak. It starts from how long one recall takes on its own, as measured, and adds query nodes until queueing at peak fits in what's left of that second. When a single Postgres recall already takes longer than a second, as on big banks, no amount of hardware fixes it, because recall runs on one primary; there the model sizes Postgres to keep up with traffic and reports how slow each recall is.

A year for a whole company, with one bank per employee or agent:

Monthly cluster cost after a year, excluding the LLM and GPUs: 1,000 employees (68M memories) $3.2k on Memlake against $4.6k on Postgres; 10,000 employees (680M) $15.1k against $58.4k; 100 autonomous agents (490M) $11.1k against $146k; 1,000 autonomous agents (4.9B) $94k against $2.46M. One recall at p99: 0.35 s on Memlake in all four, against 2.4, 9.1, 5.2 and 11.1 s on Postgres, which misses the 1 s target every timeMonthly cluster cost after a year, excluding the LLM and GPUs: 1,000 employees (68M memories) $3.2k on Memlake against $4.6k on Postgres; 10,000 employees (680M) $15.1k against $58.4k; 100 autonomous agents (490M) $11.1k against $146k; 1,000 autonomous agents (4.9B) $94k against $2.46M. One recall at p99: 0.35 s on Memlake in all four, against 2.4, 9.1, 5.2 and 11.1 s on Postgres, which misses the 1 s target every time

Almost all of the difference is the memory store itself; the rest of Hindsight costs $0.6–2k a month on either. Total memory count is not what makes Postgres expensive. Small banks cost about the same on either store, and a small deployment (around 80k memories in total) is cheaper on Postgres. Memlake's advantage shows up as banks get big, and as there are more of them. If your banks stay under roughly 50,000 memories each, Postgres is a good fit.

Caveats​

  • Head to head ran on a shared development cluster, not identical hardware: Postgres had a 4-vCPU primary shared with other development organizations. Postgres's keyword cost comes from how Hindsight's Postgres recall builds its full-text query; the runs had Hindsight's default limit of the 16 rarest query words on, and other keyword engines (vchord, pg_textsearch, pgroonga) would bring it down.
  • Engine-level benchmarks ran July–August 2026, gRPC to Memlake with no Hindsight API in front. The 100M fleet and the 10M single-bank runs used object storage in another cloud, and the fleet ran on synthetic data; the 3M and bank-growth results ran on a realistic corpus.
  • The bank-growth sweep ran on an August 2026 engine against a local S3-compatible store with 16 concurrent clients, not fully warm. It is being re-run on today's engine.
  • BEIR results are from the July 2026 engine and are being re-run.
  • Cost is modelled, not billed. Autonomous banks reach about 5M memories, past the largest bank measured through the API (151k), so their throughput is extrapolated; Postgres beyond about 210k memories per bank is extrapolated too. Recall times in the cost chart are the model's p99 for one recall: for Postgres the measured single-recall time at that bank size, for Memlake the budget of its default cache setting (measured warm recalls are faster, about 0.1 s). On the storage tier alone, our hosted fleet's shape is 2.2× cheaper on Memlake; the whole deployment, 1.5×.