Tradeoffs

Every storage design gives something up. This page lists what Memlake trades away, and when Postgres is the better fit.

Writes are durable first, fast second

Memlake keeps every write in object storage before acknowledging it, and keeps query nodes stateless. A write takes about 250 ms at p50 to commit, and p99 can reach several seconds with many concurrent writers. In return, a single bank sustained 1,200–1,500 memories per second at the storage layer in our tests, 15–27× what Postgres sustained, flat as the bank grows, and a write is searchable the moment it returns.

In practice this rarely matters: every Hindsight retain goes through an LLM first, so how fast memories land is bound by the LLM, not the store. Retains can run asynchronously (async: true), which keeps commit latency off the agent's response path.

Consistent reads have a latency floor

Every query checks object storage for the latest writes before answering. On a cached bank this is the only object-storage request, and most of the query's time inside Memlake: about 20–30 ms at p50 on GCS in the same region. Memlake does not offer an eventually consistent mode that skips it.

Cold starts are real

Object storage makes warm reads cheap and first reads slow. The first query to a bank on a query node opens it, reading its index metadata from object storage: 8.6 s at p95 for a 10M-memory bank, once per query node. Until its data blocks are cached, queries on large banks can take about a second at p99.

The layout is built for this case: a cold query needs a small, bounded number of sequential round trips, each fetching only the blocks it needs. Query nodes warm a bank in the background whenever they open it or pick up a new index, and banks stay routed to the same query node, so this happens rarely. If you know a bank is about to get busy, send it one recall ahead of time to start the warm-up.

Built for many banks, not one giant bank

Memlake scales out across banks: each bank is served by one query node at a time, and load spreads across the fleet as banks are added. Agent memory usually splits naturally (per agent, user or project), so this matches how Hindsight is used. A bank's index is never split across query nodes, so a single very hot bank is limited by the query nodes it runs on. For those, the optional balancer can serve the bank from up to three query nodes, or place it on a larger, dedicated query node.

Query cost grows sublinearly with bank size

The number of sequential storage round trips per query doesn't grow with the bank, but the CPU to scan vector clusters does, roughly with the square root of the bank: growing one bank 30×, from 100k to 3M memories, made a recall about 6× slower under heavy load. So does the number of segments each query reads, unless segment routing is on. Keeping banks scoped to one agent, user or project keeps recall fast and cheap.

Ranking policy lives in Hindsight

Hindsight sets the ranking policy (fusion, cross-encoder reranking, boosts and the token budget) and sends it with every request. Memlake runs it next to the data and returns only the results that survive, or returns each search's raw scores for Hindsight to rank. Policy changes take effect on the next request, without touching stored data or rebuilding indexes.

Two stores, not one

Memlake holds memories; Hindsight's operational data (banks and settings, the job queue, the audit log) stays in a small Postgres database. That keeps each system doing what it does best, but a deployment runs both, and the Postgres side still needs ordinary database care (backups, upgrades). It stays small: it holds no memories.

Single region, active/standby

A deployment runs in one region. Disaster recovery is active/standby: the archiver copies every bank to a separate bucket, and a restore is verified before traffic moves. On a regional failure, you lose at most about one indexing interval plus one archive pass of memories; document bodies uploaded since the last daily snapshot may also be lost.

Comparison

Hindsight supports Memlake (Hindsight Enterprise) and relational backends: Postgres by default, and Oracle 23ai. As a rule of thumb, if your banks stay under roughly 50,000 memories each, Postgres is a good fit.

Memlake excels at	A relational backend may be a better fit for
Big banks (hundreds of thousands to millions of memories), and many of them	Small deployments, banks under ~50k memories, local development
Recall that stays fast as banks grow: about 0.1 s at p99, where Postgres took 0.5 s on a 4k-memory bank and 5 s at 210k	The lowest latency on small banks, with no object-storage check on each read
Cost that follows query rate: memories stored at object-storage prices	Environments without S3, GCS, or an object store that supports conditional writes
Fast ingest: over 1,000 memories per second per bank at the storage layer, against 55–80 on Postgres	Teams that want to query memories directly with SQL
Data residency: memories stay in your own bucket, in your own account	Deployments that must keep all data in a single relational database
High availability without replicas: any query node can serve any bank

For more, see Performance and cost, Guarantees and Architecture.