Agent Memory Atlas

Pattern · Federation

Pluggable Memory Provider

Let a host runtime mount swappable memory backends behind one interface, and decide who owns trust, correction, and deletion before the first plugin ships.

Epistemic status Reporting, with one advocacy claim

Intent

Let an agent runtime treat memory as a replaceable component: one interface, many backends, chosen by configuration. Then answer the question the interface makes urgent — when memory lives in someone else's store, who is responsible for forgetting?

The page makes two kinds of claim. The first half is reported practice: host runtimes that mount memory behind one lifecycle contract are common in this atlas, from Hermes and OpenClaw to ADK, AutoGen, Agno and Strands. The second half is argued: carrying scope and deletion across that contract. ADK and MateClaw declare scope; the Pydantic AI Harness declares a targeted delete; no contract read here declares both. That half rests on one or two instances, and the page is arguing for it rather than reporting it.

The problem

A host runtime that hard-codes its memory model forces every user into one set of trade-offs. A local Markdown store cannot do cross-session user modelling; a hosted graph service is overkill for a single developer's laptop. So mature runtimes make memory pluggable.

The moment they do, memory stops being one system and becomes two: the host, which owns the conversation, the prompt, the user relationship, and any built-in memory of its own; and the provider, which owns durable storage, extraction, and retrieval.

Responsibilities that were implicit in a single system now have to cross a boundary:

  • The user tells the host to forget something. Does that reach the provider?
  • The provider extracts a claim the host never reviewed. Whose trust model applies?
  • The host has its own built-in memory and a provider is mounted. Which wins, and are they deduplicated?
  • The host wraps messages in scaffolding. Does the provider know to strip it?
  • The host is multi-tenant. Does the interface even carry a scope?

Interfaces built around the happy path — write, read, prefetch — answer none of these.

The pattern

Define a lifecycle contract the host calls at known points, and make the ownership questions explicit parts of it rather than leaving them to each plugin.

A workable contract has four groups:

Lifecycle    initialize / shutdown / on_session_start / on_session_end
Read         prefetch(query, scope) -> context block
             system_prompt_block() -> static text
Write        sync_turn(user, assistant, scope)
             on_memory_write(action, target, content)   # mirror host writes
Governance   forget(scope | id) -> result               # <-- usually missing
             scope on EVERY call                        # <-- usually missing
             capabilities() -> what this provider actually supports
Diagram — the interface is where deletion is lost — a host with no delete hook stops the request at the boundary, and a mirrored write with no reconciliation leaves two stores disagreeing
Diagram source
%% caption: the interface is where deletion is lost — a host with no delete hook stops the request at the boundary, and a mirrored write with no reconciliation leaves two stores disagreeing
flowchart TD
    User["User: forget me"] --> Host["Host runtime"]
    Host --> Built["Built-in memory"]
    Host --> Iface["MemoryProvider<br/>interface"]
    Iface --> P1["Local plugin"]
    Iface --> P2["Hosted service"]
    Iface --> P3["Graph backend"]
    Host -.->|"no delete hook<br/>= request stops here"| Iface
    Built -.->|"mirrored write,<br/>no reconciliation"| P1

The dotted edges are where real implementations break.

Why it works

A narrow interface at a real seam lets the host own conversation, prompt assembly, and policy while the provider owns storage and retrieval. Users pick a backend matching their privacy and scale needs without the host reimplementing five memory architectures. Providers reach many hosts by implementing one contract.

It also concentrates governance in one place: if the interface requires scope on every call and defines a deletion path, every backend inherits those properties whether or not its author thought about them.

Tradeoffs

  • Deletion is the hard one. Without a forget hook, a host-level erasure request has nowhere to go. The host can delete its own copy and still leave the content in a hosted provider indefinitely.
  • Trust state does not survive the boundary. A host with candidate/verified semantics cannot express them to a provider that stores flat facts, and vice versa.
  • Mirroring creates duplicates. Copying host writes into the provider is the obvious way to keep them in sync, and it silently creates two records with independent lifecycles unless removals mirror too.
  • Capabilities vary wildly. One provider does hybrid retrieval and bi-temporal validity; another does vector-only lookup. If the interface cannot report capabilities, the host must assume the weakest.
  • One provider at a time is the usual limit, chosen to avoid tool-schema bloat and conflicting backends — which means no composition, and switching backends usually means abandoning accumulated memory.
  • The host's scaffolding leaks. Providers receive whatever text the host passes; if that includes context markers, reply headers, or compaction summaries, it becomes memory.

Do not adopt this pattern to postpone deciding on a memory model. It relocates the decision; it does not remove it.

Cost to adopt

Build: an interface, a lifecycle for mounting providers, and — the part most implementations skip — scope and deletion in the contract itself.

Forces elsewhere: the contract is the ceiling on every backend's capabilities. Anything absent from it (deletion, scope, trust state) is unreachable no matter what the backend supports, and widening a published contract breaks implementers.

Ongoing: every provider needs its own tests, and cross-cutting concerns like retry and metrics belong in decorators rather than in each backend.

Skip it if you will only ever run one backend. An interface with one implementation is a guess about the future.

Seen in the atlas

Hermes Agent has the most explicit contract: a MemoryProvider ABC with more than twenty members covering identity and availability, initialization, prompt blocks, prefetch and its queued form, recall status, turn start and per-turn sync, tool schemas and dispatch, session end and switch, pre-compression extraction, delegation observation, config, backup paths, and shutdown, with a MemoryManager that mounts exactly one external provider. First-party adapters ship in-tree for Honcho, Mem0, Hindsight, Supermemory, OpenViking, ByteRover, RetainDB, and the built-in Holographic plugin — so the adapters are reviewable even when a backing service is not. The contract has no deletion hook and no per-call tenant scope: initialize(session_id, **kwargs) documents user_id among optional keyword arguments, and prefetch and sync_turn take a session_id, so a provider learns its user once at startup rather than on each read or write.

OpenClaw reaches the same shape from the other direction: the memory system is itself a plugin, memory-core, holding the index, the consolidation pass and the deletion path, and memory-lancedb is one optional vector backend beside it. The contract covers capture and injection but not inspection or repair. Its LanceDB store shows the discipline the interface itself lacks — scopedPredicate composes agent scope and operator filter into one predicate "so scope cannot be lost", and deletes are scoped too.

Holographic demonstrates the mirroring hazard concretely. Its on_memory_write copies the host's built-in memory additions into its own SQLite store, but implements only the add action — so removing an entry from the host's MEMORY.md leaves the mirrored fact in place, with no reconciliation path.

Pi is the strongest form of the argument, because it has no memory contract at all. Its ExtensionAPI exposes more than twenty lifecycle events — session_start, session_before_fork, session_before_compact, context with a result type, and more — which is ample mechanism, and none of it is memory-shaped. A plugin can capture, inject, and consolidate; it cannot be handed a scope or told to forget, because nothing in the host knows memory exists. Magic Context consequently rebuilds message indexing, FTS, embeddings, and scope from scratch on top of those events, and has to decide for itself what a forked session inherits.

TencentDB Agent Memory sidesteps the host contract altogether: at its pin it is a reverse proxy in front of the model API, reached by pointing an agent's base_url at it, with SDKs and an OpenClaw plugin shipped beside it. Because it owns the traffic rather than a slot in a host's interface, deletion and scope are its own to define, and it defines both — atomic/delete, scenario/rm and a clear-memory path, each journalled to memory_audit, with the team/user/agent/task key composed into the delete predicates as well as the reads.

MateClaw is the counterexample that shows the gap is a choice, not an inevitability. Its MemoryProvider SPI declares paired overloads — prefetch(agentId, query) and prefetch(agentId, query, ownerKey), syncTurn(...) with and without ownerKey — so scope crosses the boundary. It also handles the resilience question the other contracts leave to each plugin: spi/decorator/ wraps any provider in MetricsMemoryProvider and RetryableMemoryProvider, so retry and instrumentation are solved once for every backend rather than reimplemented per plugin. Default interface methods give implicit capability negotiation — a provider implements what it supports.

MateClaw has no deletion hook, so it closes half the governance gap and leaves the other half open.

Across the host contracts read for this page — Hermes, OpenClaw, Pi (which has none), MateClaw, ADK, AutoGen, Agno's LearningStore, Microsoft's ContextProvider, the Pydantic AI Harness's MemoryStore, CAMEL's AgentMemory, Cosmonapse's Engram and Strands' MemoryStore — two declare scope: ADK by required keyword argument, MateClaw by paired overloads. One declares targeted deletion — the Pydantic AI Harness, whose Protocol is read, get_operation, write, delete, list_paths. Everything else offers clear(), or nothing.

Agno separates two things the other contracts conflate. Its LearningStore Protocol declares recall, process, build_context, instructions and get_tools beside a learning_type property, and the three that receive the caller's context — recall, process and get_tools — take it only as **kwargs. So the contract expresses neither scope nor deletion, and both exist anyway: recall(user_id) returns None when the key is missing, retire_fact keeps a superseded row, forget archives an entity. The reason is that Agno writes the contract and six implementations behind it, so what the interface fails to require, the same team supplies. That is worth naming because it inverts the finding above — the contract is the limiting factor when the contract author is not the implementer, and a permissive interface is survivable exactly when they are the same people. It is also fragile in the predictable way: a third-party store satisfying that Protocol can ignore user_id entirely and nothing detects it. Agno's own _filter_store_kwargs exists to keep narrow third-party stores working when the framework adds context kwargs, which is thoughtful, and which also means a store that quietly drops the scope key is indistinguishable from one that never wanted it.

Cosmonapse's Engram is the only contract here with a failure vocabulary, and the omission is glaring once you have seen one. Every other interface on this page models the happy path: write, read, prefetch. This one ships five typed errors — EngramTimeout for an elapsed deadline, EngramCancelled for a task killed mid-call, EngramNotBound for storage nobody wired, EngramOverloaded for a backend that shed load, and EngramError above them — and two methods that follow from taking failure seriously. can_serve(query) -> bool lets a backend decline (the docstring's example is a BM25 engram asked for vector search, and the hosting Dendrite then skips responding), which is the only place on this page where "retrieval can return nothing" is a capability the contract can express. compensate(trace_id) reverses every journaled write for a trace, replaying inverses LIFO so nested overwrites unwind, with commit(trace_id) discarding the journal at the workflow's commit point.

Two limits are documented rather than left to be found, which is the right way to ship a rollback: compensation is best-effort, so a failing inverse is logged and the rest still run, and only Engram state is reversed — external side effects are explicitly out of scope. The limit that is not documented is where the journal lives: self._saga_journal is a dictionary on the instance, so in a protocol whose transports include NATS and Kafka precisely because workers are disposable, the rollback guarantee lasts exactly as long as the process. It also carries no scope parameter.

Outside the corpus, and read without a pinned report, LightAgent shows the pattern at its smallest — a 196-line shared_memory.py with swappable adapters shipped as examples (clawmem_memory_adapter.py, vector_memory_adapter.py). The shape survives the scale reduction; so does the missing deletion path.

SmythOS SRE is the degenerate case, and it is deliberately not counted in the tally above. LLMMemoryConnector is an abstract class extending the runtime's secure-connector base, declaring one method — load(messages) — and it is exported from the package index at packages/core/src/index.ts. There is no subclass anywhere in the tree, no registration in the connector service, and no call site. Nothing mounts it, so it is not a provider contract that anyone has had to satisfy; it is the seam without the socket. It earns a mention here because the failure it illustrates is cheap to reach: the same repository ships a working connector pattern for cache, storage, vector search, LLM providers and accounts, each with real implementations behind it, and a reader encountering MemoryManager reasonably assumes memory has one too. An exported abstract class is a promise the type system will not keep, and the distance between "we defined the interface" and "a provider exists" is the entire pattern.

Two providers in this ecosystem — Redis Agent Memory Server and OpenViking — have far richer internal scope and lifecycle models than the interfaces mounting them can express, which is the clearest evidence that the contract, not the backend, is the limiting factor.

The vendor-neutral contract nobody signed up for

Every interface above is a host's, and each was designed by people who had to make one runtime work. One is not: OpenTelemetry's GenAI semantic conventions define, at 46d43c89…, a gen_ai.operation.name enum containing seven memory operations — create_memory, search_memory, update_memory, upsert_memory, delete_memory, create_memory_store, delete_memory_store — all at development stability. It stores nothing and is not a memory system, so it gets no report here; it is an observability vocabulary, which makes it the closest thing the field has to an agreed answer about what operations memory has.

Read against this page's own tally, two things stand out.

It names deletion, twice, and at both granularities. Of the host contracts read here, one declares targeted deletion. The vocabulary written by nobody's runtime declares delete_memory for records and delete_memory_store for the whole store, and gives gen_ai.memory.record.count an explicit meaning for a delete — "the number of memory records the operation attempted to delete". It also names upsert_memory separately from create and update, defined as create-or-update "without the caller choosing which", which is the operation most of these systems actually implement and almost none of them name.

It declares no scope at all. There is no tenant, user, project or agent attribute on a memory operation — only gen_ai.memory.store.id, with the note that each component "SHOULD document what gen_ai.memory.store.id maps to". Isolation is left as an implementation detail of the store id, which is the same gap most host contracts on this page have, arriving from another direction.

The record schema is the sharper version of the same point. model/gen-ai/gen-ai-memory-records.json defines a MemoryRecord as content (the only required field), id, score, and an opaque metadata object. No validity interval, no trust state, no provenance, no supersession pointer — every mechanism the tombstone, bi-temporal validity and trust state machine pages are about falls into metadata, where it is by construction not comparable across implementations. That is not a criticism of a development-stage spec, which is right to standardise the common shape first. It is a statement about what the common shape currently is: a scored string with an id.

RushDB ships the contract without the providers. Its agent-memory contract package is provider-neutral by design — a versioned event schema, deterministic ids, a conformance fixture for other languages, and an eight-step adapter skill with a lifecycle matrix covering pre-inference recall, completed-turn capture, compaction, session end and shutdown. The two runtimes it names are first-class in the protocol's own enum and absent from the repository, occurring in exactly two lines of a reference file. Every hard part of the contract — the durable outbox, the fail-open timeout, the capture exclusions, deactivating a superseded fact — is an adapter obligation, so the conformance matrix is a promise made on an unreviewable implementation's behalf.

Strands Agents answers two of this page's tradeoffs and confirms the first one exactly. Its MemoryStore Protocol has one required method, search, and four optional ones, and the host detects which a backend implements by inspecting the type — _has_method resolves the name on the store's class and returns false when it lands on the Protocol's own stub. So the contract reports capabilities without a capability flag anybody has to keep in sync, and the host is not forced to assume the weakest backend. Nor is it one provider at a time: MemoryManager holds a list, fans searches out concurrently, and attributes each result to its store — at the documented price that results are "concatenated in store-registration order with no cross-store ranking," so the injection cap can be filled by whichever store was registered first. Extension points live in context objects (AddMessagesContext, InjectionQueryContext) precisely so the method signatures can stay fixed while the calling convention grows.

Then the ceiling. There is no deletion hook, in either the Python or the TypeScript contract, and the Storage layer the vended file store sits on declares delete that three other subsystems call and no memory path does. The design doc names the intended substitute — "Corrections are handled by storing updated facts. Newer entries take precedence via recency weighting in search results" — and of the three vended stores one implements recency weighting, as a tie-break between equal scores. That is this page's "Deletion is the hard one" in its purest form: an erasure request reaching a well-built interface with nowhere to go, and a correction story that depends on a ranking property the contract cannot require.

Resonant mounts its provider by tool name, and nothing notices when the names move. Two hooks post JSON-RPC tools/call to Resonant Mind, the same organisation's store: mind_orient and mind_ground once per session, and mind_search, mind_surface, mind_spark and mind_thread on every prompt (packages/backend/src/services/hooks.ts:248-258, :1846-1851). At Resonant Mind's pinned commit and at its 4 October 2026 head, the registry has renamed the first two to ritual_* and holds none of the last three. Each unknown name comes back as a JSON-RPC error that callMindTool returns as null without logging, and the request carries no credential although the server requires one. The agent can hold the same server's tools directly through .mcp.json with headers, so the provider works for the model and fails for the host. Check the provider's tools/list against the names the host calls, at startup.

Tests to require

  • Delete a memory through the host and prove it is gone from the mounted provider, not just from the host's own store.
  • Start the host against the provider version it documents and assert every tool name the host calls is in that provider's tool list.
  • Mirror a host write into a provider, remove it at the host, and assert the mirrored copy is removed or explicitly marked orphaned.
  • Switch providers and verify the host does not silently serve stale context from the previous backend.
  • Issue a request under one scope and assert no provider call can return another scope's memory.
  • Feed the provider a message carrying the host's own envelope markers or compaction summaries and assert none of it becomes durable memory.
  • Mount a provider that fails or times out, and assert the agent degrades rather than blocking or losing the turn.
  • Fork or branch a session and assert the memory each branch sees is what you intended — an untested question wherever the host supports branching.
  • Wrap a deliberately failing provider in the host's resilience layer and assert the turn degrades rather than stalling — and that the failure is visible in metrics, not just logged.
  • Assert that a provider advertising a capability it lacks is detected — for example, one reporting "hybrid" retrieval while running vector-only.
  • Run the host's built-in memory and a provider together, and assert the same fact is not injected twice.