Agent Memory Atlas

Pattern · Control

Memory as an Editing Surface

Treat the memory store as a document a person edits, not a database a person is shown.

Epistemic status Reporting an established practice

Intent

Make the memory a user can change the same object the model reads. Not a viewer, not an export, not a support ticket — an editor whose writes are authoritative.

The problem

Every automatic memory system is wrong sometimes. Extraction misreads a joke as a preference, a summary drops the thing that mattered, a consolidation pass merges two people into one. The corpus in this atlas shows this is not rare, and shows something sharper: most systems have no path from a user noticing to the store changing. Correction, where it exists, means telling the model in conversation and hoping the next extraction pass agrees.

The usual response is a memory dashboard. A dashboard that only displays is worse than none, because it shows the user a wrong fact they cannot fix and teaches them the memory is not theirs.

There is a second, less obvious cost. A system whose memory cannot be edited must be right, so it accumulates machinery — trust scores, confidence thresholds, verification passes — to approximate a judgement a person could have made in one click.

The pattern

Give the store an editor, and make every verb the automatic path has available to a person too:

Diagram — the automatic path and the editor write to the same rows, so a person can edit, delete, pin, merge, regenerate or suppress what extraction produced
Diagram source
%% caption: the automatic path and the editor write to the same rows, so a person can edit, delete, pin, merge, regenerate or suppress what extraction produced
flowchart TB
    A["automatic path<br/>extract / summarise / score"] --> S[("the store")]
    S --> P[prompt]
    S --> E["editor over the same rows"]
    E -->|edit text| S
    E -->|delete| S
    E -->|pin / protect| S
    E -->|merge| S
    E -->|"regenerate, then accept or reject"| S
    E -->|suppress without deleting| S

Five verbs distinguish an editor from a viewer:

  1. Edit the text in place — the stored value, not a note attached to it.
  2. Delete, and have the deletion hold against the next automatic pass.
  3. Pin, so a unit is exempt from eviction, decay, or budget pressure.
  4. Merge, because the automatic path over-splits.
  5. Regenerate with a preview — run the model again on one unit and accept or reject the result before it replaces anything.

The fifth is what separates a good editor from a dangerous one, and it generalises: any lossy rewrite a system performs automatically is safer as a previewed action a person triggers.

A sixth verb is worth having where it fits: suppress without deleting, a third state between present and gone, for material you want kept and not surfaced.

Why it works

  • It is the cheapest correction mechanism that exists. One click beats a trust state machine at fixing one wrong fact, and needs no model.
  • It puts the authority where the knowledge is. The user knows whether the fact is right; the extractor is guessing.
  • It changes what the automatic path has to achieve. Recall can be aggressive if a person can prune, so the write path does not have to be conservative.
  • It makes the store legible, which is how users come to trust it — and trust is why they tolerate an imperfect memory rather than switching.

Tradeoffs

It does not scale past what a person will maintain. This is the pattern's real boundary, and it is why systems that must learn cannot rely on it alone.

An edit with no history is a new failure mode. Overwriting in place loses what was there, and a deletion with no record can be re-derived by the next extraction pass from the same source material — so the editor appears to work and quietly does not. Pair it with the rejected-value tombstone or an append-only audit; an editor over a store with neither is a correction that does not hold.

Editing is a write path, and it needs the same scope checks as any other. A shared memory with an unauthenticated editor is worse than no editor.

In the analyzed systems

SillyTavern is the extreme case: the editor is the only write path. Nothing extracts, so there is no automatic path to correct — a person authors entries, keys and activation settings directly. It carries no rubric mechanism in this atlas, and that is the design rather than a gap: hand-authoring means the memory is exactly what the person decided.

RisuAI's HypaV3 modal is the most complete editor over machine-written memory on this page, and carries all five verbs: edit text, delete-from-here, merge with a union of source ids, pin via isImportant, and bulk re-summarise with the result previewed before it is accepted. That last one is the same operation its 2023 generation performed silently on a threshold, which makes a clear before-and-after of this pattern.

Soul of Waifu is the cautionary version. Its SoulMemoryViewer carries two of the five verbs: a person can edit the index, the profile, any topic and any diary in place, and delete a topic or diary file. Neither holds against the automatic path. A deleted topic can come back with the next create action that names it, an edit to the index survives only if the next full rewrite carries it forward, and neither reaches the log that records every automatic mutation.

Restore is the verb that went missing, and it shows both ways the gap can close. At 3d032bad… the API was right — restore_backup took a backup of the current state before overwriting, so a restore was itself undoable — and nothing in the application called it, alongside list_backups, list_topic_files and get_memory_stats. At 747048b3… all four functions are deleted and the backups they would have read are written as before, five each of the index and the user profile. The store gained a copy and lost its only reader. This pattern is a product property rather than an architectural one: unreachable correctness is not correctness, and deleting the unreachable half leaves the cost in place.

Skales's v7 snapshot, withdrawn from its repository in September 2026, shows partial adoption, which is worse than none. Two of its three stores delete correctly from the memory page; the third answers a delete request by telling the user to "ask in chat", and no such verb exists, while a chat phrase such as "remember that" can add the facts the page cannot remove. One page, three stores, inconsistent authority.

Logseq inverts the framing usefully: it is an editor that acquired an agent, rather than an agent that acquired an editor. Its conflict handling is last-write-wins by editing, which is what happens when the human path is assumed to be the only one.

The pattern is not confined to companion clients. kube-coder's dashboard has a create/edit form over the same rows its MCP tools serve. llm-wiki-memory's web editor writes through the engine rather than over the file, pinned by "PUT edits the body through the engine (re-read reflects it)". openyak lets a person replace the workspace document with a PUT, delete it, or force a regeneration with POST /refresh, over the same text the model reads. And grok-build keeps Markdown files under ~/.grok/memory/ as the source of truth, with the SQLite index derived from them.

Elsewhere in the atlas, human_review is held by a minority of systems, and most of those are approval of a queue rather than editing of the store — a reviewer says yes or no to what the extractor proposed and cannot rewrite it.

The pattern at its limit is an editing surface with nothing behind it. In the corpus, memory-compiler keeps its memory in four Markdown files — diffable, reviewable in a pull request, correctable in any editor — and opensre's /memory list tells the user they may "edit or delete the files directly". Outside the corpus, Cline's Memory Bank — six markdown files in the user's own repository, read at the start of a session and rewritten at the end — carries every verb on this page for free. A person edits the file, and that edit is the memory; the model's write goes through the same file with the same tools; a diff shows what changed; a PR review catches a bad one; git holds the history. There is no store to fall out of sync with the editor because there is no store. It gets no report — recorded as an exclusion — for the reason that makes it useful here: grepping the checkout for the mechanism returns two files and both are documentation, so what implements it is a prompt plus the agent's ordinary file tools.

Read against the systems above, it isolates what the pattern is actually buying. SillyTavern is hand-authored and needs no correction path; RisuAI built a real editor over machine-written memory and it took a modal with five verbs; Soul of Waifu wrote the API and never wired it up. Memory Bank skips all of that by making the memory a document in the first place — and pays for it in the one place a document cannot help: nothing consults the file before a write. A deletion holds until the next update pass re-derives the same paragraph from the same conversation, because no record survives saying it was removed on purpose. OpenSRE has the same gap inside the corpus: /memory forget unlinks the file, nothing records that the value was rejected, and extraction runs over the last thirty turns after every recorded turn, so the statement that produced the memory is still in the window the next pass reads. That is the rejected-value tombstone gap, arrived at from the direction of having no database at all, and it is the argument that editability and negative memory are separate properties rather than two views of one.

Windie Sandbox is the pattern taken as far as it goes, and it shows a property the five verbs do not cover: whether the editor keeps what it replaced. The whole product is an editing surface — the conversation is a SQLite tree of messages addressed by id, and a person can replace a message's text, splice one out, truncate a subtree, or fork a branch at any node. The model has no say in any of it. Two mechanisms here are worth lifting on their own: a mutation is refused while a Running or WaitingForApproval session depends on that message, with the error naming both; and any derived summary covering the edited history is deleted in the same transaction as the edit, so a stale summary cannot outlive its source.

Then replace_message is UPDATE messages SET content = ?. The prior text is gone — no version row, no supersession pointer, no mutation event, and the message's only timestamp still says when it was first written. Forking, in the same store, is perfectly lossless: a branch is one insert and every ancestor stays put. So the system has a lossless history operation and a destructive one side by side, and the destructive one is the one called "edit".

That splits this pattern's central question in two. Every system above is judged on whether a person can change machine-written memory; none of them is judged on whether the previous value is recoverable afterwards. Those are different guarantees, and a store that already keeps a tree of alternatives has no structural excuse for lacking the second — writing the edit as a sibling and moving the head would cost one insert. Editability, negative memory, and recoverable correction are three properties, not one.

In marm-memory the bundled console is not the secondary surface — it is the more capable one, and the gap is the finding. A person there creates, replaces and deletes memory rows, and adjudicates the concept graph through merge, dismiss and remove, each under the leased build lock and each aborting on a lost lease rather than half-applying. The connected agent can do none of it: its delete tool accepts only two document kinds and cannot address a memory row, and the flag that lets a caller set a memory's project and platform instead of inheriting the server process's has exactly one caller in the tree, the console's create path. The editing surface is authoritative in the sense this page means, and it is the only place the store's scope key can be written on purpose.

Wordcell is the case where the editing surface is the store: the memory is a folder of Markdown a person opens in any editor, and every index rebuilds from it, so an edit needs no second path to take effect. Deletion is where it stops holding. No command deletes a note, the migration guide tells a user leaving Supermemory to delete the file instead, and the importer keys existing notes on external_id found in the vault. A deleted imported note is therefore absent from that set and re-created as created by the next import of the same export (src/import-supermemory.ts:1134-1157). An edit is protected by a stored digest; a deletion leaves nothing to compare against.

Archiver RAG makes the editor the user's own note app and the index a cache of it. Agent memory is Markdown in an Obsidian vault; a person edits or trashes a note in Obsidian, and a watcher deletes the note's chunks by path and re-adds them on every save, so the edit reaches retrieval in seconds with no memory-specific tool. The verbs the agent holds are thinner than the person's: log_note appends -1 to a taken name instead of finding the note it restates, and its only other mutating tool, move_notes, reaches .trash/ without the confirmation the CLI's delete asks for. When the editing surface belongs to the person, give the agent a lookup before it writes, or every correction it attempts is a second note.

Binder turns the person's edit into the same typed transaction the agent's tool writes. Records render to Markdown by navigation rules; on save, the LSP diffs the file against the snapshot it rendered, matches anonymous list items to entities by a similarity score and an auction assignment, and submits one validated transaction carrying the prior value of every field it changes (packages/cli/src/lsp/handlers/save-handler.ts:41-105, packages/cli/src/diff/auction-match.ts). A person's correction is therefore invertible and schema-checked like any write. It is also unlabelled: the save and the agent's MCP transact both carry the workspace author and no channel (save-handler.ts:93-96, packages/cli/src/mcp/tools/transact.ts:101-105), so the log records that a correction happened and not that a person made it.

Tests to write first

  • Edit a unit, then run the automatic pass, and assert the edit survives.
  • Delete a unit, re-feed the source material that produced it, and assert it does not come back. This distinguishes an editor from a suggestion box, and it is the property the tombstone mark measures.
  • Assert a pinned unit survives eviction, decay and budget pressure.
  • Merge two units and assert provenance is the union, not the newer one.
  • Reject a previewed regeneration and assert the store is byte-identical.
  • Assert every editor verb is reachable from the shipped UI. A test that the function exists is not the same as a person being able to reach it — which is exactly the gap in Soul of Waifu.
  • Edit a unit, then assert the previous value is still readable from somewhere — a version row, a mutation event, a sibling node. If nothing answers, the editor is a one-way door, and the user who wanted to compare two phrasings has lost the first one. Windie Sandbox refuses a mutation a running session depends on and deletes a stale summary with the edit, and has no answer to this one.