Provem

1. Executive Summary

Provem is 21,444 lines of Python with 7,658 lines of tests, positioned not as a memory but as a governance layer over one — right-to-erasure, tenant isolation, injection defense and a tamper-evident audit, able to run over Mem0 or Graphiti or as its own store. Its README states the thesis in a sentence: the hard question is no longer "can I find it?" but "am I allowed to use it?"

It carries all seven capability marks. Where Verel arrived at them under red-team pressure, Provem arrived at them because a regulation names each one.

The evidence practice is the most disciplined in the corpus. The repository ships scripts/verify_repro.sh, described in its own header as a "self-checking EUR-0 replay of every published benchmark number" that "ASSERTS the exact published values; any missing input or drifted number exits non-zero." Run at this commit from a clean clone:

== dataset integrity ==  == baseline ==  == optimized ==  == post empty-fix ==
== headline ==  == three-system scoreboard v1 ==  == v2 ==
== neutral-prompt control (prompt-confound disclosure) ==  == manifest freshness ==
VERIFY OK (21 assertions)

and with --full, which adds the deterministic governance benchmark and the unit tests, VERIFY OK (25 assertions), exit 0. This is a stronger artifact than anything else here: not a harness a reader may run, but a regression gate that fails if the README drifts from the numbers.

It publishes its losing configurations. The tier table lists four deployments, and two of them lose: the stdlib-retrieval tier scores 0.388 "below Mem0", and the fully keyless tier scores 0.21 against a stated 0.24 no-memory baseline — a configuration that is worse than having no memory at all, published in the same table as the recommended tier's 0.614.

It discloses a confound in its own favour-adjacent result. One replay step is named "neutral-prompt control (prompt-confound disclosure)", and the README marks the abstention gap against Mem0 as "a statistical tie and prompt-confounded". It also configured the Zep comparison "following its own published evaluation checklist… to pre-empt the misconfiguration critique it raised against Mem0's paper", and closes with "We rank only what we measured."

And docs/claim_register.md records claims it has rejected. One row reads Unsupported with the evidence column literally None (ai4privacy download pending license review) and the disposition "Rejected for now; scope track runs only on committed fixtures until a licensed real PII set is approved." A claim register that carries a rejected claim is the artifact this atlas has been describing in the abstract for months.

Reservations. The governance benchmark that produces the headline compliance numbers is self-authored, which the repository says plainly. And the erasure guard is read-side suppression rather than write-side refusal, which matters in the way described below and is the sharpest thing to press on an Article 17 claim.

It is Apache 2.0 licensed, with license = "Apache-2.0" and license-files = ["LICENSE"] declared in pyproject.toml rather than left to a stray file — so the metadata and the file agree, which is more than several systems here manage.

LICENSING.md states the boundary in the project's own words: an "open engine, closed operations" model, under which "everything the published benchmarks and compliance claims rest on stays open, so anyone can reproduce and audit them" — the governed memory engine, policy evaluation, tombstones, erasure, revocation and requester authority, and the purpose stack. That is the right half to keep open for a system whose product is governance, and it is worth reading beside section 10: the claims are only checkable while the code behind them is.

2. Mental Model

A fact is admitted or quarantined, ranked or excluded with a reason, and erasure leaves behind a durable token set that outlives the records it deleted.

%% caption: erasure writes durable per-tenant token sets that every later recall consults, so forgetting outlives the rows it deleted and produces a certificate in the audit log
flowchart TB
    IN["ingest"] --> ADM{"governed admission"}
    ADM -->|"injection detected<br/>or sensitive without consent"| Q["quarantined<br/><i>quarantine_reason</i>"]
    ADM -->|"lawful basis, provenance,<br/>scope stamped"| LIVE["TemporalFact<br/>valid_at / invalid_at<br/>+ created_at"]

    LIVE --> REC{"recall:<br/>_exclusion_reason"}
    REC -->|"erased"| X1["excluded"]
    REC -->|"do_not_use"| X1
    REC -->|"wrong_tenant"| X1
    REC -->|"scope mismatch"| X1
    Q --> REC
    REC -->|"admitted"| ANS["answer or abstain"]

    F["forget(term, scope)"] --> DEL["delete matching records"]
    F --> ERT[("erased_terms[tenant]<br/><b>token sets, durable</b>")]
    F --> CERT["erasure certificate<br/>into the audit log"]
    ERT -.->|"consulted on every recall"| REC

    style ERT fill:#e7efe9,stroke:#3d6b59

The dotted edge is the mark most systems here lack. forget does not only delete: it tokenizes the term and appends the token set to a per-tenant erased_terms registry, and every later recall excludes any record whose tokens are a superset of an erased set. Deleting the rows removes what exists; keeping the token set is what survives the next ingest.

3. Architecture

Forty-eight Python modules under src/cognitive_memory/. The governance core is reliability.py (admission, erasure, exclusion), compliance.py (regime profiles, erasure modes), policy.py, scope.py, safety.py, audit.py and review.py. Around it sit retrieval.py, ranking.py, temporal.py, reflection.py, consolidation_eval.py and an answerer stack. adapters/ carries the Mem0 and Graphiti bridges plus mocks, and mcp_server.py exposes the whole thing over MCP.

docs/ is unusually load-bearing: a claim register, a research journal documenting "every synthetic suite, every negative result", an optimization log covering "every optimization iteration incl. failures and the bug post-mortem", a trust model naming what belongs in a gateway instead, a ship report with full limitations, and docs/runs/ holding the frozen artifacts and manifest.

Deployment and ergonomics

Four tiers, with their costs stated: keyless and air-gap-safe (and worse than no memory), stdlib retrieval with an LLM answerer, dense with an embeddings key ("cents per conversation, disk-cached"), and local embeddings marked "projected, unbuilt". Marking an unbuilt tier as projected in the same table as measured ones, rather than omitting it, is the same instinct as publishing the losing rows.

The compliance profiles are configuration: a HIPAA-style profile is described in compliance.py as "PHI: MRN/patient identifiers, consent required, strict erasure, long adverse-event retention", with erasure_mode validated to strict or lenient and a config error otherwise.

4. Essential Implementation Paths

A replay that asserts instead of printing

scripts/replay_report.sh is four lines that exec verify_repro.sh, kept as the documented entry point because "the replay is now self-checking — it ASSERTS every published number instead of just printing them."

The script requires each frozen input by path, runs each step, and expects verbatim strings in the output, counting assertions. A missing artifact exits 1 before anything runs; a drifted number exits 1 at the step that produced it.

The dataset is handled the way this atlas asks for and almost never sees. LoCoMo is CC BY-NC, so it is not redistributed; scripts/fetch_locomo.sh downloads it from the authors' repository, verifies it against a sha256 pinned in the reproducibility manifest, refuses to overwrite an existing file whose hash differs, and aborts on mismatch with "upstream changed?". That is a dataset checksum doing work rather than decorating a config block.

Read carefully, the "zero cost" claim has one prerequisite and no exception. Without the dataset the gate exits 1 with MISSING INPUT. With it fetched — one hash-verified download — all 21 assertions pass with no API key and no model call, and 25 with --full.

Erasure that outlives the record

def forget(self, term: str, scope: Scope) -> int:
    term_tokens = tokenize(term)
    if term_tokens:
        self.erased_terms.setdefault(scope.tenant, []).append(term_tokens)
    ...
    removed = self.backend.delete_ids(remove)
    self.audit.erasure_certificate(term, remove, scope.tenant, removed)

and on the read path, in _exclusion_reason:

if any(term <= erasure_tokens for term in self.erased_terms.get(record.scope.tenant, [])):
    return "erased"

Three things make this a rejected-value tombstone rather than a soft delete. It is keyed on the value, not the row. It is normalized — a token set with a subset test, so a later record that restates the erased term inside different surrounding text is still caught, which is a looser and more forgiving key than the exact-text hash Daimon uses. And it is tenant-scoped, so an erasure in one tenant does not silently censor another.

The limitation is the same one this atlas records for Daimon: it is suppression at read rather than refusal at write. A re-ingested erased value still lands in the backing store and is stopped on the way out, so every future read path has to consult the registry, and the store holds content a subject asked to erase. For a system whose stated purpose is GDPR Article 17, that distinction is the one a reader should press hardest on.

Exclusion with a reason, and scope that cannot be dodged

_exclusion_reason returns a string per rejected record — quarantined, erased, do_not_use, wrong_tenant, or a scope mismatch — so a recall result carries why something was withheld rather than silently returning less. The cross-tenant check is labelled "defense in depth (backends that do not pre-filter by tenant)", which is the right posture when the store is pluggable.

The subject-scope rule is the sharpest line in the file:

"a query about subject X must not be answered by a look-alike record about subject Y. A subject-less query (entity="") must NOT be served a subject-scoped record either — otherwise scope isolation is bypassable by simply omitting the entity."

Closing the omitted-parameter bypass is the kind of thing that only gets written after somebody tried it, and it is stated as the reason rather than left as a condition.

Admission that can hold a write

store applies governance before anything is retrievable: prompt-injection quarantine, a sensitive-without-consent hold, provenance and scope stamped at write. A quarantined record keeps a quarantine_reason and is excluded at recall rather than dropped, so the hold is inspectable. That is a governed write gateway whose refusal state is durable and reviewable, and it is what review.py's ReviewStatus workflow adjudicates.

5. Memory Data Model

TemporalFact is subject / relation / object plus valid_at, invalid_at, created_at, confidence, evidence, supersedes and superseded_by, scope, privacy_policy, source_type, source_trust, source_timestamp, source_conflict_policy and conflict_with.

valid_at/invalid_at against created_at is world time against record time, which is the bi-temporal mark, and source_timestamp defaults to valid_at when unstated rather than to now — a small choice that keeps an undated source from looking freshly observed.

Trust is several fields rather than one: discrete statuses (hypothesis, accepted, proposed, pending, plus quarantined as a flag with a reason), source_trust as a separate axis from confidence, and source_conflict_policy defaulting to abstain_on_conflict — a store whose default answer to a contradiction is to say nothing is unusual and is the correct default for the regulated setting it targets.

6. Retrieval Mechanics

Dense recall in the recommended tier, with the exclusion pass above between candidates and answer, and abstention as a first-class outcome rather than an empty result. The headline recall figures come from LoCoMo: 0.614 for the dense tier against Mem0's 0.565 and Zep's 0.449, with p-values reported (Provem>Mem0 p = 2.5×10⁻⁴, Provem>Zep p = 1×10⁻³⁴) and the ordering stated as identical under all three judges.

The README's framing of its own recall result is the notable part — "On recall it holds its own… roughly a tie with Mem0" — for a number that is nominally a win. A project describing its own benchmark victory as a tie is not a common failure mode.

7. Write Mechanics

Writes pass admission, are stamped with provenance and scope, and either land live or land quarantined with a reason. Erasure is synchronous: delete, register the token set, emit a certificate. Background work covers reflection and consolidation.

Operational cost

The governance path needs no model — the entire governance benchmark is "deterministic, no API key", which is why the replay can assert its numbers for free. Cost enters only at recall in the dense tier (embeddings, disk-cached) and at answering. The keyless tier removes both and is honestly reported as worse than no memory, which is the trade named rather than hidden.

8. Agent Integration

An MCP server, a CLI, and adapters for Mem0 and Graphiti. The adapter posture is the product thesis: the governance layer is meant to sit over an existing memory rather than replace it, which is why mem0_audit.py, mem0_env.py and the Graphiti bridge exist alongside its own store. docs/trust_model.md states the boundaries and "what belongs in a gateway" instead — declining scope in a security document is worth more than claiming it.

9. Reliability, Safety, and Trust

Strengths:

• A replay that asserts every published number and exits non-zero on drift; 21 assertions, 25 with --full, passing at this commit.
• A hash-pinned external dataset fetched rather than redistributed, with an overwrite refusal on hash mismatch.
• Losing configurations published in the same table as the winner, including one below the no-memory baseline.
• A disclosed prompt confound, with its own replay step, and a favour-adjacent gap called a statistical tie.
• The competitor configured to its own published checklist, to pre-empt the critique it made of a third party.
• A claim register carrying rejected and unsupported claims, with evidence level and risk per row.
• A research journal of negative results and an optimization log including failures and a bug post-mortem.
• Value-keyed, normalized, tenant-scoped erasure with a certificate in the audit log.
• Exclusion reasons returned per record, so a withheld memory is explainable.
• Scope isolation closed against the omitted-parameter bypass.
• abstain_on_conflict as the default source-conflict policy.

Gaps:

• Erasure is read-side suppression, so the store retains content a subject asked to erase and every read path must consult the registry.
• The governance benchmark is self-authored, and its 240 → 0 and 100% → 0% figures are measurements of the author's own scenario suite.
• The local-embeddings tier is unbuilt, marked projected.

10. Tests, Evals, and Benchmarks

12,111 lines of tests across 48 files, run as part of verify_repro.sh --full. The suite was not run for this reading — the dependency surface was inside the cooldown — so what follows reports the published figures and what asserts them, not a fresh verification.

The compliance headline — "Compliance violations 240 → 0, memory-poisoning success 100% → 0%, silent compounding errors 72.6% → 0.0% (paired: governance flips 497 trajectories out of 960, loses 0; deterministic, no API key)" — is reproducible without a key and is asserted by the gate. "Loses 0" is the claim worth noting: reporting that the governance layer regressed no trajectory is a stronger and more falsifiable statement than the improvement figure beside it.

Two caveats the repository states itself and this report repeats rather than discovers. The governance suite is self-authored, so it measures the failure modes its author thought of; and the LoCoMo comparison, while replayable, is a self-run evaluation of two competitors rather than a third-party leaderboard — which is why the README separates "independent third-party evaluations (quoted verbatim, their setups)" into a second ranking labelled as not comparable to the first.

The negative_eval mark rests on the compliance arm: committed, deterministic cases asserting that erased, restricted and poisoned material is not returned, wired into a gate that fails if the count moves off zero.

11. For Your Own Build

Steal

• Make the replay assert, not print. A script that re-derives your published numbers and exits non-zero on drift turns your README into something CI can fail on.
• Pin the dataset by hash and fetch it, when a licence forbids redistribution — and refuse to overwrite a local copy whose hash differs.
• Publish the configurations that lose, including any that are worse than the no-memory baseline. It costs two rows and it is the difference between a scoreboard and an advertisement.
• Keep a claim register with an evidence level per claim, and let rows say Unsupported and Rejected for now.
• Register the erased value, not just delete the row. Tokenize, normalize, scope it to the tenant, and consult it on every read.
• Return a reason for every exclusion. "Erased", "wrong_tenant" and "do_not_use" are three different silences and a caller needs to tell them apart.
• Close the omitted-parameter bypass explicitly. If a scoped record can be reached by a query that names no scope, the isolation is decorative.
• Default to abstaining on source conflict in any setting where a confident wrong answer is worse than none.

Avoid

• Read-side suppression as the whole of an erasure story. It is a real defence and it leaves the erased content in the store, which is exactly the property the regulation is about.
• Shipping without a licence file when the pitch is enterprise compliance — the one thing a compliance-minded adopter cannot do with an unlicensed repository is use it. Worth keeping as a general rule because it is the cheapest possible fix and the easiest to leave undone; here it was closed in one commit once someone pointed at it.
• Treating a self-authored governance suite as a general result. It measures the attacks the author modelled.

Fit

Right if governance is your actual problem — regulated data, multiple tenants, subject erasure requests, an audit you will be asked to produce — and you already have a memory that retrieves well enough. The adapter posture means you may be able to keep it. Seven of seven marks is a genuinely unusual position, and the evidence practice around the claims is the strongest in this atlas.

Wrong as a drop-in recall upgrade: the recall win is real and modest, its author calls it a tie, and two of the four deployment tiers are slower or worse than the baseline. And wrong wherever "erased" has to mean the store no longer holds it, rather than no longer serves it.

12. Open Questions

• Could the erasure registry move to the write path, so an erased value is refused at ingest rather than filtered at recall?
• The erased-token subset test is looser than an exact hash — what is its false positive rate on innocent records that happen to contain the term?
• The governance benchmark is self-authored. Is there an external attack suite it has not been run against, beyond the injection datasets already used?
• What does the local-embeddings tier actually score, once built?
• source_timestamp defaults to valid_at — how often is a real source timestamp available in practice, and does the fallback distort recency?

Appendix: File Index

• Governance core: src/cognitive_memory/reliability.py (forget, erased_terms, _exclusion_reason, quarantine), compliance.py (regime profiles, erasure_mode), policy.py, scope.py, safety.py.
• Audit and review: src/cognitive_memory/audit.py (erasure certificates), review.py (ReviewStatus).
• Model: src/cognitive_memory/models.py (TemporalFact), temporal.py.
• Adapters and integration: src/cognitive_memory/adapters/, mem0_audit.py, mem0_env.py, graphiti_env.py, mcp_server.py, cli.py.
• Reproducibility: scripts/verify_repro.sh, scripts/replay_report.sh, scripts/fetch_locomo.sh, docs/runs/manifest.json, docs/runs/.
• Evidence documents: docs/claim_register.md, docs/research_journal.md, docs/lager_optimization_log.md, docs/ship_report.md, docs/reliability_results.md, docs/trust_model.md, docs/locomo_benchmark.md.
• Licensing: LICENSE (Apache 2.0), declared in pyproject.toml as license = "Apache-2.0" with license-files, and the open/closed boundary in LICENSING.md.

History

2026-09-19 — f6ce1b69… — trust_state re-tested at the unchanged pin. The approved gate is where the record said and appears at six sites across the review and consolidation-eval paths. Two things are added. The vocabulary's middle is doing real work: needs_more_evidence and deferred sit between approved and rejected, so a reviewer can decline to admit an item without ruling against it, and the auto-reviewer returns a named blocker and a confidence with every verdict — a high-risk item is refused with the reason high_risk_requires_human_review rather than being silently held. And the read side has the strongest form of withholding this sweep has found: RecallResult carries an abstained flag and the recall path returns answer=None, abstained=True rather than the best of a contested set, with abstain_on_conflict as the default source-conflict policy. Declining to answer is a stronger act than filtering a list, and the suite is built so it cannot be gamed — the module notes that the equal-trust, no-history one-against-one case is deliberately kept out of the governed arm, so a policy that simply abstained on everything could not look calibrated. Nothing was installed and no suite was run.

2026-09-18 — re-read at the same commit; nothing upstream has moved. human_review stands, and the gate is tested in both directions, which is more than most holders manage: test_rejected_items_do_not_mutate_memory rejects every item and asserts controller.store.list_reflections() == [] with all items REJECTED (tests/test_review_queue.py:90-100), and test_approved_simulation_only_affects_evaluation_copy supplies the control — approve, then assert the original store still has no reflections while the evaluation copy does (:102-113). So the queue can withhold, and the pipeline can write when it does not.

One thing the record should say, because the name invites a wrong reading: simulate_review is not the review. It is a policy simulator with reviewer_id: str = "simulated_reviewer" (review.py:66-71), and its only callers anywhere are in tests/test_review_queue.py. Nothing under src/ outside review.py sets ReviewStatus.APPROVED, and there is no MCP server, no CLI verb and no HTTP route that approves an item. The approval surface is left to the embedding application. That is a normal shape for a library and it satisfies the mark's test — nothing an agent holds can approve here, because nothing in the tree can — but a reader should not mistake the simulator for a person. No marks change.

2026-09-09 — f6ce1b69… — third reading, 54 commits on: 73 files, 10,764 insertions. Screened before reading. The suite was not run this time — the dependency surface sat inside the seven-day cooldown — so unlike the previous reading this report does not carry a fresh VERIFY OK, and section 10 says so rather than letting the earlier run stand in for one.

One published claim was wrong at this commit and is corrected in both places it appeared. The core was relicensed from MIT to Apache 2.0: LICENSE is the Apache text and pyproject.toml declares license = "Apache-2.0". A new LICENSING.md states an "open engine, closed operations" boundary, keeping open "everything the published benchmarks and compliance claims rest on" — the governed engine, policy evaluation, tombstones, erasure, revocation, requester authority and the purpose stack.

All seven marks hold on the same mechanisms; forget, erased_terms, _exclusion_reason, ReviewStatus, TemporalFact and the erasure certificate are all where the appendix says. The headline figures are unchanged at this commit — 240 compliance violations to 0, poisoning 100% to 0%, silent compounding errors 72.6% to 0.0%, governance flipping 497 trajectories out of 960 and losing 0.

What is new is mostly reach rather than mechanism. reliability.py gained roughly 890 lines carrying a trajectory harness — an Agent protocol with deterministic and noisy implementations, classify_step, Scenario and run_trajectory — which is the machinery behind the paired figures rather than a new memory behaviour. Two modules arrived beside it, dsar.py and signing.py, and a ServiceNow ITSM integration with an end-to-end validation harness under qa/servicenow_e2e/ and a recorded real-instance run. Tests went from 7,658 lines to 12,111 across 48 files.

2026-08-04 — 9cc7401b… — second reading, one commit on. 9cc7401 adds an MIT LICENSE and makes the package publishable, declaring license = "MIT" and license-files = ["LICENSE"] in pyproject.toml, so the file and the metadata agree. The previous reading recorded that no licence existed anywhere in the tree and that all rights were therefore reserved; that is no longer the case and the report no longer says it. scripts/verify_repro.sh --full was re-run from the new pin after fetching the hash-pinned dataset: VERIFY OK, 25 assertions, so the published figures still hold at this commit. Nothing about the erasure path changed — forget still registers the term's tokens and excludes at recall rather than refusing at write, and the store still retains a re-ingested erased value.

2026-08-04 — fc722f06… — first reading. scripts/verify_repro.sh was executed from a clean clone after fetching the hash-pinned LoCoMo dataset: VERIFY OK, 21 assertions, and 25 with --full including the governance benchmark and the unit tests. Without the dataset the gate exits 1 with MISSING INPUT, which is the one prerequisite behind the zero-cost claim.