Zep

1. Executive Summary

This repository is not Zep. Its own README says so under About This Repository: it "is not Zep's product or service", but example code, framework integrations and tooling for Zep Cloud, the hosted agent-memory platform. The engine that used to be here — Zep Community Edition — sits deprecated and unsupported in legacy/, and the open engine that powers the hosted product is a different repository, analyzed separately as Graphiti.

So the honest description of what a reader can inspect at this commit is: a client contract for a closed temporal knowledge graph, plus the measurement apparatus its vendor points at it. That sounds like a thin subject for a report, and for the memory mechanism it is — extraction, entity resolution, edge invalidation and ranking all happen on the other side of an HTTP call. Two things in the tree repay the read anyway.

The first is benchmarks/locomo/experiments/, which holds five LoCoMo experiments of ten runs each — fifty runs, 77,000 graded question instances, with per-run standard deviations, context-token distributions and retrieval latency percentiles all committed to git. The benchmarks page uses them as its worked case of run-to-run variance. The sweep varies one thing, the retrieval budget, and it answers a question the field usually leaves as vibes: how much retrieval is enough, and what does the last increment buy?

The second is ingestion/, a bulk-loading library whose warning strings are the most candid public documentation of the hosted service's failure modes. It warns that an episode submitted without a timestamp is silently dated to ingestion time, "which corrupts fact validity timelines and invalidation ordering on backfills". It ships a helper whose entire reason to exist is that a fact is not searchable when the API says the write succeeded. It refuses to let you register "Will" as an entity alias, because rewriting it would corrupt "he will go". These are not marketing claims; they are a vendor's engineers writing down where the sharp edges are, in code, next to the guard rails.

The weakness is the obvious one and it is not fixable by reading harder: on the supported path nothing in this tree stores, extracts, ranks or forgets anything. An API key is required before a single fact can be written, there is no local mode, and the correction semantics that make Zep interesting — bitemporal edge invalidation — are visible in the API's field names and executed somewhere you cannot see. That is why this report carries no bitemporal mark (section 5). The one server in the tree that holds state, the deprecated Community Edition, keeps sessions and messages in Postgres, sends every message to a Graphiti service that extracts the facts, and flattens each fact's two time axes into one timestamp on the way back.

2. Mental Model

From inside this repository a memory has two forms and a gap between them.

The form you write is an episode: a string, a data_type of text, json or message, an optional RFC3339 created_at, and at most ten metadata keys (ingestion/src/zep_ingest/types.py). The form you read is a fact edge: a typed relationship between two named nodes, carrying fact, a relation name (written as fact_name, in SCREAMING_SNAKE_CASE), and four timestamps — valid_at and invalid_at for when the claim held in the world, created_at and expired_at for when the graph made the edge and when it retired it. Those four are the SDK's EntityEdge (zep-cloud 3.28.0, the release current at this commit), and the AutoGen integration copies all four into its memory metadata (integrations/autogen/python/src/zep_autogen/graph_memory.py:146-149). Between the two forms sits the vendor's extraction, which the client never observes.

The write side names its clock differently. An episode's created_at is the source's event time — the loaders stamp it from a Slack ts, an email Date header or, on request, a file's mtime — and add_fact_triple documents its own created_at as "The timestamp of the message". So the one record-time name a client can write is an event time.

The states a client can distinguish are therefore states of submission, not of belief:

episode constructed   -> validated locally against documented API limits
submitted             -> batch id, episode or message UUID, or task id kept;
                         untracked_items incremented when none comes back
result.wait()         -> the tail of the queue reports processed
                      -> but the fact is still not searchable
search_when_ready()   -> polls every 5s up to 120s until something comes back
retrieved             -> a fact edge with valid_at / invalid_at and
                         created_at / expired_at

Belief states are the vendor's. There is no candidate, no verified, no rejected; the closest thing is min_fact_rating, a floating-point threshold on a hosted score that search_graph accepts as a filter. A score is not a state, so this report withholds the trust_state mark. The near-miss is stated here because Zep is often described as if it had one.

How a memory dies is the more interesting half. No call on the supported path updates or deletes a single fact. A client can submit a FactTriple that carries its own closed interval, or an episode from which the service may invalidate an older edge; the only deletes are whole graphs, threads and users, as in AutoGen's clear(). Invalidation closes an interval rather than removing a row — the same move Graphiti makes, and for the same reason: a graph that overwrites cannot answer what did we believe last March. A wrong fact does not disappear; it becomes a fact that was true until a date.

Whether the hosted extractor will re-derive it from the same episode on a later pass is not answerable from this tree. There is no record of a rejected value anywhere in it, so the tombstone mark is withheld too. The deprecated Community Edition is the exception to the first sentence: it forwards a DELETE for one fact UUID to Graphiti (legacy/src/api/apihandlers/fact_handlers_ce.go:26-28).

The one place the client can cause a durable epistemic error is the timestamp. _MissingTimestampCounter in pipeline.py exists to count episodes with no created_at, and its warning is the sharpest sentence in the repository:

Zep silently defaults to the ingestion time, which corrupts fact validity timelines and invalidation ordering on backfills.

Bitemporality is only as good as the event clock it is fed. Backfill a year of Slack history without timestamps and every fact in it becomes true as of the afternoon you ran the import, in an order determined by your loader rather than by history — and because invalidation is ordered by that same clock, the corrections land backwards.

%% caption: what a client can see of a hosted graph: two lags, four timestamps, and no way to say never again
flowchart TB
    Ep["Episode<br/>data, data_type, created_at? (event time)"] --> Guard["LimitGuard splits at 9,500 chars<br/>Alias canonicalizer rewrites names"]
    Guard --> Sub["Submit: batch or sequential<br/>returns batch, episode or task handles"]
    Sub --> Opaque

    subgraph Opaque["Zep Cloud — not in this tree"]
      Extract["extraction, entity resolution,<br/>edge invalidation, ranking"]
    end

    Opaque --> Edge[("Fact edge<br/>world: valid_at … invalid_at<br/>graph: created_at … expired_at")]
    Sub -.->|"created_at absent"| Bad["dated to ingestion time:<br/>validity timeline and<br/>invalidation order corrupted"]
    Bad --> Opaque

    Wait["result.wait() reports success"] -.->|"fact still not searchable"| Lag["indexing lag<br/>search_when_ready polls 5s / 120s"]
    Lag --> Edge
    Edge --> Read["graph.search<br/>scope edges | nodes | episodes<br/>reranked, limit ≤ 50"]
    Edge --> Inject["integration hooks:<br/>thread.add_messages returns context,<br/>injected before each model call"]
    Edge -.->|"correction closes an interval,<br/>it never removes a row"| Keep["no per-fact delete<br/>on the supported path"]
    Edge -.->|"min_fact_rating is a score,<br/>not a status"| NoState["no trust state"]

3. Architecture

Six code trees share the repository, developed and released independently.

• ingestion/ — zep-ingest, about 5,200 lines of Python. A Loader → Transform* → LimitGuard → Submitter pipeline with loaders for Slack exports, .eml mail, WebVTT and speaker-labelled transcripts, text/Markdown files, and JSONL/CSV/JSON records, plus a ConcatLoader that chains loaders into one submit stream. It carries most of the tree's write-side logic: validation, chunking, alias rewriting and resumable submission.
• benchmarks/ — a LoCoMo harness (about 1,800 lines plus tests) with five committed experiments, and the DMR and LongMemEval code for the Zep paper, with no committed results (section 10).
• zep-eval-harness/ — about 4,600 lines of Python driving a smaller end-to-end ingest-then-ask loop over four conversations and four reference documents, with its runs committed under runs/ (section 10).
• integrations/ — one package per framework per language, released independently: Google ADK, Microsoft Agent Framework, AutoGen, AG2, CrewAI, LangGraph, LiveKit, Pydantic AI and Strands in Python; ADK, Mastra and Vercel AI SDK in TypeScript; ADK in Go. This is the largest tree, about half the code, and where memory meets an agent's turn (sections 6 and 8).
• mcp/zep-mcp-server/ — about 1,550 lines of Go, a self-contained module (go.mod declaring github.com/getzep/zep/mcp/zep-mcp-server) with a Makefile, a Dockerfile and a docker-compose.yml.
• legacy/ — Zep Community Edition, a Go server that keeps users, sessions and messages in Postgres via bun and sends every message to a Graphiti service for extraction and search (lib/graphiti/service_ce.go). docker-compose.ce.yaml runs it beside zepai/graphiti:0.3 and neo4j:5.22.0. Deprecated and unsupported, per the README and the linked strategy post.

examples/ adds Python, TypeScript and Go snippets and a Next.js graph viewer.

ontology/default_ontology.py is a single 139-line file of Pydantic models defining the entity types the hosted extractor is asked to classify into: User and Assistant (both declared singletons), Preference, Location, Event, Object, and the rest. The prompt engineering is in the docstrings — Preference instructs "Prioritize this classification over ALL other classifications except User and Assistant" and "Use LOW THRESHOLD for sensitivity", with trigger patterns listed. It is a useful artifact precisely because it shows what a production memory vendor decided to bias its extractor toward: user preferences, aggressively, at the expense of everything else.

The MCP server is a thin, complete Go binary over the zep-go/v3 SDK. cmd/server/main.go loads config through internal/config, and internal/server/server.go:67 registerTools() calls mcp.AddTool thirteen times against the tool values declared in internal/server/tools.go. Each handler in internal/handlers/ is thirty to ninety lines: apply defaults, build the SDK request struct, call one SDK method, marshal the response with transform.FormatJSON. Two transports are supported — StdioTransport for Claude Desktop and Cline, and mcp.NewStreamableHTTPHandler on a configurable port, chosen by TransportMode (server.go:93).

One detail in the HTTP path is a defect rather than a design choice: runHTTP's shutdown calls context.WithTimeout(context.Background(), 5) at server.go:160. The second argument is a time.Duration, so that is five nanoseconds, not five seconds — the graceful-shutdown window is effectively zero and in-flight requests are dropped on SIGINT.

Deployment and ergonomics

There is nothing to stand up and nothing to back up, which is either the whole appeal or a disqualification depending on the reader.

• An API key is required to store anything at all. Pipeline.preview() runs the full loader-and-transform chain with no API calls, so you can validate a corpus offline; Pipeline.run() cannot do anything without a Zep client.
• No local mode, no offline degradation, no self-host on the supported path. The Community Edition in legacy/ is the only self-hostable artifact and it is explicitly unsupported.
• The store is not human-readable or hand-repairable. Inspection goes through graph.search and the getters in zep_graph_inspect.py.
• Install is pip install zep-cloud for the SDK; zep-ingest and each integration package are separate installs. On the supported path the MCP server is the one thing an operator runs: make build or docker compose up, a ZEP_API_KEY in the environment, and a stateless proxy holding nothing.

The screen of this checkout found one auto-run surface — .cursor/mcp.json, which points an editor at Zep's hosted documentation server — no dependency surface inside the seven-day cooldown, 14 build-time execution points and 33 unpinned surfaces across 100 files. Nothing here was installed or run; the analysis is a read of the tree.

4. Essential Implementation Paths

• Write path — Pipeline.run() in ingestion/src/zep_ingest/pipeline.py:164. Builds a Destination, streams loader output through transforms and LimitGuard, materializes the whole stream through _validated_replay() (a SpooledTemporaryFile that spills past 8 MB, so a mid-run failure has resumable handles), then calls submit_episodes().
• Submitters — ingestion/src/zep_ingest/submitters/batch.py and sequential.py, selected by method="auto". Documented limits in types.py: 350 items per add, a 50,000-item batch cap that the submitters roll over at 10,000 by default (DEFAULT_ITEMS_PER_BATCH), 30 messages per thread.add_messages call.
• Chunking and limits — transforms/chunker.py and transforms/limits.py. LimitGuard targets SAFE_EPISODE_CHARS = 9_500 against a documented MAX_EPISODE_CHARS = 10_000, leaving headroom for context prefixes the contextualizer may prepend.
• Alias canonicalization — transforms/canonicalizer.py, the most interesting single file in the repository (see section 9).
• Explicit fact assertion — ingest_fact_triples() in triples.py:162, which bypasses LLM extraction entirely and posts graph.add_fact_triple sequentially, because "the Batch API does not accept triples".
• Retrieval — search_when_ready() in verify.py:23, wrapping client.graph.search. The benchmark's own two-scope retrieval is LocomoEvaluator._graph_search_with_retry in benchmarks/locomo/evaluation.py:63, which fires a node search and an edge search concurrently.
• Context assembly — CONTEXT_TEMPLATE in benchmarks/locomo/prompts.py, a <FACTS> block and an <ENTITIES> block with an instruction that timestamps mean event time and not mention time.
• Delete path — none exists in zep-ingest.
• Legacy purge — deleting a user in the Community Edition deletes the user's Graphiti group and each of its sessions' groups, then calls purgeDeletedResources(), which hard-deletes the soft-deleted message rows and runs VACUUM ANALYZE (legacy/src/store/userstore_ce.go:22-34, db_utils_ce.go:14-40); deleting a session calls the same purge. The transactional purgeDeleted() in purge_common.go, with its no-op tableCleanup() in purge_ce.go, is reachable only through memoryStore.PurgeDeleted, which nothing in the tree calls.

5. Memory Data Model

Episode (types.py) validates in __post_init__: non-empty string data, a data_type in the three-value enum, an RFC3339 created_at, and a metadata map of scalars capped at MAX_METADATA_KEYS = 10. A fifth field, document, is internal plumbing — the chunker stores the full source document on each chunk so the contextualizer can read it, and it is never sent to the API.

Provenance is structured rather than free text, and this is done well. Every loader stamps source_type (document, slack, transcript, email, json_record) plus source-specific keys: a document's file_name, a Slack message's channel and thread_ts. Ten keys is a tight budget for that, and it is a documented service limit rather than a library choice.

Destination is a frozen dataclass whose __post_init__ rejects any construction that is not exactly one of graph_id or user_id (bool(graph_id) == bool(user_id) raises). Every write and every read in zep-ingest carries one, which is what earns the scope_enforced mark: the scope is not a tag that happens to be stored, it is a required argument that this read path cannot omit, and test_destination_required asserts that no search is sent without one (ingestion/tests/test_verify.py:73-76). The mark covers that library and not the whole tree. The MCP server's five UUID-addressed getters take no scope key, as section 8 sets out. The Community Edition's POST /sessions/search builds its group list from an optional user_id and optional session_ids, and sends a nil list, JSON null, when both are absent (legacy/src/store/memory_ce.go:78-92). Graphiti reads a null group list as every group (search_utils.py:202 at v0.3.21); the field comment "Required on Community Edition" is the only guard (legacy/src/models/search_common.go:6).

FactTriple is the richest type and the clearest window into the hosted schema, because every documented API limit is re-validated client-side so a bad triple fails as a Python error naming the field rather than an HTTP 400 mid-run:

Two details carry weight. The single-label cap is enforced with the comment that "extraction assigns one best-match type per node" — the hosted ontology is single-inheritance, so an entity cannot be both a Person and an Organization. And the UUID endpoints exist specifically so "a re-run cannot resolve a slightly different name to a new node", which is a plain admission that name-based entity resolution is the fragile part.

The bitemporal mark is withheld, because nothing in this tree tracks either axis. The hosted contract has both: a returned edge carries world time in valid_at/invalid_at and graph time in created_at/expired_at. But the record axis is written only by the service. The one field of that name a client submits, FactTriple.created_at, is the caller's event time, and the zep-cloud SDK also accepts a caller's expired_at on a triple. What the tree itself does with the axes is read them. The LongMemEval harness and the full-example agent render valid_at … invalid_at as a date range (benchmarks/longmemeval/zep_longmem_eval.py:319-320, examples/python/agent-memory-full-example/agents.py:201-203), and the LoCoMo harness prints valid_at alone as event_time (benchmarks/locomo/evaluation.py:200). No as-of filter is issued anywhere in the tree.

The one server in the tree collapses the axes. The Community Edition's graphiti.Fact receives all four timestamps, and its public Fact keeps one: ExtractCreatedAt() substitutes valid_at for created_at, and invalid_at and expired_at are dropped (legacy/src/lib/graphiti/service_ce.go:27-42, legacy/src/store/memory_ce.go:44-54, legacy/src/models/fact_common.go:9-14). Graphiti's search at v0.3.21, the last tag of the line the compose file's zepai/graphiti:0.3 image names, returns invalidated edges unfiltered, so a Community Edition client receives a superseded fact dated like a current one. The open engine that does track both axes is Graphiti, which carries the mark.

6. Retrieval Mechanics

The surface is graph.search, assembled parameter for parameter in mcp/zep-mcp-server/internal/handlers/search.go:15 and exercised in the benchmark: a query string, a scope of edges, nodes or episodes, a limit, and a reranker chosen from rrf, mmr, node_distance, episode_mentions and cross_encoder. Filters are min_fact_rating, node_labels, edge_types, and center_node_uuid for node-distance reranking; mmr_lambda tunes diversity. The handler defaults scope to edges and limit to 10 and enforces no ceiling — the 50-result maximum that docs/TOOLS.md records is the service's, checked on the other side.

zep-ingest and the MCP server inject nothing; the integration packages do. ADK's ZepContextTool persists the user's message with thread.add_messages(return_context=True) and appends the returned context block to the system instruction before every model call (integrations/adk/python/src/zep_adk/context_tool.py:309, :385, :408). Pydantic AI's history processor, LiveKit's on_user_turn_completed and Microsoft Agent Framework's before_run do the same. Mastra and LangGraph inject through an input processor and a pre_model_hook and persist in a separate step, and AG2 exposes the call for the application to make each turn. On those paths retrieval happens without the model asking.

The benchmark's pattern is the one to copy for a hand-built read — two searches in parallel, edges and nodes, with independent limits and independent rerankers, assembled into <FACTS> and <ENTITIES> blocks.

The observable failure mode is under-recall at small budgets, and the repository measures it rather than guessing. See section 10.

The unobservable ones stay unobservable. Whether two aliases merged into one node, whether an edge was invalidated correctly, whether the cross-encoder saw the right candidate set — none of it is inspectable from a client. The zep_graph_inspect.py script in the eval harness exists because reading your own graph back is otherwise awkward.

7. Write Mechanics

The write path is asynchronous end to end, and the repository is unusually explicit about the two separate lags this creates.

Pipeline.run() is documented as "Submit the transformed stream and return immediately." The docstring instructs the caller to bind the result before waiting, so the resume handles survive a timeout:

result = pipeline.run(client, graph_id="company_kb")
result.wait()

That is lag one: the extraction work is queued and wait() blocks on it. It polls one handle per submission path — batch.get on the last batch, the processed flag of the last-submitted graph.add episode, the last message UUID of each thread, and every task id for nodes and triples. When a single queue's tail reports processed it marks everything submitted before it processed too (ingestion/src/zep_ingest/result.py:210-218), on the documented premise that plain graph.add episodes are processed in submission order. The default deadline is 60 seconds per submitted item with a 120-second floor (:82-94). Until 0.3.0 the sequential thread path looked for a task_id that thread.add_messages does not return, so every such backfill was counted as untracked and wait() refused to wait on it. Lag two is the one that matters, and verify.py states it:

Ingestion is asynchronous end to end: even after IngestResult.wait() reports success, just-written facts take a few more seconds to become searchable.

search_when_ready polls graph.search every 5 seconds for up to 120 seconds, returning the first response with any hits in context, edges, nodes, episodes, observations or thread_summaries, and returning the final empty response rather than raising, because "nothing matched" is a legitimate answer. A read-your-writes window measured in seconds, with a documented 120-second worst case, is a real constraint on any agent that writes a fact and then reasons about it in the same turn.

zep-ingest is a bulk backfill tool and sits on no agent's turn. The per-turn write path is thread.add_messages, capped at 30 messages per call, which most of the integration hooks in section 6 call before every model request. With return_context=True the context comes back in the same round trip, so the turn waits on one HTTP call per model request.

Deduplication, consolidation and conflict handling are the vendor's. The client contributes exactly one pre-ingestion normalization, the alias canonicalizer, and one explicit path around extraction, ingest_fact_triples. No code in this tree re-reads or rewrites the store in the background. Whether a pass runs on the hosted side is not knowable here; in the Community Edition that work belongs to the Graphiti service.

On the read side the benchmark measures the injection budget directly: at the default-ish 20/20 setting the assembled context has a median of 1,378 tokens and a p95 of 1,451 — a tight, predictable distribution, and small enough that the prompt-prefix cache question turns on where the caller places it rather than on its size.

8. Agent Integration

Thirteen integration packages, framework-first then language, each built and released independently: integrations/<framework>/<language>/. Python covers Google ADK, Microsoft Agent Framework, AutoGen, AG2, CrewAI, LangGraph, LiveKit, Pydantic AI and Strands; TypeScript covers ADK, Mastra and the Vercel AI SDK; Go covers ADK. That breadth is the product's real distribution strategy, and it is maintained as thirteen release trains rather than one adapter layer.

The MCP surface is thirteen tools and every one of them is a read, which is checkable against internal/server/server.go:67 rather than against a document: search_graph, get_user_context, get_user, list_threads, get_user_nodes, get_user_edges, get_episodes, get_thread_messages, get_node, get_edge, get_episode, get_node_edges, get_episode_mentions. Each handler bottoms out in exactly one SDK call and all thirteen are getters — Graph.Search, Graph.Node.GetByUserID, Graph.Episode.Get, Thread.GetUserContext and so on. The wrapped client (pkg/zep/client.go) embeds the whole zep-go/v3 client, so Graph.Add and the delete endpoints are one line away and no tool reaches them. There is no add_memory and no delete_memory, so under MCP the agent is a consumer of memory that something else wrote.

Scope is asymmetric across those thirteen. Six take a user_id (search_graph, get_user, list_threads, get_user_nodes, get_user_edges, get_episodes) and two take a thread_id (get_user_context, get_thread_messages). The remaining five are addressed by bare UUID — get_node, get_edge, get_episode, get_node_edges, get_episode_mentions (internal/handlers/types.go:61-84) — so an agent holding a UUID out of one user's search result can fetch that object without naming a user. Whether the hosted API refuses a cross-user fetch on the API key is not observable from here; the client does not attempt the check, which is the opposite of the invariant Destination enforces in zep-ingest. The eight keyed tools take their user_id or thread_id as an argument the model fills (internal/handlers/types.go:7-8, search.go:26-28), so on the MCP surface the boundary is the API key, not the user.

Agency over memory depends on the surface. Under MCP the model only reads. The integration packages hand it a write verb: AG2 registers add_memory and add_graph_data beside its search tools (integrations/ag2/python/src/zep_ag2/tools.py:782-793), and AutoGen, CrewAI and Mastra each ship a model-callable write tool. In all four the user or graph is bound when the tool is built or resolved from the runtime, never taken from the model's arguments, so the model chooses what to write and not where. The vendor still extracts, so a model's write is an episode, not a fact.

9. Reliability, Safety, and Trust

The alias canonicalizer is the safety mechanism worth the trip. Zep resolves entities by the names it sees in text, so "PROTOTYPE-202" and "ROBOT-202" do not merge, and the recommended fix is to rewrite aliases before ingestion. That fix is a text-substitution pass over a corpus about to become a permanent knowledge graph, which is a data-corruption engine if it is naive. It is not naive:

• Aliases shorter than three characters, or matching a built-in 146-word DEFAULT_RISKY_WORDS set of common English words and word-like given names (will, mark, bill, art, page, chase, hope, may, dot), are rejected at construction with an error explaining that "sentence-start capitalization defeats case-sensitive matching for word-like aliases".
• URLs and backtick code spans are matched first in the same scan and passed through untouched.
• Existing canonical mentions are protected, longest-literal-first, so an alias containing its canonical still wins.
• Word boundaries use (?<!\w)/(?!\w) rather than \b, with the comment that \b would make aliases starting or ending in punctuation — .NET, C++ — silently never match.
• A term declared as both an alias and a canonical name raises, rather than silently killing the alias.
• Per-alias replacement counts surface in preview() warnings, so a runaway alias is visible before any API call.

That is six distinct failure modes anticipated in one 227-line file, in a transform most teams would write as a str.replace loop, and it treats pre-ingestion normalization as a hazard rather than a convenience.

Against that: nothing on the write path defends the graph against prompt-injected false memory. Anything a loader reads — a Slack message, an inbound .eml, a document — is submitted as an episode and becomes graph content. The one LLM step in zep-ingest guards only its own prompt: the optional LLMContextualizer tells the model the document is data, strips its tag vocabulary from input and output, and caps the output length (transforms/contextualizer.py:8-13, :78-79, :91). The README's security notes tell the operator the rest, that graph content reaches agents' prompts and should be sanitized upstream (ingestion/README.md:581-588).

The Slack loader defaults to public channels only and reports unselected private channels and DMs in warnings, which is a privacy boundary rather than a trust one. min_fact_rating filters at read time on a score assigned by the vendor, so a caller can decline low-rated facts but cannot mark one rejected.

Provenance is a genuine strength: structured source_type plus source-specific keys on every episode, and IngestResult collects node_uuids, edge_uuids and task_ids so a run's outputs can be tied back to its inputs. Errors are collected per item as AddError(index, item_count, error) rather than aborting the run, and untracked_items counts submissions that came back without a completion id — a small, honest counter for "we sent this and cannot prove what happened to it".

The multi-tenancy boundary is the vendor's. Destination enforces the client side of it and nothing more.

10. Tests, Evals, and Benchmarks

ingestion/tests/ holds 27 test modules covering loaders, transforms, submitters, validation, limits, threads, triples and the example ontology — a well-tested client library. benchmarks/locomo/tests/ adds tests for config, persistence and common utilities. The MCP server's twelve Go tests are thinner and cover no handler: config_test.go checks that a missing API key fails and that the server-name, version and log-level defaults load, types_test.go checks struct-tag and zero-value behaviour on the input types, and helpers_test.go covers transform's JSON formatting and optional-parameter readers. Nothing exercises a tool end to end, and there are no tests for the Community Edition in this tree.

The benchmark harness is the contribution. benchmarks/locomo/ runs LoCoMo-10 (1,540 questions per run, pulled from snap-research/locomo) and grades each answer on two independent axes (evaluation.py:154-171). The completeness judge runs concurrently with answer generation, so it sees the retrieved context and the gold answer and never the model's answer; the accuracy grader runs after:

1. Accuracy — a generous CORRECT/WRONG grader against the gold answer (prompts.py, GRADER_PROMPT).
2. Context completeness — COMPLETE, PARTIAL or INSUFFICIENT, judging whether the retrieved context contained what was needed, with missing_elements and present_elements recorded per question (evaluation.py:251).

That second axis is the methodological move, and it is rare. It measures the memory layer directly instead of measuring a pipeline and attributing the score to memory. The derived metric accuracy_with_complete_context then answers the question the first axis cannot: given that retrieval worked, how often is the answer right?

Five experiments sweep the retrieval budget, ten runs each, all committed:

All five runs use gpt-4o-mini at temperature 0 for both answering and grading, with cross_encoder reranking on both scopes.

Three things fall out of that table, and they are worth more than the headline number.

Accuracy given a complete context is flat. It sits at 0.92 ± 0.01 across a 5.8x swing in retrieved tokens. Every point of end-to-end accuracy the sweep buys comes from completeness rising — 0.574 to 0.765 — and none of it comes from the answering model doing better with more material. The memory layer and the reader are cleanly separated by this design, and the separation holds.

The residual 8% is not a memory problem. When the grader confirms the context contained everything needed, the pipeline still gets one answer in twelve wrong. No amount of retrieval improvement touches that ceiling; it is the reader and the grader. A team optimizing memory against end-to-end LoCoMo accuracy is, past a point, optimizing against someone else's error bar.

The budget saturates before the top of the sweep. Going from 20/20 to 30/30 costs 45% more context tokens for 0.26 points of accuracy — a gap smaller than the sum of the two runs' standard deviations, and accompanied by accuracy-given-complete falling slightly, 0.917 to 0.915, the shape a mild distraction effect makes. The knee is at 20/20.

The second harness, zep-eval-harness/, commits one complete run chain and it is a much smaller artifact than the LoCoMo sweep. runs/chunk_sets/1_20260331T222430/ holds ten chunks from four reference documents at chunk_size: 500. runs/users/1_20260331T222436/manifest.json and runs/documents/1_20260331T222500/manifest.json record two synthetic users, four conversations, the custom entity and edge types used, and the resulting Zep episode UUIDs. runs/evaluations/1_20260331T222821/results.json grades four questions with gemini-2.5-flash-lite as both answerer and judge — 4 of 4 COMPLETE, 3 of 4 correct, accuracy_when_complete 75%, median prompt 3,144 tokens, median search 337 ms. Four questions across two users, all in one basic_facts category, is a smoke test rather than a measurement, and its timing and token statistics have a sample size of four behind them.

Two things about it are worth more than the numbers. Each run directory carries a *_config_snapshot/ — the actual Python config modules copied verbatim beside the results — so a committed result names the ontology, custom instructions and prompts it was produced under instead of pointing at a config that has since moved. And every test case in data/test_cases/ carries a needles array giving the source file, line and excerpt where the answer was stated, so a failure is attributable to a specific ingested sentence. The same two-axis grading as the LoCoMo harness is used — zep_evaluate.py runs an accuracy judge and a COMPLETE/PARTIAL/INSUFFICIENT completeness judge — so the design idea in section 11 is applied twice in this tree rather than once.

benchmarks/longmemeval/ is the experiment code for the Zep paper, arXiv:2501.13956 (Rasmussen et al., 20 January 2025), which reports 94.8% against MemGPT's 93.4% on DMR and LongMemEval accuracy gains of up to 18.5% over a baseline. Its README names two notebooks, one per experiment. The tree holds one, zep_memgpt_eval.ipynb, the DMR run over MemGPT's MSC set with a full-context and a summary baseline, committed with no cell outputs. Beside it are a LongMemEval script, a two-file test harness and a dataset statistics summary dated 2025-07-19, and a .gitignore excluding *.json, where the README says results are saved. No row behind either headline number is committed. No committed result compares Zep to anything else: the LoCoMo sweep is Zep against its own configuration, the right experiment for choosing a limit and the wrong one for choosing a vendor.

The tests a reader would want before trusting this and cannot find: anything asserting that invalid_at actually stops a superseded fact from being retrieved, anything measuring how long the post-wait() indexing lag really is, and any negative case asserting that deleted or scoped-out material must not appear in a search result. The fifty LoCoMo runs cannot speak to the first: the harness prints each fact with valid_at alone (evaluation.py:200), so the reader never sees whether a fact was invalidated. The integration suites assert request shapes against a mocked SDK. All thirteen packages add a live suite that skips unless ZEP_API_KEY is set (integrations/adk/go/live_test.go:29-31); the Python and TypeScript CI jobs pass that secret in (.github/workflows/test-integrations.yml:82-85), and a local run without it is green having reached no service.

11. For Your Own Build

Steal

Grade retrieval sufficiency separately from answer correctness, with a second judge on the same question. COMPLETE / PARTIAL / INSUFFICIENT over the retrieved context, plus missing_elements, turns an opaque end-to-end score into an attribution. It costs one extra LLM call per eval question and it is the cheapest way to stop shipping retrieval changes that were really prompt changes.

Report accuracy conditioned on complete context. It is one extra line in an aggregation and it exposes your own ceiling. If that number is flat across your retrieval sweep, more retrieval is not your problem.

Run the sweep ten times and publish σ. The gaps in the table above are two to nine standard deviations at the bottom and less than one at the top. Without the repeats, the 20/20-to-30/30 step reads as a real 0.26-point improvement.

Validate a corpus against the service's documented limits before the first API call. preview() runs the entire loader-and-transform chain with zero network calls and returns warnings; preview(limit=None) does it exhaustively. Every limit is re-checked client-side so failures name a field instead of returning an HTTP 400 halfway through a 50,000-item batch.

Treat pre-ingestion text rewriting as a hazard with a deny-list. If you canonicalize entity names before they reach an extractor, ship the risky-words guard, the URL and code-span protection, the punctuation-safe boundaries and the per-alias replacement counts. All four exist because all four failures happen.

Count the episodes with no event timestamp, and say what it will cost. A loader that silently dates history to import time produces a graph that is internally consistent and historically wrong.

Avoid

Do not assume a write is readable when the write API says it succeeded. Two lags, not one: task completion, then indexing. If your agent writes a fact and reasons about it in the same turn, you need the poll loop or you need a read-your-writes guarantee in writing.

Do not let a scope be a tag. Destination raising unless exactly one of graph_id and user_id is set is a one-line invariant that makes leaking across users a construction error rather than a filter someone forgot.

Do not read a rating as a status. A float on a fact tells you how confident something was; it does not tell you the fact was checked, and it cannot stop the extractor from re-deriving something you rejected.

Do not benchmark a memory system only against itself. Five well-run experiments on one vendor's own configuration establish where its knee is and nothing about whether the knee is in a good place.

Fit

Adopt this if memory is not the product you are building and you would rather buy the hard parts — entity resolution, temporal invalidation, reranking — than maintain them. The integration breadth is real, thirteen packages across three languages, most of which wire memory into the turn without the model asking. The client library validates every documented limit before the first call and leaves resumable handles when a wait times out.

Walk away if any of three things is true. If you need to inspect or repair the store, you cannot: the mechanism is on the other side of an API and this repository is the map, not the territory. If you need offline or air-gapped operation, there is no local mode and the self-hostable edition is deprecated. And if you want an agent that curates its own memory, the MCP server's thirteen tools are all reads, and the integration tools that do let a model write still send that write through the vendor's extractor as an episode.

The reader who gets the most from this repository may be one who never adopts Zep at all, and takes benchmarks/locomo/ to point at their own system instead.

12. Open Questions

• How long is the post-wait() indexing lag in practice? The client polls at a 5-second interval with a 120-second ceiling, which brackets it but does not measure it.
• Does the hosted extractor re-derive a fact whose validity interval was closed? Nothing in the client can express "never assert this again", so a re-extraction loop is possible and undetectable from here.
• What does min_fact_rating actually score, and on what scale? It appears only as a filter parameter.
• When a client submits a triple with its own created_at, does the hosted edge keep that value or stamp its own clock? The returned created_at is documented as the edge's creation time and the submitted one as the message's timestamp, and nothing in the tree reads one back against the other.
• Why is the LoCoMo sweep's 30/30 retrieval median (0.189 s) lower than 20/20's (0.241 s)? Most likely load rather than budget, but the harness does not record enough to say.
• Does the hosted API reject a get_node or get_edge for a UUID belonging to another user under the same API key? The MCP server's five UUID-addressed getters pass no scope key, so the answer lives entirely on the server side.

Appendix: File Index

Write path

• ingestion/src/zep_ingest/pipeline.py — Pipeline, preview(), run(), _validated_replay(), _MissingTimestampCounter.
• ingestion/src/zep_ingest/submitters/batch.py, sequential.py — submission and retry.
• ingestion/src/zep_ingest/triples.py — FactTriple, ingest_fact_triples().
• ingestion/src/zep_ingest/transforms/limits.py, chunker.py, _splitting.py — size guards.
• ingestion/src/zep_ingest/transforms/canonicalizer.py — AliasCanonicalizer, DEFAULT_RISKY_WORDS.
• ingestion/src/zep_ingest/transforms/contextualizer.py — optional LLM chunk contextualization.

Data model

• ingestion/src/zep_ingest/types.py — Episode, Destination, documented API limits.
• ingestion/src/zep_ingest/result.py — IngestResult, AddError, untracked_items.
• ontology/default_ontology.py — the extractor's entity types.

Loaders

• ingestion/src/zep_ingest/loaders/ — slack.py, email.py, transcript.py, text.py, json_records.py.

Retrieval

• ingestion/src/zep_ingest/verify.py — search_when_ready().
• benchmarks/locomo/evaluation.py — two-scope concurrent search, both graders.
• benchmarks/locomo/prompts.py — CONTEXT_TEMPLATE, RESPONSE_PROMPT, GRADER_PROMPT.

MCP

• mcp/zep-mcp-server/internal/server/tools.go, server.go — the thirteen tool declarations and registerTools().
• mcp/zep-mcp-server/internal/handlers/ — one file per tool; types.go carries the input schemas and shows which tools take a scope key.
• mcp/zep-mcp-server/pkg/zep/client.go, internal/config/config.go, internal/transform/ — SDK wrapper, config, parameter and result marshalling.
• mcp/zep-mcp-server/docs/TOOLS.md — the tool documentation, including the service-side 50-result limit ceiling.

Integrations

• integrations/adk/python/src/zep_adk/context_tool.py — ZepContextTool.process_llm_request, persist-and-inject before every model call.
• integrations/pydantic-ai/python/src/zep_pydantic_ai/history_processor.py, integrations/livekit/python/src/zep_livekit/agent.py, integrations/ms-agent-framework/python/src/zep_ms_agent_framework/context_provider.py, integrations/mastra/typescript/src/processors.ts, integrations/langgraph/python/src/zep_langgraph/hooks.py — the other injection hooks.
• integrations/ag2/python/src/zep_ag2/tools.py, integrations/autogen/python/src/zep_autogen/tools.py, integrations/crewai/python/src/zep_crewai/tools.py, integrations/mastra/typescript/src/remember-tool.ts — model-callable write tools with a construction-bound scope.
• integrations/autogen/python/src/zep_autogen/graph_memory.py — carries all four edge timestamps into memory metadata; clear() deletes the whole graph.

Evals

• benchmarks/locomo/experiments/ — five experiments, ten runs each, with configs and summaries.
• benchmarks/locomo/benchmark.py, persistence.py, config.py, ontology.py.
• benchmarks/longmemeval/ — the experiment code for arXiv:2501.13956: zep_memgpt_eval.ipynb (DMR, no outputs), zep_longmem_eval.py, Zep Test Harness/, session_analysis_summary.txt; no result rows.
• zep-eval-harness/zep_evaluate.py — the second two-axis grader; data/test_cases/ with per-answer needles.
• zep-eval-harness/runs/ — one committed chain: chunk set, user and document manifests, and a four-question evaluations/1_20260331T222821/results.json, each with its config snapshot.

Legacy Community Edition

• legacy/src/lib/graphiti/service_ce.go — the HTTP client for the Graphiti service: Fact with four timestamps, ExtractCreatedAt(), DeleteFact, DeleteGroup.
• legacy/src/store/memory_ce.go — _get and _searchSessions, where the four timestamps become one and an absent group list goes out as null.
• legacy/src/models/fact_common.go, search_common.go — the public Fact and the unenforced "Required on Community Edition" comment.
• legacy/src/store/db_utils_ce.go, userstore_ce.go, sessionstore_ce.go — purgeDeletedResources() and its two callers.
• legacy/src/store/purge_common.go, purge_ce.go — purgeDeleted() and the no-op tableCleanup(), reachable only through the uncalled memoryStore.PurgeDeleted.
• legacy/docker-compose.ce.yaml, Dockerfile.ce — the deprecated self-host path: the server, pgvector Postgres, zepai/graphiti:0.3 and Neo4j.

Engine read outside this tree

• Graphiti v0.3.21 (6536401c…), the last tag of the 0.3 line: server/graph_service/routers/retrieve.py and dto/retrieve.py (optional group_ids), graphiti_core/search/search_utils.py:202 (a null group list matches every group; no filter on invalid_at or expired_at).
• zep-cloud 3.28.0 (7f514a61…), the SDK release current on 11 September 2026: src/zep_cloud/types/entity_edge.py and add_fact_triple in src/zep_cloud/graph/client.py.

Recorded searches

Run at the tree root of the pinned checkout unless a path is given.

rg -n -i 'tombstone|suppress|blocklist|denylist|reject(ed)?_value|never again' --glob '!*.ipynb' .
rg -n -i '\b(verified|unverified|candidate|pending_review|approved|quarantin|provisional|disputed|retracted|trust_state)\b' --glob '!*.ipynb' .
rg -n 'expired_at|ExpiredAt|expiredAt' --glob '!*.ipynb' .
rg -n 'edge\.update|Edge\.Update|edge\.delete|graph\.edge\.' --glob '!*.ipynb' .
rg -n '\.delete\(|\.Delete\(|delete_' integrations ingestion/src mcp zep-eval-harness benchmarks examples
rg -n 'PurgeDeleted\(|purgeDeleted\(' legacy/src
rg -n 'lib/search' legacy/src
rg -n '//go:build|// \+build' legacy/src
rg -n -i 'prompt.?injection|untrusted|sanitiz' --glob '!*.ipynb' .
rg -n -i 'append_instructions|pre_model_hook|before_run|on_user_turn_completed|return_context' integrations
rg -n -i "not in (result|context|response|out|text|content|prompt|facts)|not\.toContain|assertNotIn|!strings\.Contains" --glob '**/test*/**' --glob '*_test.go' --glob '*.test.ts' .
rg -n 'ZEP_API_KEY' --glob 'test_integration.py' --glob 'live.test.ts' --glob 'live_test.go' integrations
rg -n -i 'arxiv|bibtex|@article|@misc|citation|doi\.org' --glob '!*.ipynb' --glob '!**/experiments/**' .
git ls-files legacy | grep -E '_test\.go$'

lib/search has no importer, so the Community Edition's MMR and RRF helpers rank nothing; no Go file carries a build constraint, so the _ce suffix is a naming convention rather than a build tag. The paper search returns nothing: benchmarks/longmemeval/README.md names the paper by title only.

History

2026-09-30 — 495bf728… — same pin; upstream had not moved. Screened again: one auto-run surface, none inside the cooldown, 14 build-time execution points, 33 unpinned surfaces across 100 files; nothing installed, built or run. bitemporal withdrawn (section 5): its record read FactTriple.created_at as record time, but that field is the caller's event time, no code here tracks either axis, and the Community Edition collapses the four timestamps into one. The other corrections were absence claims over committed code. The integrations inject context before model calls and four hand the model a write tool (section 8). The Community Edition runs on Graphiti and purges through purgeDeletedResources(), and has no build tags. The LoCoMo accuracy grader runs after the answer. benchmarks/longmemeval/ is the code for arXiv:2501.13956, without results. Census added.

2026-09-15 — 495bf728… — five commits on, to 2026-09-11. Screened at the new pin before reading: one auto-run surface, 23 dependency surfaces inside the cooldown, 14 build-time execution points and 33 unpinned surfaces across 100 files; nothing was installed or run. zep-ingest went to 0.3.0. Batches now roll over at 10,000 items by default instead of filling to the 50,000 cap. Every file or loader bound for one graph is submitted before anything waits, and wait() polls the tail of each submission path, inferring that the queue in front of a processed tail has drained, with a deadline of 60 seconds per item and a 120-second floor. Sequential thread backfills now poll the last message UUID per thread: at the previous pin that path looked for a task_id which thread.add_messages never returns, so those backfills were all counted untracked and wait() refused them. A ConcatLoader, multi-path sources and IngestResult.combine() were added, along with a production smoke script that needs a live key. The other two commits repair the Python, TypeScript and Go examples for SDK v3. Nothing in benchmarks/, mcp/, integrations/ or legacy/ changed. Both marks were re-checked and stand, now with evidence records. bitemporal rests on the triple contract and on readers that render the closed interval; no as-of filter is issued in this tree. scope_enforced rests on Destination, which search_when_ready builds before every read.

2026-08-31 — be263ee2… — same pin, two absence claims corrected, both wrong in the direction of asserting something missing that is committed. The MCP server was described as documented here and implemented elsewhere; mcp/zep-mcp-server/ is a complete Go module — cmd/server/main.go, internal/server/{server,tools}.go, thirteen handler files, pkg/zep/client.go, go.mod, Makefile, Dockerfile — and registerTools() registers exactly the thirteen tool names the report listed. The read-only finding survives on code rather than on docs/TOOLS.md; the scope asymmetry across the five UUID-addressed getters, the twelve Go tests and the nanosecond shutdown timeout at server.go:160 were not previously reported, and the open question asking where the server is implemented is answered from the tree. zep-eval-harness/runs/ was described as holding only a .gitkeep; it holds one committed chain — chunk set, user and document manifests, and a four-question evaluation under gemini-2.5-flash-lite — thin but present, and section 10 describes it instead. DEFAULT_RISKY_WORDS holds 146 words, not 150. No capability mark moved; scope_enforced was re-checked in both directions and stands on Destination in zep-ingest, which section 5 states explicitly.

2026-08-13 — be263ee2… — first reading. Screened before opening: one auto-run surface, 47 dependency surfaces inside the seven-day cooldown, 14 build-time execution points, 29 unpinned surfaces across 93 files. Nothing was installed and nothing was run; the committed benchmark artifacts were read from git, not reproduced.