The most interesting opportunity in agents right now is not a smarter model. It’s the unglamorous plumbing that lets a merely good model work on one task for thirty hours without forgetting why it started. Claude Sonnet 4.5’s internal tests reported 30+ hours of autonomous coding — one run produced an 11,000-line application. But no single context window is thirty hours long. Everything interesting happens in the gap between sessions, and the gap is bridged by exactly one thing: memory.
Here’s the mental model. An agent’s context window is a whiteboard: everything it knows about the task is written there, and when the session ends, the whiteboard is erased. For a ten-minute task, fine. For a multi-day task, your agent is a software project staffed by engineers working in shifts — and each new engineer walks in with no memory of the previous shift. Agent memory is everything you build so the next shift doesn’t start from zero. Done right, it’s the difference between thirty hours of progress and the same thirty minutes repeated sixty times.
01Session-scale memory: Anthropic’s long-running harness
The cleanest existence proof is Anthropic’s engineering write-up on harnesses for long-running agents: build a large web app — far too big for one context window — across many discrete sessions. The naive approach (re-prompt “continue” each session) fails in familiar, almost human ways: the agent bites off more than it can chew, declares victory prematurely, or leaves the code broken at the exact moment its memory is wiped.
Their fix has two roles, and both are memory infrastructure:
- An initializer agent runs once. It doesn’t build features — it builds the memory substrate: an
init.shto reproduce the environment, a progress file logging what’s been done, and an initial git commit so there’s durable, inspectable history from minute one. - A coding agent runs every session after: make incremental progress — roughly one feature at a time — verify it actually works, then write structured artifacts for the next session before the context dies.
DONE: channel list, message pane, optimistic send
IN PROGRESS: file uploads — S3 presign works, drag-drop untested
KNOWN BROKEN: nothing (all tests green as of commit 4f2a91c)
NEXT: finish upload UI, then start threads
RULE: never end a session with the app in a broken state
The key insight is that a fresh agent must quickly understand the state of work when it wakes up with an empty context. The progress file gives the narrative — what were we doing, what’s left, what’s known-broken. The git log gives the ground truth. Neither is fancy. Both are load-bearing.
The part people miss
The harness treats memory writes as part of the task, not an afterthought. The agent’s job isn’t “build the app” — it’s “advance the app one verified step and leave the campsite better documented than you found it.” That’s the single highest-leverage change you can make to a long-running agent today.
› Go deeper: why plain files beat a vector database here
Notice what the memory store is: the filesystem. Plain text, git, shell scripts. No embeddings, no retrieval pipeline. For long-running task memory, an append-only log plus version control beats semantic search, because what the next session needs isn’t “similar documents” — it’s the exact, ordered truth about the current state of the work. Vector stores earn their keep when you need similarity search over a large corpus; they are not the default. Bonus: you can read, diff, and debug a text file. Try that with an embedding.
02Lifetime-scale memory: Hermes
Where Anthropic’s harness bridges sessions within one project, Nous Research’s Hermes Agent bridges everything the agent has ever done. It’s pitched as “the agent that grows with you,” and its memory design is the most deliberate I’ve seen deployed in the open:
| Mechanism | What it does | Why it matters |
|---|---|---|
| MEMORY.md / USER.md | Durable facts, lessons learned, and a user profile live as Markdown in the workspace | Same filesystem-as-memory bet as Anthropic — human-readable, diffable, zero infrastructure |
| Periodic nudge | At intervals, a system-level prompt asks the agent to review recent activity and decide what’s worth persisting | The agent curates its own memory — the store stays small enough to actually load later |
| Skills | After a complex task, the agent writes a structured doc: the procedure, known pitfalls, and verification steps | Next similar task loads the skill instead of reasoning from scratch — and updates it if a better approach is found |
| Progressive disclosure | Skills load in stages — summary first, full detail only if the task needs it | Memory doesn’t devour the very context window it exists to protect |
| Scheduled self-review | Roughly every 15 tasks, the agent evaluates its own recent successes and failures | Conclusions fold back into memory — the loop that upgrades “persistent” to self-correcting |
The skills loop is the part worth staring at. A persistent agent remembers what happened. A self-correcting agent remembers what to do differently — and Hermes makes that a first-class artifact with its own file format, refinement step, and loading policy. That’s deployment thinking, not demo thinking.
03The landscape: eight systems, four bets
Anthropic’s harness and Hermes both bet on files. That is not the only bet on the table, and it is worth knowing what the alternatives actually do before you pick one. Here is what is genuinely shipping, with the one detail about each that changed how I think about it:
| System | Architecture | Where memory lives | The detail worth knowing |
|---|---|---|---|
| Claude memory tool | Files the agent writes itself | Your infrastructure — execution is client-side | Six commands, and /memories is a virtual prefix you map onto real storage. Anthropic ships the protocol; you own the disk. |
| Claude Code CLAUDE.md | Markdown plus an index file | Machine-local, per git repo | Only the first 200 lines or 25 KB of MEMORY.md load per session; the rest is fetched on demand. Memory has a budget whether you plan one or not. |
| ChatGPT memory | Saved memories + chat-history reference | OpenAI’s side, per account | Two separate channels with different semantics; history referencing arrived April 2025. |
| MemGPT / Letta | OS-style tiering; the agent edits its own memory via function calls | Letta server + relational and vector stores | The paper’s framing still shapes the field: context windows as a constrained resource, retrieval as a page fault. |
| Mem0 | LLM extraction into a vector store; optional graph variant | Managed cloud or self-hosted | Its 2026 rewrite is single-pass, add-only — no update, no delete. The previous version’s entire selling point was an LLM deciding when to update and delete. They abandoned it. |
| Zep / Graphiti | Bi-temporal knowledge graph | Neo4j, FalkorDB, Neptune, Kuzu | Edges carry validity windows, so facts can be invalidated rather than overwritten — the store knows what used to be true. |
| LangGraph + LangMem | Thread-scoped state snapshots plus a cross-thread key-value store | Postgres, SQLite, Redis | Checkpoints are the whole graph state per step, which buys resumption and time travel almost for free. |
| A-MEM | Zettelkasten: notes, links, and evolution | Vector store | Writing a new memory can retroactively rewrite older ones — structure emerges instead of being designed. |
Squint and there are only four bets here. Files (Anthropic, Hermes, Claude Code): human-readable, diffable, trivially auditable, and dumb about similarity. Vectors (Mem0, A-MEM): great at “what is related to this,” bad at “what is still true.” Graphs (Zep): the only ones that model time properly, at the cost of real infrastructure. Snapshots (LangGraph): not recall at all, but resumption — a different problem that people keep confusing with this one.
The convergence I did not expect: two of the most sophisticated systems on that list moved toward the boring answer. Mem0 replaced its clever update-and-delete engine with append-only accumulation. Anthropic’s memory tool ships six file operations and makes you supply the storage. When the teams with the most benchmark pressure simplify in the same direction, that is a signal.
The best line in any memory documentation
Whenever the memory tool is present, Anthropic’s API silently prepends a protocol to the system prompt. It ends like this:
ASSUME INTERRUPTION: Your context window might be reset at any moment, so you risk losing any progress that is not recorded in your memory directory.
That is the whole discipline in one sentence, and it is aimed at the model rather than at you. An agent that believes it might be killed at any instant writes things down. One that assumes it will finish does not.
04The benchmark numbers are softer than they look
Every vendor on that list has a chart showing it winning. Before you pick a system from those charts, here is what happened when two of them benchmarked each other.
Mem0’s 2025 paper reported beating a set of baselines on LOCOMO, the long-conversation memory benchmark everyone quotes. Zep responded with a post arguing both that LOCOMO is too easy to be meaningful — its conversations fit comfortably inside a modern context window, so it barely tests memory at all — and that Mem0 had benchmarked a misconfigured version of Zep. Mem0’s CTO then filed an issue on Zep’s own repository alleging Zep’s headline number was computed with a numerator and denominator that counted different question sets.
Where it landed is the part worth internalizing. Zep acknowledged a calculation error and revised its score from 84% down to 75.14%. Mem0’s independent re-run of that same system, averaged over ten runs, produced 58.44%. Nobody reconciled those two figures.
Same system. Same dataset. A twenty-five point spread.
84%, 75.14%, 58.44% — and the entire gap comes from harness choices: which question categories count, what system prompt is used, and how many runs get averaged. Not one point of that spread is about memory architecture. If you are choosing infrastructure from a benchmark table, you are choosing someone’s harness configuration.
It gets worse for the benchmark itself. An independent audit of LOCOMO’s ground truth reports 99 score-corrupting errors across 1,540 questions — hallucinated facts in the answer key, wrong temporal arithmetic, answers keyed to details that appear only in annotator notes and never in the conversation the system actually ingests. It also found the LLM judge accepting a majority of deliberately wrong answers. The auditors put the honest ceiling on LOCOMO at roughly 93–94% rather than 100%. Worth stating plainly: they disclosed that they work on a competing memory product, so read the motivation with appropriate suspicion — but the audit names specific question IDs and is checkable, which is more than most benchmark criticism offers.
Set that against Mem0’s current advertised LOCOMO score of 92.5, which sits almost exactly on that estimated ceiling. Either that is an extraordinary result, or the metric has saturated into noise. Both readings should make you want your own eval.
Two more structural problems, because they change what these scores even mean:
- LOCOMO forbids saying “I don’t know.” Several hundred of its questions are adversarial by construction — they name a real person and a real topic, then ask about something that person never said. The correct answer is a refusal. The standard harness drops those questions and instructs the model never to abstain. The benchmark the memory industry quotes systematically excludes the ability to recognize you do not know something, which is the property I would most want from a memory system. LongMemEval does test abstention, which is a real argument for preferring it.
- The benchmarks changed underneath everyone. LongMemEval’s repository records a 2025 cleanup of its history sessions “to prevent interference on answer correctness,” shipped as a separate cleaned dataset. Scores published before and after are not comparable, and almost nobody states which version they ran.
None of this means the systems are bad or the researchers dishonest. It means the field is young enough that its measuring instruments are still being built, in public, by the same people being measured. The practical conclusion is unglamorous and I believe it completely: benchmark memory on your own traffic, with your own questions, and count the ones where the agent should have said “I don’t know.”
05The retrieval function, made playable
Every memory system above eventually faces the same question: the store holds thousands of items, the context window holds a handful — which ones come back? The most influential published answer is from Stanford and Google’s Generative Agents paper, and it is refreshingly simple. Score every memory on three axes and add them up:
- Recency — an exponential decay over hours since the memory was last accessed. The paper uses a decay factor of 0.995 per hour.
- Importance — a score the model itself assigns when the memory is written: is this a mundane observation or a core fact?
- Relevance — embedding similarity between the memory and the current query.
Each component is normalized to [0,1] and combined with a weight. That is the entire algorithm, and almost every production memory system is a variation on it — which means its failure modes are your failure modes. So here it is, running, with the weights exposed. Twelve memories from a coding agent’s life, three queries, and the sliders that decide what it recalls:
Drag recency to the top and relevance to zero: the agent loads “ran ls” and “opened the settings file” — perfect recall of the last five minutes, total amnesia about the rules it needs. That is not a strawman; it is what a naive “keep the last N turns” buffer does, and it is the single most common memory architecture in production. Now drag relevance alone to the top: every retrieved memory is on-topic and none of them know that the deploy broke six hours ago — the characteristic blind spot of a bare vector store. Only the mixture recovers the behaviour you actually want.
The paper and the code disagree — and the code is the interesting one
The paper describes equal weights and a 0.995 hourly decay. The released implementation does something else on three counts: the decay constant is 0.99, an additional weight vector [0.5, 3, 2] is applied on top — making importance six times heavier than recency — and recency decays over a memory’s rank in the list rather than elapsed time. Adjacent commented-out lines show [1, 1, 1] and [1, 2, 1] were tried first.
Press “as shipped in the code” above and watch what moves: the month-old rollback procedure climbs from fourth place to second, and the repeated import error — fresh, and arguably the most actionable thing in the stream — falls out of the top three entirely. This is worth more than the algorithm itself: the published version is the one everyone cites, and the tuned version is the one that actually ran. If you are building on this — and most memory systems are — you are probably implementing the paper and wondering why your results feel different.
› Go deeper: importance is the axis everyone drops, and it is the expensive one
Recency is free to compute and relevance comes with the embedding you already have. Importance requires a model call at write time — you are paying a token cost on every observation to ask “how much will this matter later?” The pragmatic shortcut most teams take is to skip it, which is precisely why so many agents drown in their own logs: without an importance signal, a memory store cannot distinguish a rule from a keystroke, and the only remaining ranking signals both favour noise.
Set the importance weight to zero in the widget and watch which durable rules fall out of the top three. The staging-database reset, the rollback procedure, the “user prefers tabs” instruction — the facts that were expensive to learn are exactly the ones with no recency and, often, weak lexical overlap with the question being asked.
06Self-correction is memory in a loop
Put the two systems side by side and a shared skeleton appears. Every serious long-running agent runs some version of this cycle:
verify → run tests / drive the UI / check the output for real
record → write what worked, what broke, and why, to durable storage
reload → the next session (or next task) starts by reading that record
Strip out record and reload and self-correction isn’t unlikely — it’s impossible. An agent with no durable memory of its failures re-attempts them with identical confidence every time. I’ve watched an agent “fix” the same import error across three sessions because nothing told it the fix had already been tried. Anthropic’s harness closes the loop at the session scale (progress file records it; the next session reloads it). Hermes closes it at the lifetime scale (the skill records it; every future similar task reloads it). Same mechanism, different half-lives.
You can see memory working in a trace. The ideal run loads a skill before acting and checkpoints after verifying (the purple spans); the memoryless run reasons freshly before every one of its six identical failures:
Memory, visible in the trace — the purple spans are the difference
The framing I’d push
Verification artifacts are memory. A passing test suite is the agent’s memory of what “working” means. A git commit is its memory of a known-good state it can retreat to. A skill’s pitfalls section is its memory of pain. When people say an agent “self-corrects,” what they’re describing is an agent that writes its mistakes down somewhere it is guaranteed to look.
07Watch memory win: a live run
Here’s the whole argument as a running system. Three agents attack the same long task in real time. Each works inside a context window that gets wiped when full (the white flash). The only difference between them is what survives the wipe.
40-step task · each step can fail and be retried · watch what each agent does when its whiteboard is erased.
Push the failure rate up and the gap explodes: the no-memory agent thrashes forever, the progress-file agent grinds it out, and the skills agent finishes in a fraction of the sessions because it stops repeating its own mistakes — self-correction compounding in real time. That’s the entire deployment argument in one picture.
08The deployment checklist
Distilling both systems into what I’d hold a design review against:
- Make the filesystem the substrate. Progress logs, memory files, skills — plain text under version control. Reach for vector stores only when you genuinely need similarity search, not as the default.
- Write memory inside the task loop. If persisting state is a separate “cleanup” step, it gets skipped exactly when things go wrong — which is when you need it most.
- Verify before you persist. Memory is only an asset if it’s true. An agent that records “feature X done” without checking it end-to-end has poisoned every future session.
- Never end a session in a broken state. The handoff is the most dangerous moment in a long-running agent’s life. Enforce clean state at the boundary — commit working code or roll back.
- Curate, don’t hoard. Hermes’s periodic nudge is the right shape: the agent decides what crosses the threshold of future usefulness. A memory file that grows without bound just relocates the context-window problem — that’s the append-everything trap from the forgetting post.
- Load progressively. Summaries first, detail on demand. Memory that costs 50k tokens to consult isn’t memory, it’s a second task.
- Give self-correction a home. A dedicated place for “what failed and what to do instead” that is always in the reload path. This is the piece most homegrown agents skip — and the piece that compounds.
09Where memory doesn’t save you
· Memory can’t rescue a task the model can’t do. If per-step reliability is too low, perfect handoffs just document the failure beautifully. Past a point you need a better model or a simpler decomposition — pacing and memory buy room, not miracles.
· False memories are worse than no memories. One unverified “DONE” in a progress file propagates to every future session — each one trusts it and builds on sand. This is why verify-before-persist is the non-negotiable rule, not a nice-to-have.
· Curation is a judgment call the agent will sometimes get wrong. Hermes’s nudge delegates “what’s worth keeping” to the model itself. Mostly that works; occasionally it discards the one detail that mattered. Keep raw logs (git history, transcripts) as the recovery path beneath the curated layer.
And one failure mode that is categorically different from the rest, because it is adversarial:
Memory is the only component that writes to its own future prompt
Everything else in an agent reads its context. Memory writes it — which makes a memory store the most valuable thing on the board for an attacker, and the effects persist across sessions rather than ending with the conversation.
This is documented, not theoretical. Security researcher Johann Rehberger demonstrated an indirect prompt injection that wrote a persistent instruction into ChatGPT’s long-term memory, causing subsequent conversations to be exfiltrated to an attacker — surviving across sessions because that is precisely what memory is for. A follow-up against Gemini used delayed tool invocation: a poisoned document plants instructions that lie dormant until the user later types something innocuous like “yes,” sidestepping the usual defense of disabling tools while untrusted content is being processed. Google rated the impact low on the grounds that it requires user cooperation, which is itself worth noticing — the researcher and the vendor disagreed about how much a persistent, self-triggering memory write is worth.
Anthropic’s own memory-tool documentation is refreshingly blunt about the same class of problem, warning that a path like /memories/../../secrets.env can escape the memory directory and that your implementation “must validate every path in every command.” When the tool hands the model a filesystem, the model’s mistakes and its attackers both get a filesystem.
The practical rules follow from the same place as verify-before-persist: treat everything the agent writes as untrusted input on the way back in, keep memory writes reviewable, and give memories an expiry so a poisoned one cannot outlive the incident that created it.
Related and less dramatic, but more common: context poisoning, where a wrong belief gets recorded and then reinforced. The clearest public example is in DeepMind’s own Gemini 2.5 report, describing an agent playing Pokémon that became convinced it needed to retrieve an item that does not exist in that version of the game, then spent many hours pursuing it. The report also notes that past roughly 100k tokens the agent tended to repeat actions from its history rather than form new plans. A memory system faithfully preserving a false belief is working exactly as designed, which is the problem.
The takeaway
The models crossed the endurance threshold: they can sustain marathon-length work. What hasn’t caught up — and where the real opportunity sits — is the memory engineering around them. The two best working answers agree on the fundamentals: memory lives in boring, durable, human-readable files; writing it is part of the job, not an afterthought; and self-correction falls out almost for free once verified failures are recorded somewhere the agent is guaranteed to reread.
None of this is glamorous. That’s exactly why it’s the opportunity. The gap between a demo agent and a deployed one isn’t intelligence — it’s whether anyone designed what happens when the whiteboard gets erased.
Sources
Anthropic: Effective harnesses for long-running agents · Companion code (GitHub) · Nous Research: Hermes Agent · Hermes docs · Claude Sonnet 4.5 (30-hr runs) · Anthropic context management · Companion: the marathon post · Companion: memory needs forgetting
Claude memory tool docs · Claude Code memory · MemGPT · Generative Agents · its released code · the LOCOMO scoring dispute · LongMemEval · persistent memory exfiltration · context rot ·