Claude Sonnet 4.5’s 30-hour autonomous runs made headlines as a model capability. Look closer at how such runs actually complete and a different story appears: an initializer that sets up the environment, a loop that forces incremental verified progress, a progress file that survives every context wipe, git as the undo stack, and a harness that decides when to compact, when to checkpoint, when to reset, and when to give up. Anthropic’s own engineering write-up on long-running harnesses is not about the model at all — it’s about the machinery. The model runs the marathon; the infrastructure is the course, the aid stations, and the medical tent.
This post is the systems view of that machinery — the part of the stack my earlier posts kept pointing at: the marathon post proved pacing beats speed, the memory post established what must survive the wipe, the eval post defined what to measure. This one is about the loop that has to do all of it, unattended, at 3am, on step 1,847.
01The loop is an operating system
Strip any serious long-running harness — the Claude Agent SDK’s session loop, an OpenHands-style controller, your homegrown one — and the same skeleton appears, and it’s an OS kernel’s skeleton:
| OS concept | Agent-loop equivalent | The failure it prevents |
|---|---|---|
| Scheduler | Step loop with per-step timeouts and a continue/yield decision after each verified increment | One mega-step that runs forever and dies with everything in flight |
| Virtual memory | Context compaction + external memory files — RAM (context) backed by disk (filesystem) | Context overflow mid-task; the amnesia reset |
| ulimits / cgroups | Token and cost budgets per step, per task, per day — enforced by the loop, not requested of the model | The $4,000 weekend nobody noticed |
| Checkpoint / snapshot | Persist progress file + git commit at every verified milestone | A crash at hour 29 costing 29 hours |
| Watchdog timer | Progress monitor: no verified step in K actions → intervene | Doom loops and silent stalls |
| Supervisor (init/systemd) | An outer process that restarts crashed runs from the last checkpoint — Erlang’s “let it crash” applied to agents | Babysitting; 3am pages for recoverable failures |
Here’s the whole argument at span level. The ideal run checkpoints and finishes once; the uncheckpointed run pays for its first half twice; the unwatched run buys the same failure seven times:
Two traces, one crash — what checkpoints are actually for
The Erlang line deserves emphasis, because it’s the philosophical unlock: don’t build an agent that never fails; build a supervision tree where failure is cheap. Steps are idempotent, state persists at boundaries, and the supervisor’s job is not to prevent crashes but to make them cost one step instead of one run. Every technique in this post is a corollary.
02Memory management: context is RAM
- Set a high-water mark and compact before it. Reliability degrades as context fills (context rot is measurable long before overflow), so compaction at ~70–80% — summarize the history, evict raw tool output, keep the plan and recent turns verbatim — beats compaction at 99% every time. Anthropic ships this as context editing + the memory tool, and reports it lifting long-horizon eval performance ~39% combined.
- Spill to disk, deliberately. Large payloads never enter context whole: write to file, keep a path + summary in context, re-read slices on demand. The filesystem is the agent’s swap space — and unlike context, it survives the wipe.
- Design for the KV-cache. The production metric almost nobody outside agent-infra teams talks about: Manus calls KV-cache hit rate the single most important number for a production agent — cached input tokens can be ~10× cheaper and far faster. The rules it imposes: keep the prompt prefix stable (no timestamps up top), make context append-only, and prefer masking tools over adding/removing them mid-run, because every edit above a token invalidates the cache below it.
- Reset beats compact when the leg is done. Compaction preserves a thread; a fresh sub-agent with a clean context and a handoff summary preserves reliability. The marathon simulation showed exactly this: checkpoint-plus-reset beat checkpoint-plus-carry at every horizon.
03Resource management: tokens are the meter
- Budget hierarchically: per-step ceilings (a step that wants 10× the median is probably lost — that’s a watchdog signal, not a bill to pay), per-task budgets the planner can see (“you have ~200k tokens left” changes behavior), and per-day circuit breakers that yield to a human instead of dying silently.
- Treat rate limits as weather, not errors. Exponential backoff with jitter, request coalescing, and — for the long run — scheduling heavy phases off-peak. A loop that retries hot loses its budget to 429s.
- Make every step resumable. Spot-instance discipline: the run must tolerate being killed between any two steps. This is free if checkpoints are real, and it converts compute from “reserved and precious” to “preemptible and cheap.”
- Garbage-collect the workspace. Thirty hours of build artifacts, logs, and scratch clones will fill any disk. The loop owns cleanup at checkpoint boundaries — the agent, like any process, should not be trusted to pick up after itself mid-task.
04The run console: watch the infrastructure earn its keep
Below is a simulated 30-hour run under your control. Three pieces of infrastructure can be toggled: compaction (manages the context gauge), the watchdog (catches stalls), and checkpoints (bound the cost of the mid-run crash that will happen). Turn things off and watch how the run dies; turn them on and watch the same failures become log lines instead of postmortems.
Progress · context pressure · token spend — one run, hour by hour
The instructive runs: all three off (the demo configuration everyone ships first), and checkpoints-only (survives the crash, still dies of context). Infrastructure is the difference between “failed at hour 22” and “finished; two incidents, both auto-recovered.”
05Evals for runs you can’t afford to rerun
Here’s the infra problem nobody budgets for: your eval suite is now slower than your release cadence. A 30-hour scenario cannot gate a merge. The answer is the same one systems engineering always gives — a pyramid, plus recording:
| Layer | Runs in | What it checks | Gate |
|---|---|---|---|
| Step evals | seconds | Single decisions against golden traces: tool choice, compaction summaries, escalation calls | every commit |
| Replay scenarios | minutes | Recorded runs re-executed with recorded tool responses (VCR-style) — deterministic, cheap, offline | every merge |
| Checkpoint resumes | ~an hour | Boot from a stored hour-N checkpoint and run one phase live — tests the changed leg, not the whole race | daily |
| Full marathons | hours–days | 3–5 seeded end-to-end runs; trace metrics (progress-AUC, loop rate, verified pass) plus cost | nightly / weekly + canary |
Two details carry the whole scheme. Recording: every production and eval run persists its full trace and tool I/O, so any failure becomes a deterministic replay test forever — the eval set grows by incident, like the domain-agent golden set. Checkpoint fixtures: stored mid-run states are the long-horizon equivalent of database fixtures — without them, every eval of hour 20 costs 20 hours; with them it costs one. And measure the suite itself: flakiness budgets per layer, because a marathon eval that fails 20% of the time on infrastructure noise will train your team to ignore it — the rubber-stamp problem, again.
The framing I’d push
Stop asking “is the model good enough to run for 30 hours?” It is. Ask “is my loop good enough to be trusted with 30 hours of model?” — enough checkpoints that a crash is boring, enough budget enforcement that a runaway is impossible, enough recorded evals that a regression is caught at the step layer for pennies.
06Where the infrastructure doesn’t save you
· Compaction is lossy, and the loss compounds. Every summary discards something; thirty compactions in, the agent can hold a confident, internally-consistent, subtly wrong picture of its own past. Anchor compaction against ground truth that never compacts — the progress file, the git log, the tests.
· Checkpoints capture state, not the world. Resume an hour-20 checkpoint after the staging database moved on and the agent wakes into a world that contradicts its memory. Checkpoint the environment’s contract (versions, schemas, fixtures) alongside the agent’s state, or resumes become their own failure mode.
· Watchdogs can kill healthy runs. Some legitimate work looks like a stall (long builds, big migrations). Watch verified progress against the plan’s own estimates, not wall-clock silence — and make the watchdog’s first action a checkpoint-and-ask, not a kill.
Takeaways
- Build the OS, not just the agent: scheduler, virtual memory, ulimits, snapshots, watchdog, supervisor — every long-running loop needs all six.
- Let it crash — cheaply. Idempotent steps + checkpoints at verified milestones turn crashes from postmortems into log lines.
- Compact early, spill to disk, keep the prefix stable. Context is RAM; the KV-cache hit rate is the production metric that prices every decision.
- Budget tokens like cgroups: per-step ceilings, planner-visible task budgets, daily circuit breakers that yield instead of dying.
- Watch verified progress, not activity. Tokens flowing is not work happening — the watchdog triggers on the absence of verification.
- Evals are a pyramid on recordings: step evals per commit, VCR replays per merge, checkpoint resumes daily, full marathons nightly — and every incident becomes a replay test forever.
- Budget the flakiness of the evals themselves — a noisy marathon suite trains people to ignore it.
The models will keep doubling their horizon. The teams that benefit won’t be the ones with the best prompts — they’ll be the ones whose loop was already built for a run twice as long.
Sources
Anthropic: harnesses for long-running agents · Anthropic: context editing + memory tool · Manus: context engineering (KV-cache) · Erlang/OTP supervision principles · METR time-horizon · Companion: the marathon post · Companion: agent memory · Companion: long-horizon evaluation · Companion: the autonomy dial