Count the model swaps you’ve already survived. GPT-3.5 to GPT-4. GPT-4 to Claude. Claude 3 to 3.5 to 4 to whatever shipped while this post was being written. If your agent product is more than a year old, its “brain” has been replaced wholesale several times — different weights, different quirks, different failure modes, sometimes a different company — and your users mostly didn’t notice. Nothing about the product they trust survived in the model, because nothing about the product was ever in the model. The taste lives in your evals. The competence lives in your skills and memory. The safety lives in your escalation policies. The identity lives in the harness — the thing we’re all still calling scaffolding, a word that means “the part you throw away when the building is done.”
The building is never done, and we have the metaphor inverted. This post is an argument that the harness is the organism and models are its replaceable organs — taken seriously enough to also argue the other side, because the counterargument (the bitter lesson, “the model is the product”) is half right, and knowing which half is the most valuable strategic fact in agent engineering right now.
01The inversion: what actually survived your last swap
Do the archaeology on a real migration and a pattern jumps out. When teams moved from GPT-4-era scaffolds to stronger models, whole categories of code died: the chain-of-thought coaxing (“think step by step”), the JSON-begging and retry-on-malformed-output loops, the few-shot piles compensating for weak instruction-following, the “you are an expert” incantations. All of it existed to compensate for a specific model’s specific weakness, and the next model didn’t have that weakness, so the code wasn’t just unnecessary — it was often actively harmful, over-constraining a system that no longer needed the crutch.
And yet the same migrations left other things not merely intact but more valuable: the golden eval set got to prove the new model was better before anyone shipped it. The verification rules caught the new model’s brand-new failure modes on day one. The tool contracts worked unchanged. The memory and skills — distilled from thousands of production runs — transferred wholesale, because a pitfall in your domain is a pitfall for every model. The trace recordings replayed. The escalation thresholds needed re-tuning, but the policy — what’s irreversible, what stakes demand a human — didn’t move, because the world hadn’t moved.
Two categories, opposite fates, one migration. That’s not an accident of one codebase; it’s a law worth naming.
02The razor: model-shape vs. world-shape
Every line of an agent system encodes an assumption about something. The razor: is the assumption about the model, or about the world? Model-shape components encode how this model stumbles — they are loans against the current release, repaid in obsolescence. World-shape components encode what correct, safe, and valuable mean in your domain — they are deposits, and every incident, eval case, and distilled skill adds to the balance.
| Component | Encodes | Half-life | Fate at swap |
|---|---|---|---|
| Prompt phrasing & incantations | model-shape | one release | Rewritten; often deleted with relief |
| Output-format coaxing, retry-on-malformed | model-shape | already dead | Absorbed into the models themselves |
| Per-model quirk profile (temperature, tool-call style) | model-shape by design | one release | Regenerated in a day — cheap because it’s isolated |
| Golden eval set + production incident cases | world-shape | years | Appreciates — it’s how you judge the new model |
| Verification rules, domain checks, test suites | world-shape | years | Catches the new model’s new failure modes on day one |
| Memory, skills, distilled pitfalls | world-shape (mostly) | years | Transfers — a domain pitfall trips every model |
| Tool contracts & schemas | world-shape | years | Unchanged — the API didn’t hear about the launch |
| Escalation policy (stakes × reversibility) | world-shape | years | Policy holds; thresholds re-tune from fingerprints |
| The loop itself (checkpoints, budgets, watchdogs) | world-shape | years | Indifferent to which model it supervises |
Notice what the world-shape column actually is: it’s this blog’s whole series — evals, memory, tool contracts, autonomy policy, the loop. That’s not a coincidence I engineered; it’s the pattern that emerges when you keep asking “what still matters after the next launch?” The razor also explains a puzzle: why the teams most fluent in prompt tricks got the least durable advantage from them. They were accumulating the depreciating asset.
› Go deeper: the mixed cases are where the thinking gets fun
Some components refuse to sort cleanly, and interrogating them sharpens the razor. Skills: “the staging DB resets on Sundays” is pure world-shape; “phrase the query this way or the model misparses it” is model-shape hiding inside a skill file — consolidation should tag which is which, or your procedural memory silently rots at each swap. Compaction summaries: the policy (compact at 75%, keep the plan verbatim) is world-shape; the summaries themselves were written by a model, in its voice, with its blind spots — memory written by the old tenant, read by the new one. Routing thresholds (“escalate below 80% confidence”): the number is model-shape (each model’s confidence distribution differs), the variable is world-shape. The general fix is the same in every case: separate the parameter from the policy, and let the parameter be cheap to relearn.
03The adapting harness: swap as a metered, learning operation
The strongest version of the thesis isn’t that the harness survives swaps — it’s that a well-built harness metabolizes them. The swap protocol is everything this series has built, pointed at a single event:
Read the last line again, because it’s the thought I most want to leave lodged: the ability to change models cheaply is itself world-shape knowledge — a skill in the harness’s procedural memory, refined per the consolidation loop, verified per the eval gate. Teams treat each migration as a one-off crisis; the harness-first team treats it as a rehearsed operation with a runbook, dashboards, and a regression gate. Which means the moat isn’t just what the harness knows about your domain. It’s that your product improves the week a better model ships, automatically, while competitors schedule a quarter of migration work. Model progress — the thing that supposedly commoditizes you — becomes your distribution advantage.
04The simulator: three years, six models, two philosophies
Below, 36 months and six model generations, each release better and differently quirked. Two products consume them. The thin wrapper tracks the model closely — and pays for it at every dashed line: each swap breaks its model-shape scaffolding and it dips hard while re-prompting. The adapting harness accumulates world-shape capital (evals, skills, verification) between swaps and absorbs each one through the protocol above. Toggle the two properties and watch which curve you’d rather own — and find the month where the harness product rises above its own model’s raw capability, the moment the system is worth more than its brain:
Product quality vs. time — dashed verticals are model swaps · grey steps are raw model capability
Both toggles off = the thin wrapper: forever slightly below its model, resetting at every launch. Accumulation without the adapter still dips hard — capital doesn’t save you if swaps are crises. Both on is the argument of this post drawn as a line: the product detaches from the model that powers it.
05Taking the bitter lesson seriously
Now the other side, with real respect, because it’s partly right. Sutton’s bitter lesson — general methods riding compute beat human-engineered structure, eventually, always — and its product-strategy cousin, “the model is the product”, both predict harness code dying as capabilities absorb it. And they keep being right: models ate your JSON parser (structured outputs), your retrieval glue (longer context), chunks of your memory system (memory tools), your browser scaffold (computer use). The frontier labs are explicitly training against agentic tasks; every capability you scaffold is a bug report they’re fixing.
· The honest resolution is asymmetric: the bitter lesson eats the control layer of your harness — the parts that tell the model how to think. It cannot eat the truth layer — the parts that define what correct means for you: your golden set, your verification rules, your irreversibility map, your users’ trust. No amount of compute learns your staging database resets on Sundays. Plan for a harness that shrinks at the control layer and grows at the truth layer — if yours is growing at the control layer, you’re building the depreciating half.
· The harness can become the ceiling. Guardrails tuned for a weaker model can strangle a stronger one — forced decomposition where it could one-shot, mandatory checkpoints that halve its speed. This is why the swap protocol must include relaxation experiments, not just safety checks: the fingerprint diff should ask “what can we now remove?” as loudly as “what broke?”
· Portability is discipline, not default. Everything above assumes your world-shape capital isn’t secretly entangled with one vendor’s API shapes. Evals phrased as raw transcripts, tools on open contracts (MCP), memory in plain files — the boring-formats principle is what makes the house movable at all.
Takeaways — and the harness test
- Run the razor on every component: model-shape or world-shape? Loans or deposits? Sort your repo once and your investment strategy falls out.
- Separate parameter from policy — thresholds are model-shape and cheap to relearn; the policy behind them is world-shape and yours.
- Make swaps a rehearsed operation: fingerprint → quirk profile → shadow → ratchet → re-tune → record. Swap #5 should cost a fraction of swap #1.
- Let the harness shrink at the control layer and grow at the truth layer — deleting obsolete scaffolding is a feature of the strategy, not a failure of it.
- Ask for removals at every swap, not just regressions — yesterday's guardrail is tomorrow's ceiling.
- Keep the capital portable — plain files, open contracts, raw transcripts. A house you can’t move is a house the landlord owns.
And the test to carry out of this post: if a strictly better model shipped tomorrow, what fraction of your codebase would you delete? If the answer is near zero, you’ve built nothing but model-shape and the launch is a threat. If it’s near everything, you never built a product. The right answer is a number you chose on purpose — and if you can’t name yours, that’s the most useful thing this post can tell you.