Ask an agent to compare three laptops and it writes you four paragraphs you must mentally rearrange into the table it should have drawn. Ask it to help you track workouts and it describes a tracking regimen instead of handing you the tracker. We spent two years teaching agents to use tools; the frontier flipping into view — Vercel’s AI SDK streaming React components, the AG-UI protocol standardizing agent–frontend events, Google’s A2UI, a DeepLearning.AI course on agents that emit charts and forms on demand — is agents that make tools: single-purpose, just-in-time interfaces, rendered in the conversation, alive to the click.
This is the point my whole series has been circling. Seeing Agents argued pictures beat prose for humans reading agents; generative UI is the same argument for humans reading answers. And the engineering that makes it safe is the same discipline as tool design — because a UI component the agent can emit is exactly a tool the agent can call, with a schema, a contract, and an eval.
01Specs, not pixels: the architecture that makes it safe
The naive version — let the model write raw HTML/JS into the page — is an XSS generator with a design problem. Every serious generative-UI system converges on the same shape instead:
Everything good follows from that separation. Safety: the model emits data, not code — the blast radius is a rejected spec. Consistency: every generated UI is built from your design system’s components, so it looks like your product, not like a model’s HTML homework. Editability: a spec is state — the user (or the agent, next turn) can patch it incrementally instead of regenerating a blob. Evaluability: specs diff, validate, and replay — the whole trace-metric framework applies to interfaces the moment interfaces become structured output.
02The playground: the whole pipeline, live
Below is a working generative-UI agent — a compact intent compiler standing in for the LLM, emitting real specs into a real renderer. Type what you need, or tap an example. Watch the pipeline light up, read the spec it generated, then use the interface — the forms validate, the trackers check off, the timers run, the charts have tooltips. Then edit the spec JSON directly and watch the UI rebuild: that’s the editability argument, in your hands.
The spec — the agent’s entire output (edit me)
The rendered interface — live, use it
Honest architecture, small brain: intent extraction here is the same hashed-feature classifier as Prompt Studio, standing in for an LLM — the spec/validate/render pipeline is exactly what production systems run. Unknown component types are rejected at the validate stage; try breaking the JSON and watch the renderer refuse politely.
Play with the failure modes too — they teach the design rules. Ask for something outside the palette (“build me a 3D game”) and the agent falls back to the closest legal component instead of improvising: the palette is the guardrail. Change "type": "chart" to "type": "widget" in the spec and validation blocks it: unknown components don’t render, ever. That refusal is the entire security model, and it’s also why generated UIs stay on-brand — the model proposes, the design system disposes.
03The design rules
- Constrain the palette, then trust it. A dozen well-designed components cover the vast majority of answer shapes (chart, table, form, tracker, metric tiles, timer, steps, diff). Every component you add is prompt surface and failure surface — grow the palette by observed demand, not ambition.
- Components are tools — document them like it. Each component’s schema needs the seven-sins treatment: an intent-revealing name, a when-to-use contract (“table for comparisons of discrete items; chart for trends”), few required props, examples, bounded payloads.
- Keep the spec inspectable and patchable. The user should be able to say “make it monthly” and the agent should edit the spec, not regenerate the world — incremental patches preserve the user’s own edits and make the interaction feel like collaboration, not slot-machine pulls.
- Never render dead affordances. Every button the renderer draws must be wired — a generated UI with decorative controls trains users to stop trusting all generated UIs. If the palette can’t wire it, the component doesn’t offer it.
- Eval the interfaces. Spec-validity rate, render-success rate, component-choice accuracy against a golden set (“this request should produce a table”), and the widget-boot check in CI — the same gate this blog runs on itself applies verbatim.
04Where generative UI goes wrong
· Not everything wants a widget. A yes/no question answered with a dashboard is worse than a sentence. The agent’s first decision is modality — text, table, chart, form — and “text” must stay a first-class answer, not a fallback of shame.
· Hallucinated data wearing a real chart. A rendered bar chart looks authoritative even when the numbers came from nowhere. Provenance rules from the domain-agents playbook apply: generated UIs display sourced data or visibly label synthesis — the playground’s charts caption their data source for exactly this reason.
· Accessibility doesn’t generate itself. Deterministic renderers are the fix here too — bake labels, focus order, and contrast into the components once, and every generated instance inherits them. A model free-styling HTML gets this wrong at scale.
Takeaways
- Interfaces are the second output modality. Agents that only write prose make the user do the rendering.
- Specs, not pixels. The model emits declarative JSON; your renderer owns the DOM. Safety, consistency, editability, and evals all fall out of this one separation.
- The palette is the guardrail. Reject unknown components at validation — the refusal is the security model.
- UI components are tools. Same schemas, same contracts, same seven sins, same evals.
- Patch, don’t regenerate. Specs are state; incremental edits make generation feel like collaboration.
- Wire every affordance and caption every dataset — trust in generated UIs is one dead button or one confident fake chart from gone.
The chat box was never the product. It was the bootstrap. The agents worth using next will answer the way good colleagues do — with the thing you actually needed, built while you watched.