A synthetic user is a constrained AI decision agent defined by twelve fields, from functional role and context to assumptions and abandonment rules.
In the previous post I built an early, working implementation, and the next question was whether the same rules could hold up in a repeatable, automated test.
This post is that next step: how I turned the framework into a Claude Code plugin, and the technical decisions behind adapting methods designed for people into something an AI can execute without cheating.
Why “find the usability issues” is not enough
Give a model a URL and ask it to “find the usability issues.” It works halfway. And the “halfway” is the interesting part, It gives you a generic list, correct in the abstract, useless in practice.
A usability issue matters because of who encounters it and under what conditions.
Using an app from bed is not the same as using it on a factory floor. Urgency changes, lighting changes, attention changes, previous knowledge changes. The same confusing button can be irrelevant to a power user and an abandonment point for an operator wearing gloves.
The whole design comes from that observation: the AI does not evaluate the interface. It acts as a specific person in front of the interface.
The person brings the context with them. And the context turns a list of defects into a list of priorities.
Anatomy of a simulation
An orchestrator controls the browser through Playwright MCP. It reads each screen as an accessibility snapshot: text, roles, states, no guessing pixels. Then it acts on specific elements.
The decision on each screen is made by an isolated subagent, which returns a JSON for each step:
{
"action": "...",
"clarityLevel": "High|Medium|Low",
"doubtDetected": true,
"reason": "...",
"abandoned": false,
"estimatedTimeSeconds": 40,
"emotionalState": "...",
"memory": "..."
}
Two rules make this look more like a person and less like an oracle.
1. The evaluator never sees the end.
The evaluator receives one screen at a time, without knowing how many are left or what comes next in the flow.
If the interface leaves room for a mistake, the synthetic user makes the mistake. It clicks where a person would click, not where it is convenient to click in order to complete the test. This is where the framework’s forbidden assumptions live. The agent cannot assume backend logic or mentally complete what the screen does not show.
2. Emotion is memory, not decoration.
The memory field travels from one step to the next. The emotional state is inherited and accumulates. A frustration +1 persists. This detects something that is structurally invisible to any test that evaluates screens separately.
Screen five does not necessarily fail because of screen five. It fails because the user gets there with accumulated frustration.
Evaluated alone, that screen passes. Evaluated by someone carrying three doubts and one broken promise, it triggers abandonment. In the first post, I wrote that doubt is not failure. It is the signal that reveals structural friction.
Emotional memory is that idea turned into architecture.
Eight subagents, one job each
Each subagent gets a clean context. It knows the minimum required to do its job.
That ignorance is deliberate.
The agent acting as the user does not know what the orchestrator knows. It cannot compensate for bad design with knowledge a real person would not have.
Subagent
What it does
Subagent
What it does
synthetic-screen-evaluator
Acts as the user on one screen and returns the JSON for that step
synthetic-flow-synthesizer
Reads the complete run and writes the report. It never simulates again
synthetic-profile-generator
Generates a complete profile from an approved spec, choosing from a controlled vocabulary
synthetic-autopilot-synthesizer
Consolidates N runs and classifies findings by convergence across users
heuristic-persona-generator
Creates the 3 persona raters based on the business being evaluated
heuristic-expert-evaluator
Detects violations of the 10 heuristics using forced enumeration
heuristic-persona-rater
Scores each finding from the experience of ONE persona. It runs ×3
heuristic-report-synthesizer
Builds the final report using the already computed numbers
Adapting a human test: the heuristic evaluation
A textbook heuristic evaluation uses three to five human evaluators because each human finds different problems.
My first experiment was literal, and it went meh.
I iterated until I reached two synthetic detection runs with different agents, coverage was extremely high, but it exposed another problem: an unmanageable list. Dozens of valid issues, very few important ones.
The final design separates those two jobs.
1. An expert finds violations.
Based on Nielsen’s literature, an expert goes through each screen and is forced to produce a verdict for every heuristic:
Violation
Clean
Not observable
Each verdict includes textual evidence from the snapshot, forced enumeration breaks the habit of reporting only the things that stand out.
2. Three synthetic personas decide what matters based on what they bring with them: context, emotions, urgency, and constraints.
Three synthetic personas are generated according to the business being evaluated:
power user
average user
low digital literacy
They score the findings without seeing the expert’s conclusions. The same issue can matter very differently depending on what each persona brings to it.
The formula is business impact × usability impact, with agreement between personas as the tiebreaker.
This keeps issue detection and user impact as separate jobs: the expert identifies the violations, and the personas help determine which ones deserve attention first.
Three modes, and a tool for building users
The plugin currently has three modes.
simulation-run (custom)
You build a profile field by field in the Synthetic User Builder, the tool I built to materialize the framework.
First come the attributes:
Role in relation to the product
Boundaries
Initial emotional state
Context
Forbidden assumption
Only after that, and separately, comes the task.
The profile describes how someone decides, never what they have to do. That is why the same profile can be reused across tests.
simulation-auto (inferred)
You only give it the URL.
It researches the business, infers the typical roles, proposes users with tasks, and you adjust that proposal in natural language before anything runs.
heuristic-test (inspection)
The heuristic test described above, for one screen, one flow, or the entire site.
Everything run becomes a file
Every run leaves Markdown artifacts inside the project:
user-simulation-tests/
├── simulation/
│ ├── profiles/ ← users: the .md used for simulation + a .builder.json
│ │ that can be imported back into the Builder and edited manually
│ └── results/ ← one report per run + the consolidated report from auto mode
└── heuristic/
├── personas/ ← the 3 raters + business research, reused across runs
└── results/ ← reports with the prioritized findings table
Simulation reports include the full step by step flow, the emotional arc, risks, and a single “Fix this first.”
The consolidated report classifies findings by convergence: did one user suffer from this, or did all of them?
The decision to keep everything as accumulating .md files is strategic.
These are different runs, using different lenses, that can be analyzed together later, crossing heuristic violations with simulated emotions answers something no individual test gives us:
Of everything that is wrong, what actually matters?
Models and costs
What worked for me for the synthesis subagents:
For reports, consolidation, and the heuristic expert, the best available model makes sense. That is where the judgment lives.
For the screen evaluator, a medium and fast model is enough. There are many short, constrained calls, and the profile already restricts the decision.
The raters are the lightest case.
A complete run consumes between 100k and 400k tokens, depending on the model and mode, in around 20 minutes.
That is the cost of a test that previously required coordinating the schedules of three professionals, and that can now run against every iteration of the product.
See it in action
Here's a complete run against our site, kzsoftworks.com: a skeptical "Business Leader" profile, five live browser steps, and a full Markdown audit in under three minutes that names the exact moment the executive persona lost trust.
It is still early, but it already runs
Every rule in the framework became an architectural constraint: clean context, one screen at a time, emotional memory, forbidden assumptions.
Three commands, and the inferred mode only needs your URL.
If you try it and your synthetic user abandons on screen three, you already know what it means:
It is not failure. It is the signal.
A synthetic user is a constrained AI decision agent defined by twelve fields, from functional role and context to assumptions and abandonment rules.
In the previous post I built an early, working implementation, and the next question was whether the same rules could hold up in a repeatable, automated test.
This post is that next step: how I turned the framework into a Claude Code plugin, and the technical decisions behind adapting methods designed for people into something an AI can execute without cheating.
Why “find the usability issues” is not enough
Give a model a URL and ask it to “find the usability issues.” It works halfway. And the “halfway” is the interesting part, It gives you a generic list, correct in the abstract, useless in practice.
A usability issue matters because of who encounters it and under what conditions.
Using an app from bed is not the same as using it on a factory floor. Urgency changes, lighting changes, attention changes, previous knowledge changes. The same confusing button can be irrelevant to a power user and an abandonment point for an operator wearing gloves.
The whole design comes from that observation: the AI does not evaluate the interface. It acts as a specific person in front of the interface.
The person brings the context with them. And the context turns a list of defects into a list of priorities.
Anatomy of a simulation
An orchestrator controls the browser through Playwright MCP. It reads each screen as an accessibility snapshot: text, roles, states, no guessing pixels. Then it acts on specific elements.
The decision on each screen is made by an isolated subagent, which returns a JSON for each step:
{
"action": "...",
"clarityLevel": "High|Medium|Low",
"doubtDetected": true,
"reason": "...",
"abandoned": false,
"estimatedTimeSeconds": 40,
"emotionalState": "...",
"memory": "..."
}
Two rules make this look more like a person and less like an oracle.
1. The evaluator never sees the end.
The evaluator receives one screen at a time, without knowing how many are left or what comes next in the flow.
If the interface leaves room for a mistake, the synthetic user makes the mistake. It clicks where a person would click, not where it is convenient to click in order to complete the test. This is where the framework’s forbidden assumptions live. The agent cannot assume backend logic or mentally complete what the screen does not show.
2. Emotion is memory, not decoration.
The memory field travels from one step to the next. The emotional state is inherited and accumulates. A frustration +1 persists. This detects something that is structurally invisible to any test that evaluates screens separately.
Screen five does not necessarily fail because of screen five. It fails because the user gets there with accumulated frustration.
Evaluated alone, that screen passes. Evaluated by someone carrying three doubts and one broken promise, it triggers abandonment. In the first post, I wrote that doubt is not failure. It is the signal that reveals structural friction.
Emotional memory is that idea turned into architecture.
Eight subagents, one job each
Each subagent gets a clean context. It knows the minimum required to do its job.
That ignorance is deliberate.
The agent acting as the user does not know what the orchestrator knows. It cannot compensate for bad design with knowledge a real person would not have.
Subagent
What it does
Subagent
What it does
synthetic-screen-evaluator
Acts as the user on one screen and returns the JSON for that step
synthetic-flow-synthesizer
Reads the complete run and writes the report. It never simulates again
synthetic-profile-generator
Generates a complete profile from an approved spec, choosing from a controlled vocabulary
synthetic-autopilot-synthesizer
Consolidates N runs and classifies findings by convergence across users
heuristic-persona-generator
Creates the 3 persona raters based on the business being evaluated
heuristic-expert-evaluator
Detects violations of the 10 heuristics using forced enumeration
heuristic-persona-rater
Scores each finding from the experience of ONE persona. It runs ×3
heuristic-report-synthesizer
Builds the final report using the already computed numbers
Adapting a human test: the heuristic evaluation
A textbook heuristic evaluation uses three to five human evaluators because each human finds different problems.
My first experiment was literal, and it went meh.
I iterated until I reached two synthetic detection runs with different agents, coverage was extremely high, but it exposed another problem: an unmanageable list. Dozens of valid issues, very few important ones.
The final design separates those two jobs.
1. An expert finds violations.
Based on Nielsen’s literature, an expert goes through each screen and is forced to produce a verdict for every heuristic:
Violation
Clean
Not observable
Each verdict includes textual evidence from the snapshot, forced enumeration breaks the habit of reporting only the things that stand out.
2. Three synthetic personas decide what matters based on what they bring with them: context, emotions, urgency, and constraints.
Three synthetic personas are generated according to the business being evaluated:
power user
average user
low digital literacy
They score the findings without seeing the expert’s conclusions. The same issue can matter very differently depending on what each persona brings to it.
The formula is business impact × usability impact, with agreement between personas as the tiebreaker.
This keeps issue detection and user impact as separate jobs: the expert identifies the violations, and the personas help determine which ones deserve attention first.
Three modes, and a tool for building users
The plugin currently has three modes.
simulation-run (custom)
You build a profile field by field in the Synthetic User Builder, the tool I built to materialize the framework.
First come the attributes:
Role in relation to the product
Boundaries
Initial emotional state
Context
Forbidden assumption
Only after that, and separately, comes the task.
The profile describes how someone decides, never what they have to do. That is why the same profile can be reused across tests.
simulation-auto (inferred)
You only give it the URL.
It researches the business, infers the typical roles, proposes users with tasks, and you adjust that proposal in natural language before anything runs.
heuristic-test (inspection)
The heuristic test described above, for one screen, one flow, or the entire site.
Everything run becomes a file
Every run leaves Markdown artifacts inside the project:
user-simulation-tests/
├── simulation/
│ ├── profiles/ ← users: the .md used for simulation + a .builder.json
│ │ that can be imported back into the Builder and edited manually
│ └── results/ ← one report per run + the consolidated report from auto mode
└── heuristic/
├── personas/ ← the 3 raters + business research, reused across runs
└── results/ ← reports with the prioritized findings table
Simulation reports include the full step by step flow, the emotional arc, risks, and a single “Fix this first.”
The consolidated report classifies findings by convergence: did one user suffer from this, or did all of them?
The decision to keep everything as accumulating .md files is strategic.
These are different runs, using different lenses, that can be analyzed together later, crossing heuristic violations with simulated emotions answers something no individual test gives us:
Of everything that is wrong, what actually matters?
Models and costs
What worked for me for the synthesis subagents:
For reports, consolidation, and the heuristic expert, the best available model makes sense. That is where the judgment lives.
For the screen evaluator, a medium and fast model is enough. There are many short, constrained calls, and the profile already restricts the decision.
The raters are the lightest case.
A complete run consumes between 100k and 400k tokens, depending on the model and mode, in around 20 minutes.
That is the cost of a test that previously required coordinating the schedules of three professionals, and that can now run against every iteration of the product.
See it in action
Here's a complete run against our site, kzsoftworks.com: a skeptical "Business Leader" profile, five live browser steps, and a full Markdown audit in under three minutes that names the exact moment the executive persona lost trust.
It is still early, but it already runs
Every rule in the framework became an architectural constraint: clean context, one screen at a time, emotional memory, forbidden assumptions.
Generative UI can adapt interfaces to each user, but it adds risks around reliability, latency, cost, security, and accessibility. Learn the architecture that keeps those risks under control.
Generative UI assembles the interface around what each user is trying to do, instead of showing everyone the same fixed screen. That flexibility comes with real considerations: keeping the experience consistent, secure, and easy to support once it's live. This post covers what generative UI is worth building for, what it costs, and how teams keep it under control.
Generative UI works best when the experience is dynamic, but the system behind it stays tightly controlled.
Start by defining which parts of the interface can change, which cannot, and what must be validated before anything reaches the user.
TL;DR
Interfaces can adapt to user context, support more variations without designing every screen by hand, and reduce unnecessary steps in a workflow.
The trade-offs include inconsistent experiences, unreliable or unsafe output, added latency and infrastructure cost, and harder analytics and debugging.
Better prompting can reduce unwanted behavior, but it cannot guarantee reliability, security, or consistency. Those controls need to exist around the model: a stable interface shell, a closed component catalog, validation of model output, session-level logging, and model routing with fallback options.
Every control introduces a trade-off. No architecture maximizes flexibility, reliability, privacy, performance, and cost at the same time.
What does generative UI make possible?
Interfaces that adapt to context
The interface can adapt to what a person is trying to do instead of relying only on a persona defined at design time. Steps can reorder or disappear based on intent. It can change how much information it shows and what it emphasizes. Copy can adapt to the user's locale and context instead of relying on literal translation.
More interface variations with less custom development
A small set of components can support many variations without designing each screen separately. The system can also support workflows the team did not design as individual screens, as long as the required components and actions already exist.
Fewer steps between intent and action
The interface can hide controls a task does not need, reducing the number of steps required to complete it. Generative UI can also help teams test different ways of presenting the same task. Whether that improves completion or conversion depends on the workflow.
What can go wrong with generative UI?
Experience consistency risks
When layouts change between users or sessions, they can break muscle memory and make support harder. They can also drift from the design system or disrupt accessibility patterns that depend on consistent structure.
Reliability and security risks
The system should not trust model output by default. A model can render a button that does nothing, display fabricated data in a component, or produce a state the team never tested. Prompt injection can push it toward components, content, or actions the system should not allow. Weak controls can expose sensitive data or allow actions and interface states the product should block.
Performance and infrastructure risks
A generative interface also inherits the model layer's latency, cost, and availability risks. Waiting on an LLM to generate a layout adds delay before a page renders. Each generation uses processing resources, and hosted models usually add usage-based cost. Relying on one provider also exposes your product to outages, API changes, price increases, and deprecations.
Analytics and debugging risks
Standard analytics often assume a fixed set of screens. Heatmaps and funnels become harder to compare when users see different layouts. Reproducing a bug also gets harder when you cannot reopen the exact screen the user saw.
How do you control these risks?
Prompts can reduce unwanted behavior, but they cannot enforce which components the system may render or which actions it may allow. Those limits need to be enforced in the architecture around the model.
What parts of a generative interface should remain fixed?
Keep global navigation, account and security controls, primary actions, critical transaction controls, and accessibility-critical structure fixed. Let the model modify only the content and controls that benefit from adaptation.
Fixed navigation preserves familiar interaction patterns. A stable structure also makes accessibility testing, branding, and support more predictable.
How do you stop generative UI from creating broken interfaces?
Do not let the model generate arbitrary UI code. Have it return structured configuration instead. The schema should specify the component, its data, and its position. Validate that output against a closed catalog before rendering it.
The model should not write HTML, CSS, or JavaScript or choose anything outside that catalog. This reduces invalid layouts and unsupported combinations. This is the declarative approach we covered in Part 1.
How should teams test and secure generative UI?
Treat model output as untrusted input. Validate it against the schema and component allowlist, sanitize content, and keep authorization outside the model.
Add content security policies and prompt-injection defenses based on what the model can access and what actions it can trigger. Pay particular attention to user-provided content, privileged actions, sensitive data, and external tools.
Limit valid component combinations, then use visual regression and property-based tests to exercise unexpected inputs and edge cases.
Minimize sensitive data sent to the model. Mask or anonymize it before generation when the task does not require the original values.
How do you monitor a UI that looks different for every user?
Record enough context to reconstruct each generated interface. That includes detected intent, model version, generated configuration, rendered components, task completion, and errors, all tied to the session.
That record lets teams segment analytics by generated experience and reconstruct what a user saw during a specific session.
How do you control latency, cost, and outages?
Cache reusable results where freshness and privacy allow. Show a skeleton layout immediately and stream the rest in. Route simpler requests to smaller or local models, and reserve larger ones for complex requests. Put providers behind the same integration layer so you can switch models or fall back to a static experience during an outage.
What it controls
Risks it mitigates
Stable interface shell
Keeps navigation, account controls, and primary actions fixed
Muscle memory loss, brand drift, accessibility gaps, support friction
Records intent, generated configuration, rendered components, and outcome
Fragmented analytics, hard-to-reproduce bugs, support friction
Model routing and fallback
Caching, streaming, model routing, provider switching
Latency, model cost, provider downtime, difficulty switching providers
What do these controls cost you?
Keeping more of the interface fixed protects consistency but limits personalization. Limiting combinations makes the system easier to test but reduces how much it can vary. Caching lowers cost, but cached output can go stale.
Running models locally can reduce how much sensitive data leaves your infrastructure, but it adds systems your team has to operate and maintain. Detailed session logs can make support easier, but they also create storage, retention, and privacy requirements.
No architecture maximizes flexibility, reliability, privacy, performance, and cost at once. You need to decide which trade-offs matter most for each workflow and design around them.
Generative UI is a full-stack architecture that lets AI create, modify, and render user interfaces in real time, based on what each user needs at that exact moment. Instead of static, predefined screens, the interface assembles itself on the fly: a bar chart, a table, a comparison card when you're comparing things.
We've been building proofs of concept with it for the past few weeks. Most of what's written about generative UI is either too abstract or too exciting, so this is our attempt at neither: what it is, how it works, where it helps, where it doesn't, and what we learned from two demos we built.
The short version
Generative UI means the AI designs the screen that answers your question, not just the answer.
In production, most systems don't let the AI write code. It configures pre-built components. Safer, and good enough.
It shines in open-ended workflows like reporting and data exploration, where you can't pre-design every screen someone might need.
It complements standard UI. It doesn't replace it. Anyone telling you otherwise is selling something.
What is generative UI?
Generative UI is a full-stack architecture: the backend talks to the LLM, decides what the answer should look like, and picks the components, while the frontend renders them and handles how the user interacts with what’s on screen.
Compare that with how interfaces have always worked. A designer decides what goes on each screen, a developer builds it, and every user sees the same thing. Forever, or until the next redesign.
Generative UI flips that. The interface becomes dynamic and personal instead of static and universal. The AI doesn't just answer your question, it designs the screen that answers your question.
Dashboards and reporting are the most common use cases, but they're far from the only one. The same pattern works for dynamic forms, onboarding flows, and customer support, as it takes input just as easily as it presents output. It can even adjust font size, contrast, or layout for users with low vision, color blindness, or cognitive load.
The three types of generative UI
There are three levels of generative UI, from most constrained to most open (Google Cloud, 2026):
Static. Everything is pre-built. The AI picks which screen to show you from a fixed library. Low risk, low flexibility.
Declarative. The AI assembles a JSON tree that specifies which UI components to use, in what order, with what properties. It doesn't write code. It configures pre-designed widgets. This balances the AI's flexibility with the system's stability.
Open. The AI generates completely new code from scratch and the frontend renders it. Maximum flexibility, maximum risk.
Most production systems today use the declarative approach, and that's what this post assumes from here on. The AI isn't writing HTML or CSS freestyle. It selects components, fills in pre-designed widgets, and composes them into the right screen.
How does generative UI work?
Generative UI works by turning a user request into structured data that describes an interface, then rendering that data as real components. The flow looks like this:
The user asks for something, explicitly or inferred from context.
An LLM analyzes the request. It invokes tools, pulls data, and makes the design decisions: what to show and how.
The system generates structured data describing both the components and the information they'll display.
That schema travels to the frontend through the AG-UI protocol, a standard for communication between agents and frontends. It defines events that keep the agent's state in the backend synchronized with the frontend framework.
The frontend transforms the schema into actual widgets and renders them.
To the user, the result feels like magic. Behind the scenes, it's structured data flowing through a well-defined pipeline. We prefer the second description. It's the one you can build on.
Pros and cons of generative UI
Generative UI trades real personalization and faster development for added latency, inference costs, and less predictable layouts. That's the honest version. Here are the details.
What you gain
Benefit
Why it matters
Real personalization
Each user sees the view they need, not the view designed for the average user. When that happens, conversion follows.
Flexibility that scales
A small set of components combines into thousands of screens, including views you never explicitly built.
Faster development
You build the component library once. The system composes it, instead of your team coding endless specific screens.
What you pay for it
Trade-offs
What to watch
Latency
There's an LLM in the middle, and that adds response time.
Token costs
Every generated screen has an inference cost attached.
Less muscle memory
The same request won't always render the same layout. Users can't build habits around pixel positions.
Privacy
Sending data through an LLM means thinking carefully about what you send and where it goes.
None of these are dealbreakers. There are known techniques to mitigate each one.
Generative UI examples: two working demos
We built two demos. One with fictional data, one on top of a tool we use every day.
Aurora Goods: a conversational e-commerce dashboard
Aurora Goods is a fictional consumer e-commerce platform we created for the demo. The interface is simple: chat on the left, canvas on the right. You ask about the business, the LLM figures out what you need, pulls the data, and renders it visually.
Ask about 2025 sales and it shows the numbers on cards, with a short note on anything relevant. Ask it to break that down by region and it extends the same view instead of starting over, because it understands the second question builds on the first. This part took us a while to get right, and it's what makes the whole thing feel like a conversation rather than a search box.
The canvas isn't output-only either. You can click into any element and drill down: revenue by category, then inside electronics, then which products sold most.
You configure the widgets once. The system combines them and adds relevant commentary on the spot.
An internal reporting screen for our time-tracking tool
The second demo is closer to home: a generative reporting layer on top of the time-tracking tool we use every day at Kaizen. The questions in this demo are questions someone here has actually asked.
Instead of building dozens of hyper-specific reports, a small amount of code now handles virtually unlimited queries. How many hours were logged in May? Which anomalies showed up in April? How do billable and non-billable hours compare across two months? Who worked on a given project last month, and for how long? Each answer arrives as the right visualization: cards, lists, bar charts, plus a short summary that's easy to scan.
Two details won us over. The LLM suggests next steps, so exploring the data becomes a conversation. And when it's not sure, it asks instead of assuming. Ask for the hours of someone named Alex and, since we have more than one Alex on the team, it asks which one before answering.
Generative UI complements standard UI. That's the point.
Generative UI is a complement, not a replacement. Standard interfaces still win for stable, repetitive workflows where consistency matters. Nobody wants their checkout button to be creative. Generative UI wins where the workflow is complex and the questions are unpredictable.
It also changes what design systems are for. Beyond designing components and screens, teams will need to define semantic rules: how the AI should react to uncertainty, which interfaces match which intentions, and the guardrails that keep generated screens functional and safe.
That's a new kind of design work. And it's already starting.
Want to see generative UI applied to your own data?
We build working proofs of concept in two weeks. Your data, your workflows, a real thing you can click.
