What Synthetic User Testing Is
Synthetic user testing replaces human participants with AI personas. A persona is a described buyer: a role, a reason for showing up, one or two goals, a set of objections and a level of patience. The test gives that persona a task on your website or app and records what happens.
The term covers two very different methods, and most of the confusion and criticism you will read online comes from mixing them up. The next section separates them. If you only remember one idea from this guide, make it that one.
- Synthetic user testing comes in two kinds. Conversational synthetic users answer questions. Behavioral synthetic users, tested end to end, use your real product and leave a record of what they did.
- Behavioral tests are strong at finding unclear copy, dead ends, missing information and broken flows.
- They are weak at predicting conversion, emotion and culture. Synthetic personas also tend to finish tasks more often than real people do.
- A finding shared by several personas is a strong lead. A finding from one persona is a question to test.
- Use it as a fast first read and a way to test what you would never otherwise test. Confirm the findings that matter with real people.
Two Kinds Of Synthetic User Testing
Search for the term and you will find strong opinions. The Nielsen Norman Group compared output from AI generated research participants against three real user studies and concluded that synthetic users should supplement real research, not replace it. Their examples are instructive: synthetic participants tended to see every idea as a good one, listed many factors without knowing which mattered most, and praised features that real participants called contrived.
That criticism is fair, and it describes the first kind of synthetic user: one you interview. The second kind does something else entirely. It does not tell you what it thinks of your product. It uses your product, and you read the record.
| Conversational | Behavioral, end to end | |
|---|---|---|
| What the persona does | Answers questions about an idea, concept or message | Uses your live website or app in a real browser toward a goal |
| What you get | Opinions and interview style answers | Clicks, scroll depth, page changes, where it stopped and why, with notes in its own words |
| Evidence | What a language model thinks a person would say | What happened on your actual pages, recorded as it happened |
| Main failure | Agreeable answers, shallow priorities, invented enthusiasm | Finishing tasks more patiently than a real visitor, and reading pages more carefully |
| Good for | Early desk research, drafting interview guides, exploring an unfamiliar audience | Finding dead ends, unclear steps, missing information and broken flows in a product that exists |
| Needs a built product | No | Yes, a live site, staging site or running app |
This guide is about the second kind. When we say synthetic user testing without a qualifier, we mean a behavioral test run end to end: the persona starts where a real visitor starts and keeps going until the goal is done or the persona gives up.
How End To End Synthetic User Testing Works
An end to end test is harder than it sounds, because the persona has to do three things at once: behave like a particular buyer, operate a real browser, and report honestly what it noticed. A repeatable run has six stages.
- Research the business and the customersProduct, pricing, positioning and the people who buy. Personas built from real customer segments find different problems than generic ones.
- Write the personasEach persona gets a role, a reason for buying, a primary goal, a secondary goal, objections, a device, a reading style and abandon triggers. At least one should be non-technical, one cautious about data and one on a phone.
- Run each persona in a real browserThe persona types, scrolls, taps and waits at human pace on your real pages, including sign-up and setup if your product has them.
- Record behavior and reactionsClicks, scroll depth and page changes are captured from the real events. The persona leaves notes on what confused it, what worked and where it would have left.
- Analyze and rankNotes become findings, grouped by type and ranked by severity, with the element on the page each one applies to and how many personas shared it.
- Report with fixesA heat map drawn on screenshots of your own pages, ranked findings with a suggested fix each, and every screenshot and click kept as raw evidence.
Two design choices matter for trust. First, nothing the persona does is staged: clicks are recorded from the events the browser actually receives, so you can inspect what happened. Second, the recorder counts typing but never captures what is inside a field, so a test records behavior rather than content.
What It Finds Well
Behavioral synthetic testing is strongest where a problem can be noticed by trying to finish a task. In our experience that covers:
- Unclear copy. The line a persona quotes back as the one that lost them.
- Dead ends and broken flows. A step that fails with no way forward, a button that does nothing, a form that rejects a reasonable answer.
- Missing information. The question a buyer needs answered before paying, with no answer on the page.
- Pricing confusion. A price shown to the wrong person at the wrong moment.
- Mobile friction. Targets too small, content hidden, a flow that only works with a mouse.
- Sign-in and onboarding walls. Where a first session stalls before a new user sees any value.
A Real Example
Our founding customer, AutoAGI, an AI work platform, was tested by five personas built from its real customer profile. They walked the public site, sign-up, setup, a first real task and a second goal.
Of the 115 findings, 19 were drop-off risks, 40 were confusing, 23 were improvements and 33 were things that worked well. AutoAGI shipped fixes for its five highest ranked findings the day the report finished. One example: three of five personas hit the same setup step that failed after about fifty seconds with no explanation and no way forward. That is a finding several personas shared, which is why it ranked first.
The heat map layer shows the same evidence spatially: where personas clicked, how far they scrolled and the controls nobody found.
Where Synthetic User Testing Falls Short
A guide that only lists strengths is selling something. These are the limits that matter, and they are the reason the method is a first read and not a verdict.
- Personas are more diligent than people. The Nielsen Norman Group found synthetic participants outperformed most real people on tasks. A persona that completes a flow tells you the flow is possible, not that a distracted real visitor will manage it. Treat completion as an upper bound.
- It cannot measure conversion. A test shows where a persona hesitated, not what percentage of real visitors will buy. Analytics and A/B tests answer that, once you have traffic.
- Emotion and culture are approximated. A persona can report that a page felt untrustworthy. It does not feel anything, and it will miss context a person from a specific community would catch.
- It is not an accessibility audit. A browser driving persona does not use a screen reader or a switch device. Test accessibility with the tools and people built for it.
- It does not find every bug. It finds what a persona runs into on its path. Scripted tests cover paths a persona never takes.
The practical rule that follows: use synthetic findings to decide what to look at, and use real people or real data to decide what is true.
How To Read The Results
The most useful habit in synthetic testing is to ask how many personas shared a finding, and to give each finding an evidence level.
| Evidence | What it means | What to do |
|---|---|---|
| Several personas, same element | A strong lead, especially when their reasons differ | Fix it, then retest |
| Several personas, same theme | A pattern across pages or steps | Look for the shared cause |
| One persona | A question, not a conclusion | Check it against analytics or a real user |
| A recorded dead click or failed step | A fact about the page, independent of any opinion | Fix it |
Then close the loop with real people. The Nielsen Norman Group's long standing guidance is that about five users per distinct audience surfaces most usability problems, and that several small rounds beat one large one. A synthetic round before each real round means the real participants spend their time on the problems a persona could not find.
Synthetic Testing Compared With Other Methods
Synthetic testing sits between scripted tests and real research. It is not a substitute for either.
| Method | Answers | Needs traffic | Weak at |
|---|---|---|---|
| Moderated test with real users | Why a real person struggled, in depth | No, but needs recruiting | Speed and breadth |
| Unmoderated test with real users | Whether real people complete a task | No, but needs recruiting | Depth and follow up |
| Scripted QA | Whether a defined path still works | No | Judgment: a script cannot tell that copy is confusing |
| Analytics and session replay | What real visitors did at scale | Yes | Why they did it, and brand new pages with no data |
| A/B testing | Which version converts better | Yes, a lot | Finding what to test in the first place |
| Synthetic user testing, end to end | Where a defined buyer gets confused, blocked or lost | No | Real emotion, real conversion rates, accessibility with assistive technology |
Because it needs no traffic, it also reaches places the other methods cannot: a page that launched this morning, a pricing page that gets 40 visits a month, a sign-up flow that only exists on staging.
When To Use It
- Before launch. Find the dead ends before the first customer does.
- After a redesign or a big release. A fast check that the new flow makes sense to someone who has never seen it.
- On low traffic sites. You cannot A/B test with 40 visitors a month, but you can have five personas try the page today.
- On apps built quickly. Products assembled with AI builders ship fast and often skip the step where someone new tries to use them.
- Before paying for real research. Fix the obvious problems first so real participants find the subtle ones.
- Between rounds of real testing. Check the fixes before you recruit again.
It is a poor fit when the question is about taste, desirability or willingness to pay. Those need real people.
How To Run Your First Test
You can run a test yourself or order one. Either way, the setup decides how useful the result is.
- Pick one flow and one goalFor example, a first time visitor finding the right plan and starting a trial. One clear goal per persona.
- Define your real segmentsWrite down three to five kinds of buyer, using support tickets, sales calls and sign-up answers rather than guesses.
- Give each persona a different reason to buyFive personas who want the same thing find the same problems. Vary the goal, the device and the level of technical skill.
- Prepare test accounts if there is a sign-inOne throwaway account per persona, with enough trial or credit to finish the task. Never a real customer account.
- Set guard railsList any action a persona must never take, such as a destructive button or a live checkout. Use payment test mode.
- Run, then read by agreementStart with findings shared by several personas, then the recorded failures, then the single persona questions.
- Fix the top findings and run againA rerun on the same personas shows whether the fixes worked.
What A Good Report Contains
Whoever runs the test, judge the report by these eight things. A report missing most of them is an opinion piece.
- Every finding tied to a page and an element, with a screenshot.
- A severity ranking, and how many personas shared each finding.
- The persona's own words for what confused it.
- A suggested fix for each finding, written so someone can act on it.
- A heat map or equivalent showing clicks and scroll depth.
- The raw evidence kept: every screenshot and every recorded click.
- The personas, so you can judge whether they resemble your buyers.
- The limits stated plainly, including that the personas are synthetic.
For more on structure, severity and turning findings into work, see our guide to the website UX audit and report.
Synthetic User Testing For Websites And Apps
The method is the same for a marketing site and a software product, but the questions differ.
- Websites are tested on message clarity, navigation, pricing, trust and the path to a signup, booking or purchase. See website usability testing.
- Web apps and SaaS products are tested on sign-up, setup, the first real task and the second goal, the point where a new user either gets value or leaves. See app usability testing.
Glossary
- Persona
- A described buyer with a role, goals, objections, a device and a behavior profile. In synthetic testing, an AI plays the persona.
- Synthetic user
- An AI participant used in place of a human one. See conversational and behavioral above.
- End to end test
- A test that follows a persona from the first page through a complete goal, such as sign up, setup and first result.
- Heat map
- A layer drawn on a page showing where clicks, scrolling, confusion or drop-off concentrated.
- Dead click
- A click on something that looks interactive and does nothing.
- Drop-off risk
- A moment where a persona said it would leave and not come back.
- Finding
- One recorded observation, with a type, a severity, the page and element it applies to and a suggested fix.
- Agreement
- How many personas reported the same problem. The main signal for ranking findings.
Want this on your own site? We research your business, build the personas, run them in a real browser and deliver a heat map, ranked findings and fixes in 24 to 48 hours.
Order A TestFrequently Asked Questions
What Is Synthetic User Testing?
Synthetic user testing uses AI personas in place of human participants. Each persona has a role, a reason for visiting, goals and objections. In an end to end test the personas use your real site or app in a real browser and the test records what they click, where they stop and what they say confused them. The output is a ranked list of findings with fixes.
Is Synthetic User Testing Accurate?
It is accurate enough to find unclear copy, dead ends, missing information and broken flows, which are problems a persona can notice by trying to finish a task. It is not accurate for predicting conversion rates, emotional reactions or how a specific real customer will behave. Treat a finding that several personas share as a strong lead and a finding from one persona as a question to test.
Can Synthetic Users Replace Real User Testing?
No. Research from the Nielsen Norman Group found that synthetic users should supplement real research, not replace it. A synthetic test is best used before you can reach real users, between rounds of real testing, and on pages and flows you would never otherwise test. Use real people to confirm the findings that matter most.
What Is The Difference Between Synthetic Users And Synthetic Monitoring?
Synthetic monitoring runs scripted checks against a site to see whether it is up and fast. Synthetic user testing uses personas with goals and judgment to find where a human would be confused or leave. Monitoring tells you the page loaded. A synthetic user test tells you whether a first time visitor understood it.
How Many Synthetic Users Do I Need?
Five personas is a sound start for one audience, in line with the Nielsen Norman Group guidance of about five real users per distinct group. If you serve several distinct audiences, give each segment its own personas, and add personas to cover different devices, levels of technical skill and reasons for buying.
Can Synthetic User Testing Work On A Site Behind A Login?
Yes. You provide throwaway test accounts, one per persona, or authorize the tester to create them. Personas then walk sign-up, setup and a first real task. Never share a real customer login or real customer data, and use test mode for anything involving payment.
How Long Does A Synthetic User Test Take?
The personas run quickly, but good research and clear reporting take time. A SynthTest report is delivered within 24 to 48 hours of the access step being complete. A traditional study with recruited participants usually takes longer because of recruiting and scheduling.
Does Synthetic User Testing Help With SEO And AI Search?
Indirectly. A test shows where visitors leave or misread a page, which are the same problems that weaken engagement and conversion. It does not change rankings by itself and no test can promise placement or citation by a search engine or AI assistant.