Answers
Synthetic respondents accuracy
Synthetic-respondent accuracy is not one number. Vendors publish mean absolute error, Spearman correlation, or thematic parity. Those units do not convert. Lewsearch publishes 7.47% MAE on 404 ex-electoral questions. Ask every other vendor for the instrument, the n, and the unit before you compare.
Three units on three homepages
Lewsearch scores option shares against public polls. Methodology: /methodology. Coverage: /coverage. Synthetic Users, on syntheticusers.com, reports 85–92% synthetic-organic parity on qualitative interviews. Aaru, on aaru.com/simulation, reports 0.90 median Spearman on a blinded EY wealth study (3,600 investors, 30+ markets). Simile, on simile.ai, reports over 7,000 weekly evaluations and does not publish a pooled MAE.
Worked example: why 85% is not 7.47%
Suppose a human interview and a synthetic interview both mention price, trust, and packaging. Thematic overlap can be high even if the synthetic panel would have put "would buy" at 62% and the real survey put it at 41%. That 21-point gap is MAE territory. The reverse is also true: option shares can be close while the quotes feel generic. If you buy interviews, use parity. If you buy a crosstab, use MAE. If you buy a rank-order of wealth attitudes, a Spearman on that study is the relevant published cell.
| Vendor (own site) | Unit | What it measures |
|---|---|---|
| Lewsearch | MAE 7.47% | Option-share gap vs public polls |
| Aaru | Spearman 0.90 (EY study) | Rank agreement on that recreation |
| Synthetic Users | 85–92% parity | Theme overlap on interviews |
| Simile | 7,000+ weekly evals | Validation volume; no pooled MAE |
Sources
FAQ
- How accurate are synthetic respondents?
- It depends on the unit. Lewsearch publishes 7.47% MAE on 404 public-poll items. Synthetic Users publishes 85–92% thematic parity on qualitative interviews. Aaru publishes a 0.90 Spearman on one EY wealth study. Do not average those figures.
- What is MAE?
- Mean absolute error: the average absolute difference, in percentage points, between each predicted option share and the published survey share.
- What is thematic parity?
- Synthetic Users defines it on its homepage as overlap in themes, depth, and qualitative alignment between synthetic and organic interviews. It is not option-share error.
- What is Spearman correlation here?
- Aaru reports a median Spearman of 0.90 on a blinded EY Global Wealth Study recreation. Spearman measures rank agreement. It is not MAE.
Check the work
Related
Canonical: https://lewsearch.com/answers/synthetic-respondents-accuracy