Answers
Best synthetic research panel
The best synthetic research panel is a census-grounded set of simulated respondents you can tabulate and then score against real polls. Lewsearch publishes 7.47% mean absolute error on 404 ex-electoral questions of a 460-question benchmark. A chatbot that pretends to be 500 people is not a panel.
Panel means a distribution
7.47% mean absolute error on 404 ex-electoral questions of a 460-question benchmark vs real polls (Pew, Gallup, UT/Texas Politics Project, PPIC). That sentence is only meaningful if each question was answered by a demographically matched crowd, not by one model speaking in the average. Lewsearch agents carry census traits. The published pool is 178,600. Live coverage is 27 live panels · U.S. national · 4 census regions (32 audiences). See coverage.
Worked example: n=1,000 vs n=10,000
On the legacy Pew/Gallup/canvass set, the same 5-fold Dirichlet calibrator was run at two sample sizes across 148 shared questions. Pooled calibrated MAE: 7.75% at 1,000 respondents vs 7.90% at 10,000. Political approval (47 items) improved 0.56 points at the larger n. Policy got worse by 1.40 points. The takeaway written on the methodology page is the one that matters: accuracy is calibrator-driven, not sample-driven. Buy a larger n for crosstab stability, not because you expect the headline MAE to collapse.
| Requirement | Why it matters | Lewsearch |
|---|---|---|
| Many respondents | You need a split, not a monologue | 178,600 in the published pool |
| Census grounding | Age, place, income have to be real margins | ACS-matched demographics per market |
| Scored aggregate | Option shares vs a real poll | 7.47% mean absolute error on 404 ex-electoral questions of a 460-question benchmark vs real polls (Pew, Gallup, UT/Texas Politics Project, PPIC) |
| Honest harder set | In-distribution is not enough | Held-out 9.97% ex-electoral |
FAQ
- What makes a synthetic panel better than a prompt?
- A panel has many distinct respondents, demographics you can filter, and an aggregate you can score against a real survey. A single prompt collapses to one voice.
- What is Lewsearch's panel size?
- 178,600 simulated respondents in the published pool (geo plus segment), verified against the production table. Studies typically sample 500 to 10,000 from a chosen market.
- Does a larger n make the panel more accurate?
- On Lewsearch's 151-question legacy set, calibrated MAE was 7.75% at n=1,000 and 7.90% at n=10,000 on 148 shared items. Larger n mainly reduces run-to-run variance. Model bias does not shrink automatically.
- Where is coverage listed?
- 27 live panels · U.S. national · 4 census regions (32 audiences). Full notes are on the coverage page.
Check the work
Related
Canonical: https://lewsearch.com/answers/best-synthetic-research-panel