Can synthetic research track changes over time?

First, define what your tracker needs to measure

A buyer might want to know whether consideration for an offer is rising in a target market, or whether particular customers changed their minds. Those require different evidence. Define whether the target is the population at each date or a fixed group followed from the start. A population share can change as people enter or leave the group; a flat share can hide individuals moving in opposite directions. Following the same human respondents can reveal individual transitions, subject to the panel's design and limitations.

Pew Research Center's questionnaire guidance distinguishes repeated cross-sectional surveys from panels that survey the same people over time. It also notes that panel membership can change. Write the intended target into the buying brief: population trend, subgroup trend, or individual change.

Keeping the same simulated identities across runs does not establish that the system reproduces how real individuals change. For that claim, request a separate evaluation against human transitions. Aggregate agreement alone cannot supply it.

Score the movement as well as each wave

For each agreed answer share, calculate the later-wave value minus the earlier-wave value in both datasets. Compare those signed changes, then show the absolute difference between them in percentage points. Keep the original shares visible: a method can reproduce a change while remaining consistently too high or too low.

Illustrative example. All numbers below are invented to explain scoring. They are not a Lewsearch result, customer result, or observed human study.

Synthetic A matches the six-point increase but overstates the share by ten points in each wave. Synthetic B matches the first share exactly and misses the increase. Their absolute errors on the change are zero and six points, respectively. Neither comparison alone establishes fitness for a business decision.

A published example illustrates the distinction. In Pew's September 2026 analysis of timely questions, the human presidential-approval estimate was 37% in January and 34% in April 2026. The synthetic replications returned 46% in both waves. Those figures use Pew's experiment and human comparison subset. Pew's methodology records that replications ran after human fieldwork and that the January wave was used for model configuration testing. Treat this as a retrospective illustration. Pew did not evaluate Lewsearch.

Keep a method record beside the trend line

An apparent movement can be hard to interpret if the questionnaire, target audience, weighting, or generation method also changed. Pew advises retaining question wording and a comparable question context when measuring trends. For a synthetic tracker, our proposed buyer checklist adds a record of the simulation method at each wave:

• Exact question wording, answer options, stimuli, routing, and question order or randomization rules.
• Audience definitions, subgroup bases, weighting rules, and any changes to panel composition.
• Model and method version, run dates, and any changes to prompts, settings, calibration, or supplied context.
• The date the run is meant to represent, what information was available then, and when that information entered the run.

These are proposed evaluation records, not a request for a vendor's private implementation. A useful change log can describe what changed and when without disclosing proprietary code.

Model choice deserves attention. Pew's model comparison found large systematic differences between synthetic samples built with two different models using otherwise identical construction. It was not an experiment on a commercial tracker's upgrade. Our implication for buyers is to test an upgrade's effect before joining its results to an existing series.

Use an overlap check when the method changes

Plan an overlap before a method is retired: run the old and new methods against the same instrument, audience, and information cutoff for the same wave. If generation varies between runs, repeat both methods. If feasible, repeat the comparison on several relevant waves. Compare both methods with the corresponding human references. If the old method is unavailable, disclose the break and evaluate the new method separately.

That comparison helps expose differences associated with the method change. It does not prove future accuracy or justify silently subtracting a fixed adjustment from every result. Keep the original series and any revised series identifiable.

Repeated synthetic runs with the same inputs can also show how much the output varies under the run procedure. That is a repeatability check. Even highly repeatable output can miss human change. A synthetic-only repeat does not measure the uncertainty of a human population trend.

Require a test that includes rising, falling, and stable measures

Reserve target human outcomes from tuning and lock each synthetic delivery before those outcomes are revealed. Use earlier waves as inputs only as agreed, and keep later-wave information out of earlier predictions. If testing historical waves, limit supplied inputs to information available at the target date and disclose possible training-data exposure. A historical date in a prompt does not establish that later knowledge has been excluded. A prospective comparison, with outputs locked before the human findings become available, gives a clearer record of timing.

Include all agreed measures and intervals. A tracker should be evaluated for false movement on stable measures as well as missed or reversed movement. Two waves provide one interval; they offer limited evidence about performance over a continuing tracker. Seek several intervals relevant to the intended use, rather than selecting only dramatic changes after seeing the answers. For a stability claim, define a range of practically negligible changes in advance and check whether the uncertainty interval for human change lies within it. A statistically non-significant difference alone does not establish stability.

Report the human reference's recruitment, field dates, unweighted sample sizes, weights, and uncertainty. Ask for uncertainty on the change itself, using a method appropriate to the human sampling design. For repeated or partly overlapping respondents, account for the dependence between waves instead of treating the samples as independent. Report how many people answered both waves and how attrition and weighting were handled. Small observed differences may remain inconclusive. Choose decision tolerances before reviewing results; this guide sets no universal passing threshold.

AAPOR's disclosure standards cover instruments, population definitions, data-collection dates, sample sizes, weighting, and precision. Use those disclosures to understand the human comparison. They do not certify a synthetic tracker.

Make the buying rule explicit

State which movements would trigger a decision, how much error is acceptable, which audiences must pass, and what would make a result inconclusive. A failed priority segment or an unresolved method break should limit the uses you approve. If a tracker will determine spending or a launch decision, require human evidence strong enough to assess that decision. Even when synthetic and human changes agree, a trend alone does not establish that a campaign or launch caused it. Causal attribution needs an evaluation design that addresses what would have happened without the intervention.

For an initial evaluation, use the synthetic-panel validation guide. Add the wave-level checks above if the purchase is intended for ongoing tracking. For a Lewsearch discussion, read the Blind Challenge scope and agree separately on what a longitudinal evaluation would need to establish.

FAQ

Does persistent simulated identity prove longitudinal accuracy?

No. Reusing an identity links simulated records. Evidence that real individuals change in the same way requires comparison with human transitions. Evidence about population trends requires comparison across human survey waves.

Can a model update change a synthetic trend?

It can change the measurement method. Record the update and compare old and new methods under matched conditions before interpreting the joined series as population movement.

Is matching the direction of change enough?

Direction alone leaves the size of movement and the accuracy of each wave untested. Report shares, signed changes, errors on those changes, and uncertainty in the human comparison. Judge them against the intended decision.