Preference Tests are meant to help you choose between options. But a numerical lead is not always a meaningful lead.
A 12–11 split can look like a winner in a simple ranking even when the evidence is too weak to support that conclusion. Until now, you had to judge that strength yourself, then read responses one by one to understand the reasons behind the result.
The goal is to tell you what the evidence actually supports, including when the right conclusion is that the options cannot yet be separated.
Every completed Preference Test gets a Summary above the existing detailed responses. The first thing it tells you is how strong the preference difference is.
p < 0.05. The evidence supports a clear leading option.
p = 0.05 to < 0.20. There is directional evidence, but not a definitive winner.
p ≥ 0.20. The available evidence cannot meaningfully separate the leading options.
A 25–25 split is shown as No separation instead of forcing a winner, with the Summary tab explaining what the evidence actually supports.
The verdict is calculated deterministically outside the language model. AI does not decide whether a result is statistically significant.
The Summary groups the strongest reasons behind each option. Qualitative themes are tied to visible support counts and can include representative quotes, so you can trace the interpretation back to the responses.
Recurring objections and reservations are surfaced alongside the positive reasons. That makes it easier to see whether the leading concept has a fixable weakness before you ship it.
If too few participants support a qualitative theme, the Summary keeps the language directional. Audience differences are shown only when minimum sample requirements are met. Otherwise Wynter explains why the comparison is withheld.
Depending on the result, the Summary can point you toward shipping the supported option, addressing the main objection, collecting a larger sample, testing separate audiences, isolating an unresolved variable, or keeping the existing option when the alternative did not outperform it.
A Preference Test is not a failure when it returns No separation. It tells you that the evidence does not support choosing one option over another yet. The Summary then shows what participants still differed on and where the next test should focus.
No separation is a conclusion, not an error state.
Wynter will not turn the highest raw count into a winner when the statistical evidence does not support it.
The detailed option-level responses remain available below the Summary. The new layer helps you interpret the result faster without hiding the underlying evidence.
Open a completed Preference Test and start with the Summary before reading the detailed responses. Use the verdict to understand how much confidence the result supports, then use the reasons, objections and next step to decide what to do.