A ranking question asks survey respondents to put a short list of items in order, usually by importance or preference, which forces them to show what they'd prioritize when they can't have everything. Used well, ranking questions cut through the "everything is important" problem of rating scales. Used badly, with long lists and vague criteria, they produce confident-looking data that means very little.

Here's when to use ranking questions, how to design them, how to analyze them, and when to pick a different question type.

The short version

• Keep lists short. Most survey guidance tops out at 5 to 7 items. Qualtrics notes people rank the top three and bottom three items reliably, and the middle much less so (Qualtrics).
• Ranking shows order, not intensity. Item #1 might matter a lot more than #2, or barely more. Pair it with a "why" question.
• Format matters. In one Qualtrics experiment, 16% of respondents using numeric-entry ranking entered invalid answers, like ties or out-of-range numbers (Journal of User Experience). Use drag-and-drop or click-to-rank.
• Report "% ranked first" and top-3 share, not just average rank. Averages hide polarized opinions.
• For long lists, don't rank. Use "pick your top 3" or a specialist method like MaxDiff.

When to use a ranking question

Use ranking when you need a forced trade-off between a few comparable options. Good B2B examples:

• Which of these problems matters most to your team this year?
• Rank these vendor selection criteria by how much they influenced your last purchase.
• Rank these features by how much they'd change your day-to-day work.
• Rank these value propositions by how compelling they are to you.

We use this ourselves. In our 2026 study of how B2B SaaS CMOs buy software, word of mouth was ranked the #1 consideration factor by 42% of CMOs. A rating scale would've told us word of mouth is "important." Everyone says that. Ranking showed it beats everything else.

Don't use ranking when items aren't comparable (ranking "price" against "our brand colors"), when you need to know how much someone cares about each item, or when you plan to track the same items quarter over quarter. Rating scales work better for those.

Ranking vs. rating and other alternatives

Survey methodologists have argued about ranking vs. rating for decades. Early work by Krosnick and Alwin found ranking can reduce the "everything gets a 9" problem, while later studies found rating scales often have equal or better validity. The practical answer: rank when the decision is a trade-off, rate when you need a measurement. For a deeper look at when numbers beat words and vice versa, read qualitative vs. quantitative research.

How to write a good ranking question

1. Name one criterion. "Rank by importance to your team this year" beats "rank these." If people don't know what "best" means, they'll each use their own definition.
2. Use 5 to 7 items. More than that and the middle of the list turns to noise. If you have 15 candidates, cut them with an open-ended question or a pilot first.
3. Make items mutually exclusive. "Ease of use" and "intuitive interface" are the same thing. Respondents will split votes between them and both will look weaker.
4. Write items in buyer language. Use the words buyers used in interviews, not your internal feature names. Our list of 40 qualitative research questions helps you collect them.
5. Randomize the order. Items at the top of a list get chosen more just for being there. Shuffle per respondent.
6. Add "other" if the list might be incomplete, with a text box.
7. Follow with "why." One open-ended question after the ranking ("Why did you rank #1 first?") turns an order into an insight.
8. Test on mobile. Drag-and-drop on a phone is fiddly. If it's painful for you, it's painful for a VP answering between meetings.

10 ranking question examples for B2B surveys

Copy these and swap in your own items. Each one has a single criterion and a short list, and each is worth following with "Why did you rank your #1 first?"

1. Problem priority: "Rank these challenges by how much they'll affect your team's results this year." (5 to 6 problems from your interviews)
2. Vendor selection criteria: "Thinking about the last [category] tool you bought, rank these factors by how much they influenced the final decision."
3. Shortlist sources: "Rank these sources by how much they influenced which vendors made your shortlist." (peers, review sites, AI tools, analyst reports, vendor websites, sales outreach)
4. Value propositions: "Rank these statements by how compelling they'd be if a vendor said them to you."
5. Feature priority: "Rank these capabilities by how much they'd change your day-to-day work."
6. Objections: "Rank these concerns by how likely they'd be to stop you from buying a tool like this."
7. Proof that persuades: "Rank these types of proof by how much they'd increase your trust in a new vendor." (customer logos, case studies with numbers, peer reviews, free trial, analyst recognition)
8. Budget trade-offs: "If your budget were cut, rank these tools from first to cut to last to cut."
9. Content formats: "Rank these formats by how likely you'd be to read or watch them when researching a purchase."
10. Buying group influence: "Rank these roles by how much say they had in your last purchase of [category]."

Notice what the list doesn't include: "rank these 15 features." If your list is longer than seven, cut it with an open-ended question first or switch to pick-top-3.

How to analyze ranking data

Two cautions. Ranking data is ordinal, so the gap between #1 and #2 isn't the same as between #4 and #5. And with small samples, don't over-read small differences. At 100 respondents, a share like 42% carries a margin of error of roughly ±10 points (sample size math here).

Segment where it matters. In B2B, a CFO and a Head of Marketing often rank the same list completely differently. Averaging them into one ranking can hide the only finding that matters.

A worked example: reading ranking results

Here's a hypothetical result from 100 RevOps leaders ranking four value propositions. The numbers are made up to show how to read them, not real data.

What I'd conclude:

• A wins on "% ranked #1," but B is the safer lead. The gap at #1 is 4 points, well inside a ±10-point margin of error at 100 respondents. B has the best top-2 share and the best average rank. It's the message almost nobody dislikes.
• A might be a segment story. A strong #1 share with a weaker top-2 share often means one group loves it and others don't. Split by company size or role before deciding.
• C is a qualifier, not a lead. Everyone in the category claims AI. Its low ranking says buyers don't choose on it. Mention it, don't headline it. (More on this in our guide to competitive messaging analysis.)
• D is a table-stakes item. Rarely ranked first but often ranked second: buyers need it, they just don't get excited about it. Put it in the proof section, not the hero.

Then read the "why" answers. If the people who ranked A first all say something like "our board meetings are a nightmare," you've found the trigger to build a campaign around. Before you rewrite the homepage around the winner, put the new version in front of your ICP with a message test to check it reads the way the ranking suggests.

Common ranking question mistakes

• Ranking 12 items. Nobody can meaningfully order twelve things. You'll get careful top choices and random noise below them.
• Forcing a full ranking when only the top matters. If you only act on the winner, ask "pick your top 3" and save respondents the effort.
• Leaving the criterion vague. "Most important" to whom? For what? This year or ever?
• Treating average rank as the answer. An item ranked first by half and last by half averages out to "middle" and tells you nothing.
• Asking the wrong people. A perfectly designed ranking question answered by people outside your ICP is still useless. Who answers matters more than question format, which is why we built target customer surveys on a verified B2B panel. More on this in our guide to B2B research panels.

When to use MaxDiff instead

If you need to prioritize 10 to 30 items with statistical confidence (a long feature list, a set of messaging claims), MaxDiff is the standard method. Respondents see small sets of items repeatedly and pick the most and least important in each, and the analysis produces a preference score for every item. It's more work to design and analyze, so it's usually run with specialist tools or agencies. Wynter doesn't run MaxDiff studies. For fewer items, a well-built ranking question plus open-ended "why" gets you most of the way.

Frequently asked questions

What is a ranking question in a survey?

A question that asks respondents to order a list of items by a criterion such as importance, preference or likelihood, so that each item gets a unique position.

How many items should a ranking question have?

Five to seven is the practical limit. Respondents rank the top and bottom items reliably, but the middle of longer lists becomes unreliable.

What's the difference between ranking and rating questions?

Rating questions score each item independently on a scale, so many items can score the same. Ranking questions force respondents to compare items and give each a unique position, which reveals priorities but not intensity.

How do you analyze ranking question results?

Report the percentage who ranked each item first, the share who placed it in their top two or three, and the average rank. Check the distribution for polarized items, and segment by role or company size where opinions might differ.

Rank less, learn more

A short list, one clear criterion and a "why" afterward. That's a ranking question that actually informs a decision. If you want the ranking answered by people who match your ICP, not whoever's on a generic panel, run it as a Wynter survey.