Concept testing methods: how to choose the right design
Compare the four concept testing methods, see the sample size each design needs, and use a decision tree to pick the right one for your launch.
Summary:
- Choosing the right concept testing method—monadic, sequential monadic, comparative, or proto-monadic—is critical because it determines your sample size, costs, and timeline while significantly influencing the test outcome by controlling for bias.
- The primary factors for selecting a method are the need for a "clean" read versus practical constraints.
- Effective execution requires defining clear metrics beforehand, using standard sample sizes, and rigorously applying statistical analysis to ensure differences between concept scores are real rather than noise.
Most teams pick a concept testing method by accident. Someone builds a survey, drops three product ideas into it, and asks which one people like best. A winner emerges, and nobody notices that the design itself decided the outcome.
The four concept testing methods are monadic, sequential monadic, comparative, and proto-monadic. Choosing between them sets your sample size, your fieldwork cost, your timeline, and how much order bias you absorb. This guide covers the selection logic, the sample math behind each design, and how to prove the gap between your top two concepts is real rather than noise.
The four concept testing methods, and the two variables that decide between them
Every concept testing method is a rule about how many stimuli one respondent sees, and in what arrangement. That rule cascades into everything else.
- Monadic. One respondent sees one concept and answers your full metric battery on it. No comparison is possible, so no comparison bias exists.
- Sequential monadic. One respondent sees several concepts one at a time in randomized order, answering the same battery after each.
- Comparative. One respondent sees two or three concepts side by side and rates or picks between them.
- Proto-monadic. A hybrid: sequential evaluation first, then a forced choice at the end.
For worked illustrations of each type, the ultimate concept testing guide covers that ground. What it does not tell you, and what usually goes wrong in practice, is which one to run.
Two variables settle that question almost every time:
- How clean does the read on each concept need to be? A monadic read is the closest you get to a real market moment, because customers meet a new product without its rejected siblings beside it. When the decision is expensive and irreversible, such as tooling, print runs, or a brand name, you want the clean read.
- How many concepts do you have, and what is your fieldwork budget? Monadic sample requirements scale linearly with concept count. Every other design reuses respondents, cheaper and faster but with comparison effects baked in.
Metrics, panel, and incentives are secondary. Get those two right and the method chooses itself.
Skip the debate. Test the concepts.
Product Concept Testing in SurveyMonkey LaunchPad runs the monadic methodology this guide covers. Test up to 10 concepts and get a scorecard back in hours.
How to run each concept testing method, stage by stage
These stages apply to all four designs. Only stage two changes with the method you select.
Stage 1: Define the decision and pick your metrics (appeal, uniqueness, purchase intent)
Write down the decision the test will settle before you write a single question. "Which of these six packaging designs goes to print" is a decision. "How do people feel about our packaging" is not.
Then pick three or four metrics and hold them constant across every concept. The standard battery is appeal, uniqueness, purchase intent, and relevance, each on a five-point or seven-point scale.
Resist a fifth and sixth metric. Every extra metric multiplies against concept count in a sequential design, and survey length is the constraint that quietly ruins concept tests.
Stage 2: Choose your design: monadic, sequential monadic, comparative, or proto-monadic
Match your situation to a design:
- Choose monadic when concepts are similar enough that seeing one contaminates the read on the next, when the decision is high stakes, or when you need a benchmarkable score. Monadic is widely treated as the gold standard for that reason.
- Choose sequential monadic for three to six concepts on a fixed budget, distinct enough that respondents will not blur them together. Randomize order so position effects spread evenly instead of favoring whichever concept appears first.
- Choose comparative when the question genuinely is "which of these two," such as two price points on an identical offer. Cap it at two or three concepts; Contentsquare recommends that limit, and beyond three the grid becomes a chore.
- Choose proto-monadic when you need both an unbiased score and a forced choice, and want to see whether the two agree.
Randomization and rotation mechanics for the first two designs are covered in the guide to monadic versus sequential monadic survey design.
Stage 3: Size your sample per cell
A cell is a group of respondents who see the same thing. Plan on roughly 200 per cell. That is the working standard because it separates concepts that are meaningfully different without paying for precision you cannot act on.
A tighter margin of error, for instance on line extensions differing by a few points, requires more.
Monadic tests need a separate cell for every concept. Sequential monadic and proto-monadic tests get multiple concept reads out of a single cell.
| Concepts | Monadic completes | Sequential monadic completes | Comparative completes | Proto-monadic completes |
| 2 | 400 | 200 | 200 | 200 |
| 3 | 600 | 200 | 200 | 200 |
| 4 | 800 | 200 | Not advised | 200 |
| 6 | 1,200 | 400 (split rotation) | Not advised | 400 (split rotation) |
| 10 | 2,000 | Not advised | Not advised | Not advised |
Two things sit in that table.
The cost curve first: two concepts monadically means 400 completes, while a sequential monadic design covering four concepts pulls 800 concept-level evaluations out of one 200-person cell.
Then the ceiling. Sequential monadic breaks on survey length, so keep concepts multiplied by metrics under 30 questions. Four concepts across four metrics is 16 questions and works. Ten across four is 40 and will produce garbage, because respondents stop reading by concept six.
Past that ceiling, use split rotation: show each respondent a randomized subset, then raise total completes so every concept still lands about 200 reads. Six concepts, three each, means 400 completes.
Stage 4: Build the stimuli and randomization to neutralize order bias
Hold everything constant except the variable you are testing. In a packaging test, every concept gets the same product name, price, image dimensions, and length of descriptive copy.
A concept that wins because its description ran 40 words longer has told you nothing.
Where a respondent sees more than one concept, randomize position at the respondent level, not the session level. Check the distribution after fielding: if concept C appeared first for 62% of respondents, your randomization is broken and your winner is suspect.
Cap the stimuli count too. Quantilope advises six concepts at most in one test, which lines up with the survey length math above.
Stage 5: Field it and check statistical significance between concepts
This is the stage almost every guide skips.
Your test produces two numbers, and you have to decide whether the gap between them is real. Start with the Top 2 Box scores for each concept: the share of respondents choosing the top two points on your appeal or purchase intent scale.
Then run a two-proportion z-test.
- Standard error of the difference: the square root of [p1(1 minus p1) divided by n1, plus p2(1 minus p2) divided by n2], where p is each proportion and n each cell size.
- Divide the difference between the two proportions by that standard error to get a z-score.
- A z-score above 1.96 is significant at 95% confidence.
Work it through at 200 per cell. Concept A scores 55% Top 2 Box, concept B scores 45%. The standard error is about 0.050, the difference is 0.10, and the z-score is roughly 2.01.
That clears 1.96, so the 10-point gap is real. Change the numbers to 50% and 45% and the z-score falls near 1.00, which makes that five-point gap noise.
At 200 per cell, treat differences under roughly nine points in Top 2 Box as too close to call, and pick between those concepts on strategic grounds instead.
One adjustment matters a great deal. In a sequential monadic or proto-monadic design the same people rated both concepts, so the observations are paired.
Use a paired test: McNemar's test for Top 2 Box, or a paired t-test for mean scores. Paired tests carry more power than independent tests, a real and rarely claimed advantage of reusing respondents.
Run an independent-samples test on paired data and you will declare fewer winners than your data supports.
Correct for multiple comparisons too. Six concepts against each other is 15 pairwise tests, and at 95% confidence roughly one false positive is expected by chance alone. Compare every concept against the leader, or apply a Bonferroni adjustment.
Stage 6: Score with Top 2 Box and build the scorecard
Convert each metric to Top 2 Box, put concepts in rows and metrics in columns, and mark every cell significantly different from the leader. Your stakeholders can read that in 30 seconds.
Do not average the metrics into a composite score. A concept high on appeal and low on uniqueness is a different strategic bet than the reverse, and averaging destroys exactly the information you commissioned the test to get.
Validating the winner in use is a separate exercise, and product testing research picks up there.
How to choose a concept testing method for your budget, timeline, and number of concepts
Work down this decision tree in order. The first branch that matches is your answer.
1. Do you have more than six concepts?
Yes: do not run a concept test yet. Run a MaxDiff analysis to narrow the field first, then concept test the survivors.
MaxDiff shows respondents blocks of three to five items drawn from a pool of eight to 30, and needs 200 to 300 respondents per segment. It ranks a long list efficiently but gives no diagnostic read on why a concept works, which is why it screens rather than replaces the concept test.
Conjoint replaces a concept test in a different situation: when your concepts are attribute bundles such as price, size, and feature set, and you need the value of each component rather than the appeal of each bundle.
No: continue.
2. Is the decision expensive or hard to reverse?
Yes, with budget for 200 completes per concept: run monadic and accept the linear cost.
The clean read is what you are paying for. Yes, but budget is capped: run proto-monadic, which gets sequential reads plus a forced choice out of one cell.
No: continue.
3. Are the concepts so similar that seeing one would color the next?
Yes: run monadic, or cut the concept count until monadic is affordable. Sequential designs are least reliable exactly when concepts are close, because respondents anchor on the first one they see.
No: continue.
4. Is the business question a trade-off between two options?
Yes: run comparative. Two or three concepts, side by side, forced choice. No: run sequential monadic with full randomization, the default for three to six distinct concepts on a normal budget.
Timeline mostly tracks sample size: a single 200-complete cell fields fast, and 1,200 completes across six cells costs proportionally more.
Where cell size is the binding constraint, sourcing from an audience panel rather than your own list is what makes a monadic design affordable.
Concept testing in LaunchPad supports up to 10 stimuli with roughly 200 respondents each, drawing on a panel of more than 335 million people, and can return results in as little as an hour.
A worked example: choosing a design to test six packaging concepts
A beverage team has six packaging concepts for a new flavor. Print tooling is committed 10 weeks out, and the budget is 600 completes, a common ceiling for teams running market research solutions in-house rather than through an agency.
1. Rule out monadic first
Six concepts monadically is 1,200 completes, double the budget. Monadic is the right method and the wrong plan.
2. Rule out comparative
Six concepts side by side exceeds the two-to-three cap and becomes a ranking exercise.
3. Check length and similarity
Six concepts across four metrics is 24 questions plus screener, long but under the ceiling.
All six share the same brand mark and differ only in color system and illustration style, which is exactly the condition where full sequential exposure contaminates the read.
4. The design that fits
Split-rotation proto-monadic. Each respondent sees three of the six in randomized order, rates each on the full battery, then picks a favorite from the three they saw.
At 400 completes, every concept collects about 200 reads and 200 completes stay in reserve.
5. What the analysis produces
Concepts B and E lead on appeal Top 2 Box at 58% and 51%. That seven-point gap sits below the nine-point threshold at n equals 200, so it is not significant.
The 200 held-back completes then fund a follow-up monadic test on concept B alone, setting a benchmark for the next flavor launch.
But B also wins the forced choice among respondents who saw both, and McNemar's test on that paired comparison comes back significant. B goes to print, on a tiebreaker a pure sequential monadic design would not have produced.
Concept testing methods at a glance
| Method | Concepts shown per respondent | Completes required | Order bias | Best for |
| Monadic | One | 200 per concept | None | High stakes, similar concepts, benchmarkable scores |
| Sequential monadic | All, randomized | 200 total, up to six concepts | Moderate, cut by randomization | Three to six distinct concepts, fixed budget |
| Comparative | Two to three, side by side | 200 total | High by design | Two-way or three-way trade-offs |
| Proto-monadic | All sequentially, then a forced choice | 200 total | Moderate, then high in phase two | A clean score plus a tiebreaker |
| MaxDiff (screener, not a concept test) | Blocks of three to five | 200 to 300 per segment | Low | Narrowing eight to 30 ideas first |
Frequently asked questions about concept testing methods
- What is the difference between monadic and sequential monadic?
- How many respondents do you need per concept?
- Is a monadic or comparative concept test more accurate?
- How do you test statistical significance between two concepts?
- What concept testing method works for six or more concepts?
- What does a proto-monadic test design look like?
- When should you use MaxDiff instead of a concept test?
Pick your method, then run the test
The logic collapses to a short sequence.
Count your concepts and screen with MaxDiff above six. Choose monadic when the decision is expensive or the concepts are close, sequential monadic when they are distinct and the budget is fixed, comparative for a two-way trade-off, and proto-monadic when you need a score plus a tiebreaker.
Size every cell at about 200, and test the gap between your top two before you name a winner.
Apply with a template. Start from the concept testing survey templates to get the metric battery and scale structure already built, then set up your cells and randomization.
If your stimuli are finished products rather than descriptions, the product testing survey template is the closer starting point. When you need the sample and not just the questionnaire, SurveyMonkey LaunchPad handles rotation, panel sourcing, and significance testing for up to 10 concepts.