Card sorting is a research method where participants group labeled cards (one per page, product type or feature) into categories that make sense to them, so you learn how customers expect your content to be organized.
Run it before you design or rebuild navigation, then check the structure it suggests with a tree test and confirm the business impact with an A/B test.
This page sits in our customer research section.
It covers how both methods work, how many participants you need, when to pick each one, and what real navigation A/B tests from our database show when stores change their menu structure.
How card sorting works
Nielsen Norman Group defines card sorting as a method in which "study participants place individually labeled cards into groups according to criteria that make the most sense to them" (NN/g).
Each card holds one item: a product category, a help article, a plan feature.
There are three formats:
- Open card sort: participants create and name their own groups. Use it when you are building a structure from scratch and want to hear customers' own words for categories.
- Closed card sort: you give participants the categories and they place cards into them. Use it to check whether an existing or proposed set of categories holds up.
- Hybrid card sort: some categories are given and participants can add more. NN/g generally advises against it because the given categories bias the new ones.
How many participants you need
Card sorting needs more people than a typical usability test, because people's mental models vary a lot.
Tom Tullis and Larry Wood took card sorts from 168 users and compared random subsets against the full group.
The results correlated at 0.75 with 5 users, 0.90 with 15, 0.95 with 30 and 0.98 with 60 (NN/g summary of Tullis and Wood, UPA 2004).
So NN/g recommends at least 15 participants for a qualitative sort where you are learning why people group things, and 30 to 50 or more for a quantitative sort whose results you want to generalize.
Where card sorting falls short
- It strips out context. Participants see text labels, not the images, page layouts or "information scent" they would see on the live site.
- It usually gives you one level of grouping, not a full hierarchy with subcategories.
- Around 50 cards is the practical limit before participants get tired, so large catalogs have to be sampled.
- Results rarely hand you a finished menu. NN/g notes that card sorting "does not usually produce the exact categorization scheme you should follow." You still have to make design decisions.
One more caution for ecommerce: a card sort shows how people group items.
It does not show which groups make you money.
A grouping that matches customers' mental models can still bury your best sellers.
Tree testing
Tree testing checks whether people can find things in a category structure.
NN/g describes it as having "users find the locations in the tree where specific resources or features can be found" (NN/g).
Participants see only a text version of your menu hierarchy, with no visual design, and get tasks such as "Where would you find a replacement filter for your coffee maker?"
The method is also called reverse card sorting.
It grew out of a paper-based technique Donna Spencer described as "card-based classification," where index cards stood in for the branches of a site's structure (Dave O'Brien, Boxes and Arrows, 2009).
What a tree test measures
| Metric | What it tells you |
|---|---|
| Success rate | Share of participants who ended at a correct location for the task |
| Directness | Share who got there without backtracking, a proxy for how confident they felt |
| Time on task | How long it took to choose an answer |
| Paths and first clicks | Which wrong branches pulled people in, and where they started |
Sample size and task count
Dave O'Brien, whose team built one of the early tree testing tools, recommends about 50 participants per user group.
At 30, patterns start to show but small differences are hard to separate from noise.
Past a few hundred, returns diminish (Tree Testing for Websites).
O'Brien also suggests about 10 tasks per participant, shown in random order.
After several tasks, participants start to remember where things are, which inflates later scores.
A 10-task test on a medium-sized tree usually takes 5 to 10 minutes (Boxes and Arrows).
Worked example
O'Brien reports a government project where the old structure scored a 31% success rate and the redesigned structure scored 67%.
Here is how to read numbers like that.
With 50 participants per tree, 31% is about 16 of 50 people succeeding and 67% is about 34 of 50. A rough 95% interval for a proportion is p ± 1.96 × √(p × (1 − p) / n).
For 31% with n = 50 that is 0.31 ± 1.96 × √(0.31 × 0.69 / 50) = 0.31 ± 0.13, so roughly 18% to 44%.
For 67% it is about 54% to 80%.
The intervals do not overlap, so the gap is not noise.
Run the same test with 15 people per tree and the intervals widen to about ±23 to 25 points.
A 10-point gap between two trees would then tell you nothing.
That is why tree tests need more participants than a moderated usability session.
(The simple formula gets less accurate near 0% or 100% and with very small samples. Use a Wilson interval there.)
Where tree testing falls short
- The bare text tree is not your real menu. Images, mega menu layouts, search and promotional links all change behavior on the live site.
- Most tools only accept leaf nodes as correct answers, even when a parent category has its own useful landing page.
- Unmoderated tests tell you where people went wrong but not why. Add a few moderated sessions or follow-up questions if the failures are puzzling.
Card sorting vs tree testing
The short version: card sorting is generative and tree testing is evaluative.
Card sorting helps you come up with a structure.
Tree testing checks whether a structure works (NN/g).
| Card sorting | Tree testing | |
|---|---|---|
| Question it answers | How do customers group and name our content? | Can customers find things in this structure? |
| When to run it | Early, before a structure exists or during a rebuild | After you have a draft or existing structure |
| What participants do | Group cards, optionally name the groups | Click through a text-only hierarchy to complete tasks |
| Typical sample | 15 qualitative; 30-50+ quantitative | About 50 per user group |
| Main output | Groupings, labels, similarity patterns | Success rate, directness, problem branches |
| Wrong choice when | You already have a structure and need to know if it works | You have no structure yet, or your problem is visual design rather than labels and grouping |
The usual order is open card sort, then draft a structure, then tree test it (ideally against your current tree so you have a baseline), revise, and tree test again.
Neither method replaces an A/B test. Both measure findability in a lab-like setting, not revenue on the live site.
Skip both when the problem is not structure.
If customers find the right category but bounce from it, look at the product listing page.
If most of your visitors search instead of browsing, work on site search first.
What navigation A/B tests show
We have no tests in the database that report a card sort or tree test result alongside the A/B test that followed.
What we do have are many tests where a store changed how its menu groups and labels products, which is the kind of change card sorting informs. Browse them all under navigation menu tests.
Grouping by how customers think
Stone Creek Coffee, tested by Blend Commerce, replaced standard coffee categories with groups based on flavor profile (Bright, Balanced, Bold).
That variant lifted revenue per visitor 11%, conversion 2% and AOV 9% in an A/B/C test.
A second variant that added explanatory copy hurt overall purchasing.
Getting the grouping right helped, but adding more words to the menu did not.
Clearer top-level labels
Enavi tested Xtrema's menu, swapping a generic "Shop" dropdown for its top categories shown directly in the menu.
Product views rose 16.7%. That is an engagement metric; the source does not give a purchase result for this test alone.
Conversion Fanatics replaced hover dropdowns full of sub-options with a flatter menu that linked straight to the main products, removing a level of clicks.
Successful checkouts rose almost 12%, though no sample size was published.
Depth is not always the enemy
Fewer clicks is not a rule. Frictionless Commerce sent Body Ecology's "Products" link to a full category page with a short intro for each category, instead of a dropdown that listed only names.
Revenue per visitor rose 56.43% over two weeks at 96% significance, even though it added a step.
The agency noted the extra step might slow down repeat customers.
On mobile, Blend Commerce found a drill-down menu, where each category opens its own panel, beat an accordion by 7.25% in conversion for a sports nutrition brand.
A tree test can't tell you this. It tests the hierarchy, not the way the menu opens.
Results that don't repeat
Clean Commit ran the same mobile menu revert twice for one store.
One run showed +5.4% conversion but -13.8% revenue per visitor on 8,086 mobile visitors over 16 days.
The other showed -5.7% conversion and -21.1% revenue per visitor.
Neither run was significant. Navigation changes often have small effects that a short test can't measure reliably, which is one more reason to do the cheap structural research first.
SplitBase's research-led redesign for Fontana Candle Co. cut the taps needed to reach best sellers on mobile from four to one or two, and conversion went from 4.88% to 5.43%.
That was a before-and-after comparison of a full redesign, not a split test, so the menu's share of the gain is unknown.
Blend CommerceBlend Commerce
Conversion Fanatics
Clean CommitFrom research to a menu you can A/B test
- Pick your cards. Take 30 to 50 items that represent the catalog: main categories, high-traffic products, and the help pages people ask support about.
- Run an open card sort with at least 15 customers, not staff. Note the labels they invent; customers' words often beat internal category names.
- Draft two structures. One that follows the card sort, and your current menu as the baseline.
- Tree test both with about 50 participants each and about 10 tasks based on your top revenue and support journeys. Fix the branches where success or directness is low.
- Design and A/B test the winner on the live site, measuring revenue per visitor, not only clicks into the menu. Use our navigation menu element page for design patterns that have won before.