Category link bar on mobile homepage: +5% orders (inconclusive)
Test run byGrowthRockCRO agency
Commentary byLiftArchiveOur analysis, not theirsAdding a bar of nine category links to the top of the mobile homepage showed a 5% rise in orders for an apparel store with over 1,000 products, but the result was not statistically significant, so we count it as inconclusive.
GrowthRock ran this A/B test and published it on its blog.
Visits to the category pages shown in the bar did rise clearly.
- Orders: +5% at 93% one-tailed significance. Optimizely's two-tailed stats showed no significance, so this is inconclusive.
- Category page views: +10-12% at 99% or higher significance, but only for links visible without scrolling the bar.
- Links hidden off-screen in the scrollable bar showed no change in page views.
- Scale: 28 days, more than 80,000 visitors and about 2,300 conversions per version.
The change
The variant adds a scrollable bar of category links above the hero.
Simplified mockup drawn by LiftArchive from the shared screenshots.
Control
The mobile homepage led with large campaign images. Category links sat behind the hamburger menu, with more links only far down the page.
Variant
A left-right scrollable bar of nine category links, with arrows, was added at the top of the homepage. The hamburger menu stayed in place.
Why it likely came out this way
Our reading of the result, not the practitioner's claim.
- Visible links save a tap and make categories easier to find than a hamburger menu, which sent more shoppers to listing pages.
- More category visits did not clearly turn into more orders, possibly because only the first few links were seen.
- Links that required sideways scrolling were largely ignored, which limited the bar's effect.
Caveats: Not statistically significant by the testing tool's own two-tailed stats; the agency reads it as a likely winner, but we treat it as inconclusive. The test date was not shared.
Who ran it
GrowthRock ran this test for an apparel store with 1,000+ products and published the results (GrowthRock blog). The numbers are theirs; the summary and commentary on this page are ours.
Read the original case study on GrowthRock →
How to run this test on your store
- Hypothesis
- If we add visible category links at the top of the mobile homepage, then more shoppers will reach product listings and order because they skip the hamburger menu.
- Primary metric
- Orders (conversion rate)
- Guardrails
- Category page views
- Homepage bounce rate
- Revenue per visitor
- Who should test it
- Mobile stores with many products and categories that rely on a hamburger menu. Plan for a long test, as the order lift may be small. Consider a stacked layout so all links are visible, since off-screen links were ignored here.
Questions
Is a hamburger menu bad for mobile conversion?
It hides links behind an extra tap. In this test, exposing category links sent more shoppers to category pages, though the lift in orders was not significant.
Should a mobile link bar scroll sideways?
Links hidden off-screen got no extra visits here, while visible links did.
Showing all key links at once may work better.
What is the difference between one-tailed and two-tailed significance?
A two-tailed test checks for both a rise and a drop, so it is stricter.
This result reached 93% one-tailed but no significance two-tailed.
Similar tests
Blend CommerceBlend CommerceBlend CommerceBlend CommerceBlend CommerceGrowthRock