Reverting to the old mobile menu: -5.7% conversion, -21.1% RPV
Test run byClean CommitCRO agency
Commentary byLiftArchiveOur analysis, not theirsReverting the mobile menu to an earlier category-focused layout, without login and account buttons, cut conversion rate 5.7% and revenue per visitor 21.1%.
Clean Commit ran this test for an anonymized Shopify store and published it in its A/B test library.
The evidence was moderate rather than conclusive, but it was enough to keep the old layout from shipping.
- Conversion rate: -5.7%. Negative, but not statistically significant: the original had a 66.3% probability of performing better.
- Revenue per visitor: -21.1%, driven by a 16.3% drop in average order value.
- Orders: -7.0%, and add-to-cart rate fell 2.6%.
- Click-through rate: +0.5%, so shoppers used the menu about as often but spent less.
The change
The variant restores the old category-first menu and removes account links.
Simplified mockup drawn by LiftArchive from the shared screenshots.
Control
The current mobile menu, rebuilt after a winning desktop navigation test, with login and account links.
Variant
The mobile menu went back to its earlier layout and order, focused on categories, with login and account buttons removed.
Why it likely lost
Our reading of the result, not the practitioner's claim.
- Shoppers may have got used to the newer menu, so the old layout felt less familiar, not more.
- Removing account and login links may have made it harder for returning customers, who often spend more, to find their way.
- The newer layout came from a desktop win, and this test suggests it also suited mobile better than the old one.
Caveats: The evidence was moderate: a 66.3% probability that the original was better, which is not significant. Sample size and duration were not shared.
Who ran it
Clean Commit ran this test for a Shopify store (anonymized) and published the results (Clean Commit A/B test library). The numbers are theirs; the summary and commentary on this page are ours.
Read the original case study on Clean Commit →
How to run this test on your store
- Hypothesis
- If we bring back the older category-focused mobile menu and drop account links, then more shoppers will buy because the layout is simpler.
- Primary metric
- Mobile conversion rate
- Guardrails
- Revenue per visitor
- Average order value
- Returning-customer conversion rate
- Who should test it
- Stores that are unsure whether a recent navigation change helped mobile. It is a useful check before rolling back, but this result suggests keeping account links for stores with many returning buyers.
Questions
Should I remove login and account links from my mobile menu?
This test suggests caution. Removing them, along with the layout revert, went with lower revenue per visitor, possibly because returning customers lost an easy path.
Does a navigation win on desktop carry over to mobile?
Not automatically. Test mobile on its own, because screen size and shopping habits differ.
Why test a rollback at all?
Testing a rollback shows what the change is really worth.
Here it stopped a move that would likely have cost revenue.
Similar tests
@oliverkenyon@oliverkenyon@oliverkenyon@oliverkenyon@oliverkenyon@oliverkenyon