Removing a membership banner from the cart: +$5.5M revenue
Test run byOptimizelyTesting and CRO tool
Commentary byLiftArchiveOur analysis, not theirsRemoving a banner promoting a membership program from the cart page kept shoppers focused on buying, worth an estimated $5.5M in incremental revenue.
Optimizely ran the test for an online retailer and published it as an Optimizely Field Notes use case. The trade-off: new membership sign-ups fell significantly.
- Incremental revenue: about +$5.5M, as estimated by Optimizely. No conversion rate figure was shared.
- Removing the banner kept shoppers more focused on completing their purchase.
- Copy-only version: no meaningful effect. Rewording the banner was not enough.
- New memberships: fell significantly. This is a real cost to weigh against the revenue gain.
The change
The membership banner is removed from the cart page.
Simplified mockup drawn by LiftArchive from the shared screenshots.
Control
The cart page showed a banner promoting the retailer's membership program.
Variant
The membership banner was removed from the cart page. A second version kept the banner and only changed its copy.
Why it likely won
Our reading of the result, not the practitioner's claim.
- In the cart, shoppers are close to buying, and a banner for something else gives them a reason to click away.
- Removing the banner leaves checkout as the obvious next step.
- Rewording didn't help because the problem was the distraction itself, not the message.
Caveats: Anonymized vendor summary. The revenue figure is Optimizely's estimate; no sample size, duration, conversion rate change or membership figures were shared.
Who ran it
Optimizely ran this test for an online retailer (Optimizely customer) and published the results (Optimizely Field Notes use case). The numbers are theirs; the summary and commentary on this page are ours.
Read the original case study on Optimizely →
How to run this test on your store
- Hypothesis
- If we remove the membership promo banner from the cart page, then more shoppers will complete checkout because nothing pulls their attention away from the purchase.
- Primary metric
- Checkout completion rate
- Guardrails
- New membership sign-ups
- Revenue per visitor
- Long-term value of members
- Who should test it
- Retailers promoting loyalty programs, credit cards or other offers in the cart. If members are worth much more over time, value the lost sign-ups before shipping, or move the offer to the thank-you page.
Questions
Should you promote a loyalty program in the cart?
It can cost sales. In this test, removing a membership banner from the cart raised revenue but cut new sign-ups, so decide which matters more.
Is changing the banner copy enough?
Not here. The copy-only version had no meaningful effect, while removing the banner did.
Where else can a membership offer go?
The source doesn't test this. Common options are the thank-you page or account area, after the purchase is done.
Similar tests
@oliverkenyon@oliverkenyon
Surefoot