Big red Holiday Sale message vs sidebar: bounce rate 76% to 60%
Commentary byLiftArchiveOur analysis, not theirsShowing the Holiday Sale message in big red type on the homepage, instead of small text in the sidebar, cut the bounce rate from 76% to 60%.
MedaliaArt, an online painting gallery, ran this test itself, and it was published as a VWO case study.
The sample was very small, at 268 visitors.
- Bounce rate: 76% with the sidebar message and 60% with the prominent one, a 21% drop.
- Click rate: 24% (35 of 145 visitors) versus 40% (49 of 123 visitors). Any click on a homepage link counted as a conversion.
- Reported at 95% confidence after 2 weeks, but on only 268 visitors in total.
- Sales were not measured, so the effect on revenue is unknown.
The change
The variant moves the sale message from small sidebar text to a large, red, prominent spot.
Simplified mockup drawn by LiftArchive from the shared screenshots.
Control
The homepage mentioned the Holiday Sale, with 5% to 55% off paintings, in small text in the sidebar.
Variant
The same Holiday Sale message appeared in big red type in a prominent spot on the homepage.
Why it likely won
Our reading of the result, not the practitioner's claim.
- A sale of up to 55% off gives visitors a reason to look around, but only if they notice it.
- Small sidebar text is easy to miss, so many visitors may never have known a sale was on.
- The worry that a loud message would feel intrusive did not show up in the bounce rate.
Caveats: With just 268 visitors, the result could easily shift with more data. The test tracked clicks rather than sales, and no version without the sale message was tested.
Who ran it
MedaliaArt ran this test for MedaliaArt and published the results (VWO case study). The numbers are theirs; the summary and commentary on this page are ours.
Read the original case study on MedaliaArt →
How to run this test on your store
- Hypothesis
- If we show our sale message large and prominent on the homepage instead of in the sidebar, then fewer visitors will bounce because more of them will notice the discount.
- Primary metric
- Revenue per visitor
- Guardrails
- Bounce rate
- Average order value
- Margin per order
- Who should test it
- Stores running a real, sizable sale that is currently easy to miss. It may not help if the sale is small, or if a loud banner pushes your main products out of view.
Questions
Should sale messages be prominent or subtle?
In this small test, the prominent version cut bounce rate from 76% to 60%.
A sale only helps if visitors see it.
Is bounce rate a good metric for an A/B test?
It shows engagement but not buying. Sales or revenue per visitor are better ways to judge a sale message.
How many visitors does an A/B test need?
Usually far more than 268, unless the difference is very large.
Small samples can produce results that vanish later.
Similar tests
@DaveDiederen
Convertibles@DaveDiederen@DaveDiederen@DaveDiederen@DaveDiederen