'What's in the box' preview on subscription buy box: +18% conversion
Test run byConvertiblesCRO agency
Commentary byLiftArchiveOur analysis, not theirsAdding a preview of this month's box contents to a homepage subscription buy box lifted conversion about 18%.
Convertibles ran this A/B/n test for an 8-figure Australian gin subscription brand.
A second version that also showed each item's retail value did no better, so the contents preview was the lever.
- Conversion rate: 4.73% to 5.59% (+18%) without retail values, and 5.60% (+18%) with them.
- Revenue per visitor: A$4.94 to A$5.86 (+19%) and A$5.82 (+18%).
- Probability to beat control was 95.4% and 94.5%. The two variations were a statistical tie.
- Average order value and subscription share stayed flat, so the gain came from new subscribers, not deluxe upsells.
The change
The buy box adds a list of this month's items with thumbnails and an opt-in deluxe upgrade.
Simplified mockup drawn by LiftArchive from the shared screenshots.
Control
The homepage buy box showed the A$99 price, a frequency dropdown, an add to cart button and a refund guarantee. It did not say what was in the box.
Variant
Frequency tabs replaced the dropdown. An "Inside November's Box" list showed item names with thumbnails, plus a +A$30 deluxe upgrade toggle. A second version added each item's retail price.
Why it likely won
Our reading of the result, not the practitioner's claim.
- A box of named items is easier to say yes to than a mystery box.
- Shoppers needed to picture what they would get. Retail value math added little once they could.
- The deluxe toggle was opt-in, so it did not get in the way of the main decision.
Caveats: The test ran two weeks with about 7,000 visitors per arm and 1,147 orders. Frequency tabs, the contents list and the deluxe toggle changed together, so the agency's reading that the preview drove the lift is an inference.
Who ran it
Convertibles ran this test for an 8-figure Australian gin subscription brand and published the results (Convertibles case study). The numbers are theirs; the summary and commentary on this page are ours.
Read the original case study on Convertibles →
How to run this test on your store
- Hypothesis
- If we show what is inside the next box with names and photos, then more visitors will subscribe because they can picture what they are buying.
- Primary metric
- Conversion rate
- Guardrails
- Revenue per visitor
- Average order value
- Subscription share of orders
- Who should test it
- Subscription box brands whose buy box shows a price but not the contents. If contents change too late to show before launch, this is harder to run.
Questions
Should a subscription buy box show what's inside?
In this test it lifted conversion 18%. Item names and thumbnails were enough.
Does showing retail value per item help?
Not here. A version with retail values tied with the one without them.
Should an upgrade option be on or off by default?
This test used an opt-in toggle, and average order value stayed flat.
The lift came from more subscribers, not more upgrades.
Similar tests
Blend Commerce
Optimizely
Visually
@joshtCRO