Rebuy vs Shopify product recommendations: +23% conversion rate
Test run byBlend CommerceCRO agency
Commentary byLiftArchiveOur analysis, not theirsSwapping Shopify's native product recommendations for Rebuy's engine lifted conversion rate by 23% on Stone Creek Coffee's product pages.
Blend Commerce ran this A/B test in Intelligems.
Revenue per visitor rose 25%, but the test ran for only four days and ended early.
- Conversion rate: +23%, the primary metric. Intelligems gave an 89.5% probability the Rebuy version beat control.
- Revenue per visitor: +25%, with an 83% chance of being the best version.
- Subscription revenue per visitor: +84%, a large lift for a coffee brand that sells subscriptions.
- Add-to-cart rate: +12%. Average order value: +2%.
- Abandoned checkout rate: -34%, a good sign that the extra products did not distract buyers.
The change
Same recommendation block and position; only the engine behind it changes.
Simplified mockup drawn by LiftArchive from the shared screenshots.
Control
Product pages showed ten recommended products below the reviews, picked by Shopify's native recommendation API.
Variant
The same ten-product block, in the same design and position, but filled by Rebuy's recommendation engine using its own data.
Why it likely won
Our reading of the result, not the practitioner's claim.
- Only the engine changed, so the gain likely came from more relevant product matches.
- Better matches can support the choice a shopper is already making, instead of adding a new decision.
- Rebuy may draw on richer purchase data than Shopify's default logic, which helps with repeat and subscription buyers.
Caveats: The test covered 3,452 visitors over four days and ended earlier than planned, so the lifts may shrink with more data. An 89.5% probability is below the 95% level many teams use.
Who ran it
Blend Commerce ran this test for Stone Creek Coffee and published the results (Blend Commerce A/B test). The numbers are theirs; the summary and commentary on this page are ours.
Read the original case study on Blend Commerce →
How to run this test on your store
- Hypothesis
- If we fill our PDP recommendation block with a dedicated recommendation engine instead of the platform default, then conversion will rise because shoppers see more relevant products.
- Primary metric
- Conversion rate
- Guardrails
- Revenue per visitor
- Average order value
- App cost versus revenue gained
- Who should test it
- Shopify stores paying for a recommendation app and unsure if it earns its fee. Keep the design and position identical so only the engine differs. Very small catalogues may see little difference between engines.
Questions
Is Rebuy better than Shopify's free product recommendations?
In this one test, Rebuy's picks lifted conversion 23% for Stone Creek Coffee.
The test was short, so results on your store could differ.
How do I fairly test two recommendation engines?
Keep the block's design, position and number of products the same.
Then only the product selection changes, and you can credit the result to the engine.
How long should a recommendation test run?
Run it for at least one to two full weeks so weekday and weekend buyers are included.
This test ended after four days, which is why its numbers are uncertain.
Similar tests
@DaveDiederen@DaveDiederen@DaveDiederen@DaveDiederenBlend CommerceBlend Commerce