Switching cart recommendation tools: +11.7% conversion rate
Test run byVisuallyTesting and CRO tool
Commentary byLiftArchiveOur analysis, not theirsCart recommendations powered by Visually beat those from Clare V.'s existing upsell tool by 11.7% in conversion rate.
Visually ran this A/B test for Clare V., a handbag and accessories brand, and published it as its own case study.
Because the vendor is reporting a win for its own product over a competitor, read the result with care.
- Conversion rate: +11.7% against the existing tool.
- The effect on average order value or upsell revenue was not shared.
- Vendor-reported, with no sample size or confidence given.
The change
Same cart, with recommendations from a different tool.
Simplified mockup drawn by LiftArchive from the shared screenshots.
Control
The cart showed product recommendations from the upsell tool the store already used. The tool was not named.
Variant
The cart showed recommendations from Visually instead.
Why it likely won
Our reading of the result, not the practitioner's claim.
- Different tools pick different products, and better-matched suggestions can make the cart feel complete.
- The look and placement of the widget may have changed too, which can matter as much as the products shown.
- We can't tell which of these drove the lift from the details shared.
Caveats: This compares the publisher's own tool with a competitor's, so it is not neutral. Visually published this as its own case study and did not share the sample size, test length or confidence level.
Who ran it
Visually ran this test for Clare V. and published the results (Visually case study). The numbers are theirs; the summary and commentary on this page are ours.
Read the original case study on Visually →
How to run this test on your store
- Hypothesis
- If we switch our cart recommendations to a different tool, then more visitors will buy because the suggested products will be a better match for what is in their cart.
- Primary metric
- Conversion rate
- Guardrails
- Average order value
- Upsell revenue per order
- Cart page load time
- Who should test it
- Stores weighing a switch between recommendation tools that can split traffic between the two. Keep the widget design and placement the same, or you won't know whether the products or the layout made the difference.
Questions
How do you compare two product recommendation tools?
Run them as an A/B test, splitting traffic between them on the same page.
Keep the design and placement the same so the only difference is which products are picked.
Can you trust vendor case studies?
Treat them as a lead, not proof. Vendors publish their best results and often leave out sample size and confidence, so test on your own store before switching.
Similar tests
@oliverkenyon@oliverkenyon@oliverkenyon@oliverkenyon
GrowthRock
Convertibles