Moving chat pop-up off key buttons: +44.5% conversion
Test run byClean CommitCRO agency
Commentary byLiftArchiveOur analysis, not theirsMoving a chat pop-up so it no longer covered add-to-cart, cart, checkout and menu buttons raised conversion rate by 44.5% and revenue per visitor by 55.9%.
Clean Commit ran this test for an anonymized Shopify store and published it in its A/B test library.
A separate site pop-up was switched off at the same time, so the gain belongs to both changes together.
- Conversion rate: +44.5%, with a 99.9% probability of beating the original.
- Revenue per visitor: +55.9% and orders +43.9%.
- Add-to-cart rate: +27.9%.
- Mobile (28,513 of 30,122 visitors): conversion rate +34.3%.
- Average order value: +6.0%.
The change
Chat moves off key buttons and minimizes; the second pop-up is removed.
Simplified mockup drawn by LiftArchive from the shared screenshots.
Control
A chat pop-up sat on top of key buttons, including add to cart, the mini cart, checkout and navigation.
Variant
The chat moved away from those buttons, used a smaller icon on mobile, and stayed minimized by default or minimized itself after a short time. Another site pop-up was also turned off.
Why it likely won
Our reading of the result, not the practitioner's claim.
- A widget covering the add-to-cart or checkout button blocks the very action a shopper is trying to take.
- On small screens a large chat icon takes up space that matters, so shrinking it frees up the page.
- Turning off a second pop-up removed another interruption, which likely added to the effect.
Caveats: An 11-day test with only 28% statistical power, so the true lift is likely smaller than reported. A second pop-up was disabled at the same time, so this is a bundled change.
Who ran it
Clean Commit ran this test for a Shopify store (anonymized) and published the results (Clean Commit A/B test library). The numbers are theirs; the summary and commentary on this page are ours.
Read the original case study on Clean Commit →
How to run this test on your store
- Hypothesis
- If we move our chat widget away from add-to-cart, cart and checkout buttons and keep it minimized, then conversion rate will rise because shoppers can reach key actions without obstruction.
- Primary metric
- Conversion rate
- Guardrails
- Chat conversations started
- Add-to-cart rate
- Support tickets
- Who should test it
- Any store with a chat widget or pop-up that overlaps buttons, especially on mobile. Check on real phones first; if nothing overlaps, expect a much smaller effect.
Questions
Can a chat widget hurt conversion rate?
It can if it covers buttons shoppers need.
In this test moving it, along with turning off another pop-up, lifted conversion 44.5%.
Where should a chat icon go on mobile?
Over empty space, away from add-to-cart, cart and checkout buttons.
Keep it small and minimized until someone taps it.
Similar tests
@oliverkenyon@oliverkenyon
OptiMonk
ConvertiblesConvertibles
@DaveDiederen