Removing the email popup: -20.51% revenue per visitor
Test run byDave DiederenSyntra Labs
Commentary byLiftArchiveOur analysis, not theirsRemoving the email popup cut revenue per visitor by 20.51%. Dave Diederen ran this test on a client store and says it also lost when repeated on a second store. Conversion rate and order value both fell.
- Revenue per visitor: -20.51% (€2.60 to €2.07), estimated at about -€67,026 a month. The range was -48% to +19%.
- Conversion rate: -12.18% (3.07% to 2.69%). This is a loss, with a range of -35% to +17%.
- Profit per visitor: -19.99% (€1.59 to €1.27), the metric the test was judged on.
- Average order value: -9.49% (€84.78 to €76.74). Shoppers without the popup spent less per order.
The change
The variant removes the email signup popup entirely.
Simplified mockup drawn by LiftArchive from the shared screenshots.
Control
The store showed its email signup popup to visitors as normal.
Variant
The email popup was hidden for visitors in this group. Nothing else changed.
Why it likely lost
Our reading of the result, not the practitioner's claim.
- the popup's discount offer gives undecided visitors a reason to buy now, so removing it loses those orders.
- email capture brings visitors back later, and that path disappears without the popup.
- store owners see their own popup constantly and find it annoying, but shoppers see it once and often welcome the offer.
Caveats: The confidence ranges for every metric include zero, and the post does not share sample size or duration. The popup's offer and timing on the tested store were not shown.
Who ran it
Dave Diederen (@DaveDiederen) shared this test on X. The numbers and screenshots are theirs.
View the original post on X
How to run this test on your store
- Hypothesis
- If we remove the email popup, then conversion will rise because the experience is less cluttered. (In this test, the opposite happened.)
- Primary metric
- Revenue per visitor
- Guardrails
- Email signups
- Conversion rate
- Average order value
- Who should test it
- Stores thinking about removing their popup to clean up the site. Test it before you switch it off. If your popup has a weak offer or poor timing, improving it is likely the better test.
Questions
Do email popups hurt conversion rates?
Not in this test. Removing the popup lowered conversion by 12.18% and revenue per visitor by 20.51%.
Should I remove my email popup?
Dave Diederen's advice is to fix it, not remove it. The post suggests checking the offer, the timing and how easily it closes on mobile.
What makes a good ecommerce popup?
Dave Diederen describes one that appears after 8-10 seconds, offers 10% off and closes cleanly. The popup used in this test was not shown.
Similar tests
@DaveDiederen@DaveDiederen@DaveDiederen
@kasper_dolk@kasper_dolk
@peterac_