Benefit list added to newsletter box: +190.31% newsletter clicks
Commentary byLiftArchiveOur analysis, not theirsAdding a title and a bulleted list of benefits to a site-wide subscribe box lifted newsletter link clicks by 190.31%.
Dutchento, the Dutch Magento community site run by Guido Jansen, ran this test, and it was published as a VWO case study.
Clicks on the RSS link in the same box fell by 44.46%.
- Newsletter link clicks: +190.31%, a significant lift.
- RSS feed link clicks: -44.46%. This is a negative result on the second goal.
- Completed newsletter sign-ups were not shared; the metric was clicks on the link.
The change
A title and benefit list were added to the site-wide subscribe box.
Simplified mockup drawn by LiftArchive from the shared screenshots.
Control
A subscribe box on every page offering newsletter and RSS links, with no reason given for subscribing.
Variant
The same box with a title and a short bulleted list of reasons to subscribe.
Why it likely won
Our reading of the result, not the practitioner's claim.
- Guido Jansen's view was that people need a reason to subscribe beyond just visiting the site. The benefit list supplied one.
- The benefits may have made email look like the more reliable way to get updates, pulling some visitors away from RSS.
- Link position inside the box may also have played a part in the RSS drop; the case study suggests testing randomised order.
Caveats: The metric was clicks on the subscribe links, not completed sign-ups. The lift in newsletter clicks came with a large drop in RSS clicks. Sample size and test length were not shared.
Who ran it
Dutchento ran this test for Dutch Magento community site and published the results (VWO case study). The numbers are theirs; the summary and commentary on this page are ours.
Read the original case study on Dutchento →
How to run this test on your site
- Hypothesis
- If we add a short list of benefits to our newsletter sign-up box, then more visitors will subscribe because they can see what they get in return.
- Primary metric
- Completed newsletter sign-ups per visitor
- Guardrails
- Clicks on other subscribe options in the box
- Unsubscribe rate in the first month
- Email open rate for new subscribers
- Who should test it
- Content, community and lead-gen sites with a bare sign-up box that says little about what subscribers get. It won't help much if your box already makes a clear promise, and watch out if you offer several subscribe options, since one may grow at the others' expense.
Questions
Does listing benefits increase newsletter sign-ups?
Often, yes. Here a short benefit list nearly tripled newsletter link clicks. Check that clicks turn into confirmed sign-ups.
Why did RSS clicks drop?
The source isn't sure. One likely reading is that the benefits made email look more attractive, so some visitors switched from RSS to the newsletter.
What benefits should a newsletter box list?
Be specific about what subscribers get and how often, such as new articles, event news or exclusive tips.
Short bullets are easier to scan than a paragraph.