Network-size subject lines vs passion: -12.5% click rate
Commentary byLiftArchiveOur analysis, not theirsSubject lines about the size of VolunteerMatch's nonprofit network got a 12.5% lower click rate than passion-led subject lines.
MECLABS Institute ran the test with VolunteerMatch, and it was published as a MarketingSherpa case study.
Two other value themes, proof and ease of use, showed no significant difference from passion.
- -12.5% click rate for the network-size pair versus the passion pair. This variant lost.
- Proof and ease-of-use subject lines showed no statistically significant difference from passion.
- Across all lines, those starting with the word volunteer or volunteering did better.
- The test went to VolunteerMatch's large B2C volunteer list, not the B2B buyers the product is sold to.
Control
Passion-led subject lines with no product feature, such as 'Spread the Only "Good" Office Virus'. The same line was sent twice as a double control.
Variant
Subject lines about network size, such as 'CC Your Boss: 1,000+ Ways To Make A Difference (Inside)' and 'Does your company care? Thousands of ways to prove it.'
Why it likely lost
Our reading of the result, not the practitioner's claim.
- Network size was VolunteerMatch's favorite differentiator, but volunteers on the list cared more about volunteering itself.
- Numbers like '1,000+ ways' can read as generic marketing rather than something personal.
- Leading with a word the reader identifies with, such as volunteer, seems to grab attention faster.
Caveats: The B2C list was used as a stand-in for a small B2B list, so the result may not carry over to business buyers. List sizes and click rates were not shared, and the data is from early 2015.
Who ran it
MECLABS Institute ran this test and published the results (MarketingSherpa case study). The numbers are theirs; the summary and commentary on this page are ours.
Read the original case study on MECLABS Institute →
How to run this test on your site
- Hypothesis
- If we lead subject lines with a theme our readers identify with instead of our product's scale, then more will click because the email feels personal rather than promotional.
- Primary metric
- Click rate
- Guardrails
- Open rate
- Unsubscribe rate
- Downstream conversions from clicks
- Who should test it
- SaaS and mission-driven companies with an email list big enough to split into several groups. If your real buyer list is small, a proxy list like this one can point the way, but confirm on the real audience.
Questions
Should subject lines highlight how big your product or network is?
Not by default. In this test the network-size lines had a lower click rate than passion-led ones.
Can you test subject lines on a different list than your target buyers?
You can, as a rough guide, when your real list is too small.
Treat the result as a lead to confirm, since a different audience may value different things.
Why send the same control subject line twice?
A double control shows how much results vary by chance.
If the two copies differ a lot, the test setup may have a problem.