'Join the conversation' comment button: +34% comments, not significant
Test run byMarketingExperimentsResearch publisher
Commentary byLiftArchiveOur analysis, not theirsChanging the MarketingExperiments blog comment button from 'Submit Comment' to 'Join the conversation' showed a 34% lift in comments, but only at 82.9% confidence.
MarketingExperiments (MECLABS) ran and published the test. We list it as inconclusive, since the post itself says it found no significant difference at the 95% level it normally requires.
- Blog comments: +34.0% for 'Join the conversation', at an 82.9% confidence level.
- That is well below the 95% confidence MECLABS normally uses to make a decision, so the lift may be noise.
- The same test on the MarketingSherpa blog found no significant difference even at 80% confidence.
- Verdict: flat. MECLABS shipped the new copy anyway because it fit the brand and did not hurt.
The change
Only the comment button text changed.
Simplified mockup drawn by LiftArchive from the shared screenshots.
Control
The comment form ended with a standard button reading 'Submit Comment'.
Variant
The same form with the button reading 'Join the conversation'.
Why it likely came out this way
Our reading of the result, not the practitioner's claim.
- Copy that names a benefit can make an action feel more inviting than 'Submit'.
- Comment counts on a blog are low, so a 34% swing can come from a handful of extra comments.
- The weaker result on the sister blog suggests the effect, if real, is small.
Caveats: The lift reached only 82.9% confidence, and comment counts, traffic and duration were not shared. The test is from 2011.
Who ran it
MarketingExperiments ran this test and published the results (MarketingExperiments). The numbers are theirs; the summary and commentary on this page are ours.
Read the original case study on MarketingExperiments →
How to run this test on your site
- Hypothesis
- If we replace 'Submit Comment' with copy that states the value of joining the discussion, then more readers will comment because the button frames commenting as taking part, not handing in.
- Primary metric
- Comments per post view
- Guardrails
- Share of comments removed as spam
- Comment quality
- Who should test it
- Blogs and communities with steady comment volume. With only a few comments a week, plan for a long test or accept that you're testing for no harm.
Questions
What does 82.9% confidence mean?
It means the result is less certain than the usual 95% bar.
At that level there is a real chance the lift came from random variation, so treat it as a lead to retest, not a proven win.
Why did the same test differ on two blogs?
Different audiences and small comment counts can produce different results.
On the MarketingSherpa blog there was no significant difference at all.
Should I change button copy if the test is inconclusive?
You can, if the new copy fits your brand and the test shows it doesn't hurt. That is what MECLABS did here.
Similar tests
MarketingExperiments
Online DialogueMarketingExperiments