Removing mailing-list checkboxes from sign-up: no change in trials
Commentary byLiftArchiveOur analysis, not theirsRemoving two mailing-list opt-in checkboxes from the sign-up form made no significant difference to trial sign-ups.
Patrick McKenzie ran this split test on the web version of Bingo Card Creator and shared it on the Kalzumeus (patio11) blog.
The author called the test flawed, because a no-spam note was removed along with the checkboxes.
- Trial sign-ups: no significant difference. Flat.
- No sign-up numbers, sample size or test length were shared.
- Because two things changed at once, the result can't tell you the effect of the checkboxes alone.
The change
Mailing-list checkboxes and the no-spam note removed from sign-up
Simplified mockup drawn by LiftArchive from the shared screenshots.
Control
The registration form asked for an email address and showed two checkboxes for joining the mailing list, with a note promising no spam.
Variant
The same form with both checkboxes and the no-spam note removed. The email address was still required, as it served as the username.
Why it likely came out this way
Our reading of the result, not the practitioner's claim.
- Two optional checkboxes add little effort, so removing them didn't make the form noticeably easier.
- The email field stayed, so the main privacy concern for visitors didn't change.
- Losing the no-spam reassurance may have cancelled out any small gain from a shorter form.
Caveats: No numbers, sample size or duration were shared. The author said the test was flawed because the no-spam wording should have stayed in both versions.
Who ran it
Patrick McKenzie ran this test and published the results (Kalzumeus (patio11) blog). The numbers are theirs; the summary and commentary on this page are ours.
Read the original case study on Patrick McKenzie →
How to run this test on your site
- Hypothesis
- If we remove optional mailing-list checkboxes from our sign-up form but keep a no-spam note, then sign-ups will rise because the form looks shorter while still reassuring people.
- Primary metric
- Trial sign-ups per sign-up page visitor
- Guardrails
- Mailing list opt-in rate
- Email unsubscribe rate
- Trial-to-paid rate
- Who should test it
- SaaS and lead-gen businesses with extra opt-in options on the sign-up form. Change only the checkboxes, and skip it if your mailing list drives a large share of paid conversions.
Questions
Do opt-in checkboxes reduce sign-ups?
Not in this test. Removing two mailing-list checkboxes made no significant difference to trial sign-ups.
Should you keep a no-spam note on a sign-up form?
It is worth testing on its own. Here it was removed with the checkboxes, which the author said spoiled the test.
How do you avoid a flawed form test?
Change only the element you are testing. Keep everything else, such as reassurance copy, the same in both versions.