Free trial cut from 15 to 8 cards: paid conversion 1.04% to 1.89%
Commentary byLiftArchiveOur analysis, not theirsCutting the free trial limit from 15 bingo cards to 8 lifted trial-to-paid conversion from 1.04% to 1.89%, roughly an 82% relative gain.
Patrick McKenzie ran this A/B test on Bingo Card Creator and wrote it up on the Kalzumeus blog.
The test reported 99% confidence on a sample in the 5,000s.
- Trial-to-paid conversion: 1.04% to 1.89% (about +82% relative), at 99% confidence.
- Sample size was in the 5,000s, according to the write-up.
- More users reached the limit, and the extra users who hit it also bought at a good rate once in the cart, so the two funnel steps did not behave independently.
- After the change shipped, year-over-year sales rose 38% in May, 108% in June and 33% in July 2012. Those months also had a site redesign and new Stripe checkout, so they are not a clean read on this test alone.
Control
Free users could make 15 bingo cards before hitting the trial limit and being asked to buy.
Variant
The free quota dropped to 8 cards. Nothing else changed: it was a one-line code edit.
Why it likely won
Our reading of the result, not the practitioner's claim.
- A recent usability redesign meant fewer people hit the old 15-card limit, so fewer ever saw the purchase path. A lower cap put more users in front of the upgrade prompt.
- Users who needed between 8 and 15 cards were a real group. Under the old limit they could finish for free. Under the new limit they had a reason to pay.
- Patrick McKenzie found that this middle group responded well to the new, nicer checkout, which compounded the gain.
Caveats: Results are from 2012 and a single low-priced consumer product. The year-over-year sales figures overlap with a redesign and a payment change, so only the 1.04% to 1.89% figure comes from the split test.
Who ran it
Patrick McKenzie ran this test and published the results (Kalzumeus (patio11) blog). The numbers are theirs; the summary and commentary on this page are ours.
Read the original case study on Patrick McKenzie →
How to run this test on your site
- Hypothesis
- If we lower the free plan's usage cap, then more trial users will reach the paywall and upgrade, because users with moderate needs can no longer finish their work for free.
- Primary metric
- Trial-to-paid conversion rate
- Guardrails
- Trial sign-up rate
- Activation (users who complete a first real task)
- Refund or early churn rate
- Who should test it
- SaaS products with a usage-capped free trial or freemium plan where many users never hit the cap. It probably won't help if most users already hit the limit, or if the cap is so low that people can't see the product's value before it runs out.
Questions
Does a smaller free trial lower conversions?
Not always. In this test a tighter usage cap raised trial-to-paid conversion, because more users reached the point where they had to pay.
How do I pick a free trial usage limit?
Look at how much of the product a typical paying customer uses, and set the cap so users with real needs hit it.
Then A/B test the cap and track paid conversions, not just sign-ups.
Should I cap trials by usage or by time?
A usage cap works well when value comes from output, like cards made or reports run.
A time limit suits products whose value builds over days of use. Either way, test it against paid conversion.