Hard paywall with no close button: refunds wiped out trial gains
Commentary byLiftArchiveOur analysis, not theirsBoldVoice removed the close button from its onboarding paywall, and trial starts went up, but refunds rose by more than the gain, so the test lost on net revenue.
CEO Anada Lakra described it in a Sub Club podcast write-up on the RevenueCat blog. No figures were shared.
- Trial starts: up. On trial numbers alone the test looked like a win.
- Refunds: rose by more than the gain in trial starts. People who felt forced in asked for money back once charged.
- Net revenue after refunds: lower than control, so the change lost. No figures were shared.
The change
Same paywall with the close button removed
Simplified mockup drawn by LiftArchive from the shared screenshots.
Control
The onboarding paywall had a close (X) button. Users who dismissed it got a small free taste: a speech assessment, a first daily plan and a coach video, then a 7-day trial offer on day two.
Variant
A hard paywall with the close button removed, so nobody could go on without starting a trial.
Why it likely lost
Our reading of the result, not the practitioner's claim.
- Users had already invested time in onboarding, so many started the trial rather than leave.
- People pushed into a trial without wanting it tend to refund as soon as the charge shows up.
- The free taste in the control let people who weren't ready keep exploring instead of paying and refunding.
Caveats: No figures, sample size or duration were shared. BoldVoice now judges paywall tests on net revenue after refunds per exposed user on matured cohorts.
Who ran it
BoldVoice ran this test. Source: RevenueCat blog (Sub Club podcast write-up). The numbers are theirs; the summary and commentary on this page are ours.
How to run this test on your app
- Hypothesis
- If we remove the close button from the onboarding paywall, then trial starts will rise because users who finished onboarding can't skip it, but refunds may rise because some users feel forced.
- Primary metric
- Net revenue after refunds per exposed user, on a matured cohort
- Guardrails
- Refund rate
- Trial-to-paid conversion
- App Store rating and complaints
- Who should test it
- Apps with high-intent users after a long onboarding may still try it, but only if they can wait for refunds to settle. BoldVoice waited at least an extra week for trials to convert, then two more weeks for refunds. Skip it if you can only read trial start rate.
Questions
Why did more trial starts not mean more revenue?
Refunds grew faster than trial starts. People who felt forced into a trial asked for their money back when charged.
How long should we wait before calling a paywall test?
Long enough for trials to convert and refunds to land.
BoldVoice waits at least an extra week for conversions and two more for refunds.
What can we offer people who dismiss the paywall?
BoldVoice gives a narrow free taste and then offers a 7-day trial on day two.
That keeps undecided users around without forcing them.
Similar tests
Kovalee