Unclosable discount offer after dismiss: refunds beat trial gains
Commentary byLiftArchiveOur analysis, not theirsBoldVoice showed a percentage-off offer that couldn't be closed whenever someone dismissed its onboarding paywall, and trial starts went up, but refunds rose by more than the gain.
CEO Anada Lakra described the test in a Sub Club podcast write-up on the RevenueCat blog.
It lost on net revenue after refunds, and no figures were shared.
- Trial starts: up. The test looked promising on trial numbers.
- Refunds: rose by more than the gain in trial starts.
- Net revenue after refunds: lower than control, so the change lost. The discount size and all figures were not shared.
The change
Dismissing the paywall opens a discount popup that can't be closed
Simplified mockup drawn by LiftArchive from the shared screenshots.
Control
The onboarding paywall could be dismissed. Dismissing it led to a small free taste: a speech assessment, a first daily plan and a coach video, with a 7-day trial offer on day two.
Variant
The main paywall kept its close button, but dismissing it opened a percentage-off offer with no way to close it.
Why it likely lost
Our reading of the result, not the practitioner's claim.
- Users who had just finished onboarding and then hit an offer they couldn't close often took it to move on.
- A discount that traps the user feels like a trick, and people who feel tricked refund when the charge arrives.
- The discount also lowers what each paying user brings in, so it needs a large real gain to pay off.
Caveats: No figures, discount size, sample size or duration were shared. BoldVoice judged it on net revenue after refunds per exposed user on matured cohorts.
Who ran it
BoldVoice ran this test. Source: RevenueCat blog (Sub Club podcast write-up). The numbers are theirs; the summary and commentary on this page are ours.
How to run this test on your app
- Hypothesis
- If we show a discount offer when users dismiss the paywall, and let them close it, then net revenue will rise because we win back some hesitant users without trapping them.
- Primary metric
- Net revenue after refunds per exposed user, on a matured cohort
- Guardrails
- Refund rate
- Proceeds per user
- App Store rating and complaints
- Who should test it
- Apps that want to try an exit offer should keep it closable; this test suggests trapping users backfires. Check the App Store guidelines before shipping any dismiss offer on iOS.
Questions
Are exit offers a bad idea?
Not on their own. What lost here was an offer users couldn't close.
A closable offer, judged on net revenue after refunds, is a fairer test.
What metric should we use for discount tests?
Net revenue after refunds per exposed user, on a matured cohort.
Trial starts alone made this test look like a win.
Should we trust trial-start wins on discount tests?
Not on their own. Wait for trials to convert and refunds to settle; BoldVoice waits at least an extra week, then two more.
Similar tests
Superwall