Most published mobile app A/B test results are wins: 70 of the 80 subscription-app tests in our database (88%) report a winner, and 61 of 71 paywall tests do.
That number says more about what gets published than about how often app tests succeed.
Across the wins, the typical reported lift is about +50% on trial starts, +18% on conversion to paid and +20% on ARPU or proceeds per user.
Below is how we counted, where the wins cluster, and why you shouldn't expect your own win rate to look like this.
Method and sample
We counted every test in the app section of LiftArchive as of November 2026: 80 experiments on paywalls, onboarding and retention flows.
Each one is credited to the app, consultant or tool vendor that ran or published it.
You can browse them all on the app tests hub.
- Sources: 41 tests come from RevenueCat's blog and case studies, 31 from Superwall, and 8 from AppHud, Purchasely, Kovalee, an indie founder's blog and posts on X.
- Evidence type: 49 are clean A/B tests, 7 are multivariate tests, 13 are bundled tests (several changes shipped together) and 11 are before/after comparisons with no concurrent control.
- Surfaces: 71 paywall tests, 7 onboarding tests and 2 retention tests.
- Metrics: whatever the source reported. That ranges from trial start rate and install-to-paid conversion to ARPU, proceeds per user, LTV and revenue.
The brief for this page expected around 140 tests, including App Store and Google Play listing tests.
We haven't extracted store-listing experiments yet, so this page covers in-app tests only.
Most sources also don't give consistent test dates, so we don't split results by year.
Win, loss and flat split
Of the 80 tests, 70 are recorded as wins and 10 as losses.
None is recorded as flat. That is almost the reverse of what large experimentation programs report.
- Microsoft's experimentation team wrote that when ideas are tested in controlled experiments, less than a third move the metrics they were designed to improve, and fewer than that on a mature product like Bing.
- Optimizely reports that only 12% of experiments produce a statistically significant improvement on the primary metric across its customers' tests.
So "what percentage of app A/B tests win?" has two answers.
Among tests that get written up, about 9 in 10. Among all tests an app team runs, likely closer to 1 in 3 or less, if app tests behave like the web and product tests above.
We have no unpublished app data to prove that, so treat it as a working assumption.
The win rate holds up a little better when you look only at the strongest evidence.
Among the 49 clean A/B tests, 40 won and 9 lost (82%).
All 13 bundled tests and 10 of 11 before/after comparisons are wins, which is what you would expect from designs that can't separate the change from seasonality or other launches.
Many "wins" are trade-offs
At least seven of the 70 wins lost on the first metric people check and won on revenue.
These are the most useful results in the set, because they show why a single conversion number can mislead.
- BuyBye raised its yearly price by 50%: conversions fell 19%, and proceeds per user rose 25% (Superwall case study).
- Zumba added a web checkout button: immediate conversions dropped 25%, and LTV rose 17% (RevenueCat case study).
- Superwall's early app-to-web checkout tests: initial conversion fell 11.9%, while net proceeds rose about 19.85% (vendor-reported).
- Teaching.com swapped a free trial for a money-back guarantee: slightly lower conversion, and higher revenue and LTV.
The reverse happens too. Slopes replaced its paywall with a one-tap trial card and got +25% trial starts, but trial-to-paid stayed about the same.
A trial-start win only pays off if those trials convert.
SuperwallTypical lift by metric
We grouped the wins that report a clear percentage lift by metric and took the median, leaving out tests where the metric is ambiguous or covers only one plan tier.
These medians describe published wins, not what an average test will deliver.
| Metric | Wins with a clear lift | Range | Median |
|---|---|---|---|
| Trial starts / install-to-trial | 9 | +7% to about +400% | +50% |
| Conversion to paid (install-to-paid, trial-to-paid) | 8 | +8% to over +50% | +18% |
| ARPU / proceeds per user | 17 | +8.3% to +80% | +20% |
| Paywall conversion rate | 14 | +1% to +190% | +50% |
| Install conversion rate (store listing) | 0 | Not yet in the database | n/a |
The pattern matters more than the exact medians.
Lifts shrink as you move down the funnel.
Getting more people to start a trial is easy.
Getting more of them to pay is harder, and lifting revenue per user by a fifth is a strong result.
Some examples from each row:
- Trial starts: a food app's shorter paywall with a review stat lifted install-to-trial by 72% in a bundled test run by Vahe Baghdasaryan. At the low end, Ahead App's paid trial-extension offer added 7% conversion to trial.
- Conversion to paid: Zumba cut its trial from 14 to 7 days and saw +8% install-to-paid. Pixelcut tested a weekly plan against monthly and got +16% paying customers.
- ARPU: a paid 30-day trial against a monthly plan with no trial raised D90 ARPU from $2.23 to $2.89 (+30%). SellRaze lowered its annual price and saw +10% ARPU.
- Paywall conversion: Stardust added a 7-day trial for +131% paywall conversion in a multivariate test (Superwall case study).
If you plan a test around a +50% lift, you will mostly be disappointed.
Size your test to detect a realistic effect: see our guide to the minimum detectable effect, and compare your own trial numbers with our app trial conversion benchmarks.
Win rate by surface
| Surface | Tests | Won | Lost | Published win rate |
|---|---|---|---|---|
| Paywall | 71 | 61 | 10 | 86% |
| Onboarding | 7 | 7 | 0 | 100% |
| Retention | 2 | 2 | 0 | 100% |
| Store listing | 0 | - | - | Not yet counted |
Every loss in the set is a paywall test.
That doesn't mean onboarding tests always win. Paywall tests make up 89% of the sample, and the two biggest sources sell paywall tooling, so paywall losses are the ones most likely to be written up as lessons.
The onboarding wins are mostly long or personalized flows.
ASL Bloom's goal-based onboarding, built with Applica, reported +15% trial conversion, +12% ARPU and +18% 30-day retention.
Lose It! tested a longer onboarding and saw trial start rate rise by double digits.
The two retention tests are both offers. Unscripted's reactivation trial offer drove over +240% trial starts, and Lose It!'s in-app message offers brought in about 10x more money.
With two tests, that is a lead worth testing, not a rate.
Win rate by element
Tests carry one or more element tags, so the rows overlap.
Five tests (mostly web checkout experiments) have no element tag.
| Element | Tests | Won | Lost | Published win rate |
|---|---|---|---|---|
| Paywall design | 26 | 24 | 2 | 92% |
| Free trials | 24 | 23 | 1 | 96% |
| Plans and plan mix | 18 | 14 | 4 | 78% |
| Pricing | 16 | 15 | 1 | 94% |
| Discounts | 10 | 8 | 2 | 80% |
| Paywall timing | 9 | 9 | 0 | 100% |
| Freemium limits | 8 | 6 | 2 | 75% |
| Onboarding quizzes | 4 | 4 | 0 | 100% |
Paywall timing is the only element with nine or more tests and no losses.
Showing the paywall earlier, often before or during onboarding, keeps winning.
Rootd moved its paywall to the start of onboarding and reported 5x revenue, though that was a before/after comparison.
Plan mix has the most losses, and the results conflict.
Mojo made yearly the only default plan and gained 15-20 percentage points in yearly share, while Stardust's yearly-only paywall lost on both conversion and proceeds per user.
The same idea can win in one app and lose in another.
Trial length is more consistent. Zumba's move from 14 to 7 days won, and a wellness app's 14-day trial, tested by Daphne Tideman against 7 days, got slightly more trial starts but fewer paid conversions.
In both tests the shorter trial came out ahead.
Hard paywalls and forced offers account for several losses.
BoldVoice's paywall with no dismiss button and its unclosable discount both raised trial starts, but refunds rose by more than the gain.
Limits: vendor publication bias
Read every number on this page with these caveats in mind:
- Vendors publish wins. 72 of the 80 tests come from RevenueCat and Superwall, which sell subscription and paywall tools. Their case studies are vendor-reported, and a customer story about a failed test is rare marketing.
- Losses are published as lessons. The 10 losses we have mostly appear in "what we learned" posts. That makes them useful, but it doesn't make the 88% win rate real.
- No flat results. In large programs, flat is the most common outcome. Our set has none, which tells you inconclusive app tests almost never get written up.
- Weak designs inflate wins. Bundled and before/after tests won 23 of 24 times. Without a concurrent control, a seasonal bump or a new ad campaign can look like a paywall win.
- Mixed metrics. Sources pick the metric that tells the story. A test may "win" on trial starts while paid conversion stays flat, as Slopes found.
- Few significance details. Most write-ups don't report sample sizes or confidence levels. Our guide to statistical significance explains what to look for before you trust a lift.
Use this data to choose what to test and how to measure it.
Don't use it to forecast your results. If you have run an app test, win or lose, you can submit it to the archive.
Losses and flat results are the ones we need most.