Mobile app A/B testing means showing different groups of real users different versions of a screen, price or flow inside your app, then comparing what they do.
Most teams run these tests through remote config (Firebase or a paywall tool), which changes the app without a new release.
The tests that pay off most in subscription apps are on the paywall, onboarding and pricing, and you judge them on paid conversion and revenue per user, not taps.
Below is how to set up those tests, which metrics to read, how long to run them, and what real teams found when they ran them.
Every result links to an entry in our app test database.
In-app tests vs store-listing tests
There are two places to test an app, and they answer different questions.
Store-listing tests change what people see before they install.
In-app tests change what people see after they open the app.
Store-listing tests
Apple's Product Page Optimization lets you test up to three treatments of your icon, screenshots and preview videos against your original page.
A test runs for up to 90 days, new assets need App Review approval, and Apple treats a result as conclusive at 90% confidence.
Our guide to Product Page Optimization covers the setup.
Google Play has store listing experiments. You can run one default graphics experiment or up to five localized experiments at a time, testing icons, feature graphics, screenshots and, in localized experiments, descriptions.
Play measures install and open clicks, and experiments stop on their own after six months.
In-app tests
Store tests stop at the install. They can't tell you whether those new users pay.
In-app tests can, because you control the variant and you can follow each user to a purchase, a renewal or a refund.
That is why most of the money in subscription apps sits in in-app tests: the paywall, the onboarding flow, trial length, plan mix and price. The rest of this guide focuses there.
Tools: remote config, paywall tools and Firebase
You need two things to test inside an app: a way to change the experience without shipping a new build, and a way to tie each user's variant to what they buy. Three kinds of tools do this.
Firebase A/B Testing and Remote Config
Firebase A/B Testing runs experiments on top of Remote Config, Cloud Messaging and In-App Messaging.
You define variants as Remote Config parameter values, target users by Analytics audiences or user properties, and pick an Analytics event as the goal.
Per Firebase's documentation, it uses frequentist statistics, allows up to 8 variants including the baseline, and caps you at 24 running experiments and rollouts and 300 in total.
Firebase recommends two weeks as the minimum runtime for a typical Remote Config experiment.
As of November 2026, A/B Testing is listed as no-cost on both Firebase plans.
Remote Config is free up to 100,000 daily requests, but the Firebase pricing page shows per-request charges above that on the Blaze plan starting September 1, 2026.
Firebase's weak spot for subscription apps is revenue.
It sees Analytics events, not App Store or Play renewals and refunds, unless you wire that data in yourself.
It works well for onboarding and feature tests, and less well for judging a price change.
Paywall and subscription tools
Paywall tools serve the paywall remotely and connect each variant to real subscription revenue. As of November 2026:
- RevenueCat is free up to $2,500 in monthly tracked revenue, then 1% of tracked revenue. Experiments (A/B/n tests) and Paywalls are included in the Pro plan.
- Superwall offers its subscription infrastructure free, and its paywall product is free up to $10,000 a month in paywall-attributed revenue, then 1% of that revenue. It runs A/B/n tests on layout, copy, pricing, trials and offers without an app release.
- Adapty is free until $5,000 a month in revenue, then 1% of monthly tracked revenue. Its Pro plan includes a paywall and flow builder and A/B tests with custom traffic splits.
The trade-off is cost at scale: a 1% revenue fee grows with your app, and you're tied to one vendor's SDK.
These tools are a poor fit if you don't sell subscriptions or in-app purchases.
For a side-by-side view, see our app A/B testing tools and paywall tools roundups.
Build it yourself
Larger teams often assign variants on their own server and send events to their data warehouse.
You get full control and no revenue fee, but you have to build the assignment logic, the stats and the revenue joins yourself.
Metrics: trial start, install-to-paid, ARPU and retention
The metric you pick decides which variant wins.
In subscription apps, the metric closest to the screen you changed is often the most misleading one.
- Trial start rate: the share of users who start a trial. Fast to read, but easy to inflate.
- Trial-to-paid: the share of trials that turn into paying subscribers.
- Install-to-paid: paid conversions divided by installs. It combines the two above.
- ARPU or proceeds per user: revenue divided by all users in the variant. This catches price and plan-mix effects that conversion rate misses.
- Retention, renewals and refunds: the lagging metrics that show whether a win holds.
Our tests show why the gap matters. Slopes saw +25% trial starts with a one-tap trial card, but trial-to-paid stayed about the same.
Stardust's trial timeline design lifted conversion 13.6% with trial-to-paid unchanged (Superwall case study, vendor-reported).
Sometimes the top-of-funnel win is a loss. BoldVoice tested a hard paywall with no close button: trial starts went up, but refunds rose by more than the gain.
Pick a revenue metric as the primary goal and keep trial starts as a secondary one. Our subscription metrics guide defines each one.
Sample size and duration for apps
App tests need more users than most teams expect, because the metric that matters, paid conversion, is usually a small percentage.
The standard two-proportion formula gives the users you need per variant:
n = (z(α/2) + z(β))² × [p₁(1 − p₁) + p₂(1 − p₂)] ÷ (p₂ − p₁)²
At 95% confidence and 80% power, (1.96 + 0.84)² is about 7.84.
Worked example
Say 5% of new users start a trial and you want to detect a lift to 6% (a 20% relative change).
That's 7.84 × (0.0475 + 0.0564) ÷ 0.0001, or about 8,100 users per variant.
Now judge the same test on install-to-paid instead, with a baseline of 1.5% and a target of 1.8%.
You need 7.84 × (0.0148 + 0.0177) ÷ 0.000009, or about 28,000 users per variant.
Revenue goals cost more traffic, which is why small apps should test big changes.
Run your own numbers in the sample size calculator.
How long to run it
- Run at least one or two full weeks, so weekday and weekend users are both counted. Firebase's own guidance is two weeks minimum.
- For trials, wait until the trial cohort has converted or canceled. A 7-day trial needs at least 7 days after the last user enters, plus time for billing retries.
- Don't stop the first day a dashboard shows a winner. Decide your sample size first, then check the result with a significance calculator.
Test ideas from the database
These are real experiments from our database, grouped by where they ran.
Most come from case studies published by the tool vendors, so treat the numbers as vendor-reported.
Paywall and plans
Mojo saw +14% new subscription revenue when it hid credit packs and sold subscriptions only (Superwall case study).
Not every simplification wins: Stardust tested a yearly-only paywall and lost on both conversion and proceeds per user.
Trials and pricing
Zumba got +8% install-to-paid by cutting its trial from 14 to 7 days (RevenueCat customer story).
Price tests often trade conversion for revenue: BuyBye raised its yearly price 50% and saw 19% fewer conversions but 25% more proceeds per user.
Onboarding and checkout
AEZAKMI Group tested six onboarding flows in an LED controller app and moved conversion from 0.8% to 1.29%, with ARPU up from $0.12 to $0.14. Superwall's pooled tests of a paywall button that sends users to web checkout found 11.9% lower initial conversion but about 19.85% higher net proceeds.
Superwall
RevenueCatCommon mistakes
- Judging on trial starts alone. The Slopes and BoldVoice tests above show trial lifts that never reached revenue, or cost money through refunds.
- Stopping too early. Trial cohorts need time to convert. Calling a test on day three measures sign-ups, not subscriptions.
- Testing small tweaks with small traffic. If you have a few thousand installs a month, test whole new paywalls or trial lengths, not button colors.
- Mixing platforms and countries without checking. iOS and Android users, and users in different countries, often respond differently to price. Look at the split before you roll out.
- Copying another app's winner. Mojo's simpler offer won and Stardust's lost. Use other teams' results as hypotheses, then test them on your own users.
- Changing the test mid-run. Editing a variant or the traffic split after launch mixes two experiments into one result.
For the general method behind all of this, read our A/B testing guide.