Most ecommerce A/B tests should use revenue per visitor (RPV) as the primary metric. If you know your margins, profit per visitor is better still. Conversion rate is useful as a diagnostic, but on its own it can call a test a winner when the store actually makes less money.
This guide covers each metric and when to use it. It also shows how the 285 tests in our database were judged, so you can see where conversion rate and revenue tell different stories.
Conversion rate (CVR): easy to read, easy to misread
Conversion rate is orders divided by visitors. It is the first number most store owners check, and it reaches significance faster than revenue metrics because each visitor either buys or doesn't.
The trouble is that CVR ignores what people spend. A change that gets more people to buy smaller orders can lift CVR while revenue barely moves.
Dave Diederen saw exactly that when they matched the social proof line under the add-to-cart button to the ad angle. CVR rose 12.70%, but AOV fell 2.63%, so RPV grew a smaller 9.74%. That's still a win, just a smaller one than CVR suggested.
The gap can also run the other way. When Dave Diederen redesigned the bundle selector to ask who the product was for, conversion rate dropped 5.22%. A CVR-only reading would call that a loss. Revenue per visitor actually rose 9.45%, because AOV jumped 15.48%.
Use CVR when: the change can't plausibly affect order size (a trust badge, a shipping callout), or you are too low on traffic to detect a revenue change. Even then, check RPV before you ship.
Revenue per visitor (RPV): the default primary metric
RPV is total revenue divided by visitors. It equals conversion rate times average order value, so it captures both effects in one number. That is why most practitioners in our database lead with it.
Dave Diederen's checkout countdown timer test is a typical report: +2.99% conversion rate, +3.16% RPV, AOV flat to slightly down. The same idea lost elsewhere. A countdown timer for a European market cut RPV by 35.61%, and Kasper Dolk saw RPV rise 26.18% by removing a checkout timer for a Dutch audience.
RPV has one real cost. Revenue varies a lot from one visitor to the next (most spend nothing, a few spend a lot), so RPV needs more traffic than CVR to reach a reliable result. Plan for that with our sample size calculator.
Two practical rules for RPV:
- Count all visitors in the test, not just buyers. Revenue per buyer is AOV, a different metric.
- Watch for outliers. One wholesale-sized order can swing a small test. Many teams cap or exclude extreme orders, and decide how before the test starts.
Average order value (AOV): a supporting metric, never the judge
AOV is revenue divided by orders. It tells you whether people are spending more per order, which matters for bundle, upsell and threshold tests.
As a primary metric, AOV is risky because it ignores everyone who didn't buy. Dave Diederen tested skipping the cart and sending shoppers straight to checkout. AOV rose 4.08%, which looks good, but conversion rate fell 10.23% and RPV fell 6.58%. Judged on AOV, a losing test would have shipped.
The only test in our database that leads with AOV is Dave Diederen's BOGO offer for a deal-driven audience (+25% AOV over a weekend). It is a useful signal, but without CVR and margin you can't tell whether the store earned more.
Use AOV when: explaining why RPV moved. Report it next to CVR and RPV, not instead of them.
Profit per visitor: the metric closest to the business
Profit per visitor subtracts product cost (and ideally discounts, shipping costs and payment fees) from revenue, then divides by visitors. It is the right primary metric for anything that changes price, discount depth, shipping fees or free gifts, because those tests can raise revenue while giving away margin.
Dave Diederen used profit per visitor as the primary goal on a PDP layout that led with benefits instead of social proof: +35.3% conversion rate, +43.1% RPV and +44.6% profit per visitor. When Dave Diederen raised the free shipping threshold to $165, profit per visitor rose 6.27% with AOV flat.
Kasper Dolk's benefits-first hero test shows how the full set fits together. Conversion rate rose 14.41% (1.38% to 1.58%), AOV rose 7.55%, RPV rose 23.05% (€1.20 to €1.47) and profit per visitor rose 21.97% (€0.88 to €1.07). Kasper Dolk's post on that test called the result extra profit, not extra revenue.
Offer tests are where profit matters most. Milos changed a 2-pack into "Buy 2 Get 1 Free" at the same price, a deal that looks better on paper. Conversion rate fell 51.36% and profit per visitor fell 54.01%. A 30% price cut that doubled conversion rate for one brand in an Elevate case study is the opposite warning: without margin data, a CVR win on a discount can still lose money. See our guide to price testing for more.
The catch: you need cost data per product in your testing tool or analytics, and that takes setup. If you can't get it, use RPV and run a margin check by hand on any pricing or discount winner.
Guardrail metrics: what must not get worse
A guardrail metric is one you don't try to improve, but you won't ship if it gets worse. The idea comes from large-scale experimentation programs. Ron Kohavi, Diane Tang and Ya Xu's book Trustworthy Online Controlled Experiments gives a full chapter to sample ratio mismatch and other trust-related guardrail metrics.
For an ecommerce store, sensible guardrails include:
- Sample ratio: if a 50/50 test splits 53/47, something is broken and the result can't be trusted.
- Refund and return rate: urgency and aggressive offers can lift sales that come back later.
- Subscription share or repeat purchase: a test can win today and cost you lifetime value.
- Email signups: if a variant hides your capture form, short-term RPV may hide a long-term loss.
- Page speed and error rate: a heavy new widget can slow the site for every visitor.
Dave Diederen's pre-checked subscription test is the clearest case in our database. Upfront it looked like a loss: −2.86% conversion rate, −9.17% RPV and −6.50% AOV. But subscriptions rose 1,295% as a share of orders, so the decision depends on repeat revenue, not first-order RPV. That's a case where the primary metric has to be judged alongside a longer-term one.
Guardrails also protect against calling tests too early. Ruben's authority badge test showed +17% conversion rate after one week at 99.6% probability to beat control, and ended at a flat +1.66% after four weeks. Our guide to statistical significance explains why.
Pick your primary metric and guardrails before launch and write them down. If you look at twenty metrics after the fact, one of them will move by chance.
What the tests in our database used
We counted the metrics reported across all 285 tests in the LiftArchive test database. Many tests report more than one:
| Metric reported | Tests | Share of 285 |
|---|---|---|
| Conversion rate | 188 | 66% |
| Revenue per visitor | 165 | 58% |
| Both CVR and RPV | 159 | 56% |
| Average order value | 124 | 44% |
| Profit per visitor or profit | 75 | 26% |
Three patterns stand out:
- Experienced practitioners report CVR, RPV and AOV together. Most of Dave Diederen's tests list all three, and many add profit per visitor. Reporting them together is what reveals the cases where CVR and RPV disagree.
- CVR and RPV usually move together, but not always. In the bundle selector test above they pointed in opposite directions. When Dave Diederen raised the shipping price to $6.95 and $7.95, conversion rate didn't change while RPV rose 1.57% and profit per visitor rose 2.21%. A CVR-only test would have reported nothing.
- Profit shows up most on shipping, offer and pricing tests. These are the tests where revenue and profit split apart.
Many tests also report an estimated dollar figure per month. Those are projections from the test lift, not measured results, so treat them as rough. For how often tests win at all, see our research on A/B test win rates.
@DaveDiederen@DaveDiederen
@kasper_dolk@kasper_dolk@DaveDiederen
@RubendgrfA simple metric plan for your next test
- Primary: RPV for most tests; profit per visitor for price, discount, shipping and offer tests.
- Secondary (to explain the result): conversion rate, AOV, add-to-cart rate, checkout start rate.
- Guardrails: sample ratio, refund rate, subscription share or email signups where relevant.
For the full list of store metrics worth tracking outside of tests, see our guide to ecommerce KPIs.
Tracking these metrics in a testing tool
Your tool needs order revenue, not just a "purchase" event, to report RPV and AOV. In Convert, for example, setting up a revenue goal powers the RPV metric and fills in an AOV column. By default Convert counts one transaction per visitor per goal, and you have to add a parameter to count repeat orders. Check that setting if your customers often buy twice in a session.
As of November 2026, Convert's pricing page lists the Growth plan at $399 a month ($299 a month billed annually) for 100K tested users a month, with 30 advanced goals per project, which include revenue goals. That price makes it a poor fit for small stores with little traffic, which would struggle to reach a reliable RPV result anyway.