A CRO glossary defines the words teams use when they try to get more revenue from the same traffic: conversion rate, revenue per visitor, statistical significance, minimum detectable effect and the rest. Below, each term gets a plain-English definition in alphabetical order, plus a link to the page that covers it in depth. Where it helps, we show the term in action with a real result from our database of ecommerce A/B tests.
If you are new to the field, start with what conversion rate optimization is and how A/B testing works, then come back here when a word trips you up.
A–Z CRO and A/B testing terms, with links to the deeper page
A
A/A test. A test where both groups see the exact same page. It checks that your testing tool splits traffic evenly and reports no difference when there is none. If an A/A test "wins," something in your setup is broken.
A/B test. A controlled experiment that splits visitors at random between the current version (the control) and a changed version (the variant), then compares a metric such as revenue per visitor. Random assignment is what lets you credit the difference to your change rather than to seasonality or ad spend.
Above the fold. The part of a page visible before the shopper scrolls. On mobile this is small, so what goes there is one of the most tested decisions in ecommerce.
Average order value (AOV). Revenue divided by number of orders. A change can lower conversion rate and still earn more if AOV rises enough. Our guide to raising average order value covers bundles, thresholds and upsells.
B
Bayesian statistics. An approach that reports results as a probability, such as "94% chance the variant beats the control." Many store owners find it easier to read than p-values. See Bayesian vs frequentist testing for how the two differ.
Bounce rate. In Google Analytics 4, the share of sessions that were not engaged. GA4 counts a session as engaged if it lasts longer than 10 seconds, has a key event, or has 2 or more page or screen views (Google Analytics Help). Older Universal Analytics used a different definition, so old and new numbers are not comparable.
C
Cart abandonment rate. The share of shoppers who add something to the cart but do not finish the order. Baymard Institute's average across 50 studies is 70.22% (Baymard). Our cart abandonment guide covers the fixes that tend to help.
Confidence level. In frequentist testing, how sure you want to be before you call a result real. A 95% confidence level means you accept a 5% chance of a false positive. Some practitioners report results at 90%, which accepts more risk in exchange for faster decisions.
Control. The current, unchanged version in a test. Everything is measured against it.
Conversion rate (CVR). Orders divided by sessions or visitors, shown as a percentage. It is the headline metric in CRO, but it ignores order size, which is why ecommerce teams also track revenue per visitor.
Conversion rate optimization (CRO). The practice of increasing the share of visitors who buy, or the revenue each visitor produces, through research and testing rather than more traffic.
F
False positive (Type I error). Calling a variant a winner when the difference was random noise. Stopping tests early is the most common cause.
Flicker. A flash of the original page before the variant loads, caused by client-side testing scripts. It can bias results because the variant looks broken to some visitors.
Frequentist statistics. The classic approach behind p-values and confidence levels. You set a sample size in advance and only judge the result once you reach it.
G–H
Guardrail metric. A metric you watch to make sure a winning test is not causing harm somewhere else, such as refund rate, AOV or subscription cancellations.
Heatmap. A visual overlay that shows where visitors click, move or scroll. It is a research input for finding test ideas, not proof that a change will work.
Hypothesis. A testable statement of what you will change, what you expect to happen and why, usually based on research. "Because shoppers ask about returns in surveys, adding a returns line under the add-to-cart button will raise conversion rate." Our page on writing an A/B test hypothesis has a template.
K–L
KPI (key performance indicator). A number a business uses to judge whether it is on track. For stores that usually means conversion rate, AOV, revenue per visitor, cart abandonment rate and repeat purchase rate. See ecommerce KPIs for formulas.
Lift (uplift). The relative change between variant and control. If the control converts at 2.0% and the variant at 2.2%, the lift is +10%, not +0.2%. Most results in our database are reported as lift.
M
Minimum detectable effect (MDE). The smallest lift your test is set up to detect reliably. The smaller the MDE, the more traffic you need. Our MDE guide shows how to pick one that fits your traffic.
Multivariate test (MVT). A test that changes several elements at once and measures every combination. It needs far more traffic than an A/B test, so most small and mid-size stores are better off with simple A/B tests. More in our multivariate testing guide.
N
Novelty effect. A short-term lift that happens because returning visitors notice something new, then fades. It is one reason to run tests for full weeks. Ruben saw this when adding an authority badge to a product page: the result read +17% after week one, but settled at +1.66% CVR, flat after 4 weeks. The early number was noise.
Null hypothesis. The default assumption that your change makes no difference. A test tries to collect enough evidence to reject it.
P
Peeking. Checking results repeatedly and stopping as soon as they look significant. In frequentist tests, this inflates the false positive rate well above the level you set.
Power (statistical power). The chance your test detects a real effect of at least the MDE. 80% is the usual target. Low-power tests miss real winners and exaggerate the ones they do find.
Primary metric. The single metric you decide on before the test starts and use to call the winner. Picking it afterward lets you find a "win" in any test.
Probability to beat control. The Bayesian equivalent of a significance readout. Kasper Dolk reported +7.22% profit per visitor at 94.4% probability to beat control for showing a testimonial versus no testimonial.
Want results like this on your store?
Kasper Dolk at Maison Commerce. Kasper runs Maison Commerce, which builds CRO programs, landing pages and themes for fast-growing ecommerce brands such as ICON and HiStrips.
Profit per visitor. Gross profit divided by visitors. It is stricter than revenue per visitor because it subtracts discounts, shipping costs and cost of goods. Use it when a test changes prices, offers or shipping.
p-value. The probability of seeing a difference at least as large as the one you measured if the change had no real effect. A p-value below 0.05 is the usual bar for 95% confidence. It is not the probability that your variant is better.
R
Revenue per visitor (RPV). Revenue divided by visitors, which equals conversion rate times AOV. Many ecommerce practitioners use it as their primary metric because it catches changes that trade order size for conversion rate. Dave Diederen removed a free-shipping threshold and charged shipping on every order, and saw +9.17% RPV, a result a conversion-rate-only view could easily misread.
Want results like this on your store?
Dave Diederen at Syntra Labs. Dave builds and tests Shopify stores at Syntra Labs, and shares most results in public.
S
Sample ratio mismatch (SRM). When the traffic split is meaningfully different from what you set, for example 52/48 on a 50/50 test with large samples. It usually signals a bug in targeting, redirects or tracking, and the results should not be trusted until it is fixed.
Sample size. The number of visitors (or conversions) each group needs before you judge the result. It depends on your baseline conversion rate, your MDE, your confidence level and your power.
Segment. A slice of your audience, such as mobile visitors, new visitors or one country. The same change can win in one segment and lose in another. Dave Diederen's checkout countdown timer for a non-US (European) market produced -35.61% RPV, while a similar order-reserved timer for US brands gained +9.50% CVR. Compare the countdown timer results below.
Split URL test. An A/B test where each version lives at its own URL, often used for full page redesigns or landing pages built in a different tool.
Statistical significance. The point at which a result is unlikely to be random noise at your chosen confidence level. It tells you whether a difference is likely real, not whether it is large or worth shipping. Read the full explanation in our statistical significance guide.
T–V
Test velocity. How many tests a team launches in a given period. More tests means more learning, but only if each test has enough traffic to reach a clear result.
Type II error (false negative). Missing a real effect because the test was underpowered or stopped too soon. The variant gets thrown out even though it would have helped.
Variant. The changed version in a test, sometimes called the challenger or treatment. A test can have more than one variant, though each one splits your traffic further.
Win rate. The share of tests that produce a significant winner. Many changes in any program come back flat or negative, and a loss you catch in a test is money you did not lose site-wide.
To see how these terms show up in practice, browse all A/B test results in our database, filtered by page type and element.
Questions
What does CRO stand for?
CRO stands for conversion rate optimization: increasing the share of visitors who buy, or the revenue each visitor produces, through research and controlled testing rather than buying more traffic.
What is the difference between conversion rate and revenue per visitor?
Conversion rate is orders divided by visitors. Revenue per visitor is revenue divided by visitors, which equals conversion rate times average order value. RPV catches changes that raise conversion rate but shrink order size, so many ecommerce teams use it as their primary test metric.
What is statistical significance in A/B testing?
It means the measured difference between control and variant is unlikely to be random noise at your chosen confidence level, usually 95%. It does not tell you how big the effect is or whether it is worth shipping.
What is a minimum detectable effect?
The minimum detectable effect (MDE) is the smallest lift a test is designed to detect reliably. A smaller MDE needs more traffic, so low-traffic stores usually test bolder changes with a larger MDE.
What is the novelty effect in A/B testing?
A temporary lift caused by returning visitors reacting to something new. It fades over time, which is why tests should run for full weeks and why early results often shrink.