An A/B testing template is a one-page plan you fill in before a test starts: the problem you saw, the hypothesis, the change, one primary metric, the guardrail metrics, the audience, the sample size and run time, and the decision rule.
Writing these down first stops you from moving the goalposts once results come in.
Copy the template below, or use the dedicated hypothesis, prioritization and test report templates.
The A/B test template (copy this)
Every field below maps to something that, if left blank, has caused real tests in our database to end up hard to read.
Paste the table into a doc, spreadsheet or ticket and fill in the right column.
| Field | What to write |
|---|---|
| Test name | Page + change, in plain words (e.g. "Homepage buy box: contents preview") |
| Owner and date | Who runs it and the planned start date |
| Observation | What you saw in data, recordings, surveys or support tickets that prompted the test |
| Hypothesis | "Because we saw [observation], we believe [change] for [audience] will [effect on metric]." |
| Control | A short description and screenshot of the current page |
| Variant(s) | Exactly what changes; list every change if you bundle several |
| Primary metric | One metric that decides the test (conversion rate, revenue per visitor, sign-ups) |
| Guardrail metrics | Metrics that must not get worse (AOV, refunds, page speed, unsubscribe rate) |
| Audience and pages | Device, new vs returning, geography, which URLs are included |
| Baseline rate | Current value of the primary metric for that audience |
| Minimum detectable effect | The smallest lift worth acting on |
| Sample size and run time | Visitors per variant and the number of full weeks to run |
| Decision rule | Threshold to ship (e.g. 95% significance, two-tailed, on the primary metric, no guardrail loss) |
| Result | Filled in afterwards: numbers, significance, segments |
| Decision and next test | Ship, kill or iterate, and what you learned |
How to fill in the experiment plan template
Fill the fields top to bottom, and finish everything above "Result" before launch.
The order matters: the observation shapes the hypothesis, and the hypothesis tells you which metric should move.
Observation and hypothesis
Start from evidence, not opinion. A good hypothesis names the change, the audience and the expected effect, so a reader can tell later whether it was right.
Our guide to writing an A/B test hypothesis covers the format in detail.
Primary metric and guardrails
Pick one primary metric before launch, and choose it close to revenue when you can.
Clean Commit's "Shop This Color" button test shows why: click-through, the goal metric, rose 2.6% (94.9% probability), but conversion rate fell 2.6%, revenue per visitor fell 6.8% and AOV fell 7.6% across 17,746 visitors over 60 days.
Without sales as a guardrail metric, that would have looked like a win.
Sample size, run time and decision rule
Enter your baseline rate and minimum detectable effect into a sample size calculator, then round the run time up to full weeks.
Full weeks smooth out weekday and weekend differences in behavior.
Write the decision rule down too, including whether the test is one- or two-tailed.
GrowthRock's mobile homepage link bar test reached 93% significance one-tailed on +5% orders (28 days, 80,000+ visitors, 2,300 conversions per variation), but it was not significant by Optimizely's two-tailed stats.
A rule written in advance settles that kind of argument. See one-tailed vs two-tailed tests.
Variants: list every change
If the variant bundles several changes, say so in the template.
Convertibles' cart drawer test for a high-ticket equipment brand moved conversion from 0.382% to 0.444% (+16%) and RPV by +18% over 6 weeks and 84,005 visitors, but four changes were bundled and the result sat at 86-92% probability to beat control.
A complete variant field tells future readers that the lift can't be credited to any single change.
A filled-in A/B test plan template example
Here is the template completed for a real test from our database: Convertibles' "What's in the box" preview test for an 8-figure Australian gin subscription brand.
The result fields use the reported numbers; the plan fields show how the plan would have read before launch.
| Field | Example |
|---|---|
| Test name | Homepage subscription buy box: contents preview |
| Observation | The buy box showed price and a frequency dropdown, but nothing about what arrives in the box |
| Hypothesis | Because visitors can't see what they get, showing this month's box contents in the buy box will raise conversion rate |
| Control | Subscription buy box with price and a frequency dropdown, no preview of box contents |
| Variant A | Frequency tabs, an "Inside this month's box" list with thumbnails, and a +A$30 deluxe upgrade toggle |
| Variant B | Same as A, plus per-item retail values |
| Primary metric | Conversion rate |
| Audience and pages | Homepage visitors |
| Run time | 2 weeks |
| Result | +18% conversion rate for both variants; ~7,000 visitors per arm, 1,147 orders, 95%+ probability to beat control |
| Learning | The contents preview drove the lift; showing retail values added no lift on top of it |
| Next test | Test the deluxe upgrade toggle separately, since it shipped bundled with the preview |
The B variant is what makes this plan useful.
Because the retail-value version was tested as its own arm, the team learned which part of the change mattered instead of guessing.
Convertibles
GrowthRockConvertibles
Clean CommitGrowthRockMore templates for your testing program
- Hypothesis template: the observation-change-effect format with examples.
- Prioritization spreadsheet: score test ideas so the best ones run first.
- A/B test report template: write up the result, including caveats. GrowthRock's size guide test is a good model: +21% conversion at 96% significance after 10 days, with the agency noting under 200 conversions per variation and advising caution.
Store completed plans and reports in one place so past results inform the next round.
Our learning repository guide covers how, and the A/B testing hub links every method guide.