An experiment report template is a fixed set of fields you fill in after every A/B test: the hypothesis, what changed, who saw it, how long it ran, the sample size per variant, the primary metric with its interval, guardrail metrics, data checks, the decision and what you learned.
Using the same fields every time makes tests comparable, keeps losing tests from vanishing, and lets anyone check the call months later.
Below is the template as a table you can copy, field-by-field notes on filling it in, a filled-in example using a real test from our database, and a shorter monthly CRO report template for the whole program.
(If you came here for a school science lab report, the structure is similar: question, method, results, conclusion.
This page is about website and app experiments.)
The A/B test report template
Copy this table into a doc, a Notion page or a spreadsheet row.
One report per test, written the same week the test stops.
Every field should be filled; write "not measured" rather than leaving a gap.
| Field | What to write |
|---|---|
| Test name and ID | Short name plus the ID in your testing tool, so anyone can find the raw data. |
| Owner and dates | Who ran it; start and stop dates; number of full weeks. |
| Hypothesis | "Because we saw [evidence], we believe [change] for [audience] will move [metric]." Link the source research. |
| Control | What the page or flow looked like before. Add a screenshot. |
| Variant(s) | Every change in each variant. If several changes are bundled, list them all. |
| Audience and targeting | Pages, devices, countries, new or returning users, and the traffic split. |
| Primary metric | The one metric decided before launch that settles the test. |
| Guardrail metrics | Metrics that must not get worse (AOV, refunds, page speed, unsubscribes). |
| Planned sample size | Visitors or conversions per variant you needed, and the minimum effect that sample could detect. |
| Actual sample | Visitors and conversions per variant. |
| Data checks | Sample ratio check result, tracking QA, any outages, promos or launches during the run. |
| Results table | For each variant: rate, relative lift, interval, and p-value or probability to beat control. |
| Segments | Device or traffic-source splits, labeled as exploratory unless planned in advance. |
| Decision | Ship, don't ship, iterate, or rerun. One line on why. |
| Learning | What this tells you about your customers, in one or two sentences. |
| Next test | The follow-up idea this result suggests, linked to your backlog. |
The hypothesis field works best if you already wrote it in a standard format before launch; our hypothesis template uses the same wording, so you can paste it straight in.
How to fill in each section
Write the top half before launch. Hypothesis, primary metric, guardrails and planned sample size belong in the report before the test starts.
If you fill them in afterward, it is too easy to pick the metric that happened to win.
See guardrail metrics for how to choose them.
Report counts, not just percentages. "+15% conversion" means little without visitors and conversions per arm.
Counts let a reader rerun the numbers in a significance calculator.
Run the data checks. Compare the visitor split you planned with the split you got.
Harness's docs describe the usual method: a chi-squared goodness-of-fit test, flagging a mismatch when the p-value is below 0.001.
A Microsoft and LinkedIn team found that about 6% of Microsoft's experiments had a sample ratio mismatch, so this check matters.
Our SRM checker does the math. If it fails, the report says so and the result isn't trusted (more in sample ratio mismatch).
Give an interval, not only a lift. A point lift hides how uncertain it is.
Write the range as well, for example "+0.05 to +1.42 percentage points."
Say if you stopped early. Checking a test every day and stopping at the first significant reading inflates false positives.
Evan Miller showed that checking after every observation can push a nominal 5% false positive rate to 26.1%.
If you stopped before the planned sample, write that in the data checks field.
Keep the learning separate from the result. "Variant won by 15%" is the result.
"Buyers want proof of quality next to the price, not further down the page" is the learning, and the learning is what feeds the next test.
Filled-in example: a real test from the database
Here is the template filled in from a reviews retest on a whisky subscription product page, run by Convertibles.
All numbers come from the published case study.
The interval and p-value are our own calculation, assuming an even split between arms, because the source gives total visitors and orders only.
| Field | Entry |
|---|---|
| Test name | PDP: star rating under price plus member reviews (retest) |
| Owner and dates | Convertibles; May 11 to July 2, 2026 |
| Hypothesis | Reviews sat far below the buy area, with no rating near the price. Showing the rating where buyers decide should raise conversion. This was a rerun after an earlier four-element bundle lost. |
| Control | Standard product page; member reviews far below the buy area, no rating near the price. |
| Variant | A clickable "4.9 from 300+ reviews" row under the price, plus a member review carousel below the CTA. |
| Primary metric | Conversion rate |
| Guardrails | AOV, revenue per visitor |
| Actual sample | 15,615 visitors, 795 orders in total |
| Results | Conversion 4.725% to 5.460% (+15.56%); RPV +15.79%; 98% probability of being best. Our calc: two-sided p about 0.037; 95% interval on the difference about +0.05 to +1.42 percentage points. |
| Guardrail result | AOV flat |
| Decision | Ship. Reported at about +A$10,235 a month. |
| Learning | Social proof needs to sit next to the price. Testing it alone, after a bundle failed, isolated the element that works. |
| Next test | Rating placement on other templates, such as the homepage. |
Notice what the interval adds. The lift is real at conventional thresholds, but the low end is close to zero, so a "+15%" forecast for next year would be optimistic. A good report says that plainly.
CRO report template (monthly or quarterly)
A CRO report covers the whole program, not one test.
It goes to whoever funds the work, so it should fit on one page and link down to the single-test reports above.
| Section | What to include |
|---|---|
| Summary | Two or three sentences: what shipped, what it is worth, what's next. |
| Tests completed | Count, plus a list with result (won, lost, flat, invalid) and a link to each report. |
| Win, loss and flat counts | All three. Hiding losses makes the program look better and teaches nothing. |
| Shipped changes | Each winner, its measured effect with interval, and the ship date. |
| Estimated impact | Revenue or leads, labeled as an estimate, with the method stated. |
| Key learnings | Three to five customer insights, each tied to the tests behind it. |
| Running now | Live tests, expected end dates, sample progress. |
| Next up | Top backlog items with scores from your prioritization spreadsheet. |
| Blockers | Dev time, traffic, tracking problems. |
Store every single-test report in one searchable place.
That collection becomes your learning repository, and the CRO report is a monthly view of it.
For how reporting fits a wider program, see running an experimentation program.
What a good CRO report shows
The reports worth reading are the ones that include losses, flag weak evidence and break out bundled changes.
Three tests from our database show what that looks like.
Losses with a reason. Convertibles reported a strap cross-sell in a premium bag brand's cart drawer that cut revenue per visitor by 12.4% (V1) and 10.3% (V2).
The report noted basket size held and the loss came from fewer buyers, and nothing shipped.
That one line of diagnosis is worth more than the result.
Weak evidence labeled as weak. In a cart drawer test for a high-ticket equipment brand, conversion went from 0.382% to 0.444% (+16%) on 84,005 visitors and 347 orders, with an 86-92% probability to beat control.
The report said it was below 95% confidence, bundled four changes, and that mobile AOV fell 21% while desktop rose.
A reader knows exactly how much to trust it.
Stopped tests explained. A buy 3 get 1 free golf ball offer had a 2.3% take rate (94 of 4,078), -2.8% revenue per visitor and a 27% probability to beat control after 19 days.
It was stopped as not promising, and the report records that it was tested against an existing upsell widget, not against nothing.
ConvertiblesConvertiblesConvertiblesConvertiblesConvertiblesReports that include real counts and data checks also make claims easier to trust.
Anatta's tabbed buy box test reported 34,730 control and 6,144 variant visitors on an 85/15 split, a passed SRM check, and the filters applied (wholesale orders, outliers and orders under $20 removed) alongside the AOV change from $102.63 to $104.74.
Common mistakes to avoid in a CRO report:
- Reporting only the relative lift, with no counts or interval.
- Adding up the lifts from every winner into one annual figure. Effects overlap and often shrink after launch.
- Presenting segment wins found after the fact as if they were planned.
- Calling a flat test "inconclusive" without saying what effect size the test could have detected.
To see how other teams wrote up their results, browse all tests in the database or start with the basics in our A/B testing guide.