# Value proposition rewrite on product page: +445% lift not trusted

- Outcome: Flat / inconclusive
- Page type: Other (SaaS)
- Elements: Value proposition
- Date shared: 2013-09
- Test run by: [MarketingExperiments](https://liftarchive.com/practitioners/marketingexperiments/)
- Commentary by: LiftArchive (our analysis, not theirs)
- Original source: https://marketingexperiments.com/a-b-testing/truth-behind-big-lifts
- Page: https://liftarchive.com/test/tech-supplier-product-page-value-prop/

A rewritten value proposition on a top product page first showed a 445.6% conversion lift, but we treat it as inconclusive because a rerun showed the numbers were noise. MarketingExperiments (MECLABS) ran the test for a seller of mostly digital products to high-tech industries. In the follow-up, two identical copies of the control differed by 30%.

## Result

**+445% lift not trusted**

- First run: Treatment C showed a 445.6% relative lift at 99% confidence. This is flagged as unreliable.
- The control converted far below its 3.83% historical rate and had six days in a row with no conversions.
- Only 2 to 10 conversions happened per day, so a few orders swung the rate a lot.
- Rerun with a cloned control: two identical pages differed by 30%, so the original lift could not be trusted.

## Control

The company's top-selling product page as it was.

## Variant

Four treatments that reworked the page's value proposition and the order in which it made its case. Treatment C was the apparent winner.

## Why it likely came out this way (our reading)

- Too few conversions: daily rates ranged from 0.84% to 5.62%, far from the 15% the team assumed when planning.
- A statistical confidence number can look strong on tiny counts and still be noise.
- A slump in the control, not a better treatment, likely drove most of the gap.

## Caveats

Result marked inconclusive: the lift came from very few conversions and failed a cloned-control check. The team switched to a template test across the top 20 products to get roughly ten times more conversions.

## How to run this test

- Hypothesis: If we rework the value proposition on our product pages, then more visitors will buy because the page will explain more clearly why to choose us.
- Primary metric: Conversion rate across a product page template
- Guardrail: A cloned control (A/A check) to measure noise
- Guardrail: Conversion rate versus historical baseline
- Guardrail: Revenue per visitor
- Who should test it: SaaS and digital product sellers with many similar product pages. Test the shared template, not a single page, unless that page gets plenty of conversions every day.

---

Credit: MarketingExperiments ran this test and owns the result. Summary and commentary by LiftArchive (https://liftarchive.com/test/tech-supplier-product-page-value-prop/).
Usage: Free to use, quote and build on, with credit and a link: name the practitioner or organization that ran the test and link to its LiftArchive page. The results belong to the practitioners who ran and shared them; the summaries and commentary are LiftArchive's.