# Full site and app redesign: +10% trial sign-ups, purchases flat

- Outcome: Flat / inconclusive
- Page type: Site-wide (SaaS)
- Elements: Redesign, Onboarding, Progress bar
- Date shared: 2012-04
- Test run by: [Patrick McKenzie](https://liftarchive.com/practitioners/patrick-mckenzie/)
- Commentary by: LiftArchive (our analysis, not theirs)
- Original source: https://www.kalzumeus.com/2012/04/19/ab-testing-is-frustrating/
- Page: https://liftarchive.com/test/patio11-bcc-full-redesign-2012/

A full redesign of the Bingo Card Creator marketing site, app and purchasing page raised trial sign-ups on AdWords landing pages from 20.9% to 23.1%, but purchases stayed flat. Patrick McKenzie ran this sitewide A/B test and published it on the Kalzumeus blog in 2012. Many changes shipped together, so no single element can take credit for any part of the result.

## Result

**+10% trial sign-ups, purchases flat**

- Trial sign-ups on AdWords landing pages rose from 20.9% to 23.1%, about 1,400 visits per version over 7 days. Significant at 90% but not 95%.
- User success, meaning trial users who actually made bingo cards, rose about 15%.
- Purchases were flat: 50 for the old version and 46 for the redesign, a null result.
- Once users reached the new purchasing page, they bought at the same rate as before.

## Control

The old design of the marketing site, the app and the purchasing page.

## Variant

A new design across the marketing site, the app and the purchasing page and cart. The app also gained a step-by-step progress bar showing how far a user had got in making bingo cards.

## Why it likely came out this way (our reading)

- Fewer users hit the 15-card trial limit, which was the main reason people went to buy. Users got what they needed for free.
- The redesign may have drawn in more parents with small needs, who would never pass the trial limit.
- With dozens of changes bundled, any one of them could have nudged users away from the trial limit.

## Caveats

This was a bundled redesign with many changes at once, so results can't be tied to one element. Purchase counts were small, and the redesign was adopted anyway for the better user experience.

## How to run this test

- Hypothesis: If we redesign the site and app to be cleaner and show progress through setup, then more visitors will start and succeed with a trial because the path is clearer, which should lead to more purchases.
- Primary metric: Paid purchases per visitor
- Guardrail: Trial sign-up rate
- Guardrail: Share of trial users hitting the plan limit
- Guardrail: User activation rate
- Who should test it: SaaS products with a free plan or trial limit where a redesign is already planned. Track every funnel step, not just the final sale. If you need to know which change did what, test elements one at a time instead.

---

Credit: Patrick McKenzie ran this test and owns the result. Summary and commentary by LiftArchive (https://liftarchive.com/test/patio11-bcc-full-redesign-2012/).
Usage: Free to use, quote and build on, with credit and a link: name the practitioner or organization that ran the test and link to its LiftArchive page. The results belong to the practitioners who ran and shared them; the summaries and commentary are LiftArchive's.