Full site redesign and merger: more trials, but sales roughly halved
Commentary byLiftArchiveOur analysis, not theirsMerging two sites and redesigning the whole thing lifted weekday trial downloads from about 100 to 125-150, but daily sales dropped from about 2 to slightly under 1. Patrick McKenzie made this change for Bingo Card Creator and shared the numbers on the Kalzumeus (patio11) blog.
This was a bundled before/after comparison, not a split test, and the numbers came from the first week only.
- Weekday visitors: about 900 to 1,200 (compared against both old sites combined).
- Pages per visit: 2.5 to 3.5, and time per visit: about 90 to 150 seconds.
- Trial downloads: about 100 to 125-150 a day, and confirmed installs: 20-25 to 30-40. Not yet settled.
- Sales: about 2 a day to slightly under 1. Negative on the main business metric, so we mark it lost.
Control
The old Bingo Card Creator site, with a sister site, Daily Bingo Cards, running separately.
Variant
The two sites merged into one, with a full redesign of the combined site. Many changes shipped at once, so no single element can be credited.
Why it likely lost
Our reading of the result, not the practitioner's claim.
- Early sales data is noisy, and the author thought normal ups and downs might explain the drop.
- People already in the trial may have been put off by the new look and wondered if they were on the right site.
- More engagement and more trials don't guarantee more sales if the new traffic or design attracts less committed visitors.
Caveats: Before/after, not a split test, eyeballed from about one week of typical weekday numbers in 2008. Many changes shipped together, and Bingo Card Creator was downloadable desktop software at the time, not SaaS.
Who ran it
Patrick McKenzie ran this test and published the results (Kalzumeus (patio11) blog). The numbers are theirs; the summary and commentary on this page are ours.
Read the original case study on Patrick McKenzie →
How to run this test on your site
- Hypothesis
- If we merge our sites and redesign them, then paid conversions will rise because visitors will find the product and trial more easily.
- Primary metric
- Paid conversions (sales or new subscriptions)
- Guardrails
- Trial sign-ups
- Trial-to-paid rate
- Returning trial users' activity
- Who should test it
- SaaS teams planning a redesign who can split traffic between old and new versions. If you can only do a before/after switch, wait several weeks before judging sales and expect noise from existing trial users.
Questions
Can a redesign raise trials but lower sales?
Yes. Here trials and engagement went up while sales fell in the first week. Track paid conversions, not just top-of-funnel numbers.
How should you test a full redesign?
Split traffic between the old and new sites if you can.
A before/after switch mixes the design effect with seasonality and normal swings.
Why might existing trial users react badly to a redesign?
A sudden change in branding can make people unsure they are on the same site, which matters in markets where trust is fragile.