CRO best practices fail more often than most advice admits.
Of the 1,724 results in LiftArchive, 112 (6.5%) are losses, and many of them are textbook moves: countdown timers, trust bars, testimonials placed higher, a shorter path to checkout, cleaner navigation.
A best practice is a starting hypothesis, not a guaranteed lift, so test it on your own traffic before you ship it.
How we counted
We counted every result in the LiftArchive test database as of November 2026 (n = 1,724) using the direction each test was filed under: won, flat or lost.
A test is "lost" when the change performed worse than the original on its main metric, even if the gap wasn't statistically significant. Flat means no clear difference.
| Outcome | Tests | Share |
|---|---|---|
| Won | 1,447 | 83.9% |
| Flat | 165 | 9.6% |
| Lost | 112 | 6.5% |
Treat the 6.5% as a floor, not a real loss rate.
Public tests skew heavily toward wins, because agencies and brands publish what worked.
Inside large experimentation programs the picture is the reverse: the table in Kohavi, Deng and Vermeer's "A/B Testing Intuition Busters" (KDD 2022) lists success rates of 33% at Microsoft, 15% at Bing, about 10% at Booking.com, Google Ads and Netflix, and 8% for Airbnb Search.
The closest thing to an unfiltered sample in our data is Clean Commit, which publishes its losing and inconclusive tests alongside its winners.
Its 173 tests here break down as 16 won, 122 flat and 35 lost.
That is about 9% winners, which matches the big-company numbers far better than the 84% in the full database.
For more on this gap, see our A/B test win rate research.
Which best practices lose most often
We grouped tests by the best practice they tried, using the element tags on each test.
A test can sit in more than one group, so the rows overlap.
The loss rate is lost tests divided by all tests in the group.
| Best practice tried | Tests | Won | Flat | Lost | Loss rate |
|---|---|---|---|---|---|
| Change plans, pricing display or subscription defaults | 117 | 104 | 1 | 12 | 10.3% |
| Simplify checkout or forms | 91 | 76 | 6 | 9 | 9.9% |
| Redesign the layout | 227 | 173 | 33 | 21 | 9.3% |
| Add trust badges, guarantees or returns info | 112 | 92 | 10 | 10 | 8.9% |
| Add urgency or scarcity | 63 | 54 | 4 | 5 | 7.9% |
| Improve navigation, search or filters | 215 | 175 | 25 | 15 | 7.0% |
| Change the CTA button | 143 | 124 | 9 | 10 | 7.0% |
| Add social proof or reviews | 196 | 171 | 15 | 10 | 5.1% |
| All tests | 1,724 | 1,447 | 165 | 112 | 6.5% |
The differences between rows are small and the same publication bias applies to every row, so don't read this as a ranking of good and bad ideas.
What it does show is that no category is safe.
Every common best practice has public tests where it made things worse.
Redesigns and layout changes also have the highest flat share (14.5%).
Big changes bundle many edits, and the good ones often cancel out the bad ones.
Best practice myths, with the tests that broke them
Myth: urgency always lifts conversions
Countdown timers are the clearest case of a tactic that depends on context.
Dave Diederen reported that a checkout timer for US brands gained +9.50% conversion rate and +4.83% revenue per visitor.
The same team's countdown timer for a European market cost -17.35% conversion rate and -35.61% revenue per visitor, and they say they saw the pattern across 20+ brands.
Ruben saw the same split across several brands: about +10% conversion rate on a low-priced brand, no clear lift on a high-priced one, and -9% on another.
Clean Commit's low-stock announcement bar dropped conversion rate by 17.5% over 5,161 visitors.
Conversion Fanatics found the placement mattered too: an urgency message placed only above the order button did 32% worse than the control, though no sample size was published.
The research on scarcity is covered on our scarcity and urgency page.
The test evidence says it works for some audiences and backfires for others.
Myth: more trust signals means more trust
Convertibles added a trust bar under the buttons on handbag pages and saw -10.2% revenue per visitor.
The test ran 19 days with 516-547 orders per arm, below the agency's 1,000-order target, so nothing shipped.
DRIP Agency made a money-back guarantee more prominent and purchases fell 9.2%.
Clean Commit's return policy on product pages lost 6.0% conversion rate, though that result wasn't significant.
A likely reason: guarantees and return notes placed near the buy button can bring up the very doubts they are meant to settle.
See all results on our trust badges page.
Myth: put social proof above the fold
Clean Commit moved three short testimonials up next to the title, price and add-to-cart button.
Clicks rose 7.1%, but conversion rate fell 9.2% and revenue per visitor fell 15.3%, with an 89.3% probability the control was better.
Social proof had the lowest loss rate in our groups, but where it goes still matters.
Myth: remove every bit of friction
Dave Diederen's team skipped the cart and sent shoppers straight to checkout.
Conversion rate dropped 10.23% and revenue per visitor fell 6.58%.
They argue the cart works as a last sales page, with upsells and reassurance.
The same team removed an email popup that store owners assumed shoppers hated.
Revenue per visitor fell 20.51%, and they say it lost again on a second store.
Patrick McKenzie tested a trendy madlibs-style sign-up form: it converted 21.73% against 27.55% for the plain form, about -21% across 736 participants.
Myth: cleaner navigation and better search help shoppers buy
Clean Commit replaced a multi-row header with a single-row header and image mega-menu.
Conversion rate fell 22.5%, with a 99.5% probability the old header was better.
Its collection-page filters cut revenue per visitor by 33.2%, though the conversion rate change was close to a coin flip. More on our navigation menus page.
Myth: a polished page beats a plain one
MarketingExperiments' "ugly duckling" sign-up page had a white background, plain text and a typo.
It converted at about 14% against 7-9% for two polished versions with logos, testimonials and badges.
The polished versions changed many things at once, so the test can't say which change hurt.
Myth: lead with your best deals
Clean Commit replaced a trust-focused homepage hero with a carousel promoting current sales.
Conversion rate fell 46.2% and revenue per visitor fell 87.6%, with a 97.1% probability the original was better.
The test only had 1,082 visitors over 26 days, so the true drop is uncertain, but the direction is clear.
A hero that explains why to trust the brand beat one that shouted about discounts.
Myth: show the price everywhere and default to subscriptions
Adding the price to the Add to Cart button cost -1.98% conversion rate after 2,000+ orders, though Dave Diederen notes it has worked for other brands.
Pre-checking the subscription option lowered revenue per visitor by 9.17% upfront while subscription share of orders rose 1,295%.
That one may still be the better business decision once repeat orders come in, which is why the metric you pick decides what counts as a loss.
Fewer choices didn't help either. Stardust tried a yearly-only paywall in its app, and it lost on both conversion and proceeds per user against a paywall showing yearly and weekly plans.
Clean Commit
DRIP AgencyClean CommitClean CommitClean CommitWhat the failed best practices have in common
- The audience was different. Market, price point and traffic source changed the result for the same tactic, as the countdown timer tests show.
- Something was added near the buy button. Trust bars, testimonials and stock warnings all lost when they crowded the purchase area.
- Clicks went up while sales went down. Several losers raised engagement or clicks and still lost on conversion or revenue. Pick the business metric before you start and watch guardrail metrics.
- The sample was small. Some losses here, like Clean Commit's sales carousel on 1,082 visitors, are too small to trust the exact size of the drop. The same applies to the wins you read about. Use a sample size calculator before you judge any result.
How to use best practices without getting burned
Treat each best practice as a hypothesis with a reason behind it, then test it on your own traffic.
Before you run it, look up the losing tests for that element, not just the winners, so you know how it can fail.
Our losing A/B tests page lists every loss in the database, and the research hub has the win rate and lift numbers to set your expectations.
If a test loses, write down why you think it lost.
A losing test is often more useful than a winner, because it tells you something specific about your audience that no list of best practices could.