Shipping protection toggle, off by default: +23.7% conversion
Test run byConvertiblesCRO agency
Commentary byLiftArchiveOur analysis, not theirsReplacing a shipping protection widget's two checkout buttons with one toggle, set to off, lifted conversion from 1.53% to 1.90% (+23.7%).
Convertibles ran this three-way test for a consumer electronics accessories brand.
Revenue per visitor rose 16.4%, though that number was less certain.
- Conversion rate: 1.53% to 1.90% (+23.7%) for the off-by-default version, at 97% probability to beat control.
- Revenue per visitor: +16.4% for the off version, at 82% probability. That is directional, not proven.
- The on-by-default version also beat control (+6.8% conversion, +8.2% revenue per visitor), but at only about 70% probability.
- Average order value fell 6% ($148.42 to $139.58) in the off version, likely because fewer orders carried the protection fee. This is a negative to weigh.
- Among shoppers who actually opened the drawer, the on version edged ahead, so on versus off is not settled.
The change
Two checkout buttons become one protection toggle and a single checkout button with the live total.
Simplified mockup drawn by LiftArchive from the shared screenshots.
Control
The cart drawer ended with the shipping protection app's stock widget. It showed a fee row and two checkout buttons: one with protection added and one without, so every shopper had to pick before paying.
Variant
One row with the fee, an info icon and a toggle, plus a single checkout button showing the live total. One version started with the toggle off, another with it on.
Why it likely won
Our reading of the result, not the practitioner's claim.
- One button with one honest total removes the feeling of being upsold at the last step.
- The two-button layout filled most of a phone screen. Most of the lift came from mobile.
- Few shoppers changed the toggle, so the default decided what nearly everyone paid. Off meant a lower total at checkout.
Caveats: The test ran 19 days, and only a few hundred shoppers per version opened the drawer, so the drawer-level read is noisy. The agency found that the app treats a missing setting as protection on, so a default-off toggle needs a check that the off setting actually saved.
Who ran it
Convertibles ran this test for a consumer electronics accessories brand and published the results (Convertibles case study). The numbers are theirs; the summary and commentary on this page are ours.
Read the original case study on Convertibles →
How to run this test on your store
- Hypothesis
- If we replace the two checkout buttons with one shipping protection toggle and one live total, then more shoppers will check out because the price no longer looks padded.
- Primary metric
- Conversion rate
- Guardrails
- Average order value
- Shipping protection attach rate
- Checkouts charged protection with the toggle off
- Who should test it
- Stores using a shipping protection app with the stock two-button layout in the cart. If protection fees are a large share of your margin, test the on default too and weigh the lost fees.
Questions
Should shipping protection be on or off by default?
In this test, off won on store-wide conversion at 97% probability.
Among drawer viewers the on default edged ahead, but the gap was too small to call.
Why did average order value drop?
It fell 6% in the off version. The likely reason is that fewer orders included the protection fee, though the agency could not measure that directly.
What should I check before shipping a default-off toggle?
Confirm the off setting is saved before the shopper reaches checkout.
Some apps treat a missing setting as protection on, which would charge people behind a toggle that says off.
Similar tests
@oliverkenyon@oliverkenyon@oliverkenyonConvertiblesConvertibles