Free-shipping line on product pages: +21% conversion, not significant
Test run byClean CommitCRO agency
Commentary byLiftArchiveOur analysis, not theirsAdding the line "This product is eligible for free shipping" to product pages showed a 21.0% higher conversion rate, but the result was not statistically significant.
Clean Commit ran this test for an anonymized Shopify store and published it in its A/B test library.
We count it as inconclusive because the sample was small and the test had only 15% power.
- Conversion rate: +21.0%. The variant had an 84.6% probability of beating the original, which is short of significance.
- Revenue per visitor: +31.7%, and orders: +16.7%. Both point the same way but rest on the same small sample.
- Average order value: +8.8%.
- Add-to-cart rate: -2.9%, a negative signal. Clean Commit says the drop came from desktop, where add-to-cart fell 16.3% on only 453 visitors.
- Inconclusive overall: with 4,128 visitors and 15% statistical power, lifts this large usually shrink on a retest.
The change
The variant adds a free-shipping eligibility line before add to cart.
Simplified mockup drawn by LiftArchive from the shared screenshots.
Control
The product page did not say anything about free shipping. Shoppers had no way to know from the page whether delivery would cost extra.
Variant
A line reading "This product is eligible for free shipping" was added to the product page, so shoppers saw it before adding to cart.
Why it likely came out this way
Our reading of the result, not the practitioner's claim.
- Unexpected delivery costs are a common reason to abandon. Stating free shipping on the product page removes that doubt early.
- The message costs nothing to read and sits where the buying decision happens.
- The lift is probably inflated. Small, underpowered tests tend to overstate the true effect, so the real gain is likely smaller.
- The add-to-cart dip on desktop suggests the effect may not be the same on every device.
Caveats: Only 4,128 visitors over 26 days with 15% statistical power, so the result is not significant and Clean Commit labeled it a positive signal. Expect the lift to shrink if you retest it.
Who ran it
Clean Commit ran this test for a Shopify store (anonymized) and published the results (Clean Commit A/B test library). The numbers are theirs; the summary and commentary on this page are ours.
Read the original case study on Clean Commit →
How to run this test on your store
- Hypothesis
- If we state clearly on the product page that the item ships free, then more visitors will buy because they stop worrying about extra costs at checkout.
- Primary metric
- Conversion rate (orders per visitor)
- Guardrails
- Add-to-cart rate
- Revenue per visitor
- Average order value
- Who should test it
- Stores that already offer free shipping on some or all products but don't say so on the product page. It won't help if free shipping is already obvious site-wide, or if you charge for delivery.
Questions
Should I mention free shipping on product pages?
If you offer it, it is worth testing.
In this test the message pointed to a large lift, but the sample was too small to be sure.
Why is a 21% lift called inconclusive?
The test had about 4,100 visitors and 15% power.
At that size, big swings often come from chance, and the variant's 84.6% probability of winning falls short of the usual 95% bar.
Where should the free-shipping message go?
Close to the price and add-to-cart button, where shoppers decide whether to buy. That is where this test placed it.
Similar tests
@oliverkenyon
@DaveDiederen@DaveDiederen@DaveDiederen@DaveDiederen@DaveDiederen