# One price-based shipping message on product pages: -5.5% conversion

- Outcome: Lost
- Page type: Product page (Ecommerce)
- Elements: Shipping & delivery messaging, Free shipping thresholds & shipping price
- Test run by: [Clean Commit](https://liftarchive.com/practitioners/clean-commit/)
- Commentary by: LiftArchive (our analysis, not theirs)
- Original source: https://cleancommit.io/ab-tests/context-aware-shipping-message/
- Page: https://liftarchive.com/test/cc-context-aware-shipping-message/

Replacing two similar shipping messages with one message based on product price cut conversion rate by 5.5% and revenue per visitor by 9.0%. Clean Commit ran this test for an anonymized Shopify store and published it in its A/B test library. The drop was steepest on mobile, and the test stopped the store from rolling out a change that looked like a simple clarity fix.

## Result

**-5.5% conversion**

- Conversion rate: -5.5%, a negative result. The original had a 75.1% probability of doing better.
- Revenue per visitor: -9.0%, a negative result, with average order value down 4.2%.
- Orders: -4.5%, a negative result.
- Mobile conversion rate: -10.5%, where most of the loss came from.

## Control

Product pages showing two similar shipping messages.

## Variant

One message that changed with price: a reminder of the free-shipping threshold on cheaper items, or a note that the item ships free on items above it.

## Why it likely lost (our reading)

- On cheaper items, a threshold reminder may highlight that shipping is not free, which can feel like a cost.
- Two messages may have made free shipping feel more prominent than one.
- Fewer shipping reassurances on a small mobile screen may have left some shoppers unsure.

## Caveats

Across 20,974 visitors over 17 days, the loss did not reach significance, at a 75.1% probability that the original was better. The direction was consistent across conversion, orders and revenue.

## How to run this test

- Hypothesis: If we show one shipping message based on product price, then conversion rate will rise because shoppers understand the shipping benefit without mixed signals.
- Primary metric: Conversion rate
- Guardrail: Mobile conversion rate
- Guardrail: Average order value
- Guardrail: Revenue per visitor
- Who should test it: Stores with a free-shipping threshold and both cheap and expensive items. Be careful if most products sit below the threshold, since the reminder may draw attention to a shipping cost.

---

Credit: Clean Commit ran this test and owns the result. Summary and commentary by LiftArchive (https://liftarchive.com/test/cc-context-aware-shipping-message/).
Usage: Free to use, quote and build on, with credit and a link: name the practitioner or organization that ran the test and link to its LiftArchive page. The results belong to the practitioners who ran and shared them; the summaries and commentary are LiftArchive's.