Post-purchase Survey A/B Tests: What 4 Results Show

All 4 measured tests won. The most useful pattern: the survey itself is not what was tested.

Each win came from a change built on what buyers said almost stopped them.

Kasper Dolk's survey showed a checkout timer felt like too much pressure to a Dutch audience.

Removing the timer raised conversion rate 25.61% and revenue per visitor 26.18%, an estimated +€150k/month.

Four results is a small set, and three are Kasper's.

The ideas below show how to turn survey answers into tests.

measured results
0–0won vs lost or flat
ideas to test

What counts as a post-purchase survey

A short survey shown after the order, often asking "what almost stopped you from buying?" or what outcome the buyer wanted.

In this collection the survey is the research step; the tests measure the page changes it led to.

The win/loss record

4 wins, 0 losses. Three are Kasper Dolk tests and one is Oliver Kenyon's.

Published results lean towards wins, so expect some survey-led tests to lose on your store.

Where the survey-led changes were tested

  • Product page: 2
  • Collection page: 1
  • Checkout: 1

Biggest wins and their numbers

Answering objections in the hero

Kasper Dolk added benefit bullets to the hero that answered survey objections with lines like "durable", "official" and "protects camera". The result: +$763,679.14 profit per year.

Removing what buyers disliked

Kasper's checkout timer removal came straight from survey feedback that the timer felt pushy: +25.61% CVR, +26.18% RPV.

The suggestion from the test is to place urgency earlier, on the product page or cart, instead.

Reassurance on the collection page

Oliver Kenyon put three survey-based reassurance lines above the collection grid for a wild bird food and feeder brand.

The result was $44,890/month, or $0.66 per session.

A timeline from survey wording

Kasper's "how you'll feel" timeline used an outcome headline in the customer's own words, a 24-hour and 3-day timeline, and risk reversal.

CVR was up 14%, marked as still confirming.

Tests that lost or were flat

None of the 4 lost. One is unconfirmed: the timeline result was still being checked when it was shared.

Patterns that won

  • Use buyers' words. The hero bullets and timeline both reused survey phrasing.
  • Put the answer where the doubt appears. Reassurance went above the collection grid and into the hero, where visitors see it on landing.
  • Remove, not just add. The timer test shows a survey can reveal something to take away.

What the ideas suggest testing

Kasper Dolk suggests turning stated hesitations into specific product page tests: a "who this is for" block, a guarantee badge under add to cart, a comparison table or an ingredients section.

Kasper also suggests surveying 5-star reviewers on desired outcomes, rated 1-10, and showing the results as proof.

Josh Thompson claims +23% conversion rate from fixing sizing, shipping and product-info gaps found in a "what almost stopped you" survey, but showed no test data.

How to add a post-purchase survey on Shopify

Survey apps can add a question to the order status or thank-you page.

Ask one open question, such as what almost stopped the purchase, and group the answers by theme before writing hypotheses.

How to test it on your store

Hypothesis: adding the top survey objection's answer to the hero will raise revenue per visitor.

Test one objection at a time, track profit as well as conversion rate, and size it with the sample size calculator.

Questions

What should I ask in a post-purchase survey?

The tests here used questions about what almost stopped the purchase and what outcome the buyer wanted.

Kasper Dolk turned those answers into hero bullets that added $763,679.14 profit per year.

Do post-purchase surveys increase conversion rate?

Not on their own. The survey finds objections; the gains came from the page changes based on them, like Oliver Kenyon's survey-based reassurance lines worth $44,890/month.

Can a survey tell me to remove something?

Yes. Kasper Dolk's survey showed a checkout timer felt like too much pressure, and removing it raised conversion rate 25.61%.