Collection pages with landing-page content: -1.0% conversion, flat
Test run byClean CommitCRO agency
Commentary byLiftArchiveOur analysis, not theirsAdding stronger visuals, a reworked grid and landing-page style content to collection pages left conversion rate almost unchanged at -1.0%, while revenue per visitor fell 7.7%.
Clean Commit ran this test for an anonymized Shopify store and published it in its A/B test library.
With only 3,805 visitors and 26% power, the test could only have caught large changes.
- Conversion rate: -1.0%, flat. The variant had a 47.1% probability of beating the original, close to a coin flip.
- Revenue per visitor: -7.7%, a negative result, as average order value fell 5.2%.
- Add-to-cart rate: -3.1%, a negative result.
- Bounce rate: -10.3%, so more visitors stayed, but that did not turn into sales.
The change
Collection pages gain stronger visuals, a reworked grid and extra content.
Simplified mockup drawn by LiftArchive from the shared screenshots.
Control
Standard collection pages, which received most of the store's ad traffic.
Variant
Collection pages with stronger visuals, an optimized product grid and informative landing-page style content.
Why it likely came out this way
Our reading of the result, not the practitioner's claim.
- Richer content kept visitors on the page but may have slowed the path to products.
- Shoppers from ads may already know what they want, so extra content adds little.
- With 26% power, real effects smaller than a large swing would not show up at all.
Caveats: Underpowered: 3,805 visitors and 26% statistical power. This looks like a separate run of a similar redesign tested on the same store, and several changes shipped together. Duration was not shared.
Who ran it
Clean Commit ran this test for a Shopify store (anonymized) and published the results (Clean Commit A/B test library). The numbers are theirs; the summary and commentary on this page are ours.
Read the original case study on Clean Commit →
How to run this test on your store
- Hypothesis
- If we add better visuals, an optimized grid and landing-page style content to collection pages, then conversion rate will rise because shoppers can find and explore products more easily.
- Primary metric
- Conversion rate for sessions landing on collection pages
- Guardrails
- Revenue per visitor
- Add-to-cart rate
- Who should test it
- Stores with enough traffic to detect small changes on collection pages. With a few thousand visitors, expect an underpowered test like this one; lower-traffic stores may learn more from user research.
Questions
Does lower bounce rate mean a page is working better?
Not on its own. Here bounce rate fell 10.3%, but conversion was flat and revenue per visitor fell 7.7%.
What does 26% statistical power mean?
The test had roughly a one in four chance of detecting the effect it was sized for.
A flat result at that power tells you little either way.
Similar tests
@oliverkenyon@oliverkenyon@oliverkenyon@oliverkenyon@oliverkenyon@oliverkenyon