# Homepage personalized for new vs returning visitors: flat result

- Outcome: Flat / inconclusive
- Page type: Homepage (Ecommerce)
- Elements: Layout, fonts & visual design
- Test run by: [Clean Commit](https://liftarchive.com/practitioners/clean-commit/)
- Commentary by: LiftArchive (our analysis, not theirs)
- Original source: https://cleancommit.io/ab-tests/redesigned-homepage-with-extra-educational-content/
- Page: https://liftarchive.com/test/cleancommit-redesigned-homepage-with-extra-educational-content/

Showing new visitors an educational homepage and returning visitors a sales-focused one changed conversion rate by just 0.5%, with revenue per visitor up 6.7% on weak odds. Clean Commit ran this test for an anonymized Shopify store and published it in its A/B test library. Mobile and desktop moved in opposite directions, so we count it as inconclusive.

## Result

**flat result**

- Conversion rate: +0.5%. No clear difference.
- Revenue per visitor (the goal metric): +6.7%, with only a 58.6% probability of beating the original.
- Add-to-cart rate: +11.6% and click-through rate +16.7%. More browsing, not clearly more buying.
- By device: conversion rate +17.7% on mobile but -24.4% on desktop. The two cancelled out.

## Control

Every visitor saw the same homepage.

## Variant

New visitors saw a homepage that introduced the brand and explained its main benefits. Returning visitors saw product highlights, offers and direct calls to action.

## Why it likely came out this way (our reading)

- Educational content may help a first-time mobile visitor but slow down a desktop visitor who came ready to shop.
- Engagement metrics rose, which suggests the pages got attention without changing the decision to buy.
- Splitting traffic into two experiences halves the data behind each one, which makes a clear read harder.

## Caveats

The goal metric had a 58.6% probability of beating the original, which is close to a coin flip. Sample size and duration were not shared.

## How to run this test

- Hypothesis: If we show new visitors an educational homepage and returning visitors a shop-first homepage, then revenue per visitor will rise because each group gets what it needs at its stage.
- Primary metric: Revenue per visitor
- Guardrail: Conversion rate by device
- Guardrail: Conversion rate for new vs returning visitors
- Who should test it: Brands with a product that needs explaining and a healthy share of repeat visitors. Plan to read results by device and visitor type, and expect to need a lot of traffic.

---

Credit: Clean Commit ran this test and owns the result. Summary and commentary by LiftArchive (https://liftarchive.com/test/cleancommit-redesigned-homepage-with-extra-educational-content/).
Usage: Free to use, quote and build on, with credit and a link: name the practitioner or organization that ran the test and link to its LiftArchive page. The results belong to the practitioners who ran and shared them; the summaries and commentary are LiftArchive's.