Ecommerce personalization means showing different shoppers different content, offers or products based on what you know about them, such as whether they are new or returning, where they are, or what they are browsing.
In our database, the personalization tests that win most clearly are narrow ones: product recommendations in a specific spot, or one message aimed at one segment.
Broad "different site for each audience" redesigns usually land inconclusive.
This page covers what website personalization is, real examples with their numbers, how to build a strategy that holds up, and how personalization relates to A/B testing.
Every claim about what works is backed by tests you can open and read.
What website personalization is
Website personalization changes what a visitor sees based on a rule or a model.
A rule says "visitors from Canada see Canadian customer logos."
A model says "this visitor viewed three espresso grinders, so show grinders and beans on the homepage."
Most programs mix three building blocks:
- Who: the audience or segment, defined by traffic source, location, device, new vs returning status, purchase history or behavior in the session. See segmentation and behavioral targeting.
- What: the content that changes, such as a headline, a banner image, an offer, a product grid or a form message. See dynamic content.
- How: hand-written rules, or algorithms that choose per visitor, like recommendation engines and contextual bandits. See AI personalization.
Product recommendations are the oldest large-scale version. Amazon described its item-to-item collaborative filtering approach in a 2003 IEEE Internet Computing paper by Greg Linden, Brent Smith and Jeremy York.
The idea of "customers who bought this also bought" comes from that line of work.
Why stores invest in it
The most quoted industry numbers come from McKinsey's Next in Personalization 2021 report.
It found that 71% of consumers expect companies to deliver personalized interactions and 76% get frustrated when that doesn't happen.
It also reported that personalization most often drives a 10 to 15% revenue lift, with company-specific lift ranging from 5 to 25%.
Treat those as survey and consulting estimates, not test results.
They tell you shoppers notice relevance. They don't tell you which personalization will work on your site, or whether it will beat a well-built generic page.
That is what the tests below are for.
Personalization vs product customization
One naming trap: "personalized products" (engraving, monograms, custom prints) are a product feature, not website personalization.
In a Loop Earplugs test reported by Dash of CX, adding an engraved case option at checkout cost 3.04% in conversion rate.
That's a checkout upsell result, and it belongs on our upsells page rather than here.
Website personalization examples with real test results
Below are personalization tests from our database, grouped by what was personalized.
Each one links to the full write-up with the control, the variant and the caveats.
Several come from vendors who sell personalization software, so read the caveats before you copy anything.
Product recommendations
Recommendations are where personalization has the most clear wins in ecommerce.
- An Optimizely retail customer replaced static homepage products with visitor-specific recommendations and saw 136% more clicks to product pages. Note the metric: clicks, not orders.
- The same retailer sent searches with no matches to machine-matched products instead of a dead-end page, which lifted revenue 2.74% and click-through 2.11%, according to Optimizely.
- Craft Spirit Shop showed a popup with recommendations and a headline matched to the product being viewed. OptiMonk reports +38.5% purchase rate and +77.9% revenue per visitor against a no-popup control, with no sample size given.
- The Oodie added similar-product recommendations to out-of-stock pages that previously offered only a notify-me button: +9.5% conversion rate, reported by Visually.
Recommendations can also cost you. In a Clean Commit test, same-category recommendations on product pages raised average order value 19.7% but dropped conversion rate 9.9%.
Showing alternatives can pull people away from the product they came to buy.
The product recommendations page goes deeper on placement.
Location (geo) personalization
- VWO showed visitors from nine countries customer logos and results from their own country on its free trial and demo pages. The company reports +376% leads on the trial page and +247% on the demo page. VWO ran this on its own site, didn't publish sample sizes, and changed the variant mid-campaign for three countries.
- Humana put the same couple in front of a local landmark matched to the visitor's location, like the Golden Gate Bridge for the Bay Area. Banner clickthrough rose 31%.
- The Global Leadership Summit rebuilt its event landing page around each visitor's local host site. Clicks rose 32%, but registrations didn't change. This is the most common pattern in geo tests: more engagement at the top, nothing at the bottom.
New vs returning visitors
This is the most popular first personalization idea.
It's also where our evidence is weakest. Clean Commit ran several versions on Shopify stores:
| Test | What changed | Reported result | Status |
|---|---|---|---|
| Homepage, new vs returning | Educational homepage for new visitors, sales-focused one for returning | +241.1% conversion rate | Not significant; overlapping tests and holiday promotions interfered |
| Homepage, returning visitors | Highlighted offers and a shorter path to buy | +39.3% conversion rate | Not significant; the source page also cites +29.9% |
| Homepage, educational content | Brand story for new, product offers for returning | +0.5% conversion rate, +6.7% RPV | 58.6% probability; mobile +17.7%, desktop -24.4% |
| Product page, new visitors | Educational PDP for new, condensed PDP for returning | +2.4% conversion rate, -7.4% add-to-cart | 70.5% probability on 37,726 visitors |
| Product page, returning visitors | Same split | +1.0% conversion rate, orders flat | 2,581 visitors over 33 days |
The biggest numbers come from the noisiest tests.
When the test with the most traffic (37,726 visitors) shows +2.4%, the +241% headline from a test disrupted by holiday promotions is not a number to plan around.
Segment-targeted messages in lead gen
Lead-gen sites have some of the cleanest personalization wins, because one segment often has one clear fear or one unfinished task.
- A North American insurer showed returning visitors who had abandoned an application a quote reminder widget. Applications started rose 95.83% and submissions rose 23.47%.
- The same insurer showed a pop-in tailored to emotional segments to visitors who lingered without acting: +55.18% kit submits within the targeted segments.
- Groupama showed a data-privacy reassurance message only to one emotional segment on its quote form. Quote submissions rose 10% over two weeks.
All three used AB Tasty's EmotionsAI segments, and the results are measured inside the targeted group.
That's a fair way to measure targeted content, but the lift on total site leads is smaller.
Optimizely
Visually
Clean CommitEmail personalization
Email is where personalization has the longest track record, and the pattern is the same: relevant content beats a name token.
- AppVault replaced a first-name subject line with one naming the applicant tracking system already installed on the prospect's careers site. Unique opens went from 13% to 66%. That's opens only, with no sample size given.
- HomeAdvisor swapped generic follow-up emails for ones matched to each requester's remodeling project, and got 2x the conversion rate and 1.5x the clickthrough.
- The Global Leadership Summit named each past attendee's closest host site in its email, which brought 87 more registrations than the general version.
- Apollo Education Group added subscribers' names to a birthday email preheader for +9% opens, but an earlier run of the same test did poorly. Name tokens are a weak lever.
More on this in our email A/B testing guide.
When "personal" hurts
Personal is not always better. In a MarketingExperiments test, removing the writer's photo, greeting and signature from a lead page raised conversion from 34.6% to 39.9%.
That test is about tone rather than targeting.
Still, it's a useful check on the assumption that anything that feels more personal will convert better.
How to build a personalization strategy
A personalization strategy is a short list of segments, each with a specific reason to see something different, and a plan to prove the difference pays off.
Here is the order we'd follow, based on what the tests above show.
1. Start where intent is already known
The strongest results in our database target visitors whose need is obvious from their behavior: a search that returned nothing, a product that sold out, an application left half done.
Those are cheap to detect and hard to get wrong.
Weaker results come from guessing what a broad group wants, like "new visitors want education."
That guess can be right on mobile and wrong on desktop, as the Clean Commit educational homepage test showed (+17.7% vs -24.4%).
2. Pick segments big enough to test
Every segment you personalize for splits your traffic.
A segment that's 8% of visitors needs about 12 times as long as the whole site to reach the same sample size.
Run your segment's traffic through the sample size calculator before you build anything.
A practical rule: start with two to four segments that each cover a large share of revenue.
Ten micro-segments with a custom experience each sounds smart but is usually impossible to measure.
Traffic-source personalization
One low-risk segment is the ad or email a visitor clicked.
If someone arrives from an ad about a specific product or offer, the landing page can repeat that headline and offer.
This is close to ad message match, and it needs no profile data at all, just the URL parameters you already pass.
3. Change one meaningful thing per segment
The geo and segment wins above changed one element: a logo strip, a banner photo, a reassurance line, a quote widget.
The new vs returning tests rebuilt whole pages and got noisy, mixed results.
A narrow change is easier to build, easier to measure and easier to explain when it wins.
4. Measure revenue, not just clicks
Several personalization wins stop at clicks: 136% more product page clicks, 31% more banner clicks, 32% more landing page clicks with no registration lift.
Pick a primary metric close to money (orders, revenue per visitor, submitted leads) and treat clicks as a secondary signal.
5. Keep a holdout and re-check
Personalization is easy to switch on and forget.
Keep a small group that sees the default experience so you can see whether the personalized version still earns its keep months later. Our holdout groups guide covers sizing.
Checklist
| Question | Good sign | Warning sign |
|---|---|---|
| Why does this segment need something different? | Observed behavior (abandoned quote, zero-result search) | A persona guess |
| How big is the segment? | Enough traffic for a test in 2 to 6 weeks | Needs months to reach a sample |
| What changes? | One element or message | A full page redesign per segment |
| What's the primary metric? | Orders, revenue per visitor, submitted leads | Clicks or time on site |
| Is there a control? | A holdout that sees the default | A before/after comparison |
Common personalization mistakes
- Reporting the segment lift as a site lift. A 23% gain among returning abandoners is not a 23% gain in total leads. Report both numbers.
- Comparing to last month. Before/after comparisons mix the personalization with seasonality, promotions and ad changes. Use a concurrent holdout.
- Personalizing on thin data. A visitor who viewed one product isn't a "category lover." Rules built on a single signal tend to show irrelevant content.
- Too many experiences to maintain. Each segment version needs updating when prices, stock or branding change. Stale personalized content is worse than a good generic page.
- Ignoring the device split. Results can flip between mobile and desktop, so check both before rolling out.
Personalization vs A/B testing
A/B testing asks "which version is best for everyone in this test?"
Personalization asks "which version is best for each group?"
They aren't rivals. Personalization is a decision about who sees what, and an A/B test with a control group is how you find out whether that decision helps.
| A/B testing | Rule-based personalization | Algorithmic personalization | |
|---|---|---|---|
| Goal | Find one winner for the tested audience | Serve a chosen experience to a defined segment | Pick the best option per visitor |
| Who decides | The test result | You, by writing the rule | A model (recommendations, contextual bandits) |
| Control | The original version | A holdback that sees the default | A holdback or exploration traffic |
| Traffic needed | Moderate | Moderate per segment | High, and it grows with the number of options |
| Main risk | Shipping a false winner | Rules that never get re-checked | A black box you can't explain |
How the tools measure personalization
Personalization tools build the A/B test in. In Optimizely Personalization, the default holdback is 5%: 95% of campaign visitors see a personalized experience and 5% see the original, and the reported improvement is the relative difference between the two.
Wingify's (VWO's) Personalize product works the same way: with holdback on, it creates a control group and compares all experiences combined against it.
For algorithmic choices, Optimizely's contextual bandits pick a variation for each visitor based on their profile.
The exploration rate must stay between 5% and 50%.
For how bandits trade learning against earning, see multi-armed bandits.
Worked example: why a 5% holdback is slow
Say a campaign gets 200,000 visitors a month and converts at 3%.
With a 95/5 split, the holdback gets 10,000 visitors a month, or about 300 conversions.
To detect a 10% relative lift (3.0% to 3.3%) at 80% power and 5% significance, a standard two-sample calculation needs roughly 53,000 visitors per group.
The personalized side gets there in about nine days.
The 5% holdback needs over five months. Moving to a 20% holdback gets the holdback 40,000 visitors a month, so you'd reach a reading in under two months.
So the choice is: a small holdback costs less revenue (if the personalization is better) but takes far longer to tell you anything.
Size it with the power you need, not by habit.
When personalization is the wrong choice
- Your site has basic problems. Slow pages, a confusing checkout or unclear shipping costs hurt every segment. Fix them with ordinary A/B tests first.
- Your traffic is small. If one site-wide test takes a month, a segment-level test takes many months. Run broad tests until traffic grows.
- You can't explain why the segment differs. Without a reason, you're testing random variations on a smaller sample.
- You'll run other tests on the same pages. Overlapping tests can distort each other, which is one reason the +241% Clean Commit result wasn't trusted. See interaction effects.
Read segment results with care
Cutting an A/B test by segment after the fact, then personalizing for whichever segment "won," is a common mistake.
With enough cuts, some segment will look like a winner by chance.
Treat a segment result from an existing test as an idea, then run a new test targeted at that segment to confirm it.
Getting started
If you're starting from zero, a sensible first quarter looks like this: add recommendations to dead ends (zero-result searches, out-of-stock pages, empty carts), then test one geo or traffic-source message on your highest-traffic landing page, each with a holdout.
Leave full new vs returning redesigns until you have the traffic to measure them.
To compare platforms, see our website personalization tools guide.
To find more tests like the ones above, browse all test results.