To prioritize A/B tests, score every idea in your backlog on the same few criteria and run the highest-scoring idea first. ICE (impact, confidence, ease) is the fastest to use, PIE (potential, importance, ease) ranks pages instead of ideas, and PXL swaps opinion scores for yes/no questions. Whatever framework you pick, base the confidence score on evidence, such as how similar tests turned out in our database, not on gut feel.
Prioritizing well matters because most tests lose or come out flat. Optimizely's analysis of its customers' experiments found that only 12% produce a statistically significant improvement on the primary metric. With odds like that, picking the next test carefully counts for more than almost anything else in a testing program.
ICE: impact, confidence, ease
ICE was created by Sean Ellis, who coined the term "growth hacking." You rate each idea from 1 to 10 on three questions, then multiply the scores (some teams average them instead) to get one number per idea.
- Impact: if this wins, how much will it move revenue or conversion rate?
- Confidence: how sure are you that it will win, and what evidence backs that up?
- Ease: how much design, development and sign-off work does it take to launch?
ICE is quick, and that is its main strength. A team can score 30 ideas in one meeting.
Its weak point is that all three scores are opinions. Two people will give the same idea a 4 and an 8 for impact, and the loudest person in the room tends to win. Speero's write-up on prioritization also points out a contradiction: if you are already confident an idea will win, why test it? The fix is to define what each score means before you start, and tie confidence to evidence. We cover how to do that further down.
PIE: potential, importance, ease
PIE comes from Chris Goward, founder of the agency WiderFunnel. It works at the level of pages rather than individual ideas. In Goward's own explanation, the three factors are:
- Potential: how much improvement can be made on the page? Pages that perform poorly score higher.
- Importance: how valuable is the traffic to the page? Think volume and the cost of acquiring that traffic.
- Ease: how hard is the test to get running, including technical work and internal politics?
PIE is useful early on, when you are deciding where to focus: product page, cart, collection pages or checkout. For most stores, the product page wins on importance because nearly every buyer sees it. That matches what practitioners test most often: 128 of the 285 results in LiftArchive are product page tests (see the most-tested elements).
The downside is the same as ICE. "Potential" is a judgment call, and PIE tells you which page to work on, not which idea to try first on that page. Many teams use PIE to pick the page, then ICE or PXL to rank the ideas for it.
PXL: yes/no questions instead of opinions
PXL was published by Peep Laja's team at CXL to cut down on guesswork. Instead of 1-10 ratings, PXL asks mostly yes/no questions, scored 1 or 0. A few factors that matter more get double weight (2 or 0).
The questions fall into three groups:
- Visibility and impact: is the change above the fold? Is it on a high-traffic page? Will people notice it within 5 seconds? Does it add or remove an element?
- Evidence: is the idea backed by user testing, survey feedback, heatmaps or session recordings, or analytics findings?
- Effort: a time-to-implement estimate, ideally made with a developer.
Binary questions are harder to fudge. Either the change is above the fold or it isn't. PXL also rewards research, which pushes teams toward ideas grounded in customer research instead of competitor copying.
The cost is setup time, and the "above the fold" and "noticeable" questions favor big, visible changes. Those are usually the right bet, but PXL can undervalue quieter fixes like shipping copy in the cart. You can add your own questions, such as "does this address a known objection from surveys?"
Our data supports the visibility questions. Dave Diederen saw +$47k/month after adding one written customer review above the fold on a product page, and Kasper Dolk reports $823,115.54 extra revenue per year from one hero headline test. Both are above the fold and noticed in seconds.
@DaveDiederen@DaveDiederen@DaveDiederen
@kasper_dolkUsing database win rates as priors
A "prior" is what you believe before the test runs. Every framework has a slot for it: confidence in ICE, potential in PIE, the evidence questions in PXL. Most teams fill that slot with a hunch. You can fill it with the track record of similar tests instead.
LiftArchive holds 285 real ecommerce test results, tagged by page type and element. Before scoring an idea, look up how tests on the same element have gone. Three patterns are worth watching for.
1. Ideas that keep winning across stores
Some changes have won so often that they deserve a high confidence score. Dave Diederen reports that showing the price directly below the product title has a ~90% win rate across 20+ brands. Kasper Dolk saw ~10% conversion increases across multiple clients from an outcome headline that handles one objection at the top of the product page. Headline and copy tests are the most common category in our database, with 45 results, and only one of them is recorded as flat. Browse headline and copy tests for more.
2. Ideas with mixed or losing records
Other elements go both ways, and that should pull your confidence score down. Countdown timers are the clearest case. Dave Diederen saw +3.16% RPV from adding a countdown timer to checkout, but -7.1% RPV from the same idea on another store, and -35.61% RPV in a European market. Kasper Dolk went the other way and saw CVR +25.61% by removing a checkout countdown timer. Three of the 15 countdown timer results in our database are losses or mixed, the highest share of any common element.
Offer changes carry a similar risk. Milos (CROguy_) recorded -52.38% RPV (-$104,679/month) after changing a 2-pack offer to buy 2, get 1. A mixed record doesn't mean "don't test it." It means score confidence lower and expect the test to need more traffic. Our losing tests research collects more of these.
@DaveDiederen@kasper_dolk@DaveDiederen
@CROguy_@DaveDiederen3. Ideas where the direction is unclear
Sometimes the same element wins whether you add it or remove it. Dave Diederen saw +11.76% CVR from moving the review section directly below the above-the-fold area, yet also +13.36% CVR from hiding written reviews above the fold, a result Dave Diederen says won in 8 out of 10 tests. When the database shows wins in both directions, the element clearly matters, so score impact high. But your confidence about which version wins should stay moderate.
A caution about our numbers
Practitioners share wins far more often than losses. Only 15 of our 285 results are recorded as losses or flat, which is nothing like the 12% overall win rate Optimizely reports. So don't read our database as "most tests win." Use it as a relative signal: compare how often an element wins, how often it loses, and whether wins repeat across different stores. See our A/B test win rate research for the full picture.
A simple way to turn this into a confidence score (our suggestion, not a standard):
- Start every idea at 5 out of 10.
- Add 2 if three or more similar tests won on the same page type, with no losses on record.
- Subtract 2 if similar tests have lost on record.
- Add 1 for each source of your own evidence: surveys, heatmaps, session recordings or analytics.
- Cap it at 9. Nothing is certain, which is why you are testing.
A prioritized backlog template
A backlog is just a table. Every idea gets a row, every row gets scored the same way, and you sort by score. Start with a written hypothesis for each idea so everyone scores the same thing. If you need ideas to fill it, browse our test ideas.
Here is a filled-in example using ICE, with confidence set by the method above. Scores are multiplied.
| Idea | Page | Evidence | Impact | Confidence | Ease | ICE score |
|---|---|---|---|---|---|---|
| Rewrite product title as a benefit-led headline | Product page | Many wins in database; survey quotes on desired outcome | 8 | 8 | 9 | 576 |
| Add "ships tomorrow" line under add to cart | Product page | Repeated wins in database; support tickets about delivery times | 6 | 8 | 8 | 384 |
| Add countdown timer to checkout | Checkout | Mixed record, including large losses | 6 | 3 | 7 | 126 |
| Rebuild the collection page grid | Collection page | Heatmaps show little scrolling | 7 | 6 | 3 | 126 |
The headline test goes first. It has strong evidence and costs little to build. Dave Diederen saw +10.47% CVR from a benefit-led product title, one of many similar wins in our database.
Columns worth adding to your own sheet:
- Hypothesis: "Because we saw X, we believe changing Y will cause Z."
- Primary metric: conversion rate, revenue per visitor or profit per visitor. Pick one before launch.
- Traffic check: can this page reach the sample size you need in 2-4 weeks? Run it through the sample size calculator. If not, the idea drops down the list no matter its score.
- Similar tests: links to the LiftArchive results you used for the confidence score.
- Status and result: so finished tests feed back into future confidence scores.
Rescore the backlog after every test. A loss on one element should lower confidence for related ideas, and a win should raise it.
Where your testing tool fits
Prioritization happens in a spreadsheet, but you need a tool to run the tests. Convert is one option built for this kind of program. As of November 2026, Convert's pricing page lists the Growth plan at $299/month billed annually ($399 month to month) for 100K tested users per month, with a 15-day free trial and no credit card required. Multivariate and multi-page testing need the Pro plan, at $420/month billed annually.
That price is hard to justify for a small store. If you can't reach the sample size for your top-scoring idea, Convert is not for you yet; read our guide to testing with low traffic first. Our Convert review covers the tool's strengths and drawbacks.