Yes, you can usually run multiple A/B tests at once on the same site, and the results stay valid.
Random assignment spreads each test's visitors evenly across the other tests' variants, so one test can't bias another; the only real risk is an interaction effect, where two changes behave differently together than apart.
That risk is rare in practice and is concentrated in tests that touch the same page element, the same funnel step or the same goal.
This guide explains what an interaction effect is, how to check for one with a worked example, and when isolating concurrent experiments is worth the traffic it costs.
Each point is backed by tests from the LiftArchive database.
What interaction effects are in A/B testing
An interaction effect means the lift from change A depends on whether a visitor also sees change B. If a new headline lifts conversion by 10% on the old hero image but does nothing on the new hero image, the headline and the image interact.
Interactions are not the same as two tests running at the same time.
Two tests can overlap on every visitor and never interact.
Overlap is a fact about traffic; interaction is a fact about how people react to the changes together.
Why overlap alone doesn't bias your results
Say test A and test B each split visitors 50/50, independently.
Inside A's control group, about half the visitors see B's control and half see B's variant.
The same mix sits inside A's variant group.
B's influence is spread evenly over both sides of A, so it cancels out of A's comparison.
What overlap does add is a little extra noise, because B's variant moves some visitors' behavior around.
With enough traffic this is minor. It is the same kind of noise as a sale, a holiday or an ad campaign that hits both groups equally.
The thing overlap can't protect you from is a changed answer.
A's measured lift is its average effect across the current mix of B visitors.
If B later ships to 100% of traffic and the two interact, A's real-world effect will differ from what the test reported.
How to check for an interaction: a worked example
The numbers below are illustrative, not from a real test.
A store runs two overlapping tests: A is a new product-page headline, B is a free-shipping banner.
Each cell of the 2x2 gets 10,000 visitors.
| Cell | Visitors | Conversion rate |
|---|---|---|
| Control (no A, no B) | 10,000 | 3.0% |
| A only | 10,000 | 3.6% |
| B only | 10,000 | 3.5% |
| A and B | 10,000 | 3.4% |
Read A's effect separately in each B group.
Without the banner, A adds 0.6 points (3.0% to 3.6%).
With the banner, A loses 0.1 points (3.5% to 3.4%).
The interaction is the difference between those two effects:
interaction = (rate_AB − rate_B) − (rate_A − rate_control) = (3.4 − 3.5) − (3.6 − 3.0) = −0.7 points
Its standard error is the square root of the summed variances of the four cells:
SE = sqrt( Σ p(1 − p) / n ) = sqrt(0.00000291 + 0.00000347 + 0.00000338 + 0.00000328) ≈ 0.36 points
So z = −0.7 / 0.36 ≈ −1.94, a two-sided p-value of about 0.052. Even a full reversal of A's effect only just misses the usual 0.05 cut-off with 40,000 visitors.
Now look at what the overlapping test reported for A on its own.
A's arms average 3.5% (half 3.6%, half 3.4%) against 3.25% for control, a +7.7% relative lift.
That number is a fair answer for today's traffic mix, but if the banner ships to everyone, A is a slight loser.
In practice, most tools won't run this check for you.
Export visitor-level data with both test assignments, then run a chi-square test across the four cells or a logistic regression with an A×B term.
Microsoft's experimentation team used a chi-square test per pair of tests for its own interaction scan.
Why interaction checks are usually underpowered
Look at the standard error again. The interaction uses all four cells, so its standard error is about twice that of A's main effect.
To detect an interaction as large as the main effect, you need about 4 times the sample.
Andrew Gelman works through the common case where the interaction is half the size of the main effect: that needs 16 times the sample size.
This is why a "no significant interaction" result rarely proves there isn't one.
It usually means the test wasn't big enough to tell.
See statistical power for how this works for single tests.
What factorial tests in our database show
The cleanest way to measure interactions is to design for them in a multivariate (factorial) test, where every combination gets its own cell.
- For Hyundai Netherlands, Traffic4U ran a 2x2x2 multivariate test with 8 combinations of page text, extra CTA buttons and a large car photo. The best combination lifted test-drive requests by 62% and click-through to funnel step 2 by 208%.
- Convertibles ran a 2x2 of content type × placement on a luxury pajama homepage. Press logos right below the hero won at +$109,829/month, while the USP bar versions and lower placements lost. The same content won or lost depending on where it sat, which is the kind of dependency an interaction describes.
- Online Dialogue's test for Van der Valk hotels ran one arm with all changes combined and separate arms for each change alone. A clear "book a hotel room" button gave at least +13% conversions, promoting last-minute deals cut conversions, and free coffee and Wi-Fi had no significant effect. A combined-only test would have hidden that one change was working against the others.
Combining separate winners also doesn't simply add their lifts.
Stardust's paywall tests won separately on a free trial and on pricing display.
When Stardust combined those winners, the version without a video background gave +55.7% paywall conversion and +9% proceeds per user, and the video version gave +67% and +12.5%. If you stack winners, test the stack.
Factorial designs need traffic. MarketingExperiments ran a six-factor test on a free-download page with only about 70 to 76 views per factor level.
The publisher called it a micro-test, and results that small shouldn't drive decisions.
Convertibles
Blend Commerce
Online DialogueRunning concurrent experiments safely
Concurrent experiments are A/B tests that run at the same time on the same audience.
Large testing programs run them by default, because waiting for each test to finish before starting the next would cut their test volume by a large factor.
What large testing programs found
In a 2023 analysis, Monwhea Jeng of Microsoft's Experimentation Platform checked every pair of A/B tests running on one day in four major Microsoft products, where tests are assigned independently and not isolated.
Three products showed no detectable interactions. The fourth showed a tiny number: 0.002% of test-pair metrics, or 1 in 50,000.
None of those few interactions showed two significant effects moving in opposite directions.
The analysis found that the two treatment effects either pointed the same way at different sizes, or one of them wasn't statistically significant.
Google built its experiment system around overlap. The 2010 KDD paper by Diane Tang and colleagues describes splitting the system into layers with independent diversion, so a single request can be in several experiments at once.
Changes that could conflict, such as two tests on the same parameter, go in the same layer and stay mutually exclusive.
Your store is not Microsoft or Google, and a small site has fewer tests competing for the same page.
The lesson still carries over: overlapping tests are the normal case, and isolation is the exception you choose on purpose.
When to isolate tests
Optimizely's Web Experimentation docs recommend mutually exclusive experiments when you are testing elements on the same page with the same goal, or subsequent funnel pages with the same goal.
Tests on different areas of the site, or with different goals, are less likely to interact.
The same page warns that mutual exclusion adds implementation effort and can significantly slow your testing cycle.
| Approach | Use it when | Cost | Wrong choice when |
|---|---|---|---|
| Overlap (independent splits) | Tests sit on different pages, elements or goals | A little extra noise | Two tests change the same element, or one variant breaks the other's layout |
| Mutual exclusion | Same page and same goal, or consecutive funnel steps with the same goal | Each test gets only part of the traffic, so it runs longer | You plan to ship both changes together, since you never measure the combination |
| Sequential | The second test depends on the first one's winner | Calendar time | Traffic is high enough to run both now with a clean split |
| Factorial / multivariate | You expect the changes to interact and want to measure it | Needs much more traffic per cell | Low traffic: cells end up too small to read, as in the micro-test above |
Here is the traffic math for exclusion. Say your checkout gets 40,000 visitors a week and each of two tests needs 40,000 visitors in total.
With overlap, both finish in one week. With mutual exclusion, each test gets 20,000 visitors a week, so both take two weeks.
Use the sample size calculator to run your own numbers.
If you use exclusion groups, follow Optimizely's rule from its Feature Experimentation docs: start every experiment in the group at the same time, and don't add or remove running experiments, because it may have unintended effects on your results.
How overlap shows up in real tests
Several tests in our database carry an overlap caveat.
None were thrown out, but each result is weaker because of it.
- Blend Commerce's image-led mobile menu for Fame showed +7.75% revenue per visitor, but briefly overlapped a checkout test. Blend labeled the result directional.
- Blend's category tiles test for a travel brand overlapped a product launch. Excluding the launch page it showed +13% RPV; across the full period, conversion was down 2%. A launch isn't a test, but it shows how outside changes can swing a short result.
- Patrick McKenzie's madlibs sign-up form test for Bingo Card Creator lost at 21.73% vs 27.55% (about −22%), with another A/B test running on the same page at the same time. Overlap on the same page is the case where an interaction is most plausible.
- Clean Commit's personalized homepage for new vs returning visitors reported +241% conversion rate but didn't reach significance, and Clean Commit noted that overlapping tests and holiday promotions interfered.
A practical checklist for concurrent tests
- List every live test with its page, element and primary metric. Overlap freely when two tests share none of these.
- Isolate, or run in sequence, when two tests change the same element, or when one variant can visually break the other (for example, two tests both editing the add-to-cart area).
- QA the combinations, not just each variant. A 2x2 overlap means four page versions are live.
- After the test, cut each result by the other test's assignment. A large gap is worth a look, even if it isn't significant.
- Before shipping two winners together, run the combination against control or keep a holdout group to confirm the combined effect.
For the wider program view, see building an experimentation program, and the A/B testing hub for the rest of this series.