Google Ads experiments let you split a Search, Display, Video or Hotel campaign's budget between the original and a modified copy, then compare results before applying the change.
Meta's A/B test tool does the same for Facebook and Instagram ads, keeping the two audiences apart so nobody sees both versions.
Both tools tell you which ad won on that platform, but neither fully isolates the creative from the algorithm's choice of who saw it.
That last point matters more than most guides admit.
Below we cover how each tool works, what real ad tests in our database show, and where the click stops being the right thing to measure.
Facebook ad testing: what the platform can and can't tell you
Meta describes its A/B test as a way to compare two versions of an ad strategy by changing one variable, such as images, text, audience or placement.
According to Meta's help page on A/B testing, it shows each version to a segment of your audience and ensures nobody sees both.
Meta also warns against informal testing, such as turning ad sets on and off by hand.
That leads to overlapping audiences and unreliable results.
Comparing last week's ad with this week's is not a test either, because seasonality, auction pressure and learning-phase effects all change at once.
The divergent delivery problem
Even a proper platform test has a catch.
In a 2025 Journal of Marketing paper, Michael Braun and Eric M. Schwartz showed that ad platforms deliver each version to a different, algorithm-chosen mix of users.
As the American Marketing Association's summary puts it, the "winning" ad may have won because it was shown to people more likely to respond.
Their advice splits by goal:
- Predicting performance on the same platform: the built-in tests are fine, because the algorithm will keep targeting that way.
- Learning what your customers respond to: treat the result as a hint, not proof, since the two audiences differed.
- Reusing the message elsewhere (your site, email, another ad network): don't treat the result as causal evidence about the creative.
If you want to know whether a message works for your customers in general, test it on your own site with a randomized A/B test, where you control who sees what.
Ad creative testing: what the tests in our database show
Our database holds a small set of ad-level tests and a much larger set of tests on the pages ads point to.
You can browse them on the ad tests hub or in the full test library. Four patterns stand out.
1. A higher click-through rate can lose money
Patrick McKenzie's 2006 search ad tests for Bingo Card Creator are the clearest example.
A short ad offering free word lists lifted click-through rate from about 1.1% to 5.8%.
But the trial conversion rate fell from 22% to 14%, and cost per trial rose from about 32 cents to about 43 cents.
A rhyming-poem ad in the same ad group roughly doubled CTR (1.1% to 2.13%) with cost per acquisition about flat.
Both tests ran for about a week on under $5 of spend, so the samples were tiny.
The lesson still holds: the metric you pick decides the winner.
| Per 1,000 impressions (worked from the reported rates) | Plain ad | Free word-list ad |
|---|---|---|
| Click-through rate | 1.1% | 5.8% |
| Clicks | 11 | 58 |
| Trial conversion rate | 22% | 14% |
| Trials | about 2.4 | about 8.1 |
| Reported cost per trial | about $0.32 | about $0.43 |
The word-list ad produced more trials per impression but each trial cost more.
Which one wins depends on whether you're limited by budget or by impressions.
Decide that before the test starts, as part of your hypothesis.
2. Fewer clicks can mean more customers
A large financial institution tested a clearer credit card banner, run by MarketingExperiments.
The new design made it obvious the click led to a card application.
Banner clicks fell 35%, but applications per day went up.
The control had been buying curious clicks from people who never meant to apply.
3. The angle matters more than polish
Oliver Kenyon tested a pain-relief angle against a longevity angle for the same dog supplement.
The new angle brought +28% CTR and an estimated +$116k a month.
HomEquity Bank found that longer Facebook videos with larger text, built to play without sound for an older audience, beat its previous best ads by 30%, though the source doesn't name the metric.
Dashlane saw a 93% higher clickthrough rate once review stars appeared on its paid search ads.
That was a before-and-after comparison, not a split test, so read it as a strong signal rather than a measured effect.
4. The landing page has to keep the ad's promise
Many of the biggest "ad" wins in our data came from changing where the click lands.
Oliver Kenyon added the founder story from the ads to the product page and cut CPA by 15%.
Dave Diederen matched the social proof line under the add-to-cart button to the ad angle each visitor came from, for +9.74% revenue per visitor.
Journey Further sent High Street TV's paid traffic to a dedicated PPC landing page instead of the product page and got +32.62% purchases, over six weeks with about 7,500 users per arm.
See ad message match for more tests like these, including advertorial and listicle pages built for cold traffic.
@oliverkenyon@oliverkenyon
MarketingExperimentsMeta A/B test vs Google Ads experiments: how to run each one
Setting up a Meta A/B test
Meta's guide to A/B test types recommends starting from the Ads Manager toolbar: tick the campaign or ad set, click A/B test, and pick a variable.
You can also build tests in the Experiments tool or duplicate an existing campaign, ad set or ad. Finished results show up in Experiments.
Meta recommends the same budget for both versions and measures them on cost per result or cost per conversion lift.
Change one variable at a time, or you won't know which change caused the difference.
Setting up a Google Ads experiment
Google's custom experiments help page says these are available for Search, Display, Video and Hotel Ads campaigns, not App or Shopping.
You pick what share of the original campaign's budget goes to the experiment arm.
When it ends, you can apply the change to the original campaign or turn the experiment into a new campaign.
You can schedule up to five experiments, but only one runs at a time.
For Search campaigns you choose how traffic is split:
| Split type | How it works | Use it for |
|---|---|---|
| Cookie-based | Each user sees only the original or only the experiment | Landing page changes, bidding toward conversions that happen days later |
| Search-based | Each search is assigned at random, so one person can see both | Changes that act on a single query, such as match type |
Display experiments always use cookie splits. If your change affects anything after the click, use a cookie split, or the same person can land in both arms and blur the result.
Rules that apply to both platforms
- Judge on cost per acquisition or revenue, not CTR. The Bingo Card Creator and credit card tests show CTR can point the wrong way. Pick one primary metric before launch and treat CTR as a diagnostic, not the goal.
- Plan the sample size before launch. Clean Commit's test that sent ad traffic to the homepage showed +103.9% conversion on just 2,777 visitors, with 7% power. That's noise, not a result. Use a minimum detectable effect to work out the traffic you need.
- Run full weeks. Weekday and weekend buyers behave differently. Run at least one full week, and two if your sales cycle is longer than a day or two.
- Re-check wins as spend grows. A result from a small budget may not hold once the platform reaches colder audiences, so rerun or monitor the winner at the new spend level before you call it settled.
- Measure the page, not just the ad. When you change the destination, a split URL test on your own site, with visitors randomized by your testing tool, gives a cleaner read than an ad-platform test.
When a platform test is the wrong tool
Platform A/B tests answer "which ad does better on this platform right now?"
They are the wrong choice when you need the incremental effect of advertising itself (use a holdout group or a lift study).
They're also wrong when you want to know if a message works on your own site, because divergent delivery mixes up the creative with the audience.
If your ad tests keep winning on CTR while CPA stays flat, the problem is usually the page after the click.
That's where landing page work pays off: match the headline, offer and proof to what the ad promised.