Attribution Models and How They Skew A/B Test Results

Multi-touch attribution splits the credit for a conversion across several marketing touchpoints (an ad, a search, an email), instead of giving all of it to one.

It is a rule for sharing credit, not a measure of cause and effect.

That is why attribution reports and A/B tests often disagree, and why a randomized test or holdout is the better judge of what actually lifted sales.

This page covers how the main attribution models divide credit, where marketing attribution goes wrong, and how it can skew the results of tests in our A/B test database.

Attribution models: how each one splits the credit

An attribution model is the rule that decides which touchpoints get credit for a conversion.

Single-touch models give everything to one touchpoint. Multi-touch models spread it out.

Take one shopper. They click a Meta ad, come back a few days later through a Google search, then buy $120 of product from an email link.

Here is how common models would credit that order.

ModelRuleMeta adGoogle searchEmail
Last clickAll credit to the final touchpoint$0$0$120
First clickAll credit to the first touchpoint$120$0$0
LinearEqual shares$40$40$40
Position-based (40/20/40)40% first, 40% last, 20% split across the middle$48$24$48
Time decayMore credit to touchpoints closer to the saleLeastMoreMost
Data-drivenCredit estimated by a model from converting and non-converting pathsVaries by account

Same order, same $120. Depending on the model, the Meta ad is worth everything or nothing.

None of these rows tells you whether the order would have happened without the ad.

Which models GA4 and Google Ads still offer

Google removed first click, linear, time decay and position-based models from Google Ads and Google Analytics 4 in 2023. New selections stopped in June 2023, and conversion actions still on those models were switched to data-driven attribution in September 2023.

GA4 now lists three models: data-driven, paid and organic last click, and Google paid channels last click.

Google describes data-driven attribution as a counterfactual approach that compares what happened with what could have happened.

It is still a model fitted to observed paths, not a randomized experiment.

Our GA4 for CRO guide covers how to set this up for test analysis.

Single-touch vs multi-touch

Last click is simple and stable, but it overcredits the channels that close sales, like branded search and email.

First click overcredits discovery channels. Multi-touch models look fairer, but the split is still a choice someone made, or a model trained on correlations.

A good rule: pick one model, keep it fixed, and use it to compare channels over time.

Do not use it to decide whether a page change worked.

When attribution models are the wrong tool

  • Judging an on-site change. A new product page doesn't belong to one channel. Run an A/B test and compare variants on revenue per visitor.
  • Proving a channel is incremental. Attribution counts touchpoints on the path. It can't tell you whether those buyers would have bought anyway. Use a holdout group or a platform lift study.
  • Low-volume accounts. Data-driven models need lots of paths. With few conversions, the credit split is noisy.

Marketing attribution and how it skews A/B test results

Marketing attribution is the wider practice of tying revenue back to channels, campaigns and touchpoints.

It breaks in a few predictable ways, and each one can make a test look better or worse than it was.

Observational attribution overstates or misses the real effect

The strongest evidence here comes from Gordon, Zettelmeyer, Bhargava and Chapsky, published in Marketing Science in 2019. They took 15 US ad campaigns that Facebook ran as randomized experiments, about 500 million user-experiment observations and 1.6 billion impressions.

Then they estimated the same effects with matching and regression methods of the kind advertisers use.

The observational methods often failed to match the experimental results, even with extensive demographic and behavioral data.

The reason is that likely buyers are more likely to see ads in the first place.

Attribution can't separate "the ad caused this" from "the ad found someone who was going to buy."

Platforms grade their own homework

Each ad platform counts conversions inside its own windows.

Meta's attribution settings count events within 1 or 7 days after a link click, and within 1 day after an ad view.

Google Ads can claim the same order through its own click.

Add up the platform reports and you will often get more conversions than your store actually recorded.

Tracking gaps make this worse. Since iOS 14.5, apps need permission through Apple's App Tracking Transparency framework before they can track users across other companies' apps and sites.

Many paths now have missing steps that platforms fill in with modeling.

See our page on App Tracking Transparency for the app side.

Landing page tests reported in ad-platform metrics

Many paid-traffic tests in our database report CPA, CAC or ROAS.

Those numbers come from the ad platform's attribution, so they move with attribution windows and tracking quality as well as with the page.

These are useful results. But a CPA drop measured in Ads Manager is not the same as a randomized split of site visitors.

If the attribution setting, the tracking setup or the audience changed during the test, the CPA moves too.

When you can, confirm with a split test on the site and check store-level revenue, not just platform-reported conversions.

The metric you credit decides the winner

Attribution isn't only about channels. Inside a test, you also choose which step of the conversion funnel gets credit.

Crediting an early step can crown the wrong variant.

  • Clean Commit added a 'Shop This Color' button to product pages. Click-through, the goal metric, rose 2.6% at 94.9% probability. But conversion rate fell 2.6% and revenue per visitor fell 6.8% across 17,746 visitors over 60 days.
  • Clean Commit's filter panel on collection pages showed +10.1% conversion rate and +12.8% add-to-cart, while revenue per visitor fell 10.3% and AOV fell 16.1%.
  • Hubstaff's homepage redesign lifted visitor-to-trial 49% in a three-month split URL test with 259k visitors. Trial-to-paid fell 7%, which the team attributed to factors outside the homepage.

The fix is the same as with channel attribution: credit the outcome you care about.

Use revenue per visitor or paid conversions as the primary metric, and watch guardrail metrics downstream.

More on picking them in our guide to A/B testing metrics.

Channels that pay off later look worse in-session

Some on-site elements feed another channel. An email popup turns visitors into subscribers, and some of them buy later from an email.

In a channel report, that later sale belongs to email, not the popup.

A randomized test avoids that trap, because it credits the variant a visitor was assigned to, whatever brought them back.

Dave Diederen removed an email popup on a client store and it lost: -12.18% conversion rate and -20.51% revenue per visitor.

Dave Diederen reported the same result when repeating the test on a different store.

The limit is the test window. If most of the value arrives weeks later, a short test still undercounts it.

Holdouts measure what attribution can't

A holdout keeps a random slice of visitors away from a tactic and compares outcomes.

That gives you incremental lift, which no attribution model can.

OptiMonk ran a conversational popup for NuBest against visitors who didn't see it.

Orders were up 20.87% versus the holdout, with a 7.58% email signup rate.

The sample size wasn't given, so treat it as directional.

Self-reported attribution has a cost too

Some stores ask buyers "How did you hear about us?" to fill the gaps in tracked data.

That question is friction. Bonafide Health's cart page held a how-did-you-hear dropdown before checkout.

Visually tested a redirect that skipped that step and reported +5.6% checkout rate, with more shoppers filling in the dropdown.

It's vendor marketing with no sample size, but it shows that where you collect attribution data can affect conversion.

Segment by source instead of crediting by model

The most useful way to bring attribution data into testing is as a segment, using UTM parameters or referrer, not as a success metric.

Dave Diederen hid the 'As Seen On' press bar on the homepage for paid traffic and got +3.55% conversion rate and +2.61% revenue per visitor.

Visually showed Bonafide's Facebook ad traffic a reminder of their 20% off code in cart, for +3.5% conversion rate and +5.5% profit per session.

In both cases, attribution data decided who saw the variant. Randomization decided whether it worked.

A practical setup

  1. Use one attribution model in GA4 and leave it alone, so channel trends stay comparable.
  2. Judge on-site changes with A/B tests on revenue per visitor or another bottom-line metric.
  3. Judge channels and always-on tactics with holdouts or lift studies.
  4. Reconcile platform-reported conversions against store orders. A tool like Triple Whale can help with this on Shopify, but it's still attribution, not an experiment.
  5. Map the paths people take with customer journey mapping, then test the steps where they drop off.

Questions

What is multi-touch attribution?

Multi-touch attribution is a way of splitting credit for a conversion across several marketing touchpoints, such as an ad, a search and an email, instead of giving it all to one.

It shows how credit is shared, but it doesn't prove that any touchpoint caused the sale.

What are the main attribution models?

The common ones are last click, first click, linear, position-based, time decay and data-driven.

Google removed first click, linear, time decay and position-based from Google Ads and GA4 in 2023, so GA4 now offers data-driven and two last-click options.

Is multi-touch attribution better than last click?

It spreads credit more evenly, but it is still based on observed paths, not experiments.

For deciding whether a page change or a channel actually adds sales, an A/B test or holdout group is more reliable than either model.

How does attribution affect A/B test results?

Tests reported in ad-platform metrics like CPA or ROAS move with attribution windows and tracking gaps, not just the page.

Tests that credit an early step, like clicks, can also pick a variant that loses revenue.

Measure on-site tests with randomized splits and revenue per visitor.

What is marketing attribution?

Marketing attribution is the practice of tying revenue back to the channels, campaigns and touchpoints that came before it.

Ad platforms, analytics tools and post-purchase surveys each give a different view, and their totals rarely match.