Incrementality testing is the method with the best reputation in marketing measurement and the smallest share of the shopping. Buyers describe it as the thing finance believes, the referee when the dashboards disagree, the only number that survives a budget defence — and then they go to market for attribution three times as often. Both of those things are true at once, and the gap between them is what this page is about.
What is incrementality testing?
Incrementality testing measures whether marketing caused outcomes that would not have happened without it. It works by deliberately varying marketing exposure between comparable test and control groups and measuring the difference in outcomes. Common approaches are audience holdouts and geo-lift tests.
In an ecommerce context the outcome is usually sales, but the method applies equally to leads, signups or installs.
That makes it an experiment rather than a model. Attribution assigns credit for conversions that already happened. Incrementality testing asks whether those conversions needed the advertising at all.
The distinction matters because the two can disagree sharply and both be internally consistent. An attribution tool can confidently credit a retargeting campaign for thousands of conversions that a holdout test shows would have happened without it. Neither tool is broken; they are answering different questions.
What is the difference between incrementality testing, MTA and MMM?
The methods are often grouped together, but they answer different measurement questions.
Which touchpoints deserve credit for this conversion? A model built on observed customer journeys — useful for journey-level optimisation, but limited to the touchpoints it can observe.
Did this spend cause anything? The experimental approach of the three: exposure is deliberately varied between comparable groups and the difference is measured. Narrower in scope, but valued by buyers because the causal result is harder to dispute.
How much did each channel contribute overall? Models historical spend against outcomes, including offline and channels no tracker can see. Broad, but dependent on sufficient historical variation in spend and outcomes to model against.
Of these three methods, incrementality testing is the experimental approach. That property is why buyers reach for it to check the other two, and it is also what creates its practical constraints: tests need enough volume, enough time, and enough opportunities to run them.
Is incrementality testing replacing attribution?
No — on the evidence of what buyers are actually in market for. Roughly three times as many buyers in this corpus are shopping for attribution as for incrementality testing, and roughly two in three of those shopping for incrementality testing are simultaneously in market for attribution or marketing-mix modelling.
The pattern is addition, not substitution. What is genuinely shifting is trust, not the shopping list.
The replacement story is intuitive enough that it is worth testing directly. If incrementality testing were displacing attribution, three things would show up in buying behaviour. None of them does.
| If incrementality were replacing attribution… | What the interviews show |
|---|---|
| Buyers would be leaving attribution for it | They shop both at once. Roughly two in three buyers in market for incrementality testing are also in market for attribution or marketing-mix modelling. |
| Demand would be moving toward it | Attribution is the larger market by some distance — roughly three buyers in market for attribution for every one shopping incrementality testing. |
| It would stand alone in the stack | Buyers describe it as the referee, not the only measurement system — the method that settles disputes between the dashboards and the models, which presupposes the others are still running. |
What the interviews do support is a shift in authority. Attribution is treated as one input rather than the answer, and the lift-based methods are what buyers reach for when a number has to survive a budget defence. That is a real and consequential change. It is just not the same thing as one category eating another.
The detail behind that shift — why the trust collapsed, what buyers replaced it with, and how the methods reconcile in practice — is the subject of nobody believes their attribution anymore and what attribution-tool buyers wish they'd known.
How does incrementality testing work?
Incrementality testing compares outcomes between groups that receive different levels of marketing exposure. The two approaches buyers most often describe are audience holdout tests and geo-lift tests.
A holdout test excludes a randomly selected audience from a campaign and compares their behaviour with the exposed group. A geo-lift test does the same thing geographically — turning exposure off or scaling it in some regions while comparable regions continue as normal. In these interviews buyers tend to discuss the credibility of the resulting number more often than the distinction between the two experimental designs; both are usually described simply as testing incrementality.
Both approaches need enough volume, exposure and time to detect a meaningful difference — a question of statistical power. Below that threshold tests can come back inconclusive, because the lift being looked for is too small relative to the noise around it. That constraint is the source of the category's most-cited practical problem, covered on the attribution lessons page.
What do you need before incrementality testing works?
Three things. Volume — can the experiment detect a lift at all? Cadence — can you run enough tests to justify the contract? Data — can you trust the outcome being measured?
Volume is the one buyers tend to express as spend, because for most of them the two move together; technically, what matters is whether the experiment has enough statistical power to detect the expected lift.
The sequencing around data in these interviews is telling. One apparel brand is standing up a data warehouse and incrementality testing in the same motion — centralising reporting first, so the tests have something trustworthy to measure against. That is the right order. A well-designed experiment still needs trustworthy outcome data; inconsistent inputs can make an otherwise rigorous test difficult to interpret or defend.
Do ad platforms offer their own incrementality testing?
Yes, and it is now a live alternative to buying a standalone tool — which is the competitive pressure the category faces that its own marketing rarely mentions.
One large footwear retailer offboarded its dedicated incrementality vendor over cost and effectiveness, and is evaluating the measurement functionality in the ad platform it already runs most of its spend through. The appeal is straightforward: the platform's lift tooling comes bundled with budget the buyer has already committed, removing a separate line item.
The trade-off is independence. A platform measuring the incrementality of spend on that same platform is grading its own homework — so the trade-off is lower tooling cost against independent measurement. Whether it is worth it is a judgement about how contested the number needs to be, and buyers here are visibly making it in both directions.
Want this read against your own stack?
Get my read →Should you buy a dedicated incrementality testing tool?
Buy it when your numbers are being argued with and your marketing volume is large and steady enough to run statistically readable tests. Below that, the contract outlasts the testing.
The case is strongest where measurement is already contested — particularly in omnichannel or offline-heavy businesses, where the click-path story cannot capture the whole purchase journey. If your existing measurement is sufficient for the decisions you actually need to make, the added cost and testing requirements of independent incrementality are harder to justify.
Two cautions come out of these interviews specifically. The first is cadence: several buyers describe annual contracts that sit idle through periods when they are not running enough tests to justify them. The second is churn — several buyers in these interviews have discontinued an incrementality provider and returned to market, so treat the first contract as an experiment rather than an infrastructure decision, and keep the exit cheap.
The framing that survives the evidence is this. Incrementality testing is winning the argument about which number to believe, and it is not winning the budget line away from anything — it is being added to a stack that still contains everything it referees.
Common questions
What is incrementality testing?
Incrementality testing measures whether marketing caused outcomes that would not have happened without it. It works by deliberately varying marketing exposure between comparable test and control groups and measuring the difference in outcomes; common approaches are audience holdouts and geo-lift tests. In an ecommerce context the outcome is usually sales, but the method applies equally to leads, signups or installs. That makes it an experiment rather than a model. Attribution assigns credit for conversions that already happened; incrementality testing asks whether those conversions needed the advertising at all.
Is incrementality testing replacing attribution?
No, on the evidence of what buyers are in market for. Roughly three times as many buyers in this corpus are shopping for attribution as for incrementality testing, and roughly two in three of the buyers in market for incrementality testing are simultaneously in market for attribution or marketing-mix modelling. The pattern is addition rather than substitution: incrementality is being adopted as a check on the other methods, not as a replacement for them. What is genuinely shifting is trust — buyers treat attribution as one input rather than the answer.
What is the difference between incrementality testing and attribution?
Attribution assigns credit for conversions that occurred; incrementality testing measures whether marketing caused conversions that would not otherwise have occurred. Attribution uses observed customer journeys to distribute credit among touchpoints, while incrementality deliberately varies exposure between test and control groups to measure causal lift. In practice, buyers in these interviews use incrementality to check attribution rather than to replace it.
What is the difference between incrementality testing, MTA and MMM?
They answer three different questions. Multi-touch attribution asks which touchpoints deserve credit for a conversion that already happened. Incrementality testing asks whether the spend caused any conversion at all, by deliberately varying exposure between test and control groups and measuring causal lift. Marketing-mix modelling asks how much each channel contributed overall, including offline, by modelling historical spend against outcomes. Of these three methods, incrementality testing is the experimental approach; the other two are models built on observed data, which is why buyers use it to check them.
How does incrementality testing work?
Incrementality testing compares outcomes between groups that receive different levels of marketing exposure. The two approaches buyers most often describe are audience holdout tests, where a randomly selected audience is excluded from a campaign, and geo-lift tests, where whole regions have exposure turned off or scaled while comparable regions continue as normal. Both need enough volume, exposure and time to detect a meaningful difference — a question of statistical power. Below that threshold tests can come back inconclusive, because the lift being looked for is too small relative to the noise around it.
What do you need before incrementality testing works?
Three things: enough volume and statistical power to detect a lift, enough testing cadence to justify the tooling, and trustworthy outcome data. Buyers often describe the first requirement as media spend, but spend is a proxy for whether the test can generate a readable signal. One brand in these interviews is standing up a data warehouse and incrementality testing in the same motion, centralising reporting so the experiments have trustworthy outcomes to measure against.
Do ad platforms offer their own incrementality testing?
Yes, and it is now a live alternative to buying a standalone tool. One large footwear retailer offboarded its dedicated incrementality vendor over cost and effectiveness and is evaluating the measurement features in the ad platform it already uses. That is a meaningful competitive pressure on the category: the platform's own lift tooling is bundled with spend the buyer is already committed to, but it means the platform is grading its own homework. The trade-off is lower tooling cost against independent measurement.
Should you buy a dedicated incrementality testing tool?
It depends on whether you need an independent causal answer, whether you have enough statistical power to run readable tests, and whether you will run them often enough to justify dedicated tooling. The case is strongest where existing measurement is already disputed. Below the power threshold tests come back inconclusive and the contract outlasts the testing. Several buyers in these interviews have discontinued an incrementality provider and returned to market, so treat the first contract as an experiment rather than an infrastructure commitment.
Get this research made for your stack
Whether incrementality testing earns a line in your budget depends on your media spend, your cadence, and how much your current numbers are already being argued with. Do a 15-minute interview and get the version of this that applies to you: what peers at your spend level run, what they dropped, and what finance actually accepted.
Get my personalized read — 15-min interviewNo password · your interview is anonymized before it ever informs a page like this one.