What is incrementality testing?
Measuring whether marketing actually caused additional sales — as opposed to taking credit for sales that would have happened anyway, which is what most attribution does.
The problem it solves. Standard attribution assigns credit to whichever touchpoints preceded a conversion. But someone who was already going to buy might click a paid search ad for the brand name on the way. The ad receives the credit; it caused nothing. That gap — between attributed and incremental — is the central measurement problem in marketing.
It is largest exactly where reported performance looks best: branded search, retargeting existing customers, and audiences already intending to purchase. Channels that appear most efficient are frequently the least incremental, because they intercept demand rather than creating it.
How incrementality is actually measured. By creating a control group that does not see the advertising and comparing outcomes:
Geographic (geo) tests. Run the campaign in some regions and withhold it in comparable ones, then compare. Practical, robust, and the most widely used method, since it works without user-level tracking — which matters increasingly as tracking becomes harder.
Holdout groups. Randomly exclude a proportion of the addressable audience.
Ghost ads / PSA tests. The control group goes through the same auction and targeting but is served an unrelated or public-service ad, so the two groups are genuinely comparable in intent rather than merely in demographics.
Switchback tests, alternating on and off periods.
What the results typically show. Incrementality tests frequently find platform-reported conversions substantially overstate true contribution, and the discrepancy is largest for lower-funnel and retargeting activity. Several widely cited experiments have found the incremental effect of some campaigns close to zero.
The practical difficulties: tests need enough scale to detect an effect; they cost money, since the holdout forgoes revenue; contamination occurs when control regions are exposed anyway; and effects can be slow to appear.
Why it is worth it. It is the only method that establishes causation rather than correlation, and it is the natural complement to marketing mix modelling, which estimates contribution at an aggregate level.