Incrementality Testing
Incrementally testing measures the actual impact of a marketing channel or campaign by comparing results from people who were exposed to it with results from a similar group who were not. This helps identify conversions and other outcomes that happened because of the marketing activity, rather than those that would have happened anyway.
Why it matters
Attribution models can sometimes give too much credit to channels that capture demand that already existed. Branded search and retargeting are common examples. Someone who clicks a branded ad or a retargeting ad may have converted even without seeing the ad. Incrementally testing helps separate these conversions from the ones the channel actually influenced.
This is important when deciding where to spend a marketing budget. A channel may show strong attributed conversions but contribute very little additional pipeline. Looking at incremental results helps teams understand which channels are genuinely driving new demand and which ones are mainly capturing existing demand.
A common way to run an incrementally test is through a holdout experiment. One group is exposed to the marketing activity while a similar group is not, and the results are compared over the same period. Having performance data from different channels in one place can make these tests easier to set up and analyse. Platforms like Neo can help bring this channel-level data together, reducing the need to combine data manually.
Example
A D2C ecommerce brand suspected that its retargeting ads were getting too much credit for conversions. An incrementality test found that only 30% of the conversions attributed to retargeting were actually incremental. The team then moved the remaining budget to top-of-funnel channels that were generating new demand.