Incrementality testing is a controlled experiment that shows how many conversions your marketing caused, as opposed to how many would have happened anyway. You split an audience or a set of markets into a group that sees the campaign and a group that does not. Then you compare outcomes. The gap between the two is your incremental lift, and it is the number to trust when platform dashboards and your finance team disagree.
TL;DR
- Incrementality measures the additional outcomes a campaign caused, compared with what would have happened without it. Attribution only assigns credit to touchpoints.
- A valid test needs a credible control group, a long enough window to catch delayed conversions, and enough volume to separate real lift from noise.
- Lift is (test rate minus holdout rate) divided by the holdout rate. Incremental ROAS is incremental revenue divided by spend.
- Run tests when a channel takes a big budget, when attribution sources disagree, or when someone proposes cutting or scaling spend.
- Use tests to calibrate attribution and media mix models, not to replace them.
Table of Contents
- What incrementality testing measures
- Incrementality vs attribution
- The main test designs
- How to design a test that holds up
- How to calculate lift and incremental ROAS
- When to run a test
- What tests can and cannot tell you
- Common mistakes
- Our perspective
- Put the supporting data in one workflow
- FAQ
- Sources
What incrementality testing measures
The IAB’s November 2025 guidelines define incrementality as the causal impact of marketing: the additional business outcomes a campaign or tactic drives, compared with what would have occurred without it. That counterfactual is the whole idea. A customer who clicked your retargeting ad and then bought may have bought anyway. A last-click report counts that sale as yours. An incrementality test asks how many of those sales disappear when the ad is withheld.
The outcome you measure can be sales, revenue, sign-ups, store visits or any site or app action that creates value. Google’s guide to incrementality testing notes that tests can run on any advertising channel and can focus on revenue, profit or other value-driving actions.
Four questions come up most often in practice:
- Does this channel work at all? A lift test turns the channel off for a holdout group.
- Should we add a new channel? A test in a few markets shows whether it brings new demand before you commit.
- How much more spend can we absorb? A marginal spend test raises budget for part of the audience and checks whether returns hold.
- Is branded search just capturing people who would have found us anyway? A branded search test pauses the ads in some regions and watches what happens to organic traffic.
Incrementality vs attribution
Attribution and incrementality answer different questions. Attribution distributes credit for conversions that happened across the touchpoints a user saw, using a rule or a model such as last click or data-driven. It describes paths. Incrementality tests causation: it removes the marketing for a comparable group and measures what changes.
That difference explains why the two so often disagree. Platforms tend to report every conversion they touched, while a holdout shows only the conversions that depended on the ads. Channels near the bottom of the funnel, such as retargeting and branded search, usually look strongest in attribution because they reach people who were already about to buy. Upper-funnel channels often look weak in last-click reports even when they create demand that other channels then collect.
| Attribution | Incrementality testing | |
|---|---|---|
| Question answered | Which touchpoints get credit? | What would not have happened without the marketing? |
| Method | Rules or models applied to tracked journeys | Controlled experiment with a control or holdout group |
| Speed | Continuous, near real time | Needs a test window of weeks |
| Strength | Daily optimization and creative comparison | Causal proof of a channel or tactic |
| Weakness | Counts conversions that would have happened anyway | Covers one channel or tactic at a time, at a cost |
The practical answer is to use both. Attribution runs day to day. Incrementality tests periodically check whether attribution is telling the truth, and the results can be used to adjust how much you believe each channel’s reported numbers. If you want a deeper look at fixing attribution inputs before you test, our guide on marketing attribution modeling for analysts walks through it.
The main test designs
There is no single method. Ipsos MMA’s overview describes marketing mix modeling as the main way to measure incrementality at the portfolio level, geo holdout tests as direct causal evidence for a specific channel, and platform lift studies as a measure of one channel in isolation. Here is how each fits.
Platform conversion lift
Ad platforms can randomize users into a treatment group that sees ads and a control group that is held back. Google’s own documentation on Conversion Lift based on users describes this setup and the metrics it returns: incremental conversions, incremental CPA, relative lift, incremental conversion value and incremental ROAS. It is the fastest design to launch, but it measures only that platform and does not show how the channel interacts with the rest of your mix.
Geo holdout tests
You pause or reduce a channel in a set of regions and compare them with matched regions where it keeps running. Geo tests work for channels where you cannot randomize individual users, such as TV, out-of-home, audio or broad digital buys. They are slower and need careful market matching.
Known-audience holdouts
You split your own customer or prospect list, suppress the campaign for one half and compare conversion. This works well for CRM, email, paid social retargeting and any channel where you can match on a list.
Matched or synthetic controls
When true randomization is impossible, analysts build a control from similar markets or time periods. The result is only as good as the match, so treat it as supporting evidence and confirm with a randomized test where you can.
How to design a test that holds up
A weak design produces a confident wrong answer, which is worse than no answer. The IAB guidelines say a method can credibly claim causality only when it meets three requirements: a credible counterfactual or intervention, control for bias such as seasonality, pricing, promotions and competitor activity, and statistical robustness that separates true effects from randomness, such as confidence intervals that exclude zero.
Turn that into a checklist:
- Write the hypothesis and the decision first. For example: if incremental CPA is under our target, we raise budget 20%; if not, we cut it. Without a decision rule, results get argued over instead of used.
- Pick one primary metric. Sales, new customers or leads. Add secondary metrics such as profit or branded search volume, but do not shop for whichever one looks best.
- Build a clean control. Randomize where you can. For geo tests, match markets on historical sales level and trend.
- Freeze the confounders. Avoid launching promotions, price changes or big creative swaps in only one group during the window.
- Size the test before launching. Smaller expected lift needs more users or more markets.
- Set the duration to your sales cycle. Include enough weeks to capture delayed conversions and weekday patterns.
- Do not peek and stop early. Ending a test the day it looks good inflates false positives.
Duration and sample size
Published rules of thumb vary by channel and vendor, so treat them as starting points for your own power calculation. One practitioner guide on geo holdout design suggests that paid media geo tests usually need four to eight weeks at minimum, 20 or more markets per arm for reasonable power on national channels, and a pre-period at least as long as the test period. For user-level social tests, one guide on Meta holdout groups recommends at least 200,000 users per group and two to four weeks minimum, longer for products with long consideration cycles.
The reason sizes get large is simple. Small lifts are hard to see against everyday noise. If you expect a 5 to 15% lift, you need thousands of users per group at a minimum, and often far more when your baseline conversion rate is low. When your volume cannot support a clean test, test a bigger change (turn the channel fully off in a region rather than trimming 10%) or lengthen the window.
How to calculate lift and incremental ROAS
The math is short. Suppose you run a user-level holdout for a paid social campaign. The numbers below are illustrative.
| Item | Test group | Holdout group |
|---|---|---|
| Users | 500,000 | 100,000 |
| Conversions | 1,500 | 240 |
| Conversion rate | 0.30% | 0.24% |
Relative lift = (test rate minus holdout rate) divided by holdout rate = (0.30% minus 0.24%) divided by 0.24% = 25%. The same formula appears in Upstack Data’s lift guide.
Incremental conversions = test conversions minus the conversions you would expect at the holdout rate. At 0.24%, 500,000 users would have produced 1,200 conversions. So 1,500 minus 1,200 gives 300 incremental conversions. Only 20% of the 1,500 conversions the campaign touched were caused by it.
Incremental CPA = spend divided by incremental conversions. With $30,000 of spend, that is $100 per incremental conversion.
Incremental ROAS (iROAS) = incremental revenue divided by spend. As Google’s guide describes, you divide the newly discovered incremental revenue by the campaign’s media spend. At $120 average order value, 300 incremental orders bring $36,000, so iROAS is 1.2.
Compare that with what a dashboard would say. If the platform claimed all 1,500 conversions, its ROAS would read 6.0. The incremental view is five times lower, and that gap is what the test exists to expose.
Read the result as a range
A single lift number is an estimate. Report it with its confidence interval. If the interval crosses zero, you cannot say the campaign worked, and you cannot say it failed. Ask for the range, decide whether even the pessimistic end clears your breakeven, and plan a follow-up when it does not.
Also look beyond revenue. Two channels can deliver the same incremental revenue and very different profit once cost of goods, fulfillment and returns are subtracted, and they can bring in very different shares of new customers. Track incremental revenue, profit after those costs and the cost of acquiring a new customer for each test.
When to run a test
Testing costs time and sometimes lost sales in the holdout, so spend it where the answer changes a decision. New Engen’s guide lists the situations well. Good moments include:
- A channel takes a large share of budget.
- Attribution systems disagree with each other.
- Platform-reported results look stronger than total business growth.
- A channel looks weak on last click but may feed other conversions.
- Leadership is deciding whether to cut or scale a line.
- Several platforms may be claiming the same customers.
- Finance wants clearer evidence of marketing’s contribution.
Skip tests on tiny budgets, on channels you are about to retire anyway, and during a period of unusual disruption such as a major product launch or site migration.
What tests can and cannot tell you
A test gives strong evidence about one tactic, in one period, for one audience. It does not give you permanent truth. The IAB guidelines stress that incrementality is not static: changes in competition, consumer behavior and tactics shift outcomes, and repeated measurement across periods shows whether a lift persists or was random fluctuation. A channel that tested at 1.8 iROAS last spring may test differently after your competitors change their bidding.
It also covers only the slice you tested. Ipsos MMA’s guidance is that no single method is free of limitations and that the most robust programs combine approaches, with each validating and calibrating the others. In practice that means a loop: media mix modeling sets the portfolio view, tests verify the biggest or most doubtful channels, and attribution handles daily optimization using the corrections the tests suggest.
For a wider picture of the challenges teams face putting measurement like this into practice, see this piece on digital marketing challenges in 2026.
Common mistakes
- Stopping at the first good-looking day. Run to the planned end.
- Testing during a promotion in only one group. The effect gets mixed up with the discount.
- Ignoring spillover. In geo tests, people cross regional borders, and ads can reach holdout users through other channels.
- Judging upper-funnel media on short windows. Delayed conversions land after the test ends.
- Reporting lift without cost. A 20% lift that costs more than the profit it brings is a loss.
- Treating one result as law. Retest big decisions every so often.
Our perspective
Most teams we talk to are not short of data. They are short of time to line it up. A proper incrementality readout needs ad platform spend, analytics conversions, search demand, competitor activity and pricing context in one place, and then it needs someone to cross-check them before a decision gets made.
That is the problem Prowl was built around. We connect any AI agent or workflow to 444 marketing intelligence tools through a single endpoint, so an analyst or an agent can pull cross-provider data without wiring up each integration by hand. Our approach is to make every number in a report traceable to its source, so when a lift estimate goes to a CFO, the inputs can be checked.
We think the best measurement practice is a loop: design the test carefully, gather the surrounding market context quickly, and write up the result in the format the room needs, whether that is an interactive report, a PDF or a slide deck. Our goal is to let a team spend its hours on the experiment design and the decision, not on stitching exports together.
Prowl Team
Put the supporting data in one workflow
An incrementality test rests on context: what spend and conversions looked like before the test, what competitors did during it, and how the channel compares with benchmarks. Prowl gives your agent one connection to 444 marketing intelligence tools and generates cross-referenced reports from live data, so you can prepare the pre-test baseline and the post-test readout without building each integration separately.
Pricing is metered. You can top up credits ($10, $50 or $200, one-off) or choose a monthly plan: Exploit at $60 per month, Blackops at $120 per month or Syndicate at $240 per month. See the Prowl pricing page for what each plan includes, or browse Prowl’s use cases to see how teams connect it to their agents. For the reporting side of testing, our guide to ad performance analysis and the cross-channel KPI guide are good next reads.
FAQ
What is incrementality testing in simple terms?
It is an experiment that compares a group exposed to your marketing with a similar group that is not. The difference in outcomes is the effect your marketing caused. It strips out sales that would have happened regardless, such as loyal customers who were going to buy anyway.
How is incrementality different from attribution?
Attribution divides credit for conversions among the touchpoints in a customer journey. Incrementality measures whether those conversions depended on the marketing at all. Use attribution for daily decisions and incrementality tests to check whether the credit it hands out is deserved.
How long should an incrementality test run?
It depends on your sales cycle and volume. Published guidance for user-level social tests is a minimum of two to four weeks, and for geo tests four to eight weeks or more, with a pre-period of similar length. Products with longer consideration cycles need longer windows to capture delayed conversions.
What is a good incremental ROAS?
There is no universal number. Compare incremental ROAS with your breakeven, which depends on gross margin, returns and the value of repeat purchases. An iROAS below your breakeven can still be worth running for new-customer acquisition if lifetime value is high, but make that call on profit data, not on revenue alone.
Sources
- IAB and IAB Europe: Guidelines for Incremental Measurement in Commerce Media (November 2025)
- Google Ads Help: Understand your Conversion Lift based on users measurement
- Google: Use incrementality testing for effective marketing (global)
- Google: Use incrementality testing for effective marketing (US)
- Ipsos MMA: What Is Marketing Incrementality?
- Designing defensible geo holdout tests for incrementality measurement
- Incrementality testing for Meta ads: holdout groups guide
- Upstack Data: How to measure and calculate lift
- New Engen: What Is Incrementality Testing in Marketing?