What Is Incrementality Testing? How to Measure Marketing’s True Impact
Your advertising platforms report strong results. Meta claims conversions. Google claims conversions. Your attribution platform assigns credit across several touchpoints.
But how many of those customers would have purchased anyway?
That is the question incrementality testing is designed to answer.
Incrementality testing measures the additional conversions, revenue, or other outcomes caused by a marketing activity. It compares people or markets exposed to marketing with a similar group that was not exposed, helping brands estimate what would have happened without the investment.
Attribution tells you where credit was assigned. Incrementality testing tells you whether marketing changed the outcome.
For brands deciding which channels to scale, cut, or defend, that difference matters.
What Is Incrementality Testing in Marketing?
Incrementality testing is a form of marketing experimentation used to determine whether an ad, campaign, or channel caused an outcome that would not have occurred otherwise.
The basic structure involves two groups:
A treatment group that receives the marketing
A control or holdout group that does not
The difference in outcomes between those groups represents the campaign’s incremental lift, assuming the groups are comparable and the test is designed correctly.
For example, imagine that:
10% of customers exposed to a campaign convert.
8% of similar customers who were not exposed convert.
The campaign’s estimated incremental lift is 2 percentage points.
The campaign may receive credit for every conversion in the exposed group, but incrementality testing indicates that only the additional difference was likely caused by the advertising.
This is the counterfactual at the center of incrementality measurement:
What would have happened if we had not run the marketing?
Why Is Incrementality Testing Important?
Most marketing measurement systems are good at reporting activity. They can show impressions, clicks, attributed conversions, cost per acquisition, and return on ad spend.
Those metrics do not always establish causation.
Someone may click an ad after already deciding to buy. A branded search campaign may capture a customer created by another channel. Multiple platforms may claim the same conversion. A channel without a final click may influence the sale without receiving credit.
As a result, brands can make costly decisions based on an incomplete picture:
Scaling channels that are capturing existing demand
Cutting channels that influence purchases without generating final clicks
Overestimating platform-reported performance
Underinvesting in upper-funnel marketing
Giving multiple channels credit for the same revenue
Optimizing toward customers who were already likely to convert
Incrementality testing helps separate correlation from causation. It gives brands a better estimate of what marketing actually added to the business.
Incrementality Testing vs. Attribution
Attribution and incrementality answer different questions. Brands often need both, but they should not use the terms interchangeably.
| Attribution | Incrementality testing |
|---|---|
| Identifies touchpoints associated with a conversion | Estimates whether marketing caused an additional conversion |
| Asks who interacted with an ad before purchasing | Asks who purchased because of the ad |
| Assigns credit across channels or touchpoints | Compares exposed and unexposed groups |
| Helps optimize campaigns and customer journeys | Helps validate larger investment decisions |
| Reports what happened | Estimates what would have happened otherwise |
Attribution can help teams manage day-to-day performance. Incrementality testing is better suited to questions about true business impact.
If a platform reports 10,000 conversions, attribution helps explain which campaigns received credit. Incrementality testing helps estimate how many of those conversions would not have happened without the advertising.
How Does Incrementality Testing Work?
A reliable incrementality test creates a credible comparison between a group that receives marketing and a group that does not.
1. Define the business question
Start with a decision, not a methodology.
Useful incrementality questions include:
Is paid social generating additional sales?
Does upper-funnel video create demand beyond existing customers?
What happens if we add or remove a channel?
Is a campaign increasing new-customer acquisition?
Are branded search ads creating conversions or capturing existing demand?
Does a major partnership generate incremental revenue?
Should we increase investment in this market?
The question should be specific enough that the organization knows what it will do with the answer.
2. Select a treatment and control group
The treatment group receives the campaign or channel being evaluated. The control group does not.
Depending on the test, groups may be composed of:
Individual customers
Platform users
Geographic markets
Stores
Regions
Matched audience segments
The groups should be as comparable as possible before the test begins. If one group already purchases at a higher rate, that underlying difference can distort the result.
3. Establish the outcome
The test should have a clearly defined primary outcome, such as:
Conversions
Revenue
New customers
Store visits
App installs
Leads
Brand searches
Brand lift
Defining the outcome in advance reduces the temptation to search for a favorable result after the test ends.
4. Run the experiment
During the test, the treatment group receives the marketing while the holdout group does not.
Teams must monitor the test for problems such as:
Audience overlap
Campaign leakage
Uneven promotions
Inventory changes
Seasonal events
Pricing differences
Other media that affects only one group
These factors can make a difference between the groups look like marketing lift when another variable caused it.
5. Calculate incremental lift
Incremental lift is the difference between the outcome for the treatment group and the expected outcome based on the control group.
A simplified formula is:
Incremental conversions = treatment conversions − expected baseline conversions
Lift can also be expressed as a percentage:
Incremental lift = (treatment conversion rate − control conversion rate) ÷ control conversion rate
If the treatment group converts at 12% and the control group converts at 10%, the campaign produced a 20% relative lift.
6. Make a decision
The final step is often the hardest.
A test should inform whether the brand will:
Scale the investment
Maintain it
Reduce it
Reallocate budget
Redesign the campaign
Run a more rigorous follow-up test
Change its assumptions about the customer journey
Incrementality testing should produce action, not another dashboard.
What Are the Main Types of Incrementality Tests?
The right method depends on the question, available data, channel, audience size, and level of investment.
| Test type | How it works | Best suited for |
|---|---|---|
| Randomized holdout test | Randomly withholds marketing from part of an eligible audience | Establishing causal lift when user-level randomization is possible |
| Conversion lift study | Compares conversions among platform-defined exposed and holdout groups | Evaluating campaign or platform impact |
| Geo experiment | Runs marketing in selected regions and compares results with similar untreated regions | Channels that are difficult to randomize at the individual level |
| Matched-market test | Pairs similar markets, stores, or regions before applying different treatments | Retail, local media, product launches, and regional campaigns |
| On/off test | Compares performance while marketing is active and inactive | Directional learning when stronger controls are unavailable |
On/off tests are generally less reliable because performance can change over time for many reasons. Seasonality, promotions, competitor activity, economic conditions, and changes in demand may all affect the result.
Randomized tests offer stronger causal evidence when they are feasible. Geo and matched-market experiments can provide useful answers when individual-level holdouts are impractical.
Incrementality Testing vs. A/B Testing
An A/B test commonly compares two versions of an experience to determine which performs better. Incrementality testing determines whether marketing created an outcome beyond what would have happened without it.
For example:
An A/B test might compare two headlines to identify which generates a higher conversion rate.
An incrementality test might compare advertising against no advertising to determine whether the campaign creates additional conversions.
New Engen’s 2026 Growth Playbook separates marketing experimentation into two categories.
Rapid, directional experiments
These tests help teams adjust day-to-day creative and media decisions.
They can answer:
Which message resonates?
Which hook captures more attention?
Does Feature A outperform Feature B?
Which creative format is more efficient?
Structural, causal experiments
These tests address larger business questions:
Is paid media generating incremental demand?
What happens if we remove or add a channel?
How much growth would have happened anyway?
Is the investment producing a causal return?
The playbook recommends a portfolio in which rapid tests account for most experimentation volume, while a smaller number of rigorous causal experiments address the most valuable questions.
Many brands run zero to a handful of experiments in a year. But velocity without direction or clear bearings is dangerous and can lead to wasted energy or decisions that negatively impact business.Kevin Goodwin, SVP of Strategy & Growth
Brands need both types of marketing experimentation. Treating every directional test as causal can create false confidence. Requiring a complex incrementality study for every small optimization can slow the organization unnecessarily.
When Should Brands Run an Incrementality Test?
Incrementality testing is most useful when the answer could materially change a business decision.
Consider running a test when:
A channel receives significant budget.
Attribution systems disagree.
Platform-reported performance appears stronger than total business growth.
A channel has weak last-click results but may influence other conversions.
Leadership is considering cutting or scaling an investment.
The brand is entering a new market.
A large campaign, sponsorship, or partnership needs to be evaluated.
Upper-funnel media is being judged by short-term conversion metrics.
Multiple platforms may be claiming the same customers.
Finance needs clearer evidence of marketing’s contribution.
Not every campaign requires a structural causal test. The cost and complexity should be proportional to the decision.
How Do You Measure Incremental ROAS?
Incremental return on ad spend, or incremental ROAS, evaluates revenue caused by the marketing rather than all revenue attributed to it.
A simplified formula is:
Incremental ROAS = incremental revenue ÷ advertising cost
Suppose a campaign costs $100,000 and platform attribution reports $500,000 in revenue. The platform-reported ROAS is 5.0.
An incrementality test finds that only $250,000 in revenue was caused by the campaign. The incremental ROAS is therefore 2.5.
The campaign may still be profitable. But the incrementality result gives the brand a more realistic basis for deciding whether to scale.
Why Might Incrementality Results Disagree With Platform Reporting?
It is common for an incrementality test to produce a different result from platform attribution.
Potential reasons include:
The platform is receiving credit for existing demand
Customers who were already likely to buy may see or click an ad before converting. Attribution assigns credit to the interaction even if the purchase would have occurred anyway.
Multiple platforms claim the same conversion
A customer may interact with several ads before purchasing. Each platform sees its own interaction and may claim the full conversion.
The channel influences behavior without receiving a click
Video, creator content, audio, connected television, and other upper-funnel media can influence demand without producing a traceable final click.
The measurement windows are different
A platform may use a short attribution window while the campaign affects customer behavior over a longer period.
The test measures a different business outcome
Platform optimization may focus on reported conversions while the incrementality test evaluates new customers, total revenue, or another business-level metric.
The purpose of incrementality testing is not to make every measurement system agree. It is to understand what each system can and cannot see.
What Makes an Incrementality Test Reliable?
A useful incrementality test requires more than creating a holdout group.
Sufficient sample size
The test needs enough observations to distinguish actual lift from random variation. Small campaigns or rare conversions may not generate a clear result.
Comparable groups
The treatment and control groups should have similar behavior before the test. Major differences weaken the comparison.
A meaningful test period
The test must run long enough to capture the expected customer response while limiting interference from unrelated changes.
Clean separation
People or markets in the control group should not be heavily exposed to the treatment through another route.
Predefined success criteria
Teams should agree on the primary metric, test duration, confidence requirements, and decision rules before seeing the results.
Business-level alignment
The metric should connect with revenue or another outcome leadership and finance recognize. A statistically valid result may still fail to change behavior if stakeholders do not trust what it represents.
[INFOGRAPHIC: Six Requirements for a Reliable Incrementality Test]
A decision-focused question
Comparable treatment and control groups
Sufficient sample size
A clean testing window
Predefined success criteria
Organizational agreement on the next action
What Should Brands Do When a Test Challenges the Strategy?
This is where incrementality testing becomes more than a measurement exercise.
New Engen’s 2026 Growth Playbook argues that many brands use measurement tools to justify decisions they have already made. Incrementality tests are rerun, narrowed, or dismissed when the answer conflicts with the existing plan.
“We never actually escaped the performance trap. We just rebuilt it with better dashboards and more data.”
—Kevin Goodwin
The intended role of incrementality testing is to find causal truth and help the organization adapt when evidence challenges its strategy.
That requires:
Agreeing on decision rules before the test
Involving finance early
Connecting results to business outcomes
Documenting assumptions
Separating test quality from whether the answer is convenient
Giving teams permission to change course
Using one test as part of a larger body of evidence
As Chandler Dutton, Measurement Strategy Team Lead at Haus, explains in the playbook:
“The way teams operationalize measurement is the product of their patterns, their fears, and their incentives.”
A sophisticated testing program will not improve growth if every team is rewarded for defending its existing budget.
Frequently Asked Questions About Incrementality Testing
What is incremental lift?
Incremental lift is the additional outcome caused by a marketing activity compared with what likely would have happened without it.
Is incrementality testing the same as attribution?
No. Attribution assigns credit among marketing touchpoints. Incrementality testing estimates whether marketing caused an additional outcome.
Is incrementality testing the same as A/B testing?
Not necessarily. An A/B test usually compares two executions or experiences. An incrementality test typically compares marketing exposure with a holdout condition to measure causal lift.
How long should an incrementality test run?
The appropriate duration depends on purchase frequency, conversion volume, media spend, seasonality, and the expected response window. The test should be designed around the customer’s actual decision cycle rather than a universal number of days.
Can incrementality testing measure upper-funnel marketing?
Yes, although the outcome and testing window must reflect how upper-funnel media works. A brand may evaluate changes in awareness, search behavior, new customers, or revenue over a longer period instead of expecting immediate last-click conversions.
What is a marketing holdout group?
A holdout group is an eligible group of customers, users, or markets intentionally excluded from a marketing treatment. Its behavior provides an estimate of what may have happened without the campaign.
How often should brands run incrementality tests?
Brands should prioritize incrementality testing around consequential questions and major investment decisions. Rapid experiments can guide ongoing optimization, while rigorous causal tests should establish whether larger strategies and channels are creating genuine business value.
Incrementality Testing Should Create Conviction
The most valuable outcome of incrementality testing is not a lift percentage. It is the confidence to make a better decision.
Attribution helps marketers understand the paths associated with conversion. Incrementality testing goes further by estimating whether marketing changed what the customer did.
That answer can reveal that a high-performing channel is receiving credit for demand that already existed. It can also show that a seemingly weak channel is creating value that last-click reporting cannot see.
Either result is useful if the organization is prepared to act on it.
Incrementality testing should not be used to make an existing strategy look correct. It should help brands understand what is genuinely driving growth, invest with greater confidence, and change direction when the evidence demands it.
Learn how incrementality testing, marketing mix modeling, experimentation, and creative performance work together in New Engen’s 2026 Growth Playbook.
