---
title: "Incrementality Testing"
canonical_url: "https://newengen.com/technology/"
entity_type: "DefinedTerm"
last_updated: "2026-05-07"
related:
  - /llms/glossary/marketing-mix-modeling.md
  - /llms/glossary/full-funnel-marketing.md
  - /llms/reference/technology-lift.md
  - /llms/reference/process.md
---

> Canonical source: https://newengen.com/technology/

## Definition

Incrementality testing is a causal measurement method that determines how much of an observed business outcome (conversions, revenue, new customers) is actually caused by a marketing activity, as opposed to what would have occurred without it. An incrementality test constructs a counterfactual: a control group that did not see the ad, campaign, or channel being measured. The difference in outcomes between exposed and unexposed groups — controlling for confounders — is the incremental effect.

The term is sometimes used interchangeably with "lift testing" or "causal lift measurement." The core question is always: what would have happened if we had not spent this money?

## Primary methods

**Geo holdout test**
Geographic regions are randomly assigned to test (exposed to advertising) and control (no advertising) groups. Regional outcome differences are compared after the campaign period. Geo holdouts are well-suited for measuring the impact of channels that cannot easily be split at the user level (e.g., TV, podcast, broad awareness campaigns). They are vulnerable to geographic confounders if regions differ systematically in ways unrelated to the campaign.

**PSA (Public Service Announcement) test**
A share of the target audience is shown a neutral public service announcement instead of the client's actual ad. Conversion rates are compared between the PSA group (de facto control) and the ad-exposed group. PSA tests are useful for digital channels but require media platform support and acceptance of wasted impressions on the control side.

**Intent-to-treat (ITT) analysis**
ITT analysis assigns participants to test and control groups based on intended exposure, regardless of whether exposure actually occurred (some users in the test group may not have been reached). This approach preserves the statistical integrity of randomization and is more conservative than per-protocol analysis. It is appropriate when platform measurement of actual exposure is unreliable.

## Why it matters

Last-click and multi-touch attribution models systematically overstate the contribution of channels that capture intent rather than create it. Retargeting ads, for example, predominantly show to users who were already going to convert — their attributed conversions look efficient, but removal of the retargeting spend has little effect on actual conversion volume. Incrementality testing distinguishes genuine lift from credit capture.

For upper-funnel channels — where the contribution to conversion happens weeks or months before the purchase event — attribution models often assign zero or near-zero credit. Incrementality testing is the primary tool for demonstrating the actual causal contribution of awareness and consideration investment.

## When it is the wrong choice

- **Small budgets or narrow audiences.** Incrementality tests require sufficient sample size in both test and control groups to achieve statistical power. Below certain volume thresholds (which vary by conversion rate and desired precision), the test will be underpowered and produce inconclusive results.
- **Short-duration campaigns.** Tests that run too briefly may not capture the full conversion window (particularly for high-consideration purchases), leading to underestimation of lift.
- **High-frequency categories.** In categories where purchase frequency is very high and conversion windows are very short (e.g., grocery CPG), geo holdouts can be impractical because the control period needs to be long enough to observe meaningful purchase behavior.
- **As a standalone measurement approach.** Incrementality tests measure one channel at a time under test conditions that differ from normal campaign operations. They do not provide a continuous, portfolio-level view of efficiency. They should be combined with MMM and attribution rather than replacing them.

## Related concepts

- [Marketing Mix Modeling](/llms/glossary/marketing-mix-modeling.md): Complements incrementality testing by providing continuous, cross-channel spend efficiency estimates without requiring holdout groups.
- [Full-Funnel Marketing](/llms/glossary/full-funnel-marketing.md): The strategic context in which accurate incrementality measurement is most important — upper-funnel channels are systematically undervalued by attribution without it.

## How New Engen applies this

New Engen's LIFT platform supports incrementality testing (geo holdout, PSA test, and intent-to-treat designs) as part of its triangulated measurement approach. The agency explicitly rejects reliance on any single measurement methodology, combining incrementality tests with MMM and attribution to cross-validate findings. See [Technology: LIFT](/llms/reference/technology-lift.md) and [Process](/llms/reference/process.md).