Question 1

What is incrementality testing?

Accepted Answer

Incrementality testing is a method to measure the true causal impact of marketing activities by comparing a test group (exposed to marketing) against a control group (not exposed). It helps determine what would have happened without your marketing efforts, revealing the actual lift or incremental value generated by your campaigns. Unlike correlation-based methods, incrementality establishes causation through experimental design, making it the gold standard for measuring marketing effectiveness.

Question 2

How is incrementality different from attribution?

Accepted Answer

Attribution assigns credit for conversions to specific touchpoints in the customer journey, while incrementality measures the true causal impact of marketing by comparing test and control groups. Attribution models (last-click, multi-touch, etc.) rely on correlations and can be misleading due to selection bias—they often give credit to channels that users were going to use anyway. Incrementality solves this by measuring the counterfactual: what would have happened without the marketing activity. For example, retargeting often shows high ROAS in attribution but low incrementality because it targets users already likely to convert.

Question 3

What is a good incrementality lift percentage?

Accepted Answer

A good incrementality lift varies by industry, channel, and campaign objectives. Generally, a relative lift of 10-30% is considered good, while anything above 30% is excellent. However, even a 5% lift can be valuable if it's statistically significant and the campaign has large scale. Upper-funnel brand campaigns typically show 5-15% lift, mid-funnel consideration campaigns 15-25%, and lower-funnel conversion campaigns 20-40%. Mature markets tend to have lower incrementality than emerging ones. The key is comparing your lift to your cost—a 10% lift that costs 5% of revenue is better than a 20% lift that costs 25%.

Question 4

How do I set up an incrementality test?

Accepted Answer

To set up an incrementality test: 1) Define clear objectives and success metrics (conversion rate, revenue, etc.), 2) Create truly randomized test and control groups using methods like tests, ghost ads, or geographic isolation, 3) Ensure the only difference between groups is exposure to the marketing activity being tested, 4) Determine appropriate sample sizes using power analysis (our calculator can help), 5) Run the test for a sufficient duration (typically 2-4 weeks), 6) Control for external factors like seasonality or other marketing activities, and 7) Analyze results using this calculator. For platform-specific tests, use Facebook's Conversion Lift, Google's Randomized Controlled Experiments, or TikTok's Brand Lift studies.

Question 5

What sample size do I need for an incrementality test?

Accepted Answer

Sample size depends on your baseline conversion rate and the minimum lift you want to detect. For a baseline conversion rate of 1% and detecting a 20% relative lift (to 1.2%), you'd need approximately 35,000 users per group for 80% statistical power. For a 5% baseline rate, you'd need about 6,500 per group. Generally, you need larger samples for lower conversion rates or to detect smaller lifts. For most tests, aim for at least 1,000 users per group, though high-traffic campaigns may require 10,000+ per group for statistical validity. Use our calculator's sample size feature to determine your specific requirements based on your expected conversion rates.

Question 6

What are the most common incrementality testing methodologies?

Accepted Answer

The most common incrementality testing methodologies include: 1) PSA (Public Service Announcement) Tests - where the control group sees charity/PSA ads instead of your ads, 2) Ghost Ads - where ad platforms record impressions that would have been served to control users without actually showing them ads, 3) Geographic Testing - comparing similar markets where one receives the marketing treatment and one doesn't, 4) Intent-to-Treat Analysis - analyzing users based on their assignment to test/control regardless of whether they actually saw ads, and 5) Holdout Tests - where a portion of your audience is excluded from specific marketing activities. Each method has trade-offs between accuracy, cost, and implementation complexity.

Question 7

How do I interpret incrementality test results?

Accepted Answer

When interpreting incrementality test results, focus on: 1) Statistical Significance - ensure your p-value is <0.05 (95% confidence) before making decisions, 2) Practical Significance - a statistically significant result may still be too small to justify the marketing investment, 3) Incrementality ROI - compare the incremental revenue to the cost of the campaign, 4) Segment Analysis - examine how incrementality varies across different audience segments, 5) Channel Interactions - understand how this channel affects performance in other channels, and 6) Trend Analysis - track how incrementality changes over time or with scale. Remember that incrementality is not static—it can change with audience saturation, creative fatigue, or competitive activity.

Incrementality Calculator

Control Group

Test Group

Business Metrics

Frequently Asked Questions

Marketing Incrementality Calculator

Free Calculator

Incrementality Calculator

Control Group

Test Group

Business Metrics

Frequently Asked Questions