Incrementality Testing for DTC: 5 Tests We Run (2026)

Every ad platform grades its own homework. Meta and Google can both claim the same Shopify order. Klaviyo then logs it again as flow revenue, and the combined total runs well past what reached your bank account. Incrementality testing is how you find out which of those sales your ads caused. It's also how you stop paying for customers who were going to buy anyway.

Most teams already suspect their numbers. In the IAB's State of Data 2026 report, 75% of US buy-side marketers said their current measurement approaches fall short of what they need. Few DTC brands have turned that doubt into a test plan.

This post covers the 5 incrementality tests we run for DTC brands, in the order we usually run them. Each comes with the sample size and budget we plan around, and the rule we use to turn a result into a spend decision.

TL;DR

  • Incrementality testing compares people who see your ads with a random holdout, so you measure caused sales.
  • Detecting a 20% lift needs roughly 430 purchases in the control group at 80% power.
  • Google now runs conversion lift tests from about $5,000, and Meta trains Incremental Attribution on lift studies.
  • At 50% margin, a 4x reported ROAS only pays if at least half those sales are incremental.

What is incrementality testing?

Incrementality testing measures the sales your marketing caused by comparing an exposed group with a random control group that sees no ads. The gap between their conversion rates is the incremental lift. DTC brands use it to check whether platform-reported ROAS reflects new revenue or purchases that would have happened anyway.

How do you calculate incrementality?

Subtract the control group's conversion rate from the test group's, then divide by the test group's rate. If the test converts at 2.0% and the control at 1.5%, 25% of test conversions were incremental. Multiply revenue by that share for incremental revenue, then divide by spend for incremental ROAS.

How long should an incrementality test run?

Plan for at least two full weeks plus your typical purchase cycle, and keep going until each group reaches its target purchase count. For most DTC brands that means two to six weeks. Avoid launching during BFCM or a sitewide sale, because results from a promo week rarely hold at full price.

What is incrementality testing?

Incrementality testing splits your audience into two random groups. The test group sees your ads as normal. The control group, usually 10% to 20% of the audience, sees nothing from that campaign. Any difference in purchase rate between the two is what the ads caused.

Attribution answers a different question. It hands out credit for orders that already happened, using clicks and views, and each platform writes its own rules. One Shopify order can be claimed by Meta and Google at once, then logged again as Klaviyo flow revenue.

The error runs in both directions. Meta's own measurement research found that last-click models undervalue Meta by 31% at the median, while retargeting campaigns routinely overclaim. A holdout is the only method we trust to settle which way a given campaign leans.

The math is short. Incremental lift equals the test conversion rate minus the control rate, divided by the test rate. Incremental ROAS (iROAS) is incremental revenue divided by spend.

"Last-click attribution undervalues Meta by 31% at the median, according to Meta's own measurement research." — PPC Land on Meta’s “Suite of Truth” white paper

How big does an incrementality test need to be?

Most failed tests we review were too small to say anything. An inconclusive result still costs you a month of holdout revenue, so size the test before you launch it.

Start from the smallest lift worth acting on. At 95% confidence and 80% power, the control group needs about 1,650 purchases to detect a 10% lift and about 430 to detect 20%. A 30% lift needs about 200, and a 50% lift needs about 80.

The test group needs a little more because it converts higher. For a 20% lift that's roughly 520 purchases. At the $33.80 median Meta cost per purchase in our 19-brand benchmark, that's about $17,500 of spend in the test cell.

Run at least 14 days so two full weekends land in the data, then add your typical days to purchase. Our ceiling is six weeks. Past that, audiences and creative drift enough to muddy the read.

Bar chart of control-group purchases needed for incrementality testing at 95% confidence and 80% power: about 1,650 for a 10% lift, 430 for 20%, 200 for 30% and 80 for 50%

Test 1: Meta Conversion Lift for prospecting

We start with Meta because it usually holds the biggest share of a DTC budget and the most contested credit. Conversion Lift is Meta's user-level holdout. Meta withholds ads from a random control group and compares purchases from both groups through your pixel and Conversions API.

How we set it up

Run it from the Experiments tool in Ads Manager or through your Meta rep. Test one prospecting campaign or a group of them, keep the control at 10% to 20%, and lock budget and creative for the full window. Check event match quality on your Conversions API events first, since missing server events shrink both groups' counts.

Test Incremental Attribution alongside it

Meta now uses lift data to power its Incremental Attribution setting, which steers delivery toward conversions that would not have happened without the ad. In 37 Conversion Lift studies run in 2024, Meta reported a 46% average lift in performance for campaigns using it. We switch it on for one campaign and keep a lift study running so the claim gets checked in your own account. Our Meta Andromeda breakdown covers how this fits the rest of the 2026 delivery changes.

"75% of US buy-side marketers say their current measurement approaches fall short of what they need." — IAB State of Data 2026, via MarTech

Test 2: Google Ads conversion lift and a brand search holdout

In 2025 Google cut the minimum for its incrementality tests from about $100,000 to $5,000, with new statistical models it says make results up to 50% more conclusive, as Search Engine Land reported. For a mid-size Shopify brand, that puts a lift test inside one month of normal Google spend.

Point it at Performance Max first

Performance Max mixes new-customer prospecting with branded queries in one ROAS figure, which makes it the Google campaign we trust least on reported numbers. A lift result tells you how much of that blended ROAS is new demand. Our Performance Max guide covers how we split brand traffic out before testing.

Pause brand search in matched regions

Brand search is the cheapest test on the account. Pick a set of matched regions, pause the branded campaign there for two to three weeks, and compare total Shopify orders and organic brand clicks with the rest of the country. If organic picks up most of the lost clicks and orders hold steady, lower the brand budget and move the money to prospecting.

Test 3: Geo holdout for a whole channel

Platform lift studies run inside each platform's walls, so they can't tell you what happens when a whole channel goes dark. A geo holdout can. You switch a channel off in some regions, keep it running everywhere else, and compare orders from Shopify, which no ad platform controls.

Match the regions first

Pull 12 months of Shopify orders by shipping state. Pair states with similar order share and seasonality, then randomly assign one from each pair to the holdout. Holding out 15% to 25% of national orders gives most brands enough signal without putting the quarter at risk.

Read it against the baseline

Run four weeks with the channel off, then watch two more weeks for lagged orders. Compare the holdout states against their own pre-period and against their matched pairs. Meta's open-source GeoLift package handles matching and confidence intervals if you have an analyst, and a spreadsheet is fine for a first pass.

Budget sets the clock. The chart below shows how long a test takes to collect about 520 test-cell purchases at our median $33.80 CPA, with 14 days as the floor.

Bar chart of days an incrementality test needs to collect about 520 test-cell purchases at a $33.80 cost per purchase, from 70 days at $250 a day to the 14-day floor at $2,500 a day

Test 4: Retargeting holdout before you scale it

Retargeting looks like the best money in most accounts. In our DTC benchmarks, cold Meta prospecting typically runs 1.5x to 3x ROAS while retargeting often clears 5x. Much of that gap comes from people who were already on their way to checkout.

We hold out a random 20% of the retargeting audience for three weeks. On Meta that's a Conversion Lift study on the retargeting campaign alone. On other channels, an excluded custom audience built from a random split of customer IDs does the job.

In our experience scaling DTC brands, this is the test that moves the most budget. For a home-goods brand we took over at a 1.9x blended MER, we reached 2.7x in 90 days without extra creative budget, mostly by cutting a 6x-ROAS retargeting campaign and moving that spend to cold prospecting (the benchmark report has the detail). Catalog retargeting carries the same risk, which we cover in our Meta catalog ads guide.

"On Meta, cold prospecting typically runs 1.5x to 3x ROAS, while retargeting warm shoppers often clears 5x." — Top Growth Marketing, DTC Benchmarks 2026

Bar chart of a home-goods DTC brand's blended MER rising from 1.9x to 2.7x in 90 days after a retargeting holdout led to moving spend into cold prospecting

Test 5: Klaviyo holdout for email and SMS

Email and SMS get the least scrutiny because sends cost almost nothing. The hidden costs are discount margin and list fatigue. Klaviyo credits a flow with any order placed inside its attribution window, so a winback flow can claim buyers who came back on their own.

Build the holdout

Export your active profiles, give a random 10% a profile property such as holdout = true, and import the file back. Build a segment on that property and exclude it from campaigns and from the flows you're testing. Keep order confirmations and other transactional messages running for everyone.

Compare revenue per profile

After 30 to 60 days, compare Shopify revenue per profile for the two groups. If the holdout lands within a few percent of the mailed group, your discount flows are giving away margin to people who would have bought at full price. Our Klaviyo team runs this before every major flow rebuild.

How do you turn incrementality results into budget decisions?

A lift result only matters if it changes where money goes. We convert each test into one bar: the share of reported conversions that must be incremental for the campaign to pay. That share equals one divided by reported ROAS times gross margin.

At 50% margin, a campaign reporting 2x needs every sale to be incremental. At 3x it needs two thirds, at 4x it needs half, and at 6x a third is enough. Put your measured lift next to that bar and most decisions make themselves.

Rebuild the plan around iROAS

Set channel targets on incremental ROAS, then use platform ROAS for day-to-day moves inside each channel. Retest every major channel every six months, or after a big creative or budget shift. Track blended MER in our MER calculator to confirm the reallocation shows up in total revenue.

Bar chart of the share of reported conversions that must be incremental to break even at 50% margin: 100% at 2x reported ROAS, 67% at 3x, 50% at 4x, 40% at 5x and 33% at 6x
✅ Do❌ Don't
Size the test from the smallest lift worth acting on.Launch a two-week test and hope it reaches significance.
Hold out 10% to 20% of the audience, or 15% to 25% of national orders for geo tests.Hold out 2% because it feels safer for revenue.
Lock budget and creative for the full test window.Swap creative or double spend halfway through the test.
Read results in Shopify orders and revenue.Judge a geo test on platform-reported conversions.
Schedule tests outside BFCM and sitewide sales.Run your first holdout during a Black Friday promo.

The TGM Take

The common line is that incrementality testing belongs to big brands with seven-figure budgets and a data science team, and that smaller DTC brands should watch blended MER and move on. We think that advice costs mid-size brands more than anyone.

A brand spending $40,000 a month can lose a quarter of it to retargeting and discount flows that mostly reach people who were already buying. MER will show the drag, but it can't tell you which line to cut. A Klaviyo holdout costs nothing to run. A retargeting holdout costs a few weeks of reduced reach, and Google's floor for a lift test is now $5,000.

Our position: run one holdout every quarter starting with retargeting, and don't scale any channel past 30% of spend until a test shows its sales are real.

— Jack Paxton, Founder, Top Growth Marketing

Where to start with incrementality testing this quarter

Incrementality testing replaces the platforms' grading with your own. Size every test from the lift you care about, so it can return an answer. Start with the cheap holdouts in retargeting and Klaviyo, where the overclaiming is usually biggest. Then move budget on the break-even share of incremental sales, and confirm it in blended MER.

If you want help planning your first test, book a growth strategy call. Our Facebook ads team builds a holdout into every new account, and our Facebook ads for ecommerce playbook covers the rest of the funnel.

Frequently Asked Questions

What's the difference between incrementality and attribution?

Attribution assigns credit for orders that happened, based on clicks and views, and each platform uses its own rules. Incrementality measures what would not have happened without the ads, by comparing an exposed group with a control. Attribution is useful for daily optimization, while incrementality tells you what a channel is worth.

How much budget do you need for an incrementality test?

It depends on your cost per purchase and the lift you want to detect. Detecting a 20% lift needs about 520 test-cell purchases, which is roughly $17,500 of spend at a $33.80 CPA. Google's own lift tests start from about $5,000, and Klaviyo holdouts cost nothing beyond the held-out revenue.

Can a small DTC brand run incrementality tests?

Yes, if it starts with the cheap tests. A Klaviyo holdout and a brand search pause work at almost any size. Below about $500 a day of Meta spend, user-level lift studies struggle to reach enough purchases, so test for larger lifts or run a longer geo holdout instead.

What is a good incremental ROAS?

Any incremental ROAS above one divided by your gross margin is profitable on first purchase. At 50% margin that's 2x. Brands with strong repeat purchase can accept a lower first-order iROAS if their 90-day customer value covers the gap.

How often should you run incrementality tests?

We retest each major channel every six months and after any large budget or structure change. Running one holdout per quarter keeps the program manageable and builds a record you can use to calibrate a marketing mix model later.

Last updated:

Jack Paxton
Written by
Jack Paxton is the founder of Top Growth Marketing, a DTC and eCommerce growth agency. He works hands-on with Shopify and DTC brands on paid social, Google Ads, and Klaviyo email and SMS.
Founder of Top Growth Marketing · $336M+ managed ad spend · 218+ DTC brands scaled

What are you looking for?

About

Top Growth Marketing

TGM has spent more than $300 Million across social & search advertising platforms. Let us help grow your business using the best, performance-based customer acquisition strategies. 

Our Partner Brands

Growing 200+ ecom brands and counting

Walmart Playboy Gibson Hard Rock Fatburger Atlas Coffee Pixi Beauty Ubersuggest Joovv Waterbox Taste Salud Saweetie
View All Clients
OR
Book a Call

$314M+ managed ad spend · 200+ brands scaled

You May Also Like...

0 Comments

Submit a Comment

Your email address will not be published. Required fields are marked *