Incrementality: The Definitive Guide

Kevel Team

Written by the people building the future of retail media

Table of Contents

What is incrementality?

How do I measure incrementality?

Incrementality testing examples

Why do advertisers care about incrementality?

How retailers can increase incrementality

Incrementality vs. attribution

Why incrementality matters in retail media

What incrementality is and how to measure it.

Incrementality has quickly risen to the forefront of the retail media conversation, becoming a crucial metric that advertisers are increasingly expecting from retail media networks. Publishers have measured incremental lift for a long time, but with marketing budgets shrinking YoY, according to Gartner’s CMO Survey, advertisers are more stringent than ever about where they spend their money.

Many leading retail media networks (RMNs) have already started investing in measurement and showcasing incrementality to advertisers. Instacart’s VP of Ads Product, Ali Miller, has dubbed incrementality the “gold standard in terms of showing the actual impact or casual impact of ads on driving sales for our advertisers and brands.”

Albertsons has also embraced incrementality in recent times. Evan Hovorka, Albertsons Media Collective’s VP of Product and Innovation, recently stated that incrementality helps retailers justify the effectiveness of campaigns run on their RMNs.

"Incrementality tells you not only did you make a sale, but how much of that total sale amount came as a result of the retail media network advertising."

- Evan Hovorka, VP of Product and Innovation at Albertsons Media Collective, Chain Store Age 2023

What is incrementality?

According to the Interactive Advertising Bureau (IAB), incrementality in retail media “...quantifies the causal impact of marketing strategies on outcomes such as sales, isolated from other business influences and attributed to advertising campaigns.” The IAB’s Jeff Bustos also explains incrementality as the ability to isolate the media spend when looking at things like sales, ROAS, or any other kind of conversion measure a brand is tracking.

Put simply, incrementality, sometimes also called “incremental lift”, is a measure of true value; the potential causal impact of marketing. In formulaic terms, incrementality is the difference between the number of sales that would have occurred without the campaign and the number of sales or engagements that actually occurred with the campaign:

‍ (Test Conversion Rate - Control Conversion Rate) / (Test Conversion Rate) = Incrementality

How do I measure incrementality?

Measuring incrementality is crucial to understanding the impact of marketing campaigns, but it can be challenging due to the various methodologies, formulas, and data sets involved. It’s rarely guaranteed that two measurements of incrementality use an identical system. In fact, Jason Goldberg, Chief Commerce Strategy Officer at Publicis, put it like this: “Marketers and CFOs want to compare these things apples to apples, and that’s not possible.”

To better understand how to measure incrementality, let’s go over the various methodologies, data requirements, and best practices for transparency according to the IAB.

Measuring Incrementality: Methodologies

Here are the IAB-recommended methods for measuring incrementality:

• Randomized control trials (RCT). Also called A/B tests, this method randomly assigns individuals to different conditions to assess their impact using statistical techniques (e.g. t-tests, covariance analysis, and regression analysis) to compare effectiveness across two groups.
• Synthetic controls. Synthetic controls are used to calculate campaign sales lift by creating a post-campaign control group of non-exposed consumers with characteristics similar to those of exposed ones. When employing such controls, remember to consider factors like purchase and digital behavior, employ propensity scoring, and criteria to avoid bias.
• Matched-market tests. This method is common for in-store media, comparing two similar markets - one targeted in a marketing campaign and the other untargeted - to observe incremental impact. Methodological rigor in order to isolate the incremental sales effect and minimize data bias is important.
• Other machine learning models. Other ML models are used to measure the increase of specific outcome metrics caused by a campaign. (This usually involves building a more complex predictive model.)

Emerging methods

New methods for measuring incrementality in retail media are still being developed. This includes:

• Closed-loop measurement. This method measures the impact of a campaign by tracking the entire customer journey from awareness to purchase. This is a highly accurate way to measure incrementality.
• Econometrics. This is a statistical method that can be used to estimate the impact of a campaign on sales. Econometrics is more complex than closed-loop measurement, but in some cases, it can offer more accuracy.

Measuring Incrementality: Data

Different types of data are necessary for accurately measuring incrementality.

These data types include:

• Deterministic data. Deterministic data is highly accurate, using precise user data, like login data, user IDs, or info pulled from loyalty programs. This data type should be prioritized to form the baseline of incremental analysis due to its accuracy.
• Probabilistic data. Probabilistic data predicts user behavior using statistical techniques and should be used in multi-platform scenarios -- and ideally, only when deterministic data is insufficient.

Key data sources for measuring incrementality include:

• Media spend data. One of the most commonly used data sources. This is the spend on each channel from impressions or clicks, and is a kind of deterministic data.
• Sales data. The other most commonly used data source. This includes marketing KPIs and is a kind of deterministic data.
• Competitor and market conditions data. This data is usually sourced from third parties or public databases.
• Customer attitudinal data. This means things like customer sentiment and brand awareness, and is a kind of probabilistic data.
• Digital data. This is data from digital platforms, including click-through rates (CTRs) and web analytics data.

Remember: Incrementality within the retailer’s ecosystem may not consider broader market impact.

Measuring Incrementality: Transparency

MRC standards stress transparency in incrementality methodology, assumptions, and limitations. Here are some IAB-recommended best practices for transparency:

• Ensure accuracy in data reporting and run periodic validation. For deterministic data, validate data completeness, consistency, and reliability. For probabilistic/synthetic data, use well-established statistical methods and validate models with actual data.
• Retain raw data for auditing. Document data processing procedures.

For more in-depth information, download the IAB and MRC’s Retail Media Measurement Guidelines here, or browse the IAB’s Retail Media Measurement Guidelines Explainer here.

Incrementality testing examples

Incrementality Example #1: Test Calculation

To calculate incrementality using an A/B method, you’ll want to find the conversion difference between a test group and a control group. This formula looks like:

‍ (Test Conversion Rate - Control Conversion Rate) / (Test Conversion Rate) = Incrementality

To see this formula in action, let’s imagine your test group saw 1.5% conversion and your control group saw a 0.5% conversion -- “conversion” meaning leads, sales, profit, or whatever metric is relevant to your business.

‍ (1.5% - 0.5%) / 1.5% = 66.7% incrementality in conversions

Incrementality Example #2: Step-By-Step

Here’s a more functional example of what incrementality testing can look like:

1. You’ll select a market or region in which you want to examine the effectiveness of an advertising campaign. This is the treatment group.
2. Select a market or region, or derive an artificial market or region, which serves as a baseline for comparison. This is the control group.
3. Run ads in the treatment group only, ensuring everything in the control group remains as constant as possible.
4. Once the test has finished, analyze results and calculate the incremental uplift and its associated ROI.

Why do advertisers care about incrementality?

Incrementality has become crucial for advertisers because it allows them to measure the true impact of their retail media campaigns. According to a report by the Association of National Advertisers (ANA), advertiser reception to RMNs has been relatively lukewarm, with 42% of advertisers feeling “on the fence” regarding their RMN investments. The same report found that brands believe a lack of measurement standardization and transparency is “preventing advertisers from deriving full value from their RMN investments.”

That’s where incrementality plays an important role:

Unlike return on ad spend (ROAS), which only measures total sales or engagement generated by a campaign, incrementality focuses on the additional sales or engagement that is directly attributable to the campaign. This distinction is why some feel that a high ROAS can be misleading -- a sentiment reflected in the 56% of advertisers surveyed who cite incremental sales as their most used KPI for RMNs, just behind 58% that cite ROI and ROAS.

"Incrementality is a measure that can show brands 'real math' that proves investing in a retail media network will make them money, and not just hypothetical money that you'll never see, but money that actually shows up in your GAAP reported; income statement."

- Jason Goldberg, Chief Commerce Strategy Officer at Publicis, Modern Retail 2023

Another reason for incrementality’s rise in popularity among advertisers is owed to the broader economic environment. Publicis’ Jason Goldberg explains that people get more interested in incrementality when budgets are tight, since it can help control costs by ensuring that every dollar spent on marketing is driving actual, measurable value -- minimizing media waste and concentrating on which ads are truly driving revenue.

Put simply, advertisers care about incrementality because it offers a precise and accurate measure of a campaign’s effectiveness, and helps advertisers make better informed decisions.

How retailers can increase incrementality

Improving incremental sales lift will involve stepping up your overall advertising and marketing efforts. While some factors that impact incrementality are out of your control, like market trends or your competitors, there are some straightforward ways retailers can help increase incrementality.

Here are some of them:

• Offer unique ad units. Engaging ad units, like native ads or video shelf placements, are far more likely to capture user attention than traditional ad formats. If you want to increase the likelihood of consumer engagement (and conversion), launching standout ads is a good place to start. Additionally, this allows for brands to appear where they might not organically, promoting new-to-brand customers.
• Optimize audience segmentation. Segmenting users based on incremental visits will provide a better understanding of a campaign’s impact on each consumer and reveal how each channel may have influenced users differently. For instance, if you find that a campaign was more effective on mobile than TV, you may want to adjust your campaign parameters to focus more on mobile in the future.
• Utilize targeting. Targeting the right audience with tailored messaging can not only lead to higher conversions but also help brands showcase their lifetime value.
• Understand campaign frequency. Optimizing the frequency of your campaigns can encourage users and drive action, so long as you’re careful to avoid inundating users and generating ad fatigue. It can also help to maintain campaign consistency across all channels to reinforce brand messaging and enhance marketing efforts.

Incrementality vs. attribution

When it comes to measuring the impact of marketing activities, two key concepts often come into play: attribution and incrementality. While both are essential for understanding campaign effectiveness, they serve different purposes and offer unique insights.

Attribution

Attribution is primarily used for short-term measurement, providing frequent, granular data in real-time. It involves matching two data points -- like clicks to installs, or impressions to installs -- to understand which marketing touchpoints contributed to a conversion. Attribution helps marketers track user interactions across various channels and determine which efforts are driving immediate results.

But attribution has its limitations, focusing more on association rather than causation. This means that while attribution can tell you which touchpoints were present in the user journey, it can’t infer whether those touchpoints were the actual cause of the conversion. This usually leads to questions about the true impact of each marketing touchpoint: what if a tracked marketing touchpoint had no influence on the conversion? Without understanding this, marketers can’t be certain their investments are truly driving value.

Incrementality

Incrementality, on the other hand, is used for mid-term measurement and focuses on the true effectiveness of advertising activities. It can help determine whether a marketing effort has brought additional value that would not have otherwise taken place.

Unlike attribution, which measures association, incrementality measures causation. It helps marketers understand the immediate impact of new campaigns and strategies by evaluating whether observed outcomes are directly attributable to a campaign.

Media Mix Modeling (MMM)

If you’re looking into attribution and incrementality, chances are you’ll also see media mix modeling (MMM) pop up. MMM is used for long-term measurement, allowing marketers to analyze the impact of various marketing channels over an extended period.

Why incrementality matters in retail media

Advertisers want concrete evidence that an investment in a retailer’s RMN will bring financial returns. Incrementality metrics are a powerful tool retailers can use to demonstrate the true value of their RMN, focusing on outcomes that are directly attributable to an advertiser’s marketing efforts.

The future of incrementality measurement in retail media is promising. Emerging methods are becoming more sophisticated, measuring the impact of campaigns across multiple channels and touchpoints, and by focusing on incrementality, retailers can help brands optimize marketing spend, minimize waste, and ultimately improve profitability.