The short answer
MMM should separate email marketing from service messages using each message's purpose, sending trigger, and intended sales outcome. Review the mapping with email and CRM teams, distinguish purchase-driven messages from campaigns intended to generate additional purchases, and use experiments for uncertain cases. Automated emails and service messages containing promotions need individual assessment.
This article is part of Asked by Marketers, a series answering real questions from marketing leaders.
Sellforte's team holds more than 1,450 meetings each year with marketing leaders in Ecommerce and Retail about Marketing Mix Modeling and incrementality testing. Each week, we anonymize at least one question from those conversations and answer it in depth, based on what marketers are actually struggling with, not what keyword tools suggest. About the series →
Why marketers ask this
An email channel can contain newsletters, promotional campaigns, account updates, and messages sent because somebody has already bought something. When these all enter the same reporting category, a large email contribution is difficult to interpret. Which activity should the team increase, and which part of the result simply reflects an active customer base?
The people interpreting the model may not know what each source-system code means. A category labeled "service" can include product recommendations. A triggered message can be a sales campaign. Email and app push notifications may even have been grouped together during data preparation.
In one model, an email classification review led to a larger estimated baseline in MMM. A larger baseline alone does not validate the revised model. The team still needs to establish what changed and whether the new treatment answers the business question. Our article on a small media share of total sales explains how to interpret contribution alongside model scope.
Which emails should MMM treat as marketing?
Classify emails by the customer behavior they are intended to change and the event that causes them to be sent. A platform label or an automation rule is insufficient on its own.
For Marketing Mix Modeling, the timing matters. An order confirmation sent after a purchase cannot have caused that purchase. A promotional message sent to an existing customer could influence a later purchase, even if the sending process is automated. The model needs to distinguish those outcomes.
On a small screen, scroll sideways to read the full table.
| Message | Measurement question | Starting treatment |
|---|---|---|
| Promotional newsletter | Did sending it generate additional purchases? | Include as a candidate marketing activity. Keep campaign and promotion details. |
| Order confirmation with only order details | Does its volume mainly follow orders already placed? | Keep it separate from the marketing activity credited with generating those orders. |
| Automated product recommendation | Does it change purchasing beyond the customer's existing intent? | Assess as marketing. Automation does not establish or rule out incrementality. |
| Service email with a next-purchase voucher | Does the promotional content increase subsequent purchases? | Flag as mixed purpose and test the promotional addition where feasible. |
Ask the channel owner to explain ambiguous categories using actual message examples. "Trigger," "info," and permission-based labels may describe how the system operates without explaining what the message does. Record the classification, its rationale, and any unresolved cases.
The same review should establish whether a message targets an existing customer, a prospect, or someone who has already completed the outcome being modeled. Email can contribute to repeat purchasing while being an implausible explanation for the earlier event that made someone eligible to receive it.
How should the classification feed into MMM?
Apply the agreed categories consistently to the historical data and document how each category enters the model. The analytics team needs to trace the email contribution back to the messages included in it.
Start with a mapping that connects campaign or message IDs to purpose, trigger, audience eligibility, and promotional content. Keep email separate from push notifications where the data allows. Preserve the dates when definitions changed so a reporting change does not look like a sudden change in marketing effectiveness.
Specify the input metric as well. Depending on the methodology, a model may use activity measures or attributed conversions and revenue. A send count describes volume; attributed revenue describes credit assigned by a tracking system. Neither is automatically a measure of additional sales. If attributed data is used, the classification must carry through to that data, and its credit must still be evaluated against causal evidence.
Service activity that follows purchases should not automatically become a control variable just because it has been removed from the marketing category. Feeding an outcome-derived signal back into the model can distort the effects being estimated. The modeler needs to specify the causal role and timing of the variable.
Where a category's contribution cannot be identified reliably, document that limitation and its treatment in the baseline or reporting scope. A baseline classification is a modeling choice, not experimental evidence that the messages have zero effect on all future purchases.
After changing the mapping, compare model versions over the same dates and sales definition. Inspect the movement in email contribution, baseline, and other channels. Check it against message content and available experiments. The team needs to defend the revised estimate, even when it is less favorable than the original result.
How can you measure emails that combine service and marketing?
Use a test that varies the promotional content while keeping the essential service message consistent. That comparison can estimate what the promotional addition contributes to later purchases.
For example, randomly assign eligible customers to receive either a service email with a product recommendation or the same service email without it. Define the purchase outcome and follow-up period before starting. Track outcomes for everyone assigned to each group, including customers who never open or click the message.
An incrementality test estimates the difference in outcomes caused by the tested change. Comparing openers with non-openers would mix the effect of the email with differences in customer interest. Random assignment provides a more credible comparison, provided delivery, measurement, and other campaign exposure are handled consistently.
A voucher requires an additional distinction. If one group receives a voucher and the other does not, the result measures the combined effect of the offer and its communication. It does not isolate the email's effect from the discount. Choose the comparison to match the decision, and evaluate margin after discounts and returns when deciding whether to continue the activity.
When using the result to inform MMM, retain the message category, eligible audience, outcome definition, and observation window. A test of one promotional module does not establish the incrementality of all newsletters or all service messages. Report the uncertainty, particularly when the test contains few repeat purchases.
How this looks in practice
Consider a hypothetical retailer whose order confirmation includes a recommendation for a complementary product. Its reporting system groups these messages with promotional newsletters.
The team first separates confirmations from newsletters and checks which purchase each message could have influenced. The original order occurred before the confirmation was sent, so it cannot be evidence of the confirmation's marketing effect. A subsequent order is a relevant outcome.
For the next test, both randomly assigned groups receive their order details. One group also sees the recommendation. The retailer compares subsequent net revenue per assigned customer over a predefined period, with an uncertainty interval. That result informs the treatment of the recommendation in measurement. It does not measure the total value of sending an order confirmation.
The team records the distinction in the data mapping, checks that it is applied across the relevant history, and reviews the revised email contribution before changing campaign plans.
Related questions
How large should an email holdout be?
Size it around the expected effect, outcome variability, measurement period, and business cost of withholding the tested marketing. There is no universal percentage. Our guide to incrementality-test holdout size explains the tradeoffs.
How Sellforte helps
Sellforte combines MMM, incrementality testing, and attribution to help teams assess the additional sales their marketing generates. For email measurement, that work includes reviewing channel definitions and aligning experimental evidence with the activity being modeled. Book a demo to discuss your measurement setup.
Authors
Lauri Potka is the Chief Operating Officer at Sellforte and has over 15 years of experience in Marketing Mix Modeling, marketing measurement, and media spend optimization. Before joining Sellforte, he worked as a management consultant at the Boston Consulting Group, advising some of the world's largest advertisers on data-driven marketing optimization. Follow Lauri on LinkedIn, where he is one of the leading voices in MMM and marketing measurement.