Calibrating your MMM with experiment results
How Sellforte uses geo lift and conversion lift results as evidence in your marketing mix model, and what changes when it does.
A marketing mix model (MMM) estimates each channel's incremental impact from your full sales and media history. An incrementality experiment measures one thing causally: a specific channel, in a specific period and set of markets, compared with a control group. Calibration connects the two. Sellforte uses the experiment result as evidence about what the model should consider plausible for that channel (in Bayesian terms, a prior) before the model is fitted to your data.
Which evidence can be used
- Geo lift tests analyzed in Sellforte Experiments. A geo lift test can calibrate any channel, including offline media.
- Conversion lift studies, such as the Meta Conversion Lift studies in your Experiment Library.
- Attribution data combined with incrementality benchmarks. This gives an informed starting point for channels that have no test evidence.
How several tests are combined
When several tests cover the same channel, Sellforte combines them into a weighted view. The latest result doesn't simply overwrite the earlier ones, and the results aren't simply averaged: newer tests and tests with tighter confidence intervals carry more weight. A single new test therefore doesn't swing the model to its own number, but a consistent pattern across tests moves the model over time.
What makes a test useful for calibration
- The same KPI as your model. A test that measures purchases lines up poorly with a model that measures revenue. Agree the KPI before the test starts.
- A control group. A geo lift test compares test regions with comparable control regions. Switching a channel off in a single market with nothing to compare against gives no counterfactual, so it can't be analyzed this way.
- Enough spend and time. The tested activity needs to be large enough, and to run long enough, to move the KPI above its normal variation. See How to Design a Geo Lift Experiment.
- A precise result. A test can be good enough for a campaign decision and still be too uncertain to calibrate a model with. Results with wide confidence intervals carry little weight.
You may not need a new test
Calibration can use tests you have already run. Tests in your Experiment Library can be used as they are, and tests run outside Sellforte can be added: Meta Conversion Lift studies can be imported, and a historical geo test can be re-analyzed in Sellforte Experiments when you know its timing and its test and control groups. A new test is needed only when a channel has no usable evidence. Test Prioritizer shows where your test evidence is missing or out of date.
What changes in your results
- Incremental ROAS can change for the tested channel, upward or downward. Test results are direct causal evidence, so they typically carry more weight than priors based on attribution data.
- Media Optimizer follows the calibrated model. If a channel's incremental ROAS rises, the optimizer will start recommending more budget for it.
- The model and the test will still show different numbers. The test measured specific activity in one period and set of markets. The model estimates the channel's ongoing contribution alongside every other channel, promotion and seasonal effect. Calibration makes the model take the test seriously without discarding everything else it sees.
Getting started
Calibration with experiment results is part of Sellforte MMM plans. To use your test results in calibration, or to plan a test that will produce usable evidence, contact your Customer Success Manager. The Sellforte team reviews which tests are strong enough to use before they affect your model.