Test Prioritizer: Overview

Test Prioritizer helps you decide which incrementality test to run next. It looks at your media investment and your previous testing activity, finds testing opportunities across online and offline media, and ranks them with a priority score. You find it in the Experiments section of Sellforte.

Most teams have more possible tests than time and budget to run them, and a test typically takes four to eight weeks. Test Prioritizer gives you a shared, data-based starting point for the testing roadmap, so the next test answers a question that matters for an upcoming media decision.

Test Prioritizer is switched on per workspace. If you don't see it in Experiments, contact your Customer Success Manager.

How the priority score works

Each opportunity is a media feature: a country, an ad platform and an advertising channel, for example Meta Sales in the United States. It gets a priority score between 0 and 10, built from three components (the Impact, Confidence, Ease framework):

• Impact reflects how important the opportunity is, based on its media investment over the last 12 months and its rank in your media mix. If you use Sellforte Marketing Mix Modeling (MMM), the model's incremental ROAS estimates also highlight channels with unusually high or low returns compared with the country average.
• Confidence shows gaps in existing experimental evidence: how many tests have covered the media feature and how recently.
• Ease reflects how straightforward the suggested test method is to run. The suggested method is a conversion lift study on ad platforms that run their own lift studies, and a geo lift test elsewhere, including offline media. A conversion lift study scores higher, because it needs only a small holdout group.

Each component contributes up to a third of the score. Open an opportunity to see the details on three tabs:

• Score: the value and points for Impact, Confidence and Ease.
• Campaigns: the campaigns that make up the spend, whether they are active, and when each was last tested.
• Past tests: the experiments that were counted and how much of the spend they covered.

Keep in mind when you read the score:

• The score is a planning aid. A high score does not predict a high incremental ROAS, and it does not guarantee a conclusive test result.
• Confidence here means confidence in the existing evidence. It is separate from the statistical confidence or credible interval reported for a completed test. A media feature that has never been tested gets a high Confidence score, because new evidence would be valuable.
• Test Prioritizer doesn't consider timing. Plan the test around peak seasons and campaigns, and remember that running several tests at the same time can affect their results.

Example: reading a recommendation

Imagine an ecommerce business that advertises in several markets. Test Prioritizer ranks Meta Sales in the United States second, with a priority score of 9.2:

• Impact 2.5 of 3.3: the channel received $185,950 in media investment over the past 12 months, which puts it among the top third of media features by spend. Sellforte MMM also estimates its ROAS at 96% above the country average.
• Confidence 3.3 of 3.3: the media feature has never been tested, so there is no direct experimental evidence of its contribution.
• Ease 3.3 of 3.3: a conversion lift study is available, with a quick setup and a small holdout group.

The component scores are rounded, so they don't always add up exactly to the total. The team now knows what to test next and why, and can set up the conversion lift study through its usual testing workflow.

Test coverage

The coverage cards show how broadly your testing effort reaches:

• the share of media spend with test evidence from the last 12 months
• the number of media features tested

For example, tests might cover 29% of media spend and 8 of 65 media features. The coverage cards describe your whole media mix, while the ranking reflects the filters you select.

Coverage and the Confidence score count the tests in your Experiment Library. If an earlier test is missing from the library, the channel it covered is ranked as untested, so add your past tests to the library first.

Filtering the ranking

Filter the opportunities by country, ad platform and suggested test method. This is useful when you prepare a shortlist for the team responsible for a market or a channel. Use the ranking together with your campaign calendar, available budget and operational constraints to choose the next test.

Hiding features you can't test

If a media feature can't be tested, for example because there's no budget to hold back or the channel is ending, mark it as untestable. Features with no spend in the last three months are hidden automatically. You can show both again when you need them.

From a priority to a test

Selecting an opportunity is the starting point for experiment planning. A simple workflow is:

1. Decision: agree which budget decision the test should inform, and what you would do differently after a credible result.
2. Scope: choose the channel, the market and the test method.
3. Design: agree the KPI, the intervention, the timing and the data needs.
4. Readout: review the result and decide what to do next.

Test Prioritizer doesn't set up the test. Set it up through the existing Experiments workflows: How to Design a Geo Lift Experiment, Sellforte Experiments: Conversion Lift and Sellforte Experiments: A/B Test. The Design test option that you may see in Test Prioritizer is part of upcoming functionality and is not active yet.

Test Prioritizer and Sellforte MMM

If you use Sellforte MMM, model results add evidence to the Impact score. Completed tests can also be used to calibrate the MMM, and the calibrated model feeds Media Optimizer: prioritize what to test, measure the incremental impact, calibrate the MMM, and optimize your media investment. See Calibrating your MMM with experiment results.