Sellforte Solution Research helps marketers, marketing analytics leaders and procurement teams review and evaluate marketing measurement and optimization tools, solutions and services.
We conduct primary research to understand what advertisers need and turn those needs into practical vendor evaluation criteria. We then evaluate vendors against those criteria using publicly available information. Advertisers can use the findings as a starting point for their own evaluation.
We publish two types of research:
- Evaluation guides that explain what to assess, why it matters and what to ask vendors.
- Vendor comparisons that apply those frameworks and show the evidence behind the assessments.
Guiding principles for creating evaluation criteria
1. Start with advertisers' needs
We develop criteria from the questions advertisers ask, the work they need to complete and the requirements they bring to procurement.
Our research draws on the constant interactions we're having with marketers and analytics professionals and from the experience we gain from participating in tender processes. This helps us understand both what buyers ask for and what teams need during implementation and ongoing use.
2. Study actual usage and unmet expectations
Where usage data is available, we examine what people ask tools to do, including requests the tools cannot answer.
As an example, for our conversational AI evaluation framework, we analyzed 1,660 real prompts submitted to Sellforte AI, including unsuccessful requests. We combined this with advertiser discussions and research across 78 vendors. Unmet expectations help identify capabilities buyers need beyond what products currently deliver.
3. Define capabilities buyers can assess
Each criterion should describe a capability clearly enough for a buyer to investigate and a vendor to demonstrate.
For example, evaluating AI requires distinguishing between reporting historical results, recommending budget allocations and executing changes. The framework should also examine the analytical models supporting those answers and whether users can inspect their assumptions.
4. Cover the full working process
Evaluation criteria should cover how teams use a tool beyond an individual analysis: how they access results, keep track of earlier findings and use measurement to make decisions.
As an example, for incrementality testing tools, this includes support for different experiment types, a shared experiment library and integration with Marketing Mix Modeling. Enterprise requirements such as security, access controls and deployment options also belong in the evaluation.
5. Allow buyers to set their own priorities
The importance of each criterion depends on the organization's measurement program, internal resources and procurement requirements.
A team evaluating geo testing may prioritize different capabilities from one managing experiments across multiple countries. Buyers should identify mandatory requirements early and use realistic demonstrations to investigate the capabilities that matter most to them.
Guiding principles for vendor evaluations
1. Apply consistent rules to every vendor
Within each comparison, we apply the same criteria, scoring instructions and evidence standards to every vendor, including Sellforte.
The published methodology explains the assessment's scope and any assumptions or exceptions. As an example, our AI vendor comparison includes specific rules for distinguishing capabilities evidenced in a conversational interface from those documented elsewhere in the platform.
2. Use public evidence and explain its limits
Our comparisons assess vendor websites and publicly accessible technical documentation. This means we're evaluating the same information that is available to a prospect before taking a demo or sales call. Every vendor has a similar opportunity to influence their scoring by adapting their public documentation.
In terms of source materials, technical documentation takes priority, followed by product pages and marketing materials.
Scores describe the evidence found under those rules. A zero means that the research did not find supporting evidence; it does not establish that the capability is absent. Documentation quality and discoverability can affect results. Our incrementality testing vendor comparison explains these evidence standards.
3. Using AI in scoring, to minimize Sellforte bias
In our current comparisons, Claude and ChatGPT separately assess vendors, and we average their scores.
We also disclose the role of AI and Sellforte. Sellforte develops the framework and publishes the research while also participating in the market being evaluated. AI-assisted scoring does not eliminate potential bias in the framework or errors in finding and interpreting evidence. We disclose those limitations.
4. Make assessments open to inspection
We publish criteria, scoring instructions and research records, including rationales and source links. Readers can examine individual judgments and repeat the process.
We also record evaluation dates and model versions. A repeated assessment may produce different results as documentation, models and search results change.
5. Keep scores in context
Total scores summarize performance against a particular framework. Category results help buyers investigate their specific priorities.
In our current comparisons, criteria have equal weight, so categories containing more criteria contribute more to the total. These assessments use public sources and do not include hands-on product testing. Buyers should verify capabilities and investigate pricing, services, support and integration requirements directly.
6. Revise findings when evidence changes
Vendor offerings and documentation evolve. We record evaluation dates and changes, and welcome corrections supported by documentation. Vendors and readers can submit evidence for review at research@sellforte.com.
How advertisers can use the research
Use the evaluation guides to define requirements and prepare questions. The vendor comparisons can help you build a shortlist and identify what needs verification.
Base your final choice on your organization's objectives and the evidence gathered during your own evaluation, including vendor discussions, demonstrations and procurement review.
Authors
Lauri Potka is the Chief Operating Officer at Sellforte, with over 15 years of experience in Marketing Mix Modeling, marketing measurement, and media spend optimization. Before joining Sellforte, he worked as a management consultant at the Boston Consulting Group, advising some of the world’s largest advertisers on data-driven marketing optimization. Follow Lauri in LinkedIn, where he is one of the leading voices in MMM and marketing measurement.
Marketing budgeting season is here with five best practices to follow.
Bid Optimization: How to Calculate the Optimal Bid Values for Your Campaigns Using miROAS
