MASS Analytics review

Strong — 4 of 5 on this rubric’s scale, from Poor (1) to Excellent (5). This dimension carries 25% of the total score.

Benchmark — across the 16 other agencies evaluated in this discipline, this dimension runs Excellent 1 · Strong 3 · Adequate 12. The typical agency here scores Adequate, and 1 of them score higher than this one.

What this dimension measures: The heaviest weight, because an unvalidated model is an opinion with decimal places. Look for a described methodology, stated assumptions, and above all how the model is VALIDATED: holdout periods, backtesting, or reconciliation against a real experiment. A described method with no validation story scores 3.

Scores high — A firm that publishes its validation approach and names its limits scores 4-5.

Scores low — 'Proprietary algorithm' with no methodology at all scores 1-2 — proprietary is not a method, and in this category it is the single least checkable claim a vendor can make.

What we found — Publishes real, ungated methodology rather than a proprietary-algorithm claim. The marketing-mix-modeling solution page states models are 'either Frequentist or Bayesian' built on 'sound statistical principles'; the models-plus-experiments white paper sets out an explicit validation philosophy (experiments anchor uncertain modelled coefficients, narrow confidence intervals and become priors for future planning, with 'meta-analytic benchmarking over time' across channels and regions as the accumulating evidence base). It names its own limits: it states plainly that MMM estimates correlation rather than causation, and cautions that quasi-experimental methods such as synthetic controls and matched markets 'depend heavily on modeling assumptions'. The Kellogg's case study names a concrete technique - log-linear modelling fitted via genetic algorithms over auto-generated candidate variables. Held below Excellent for two reasons read first-hand: no in-model fit-validation protocol is published anywhere on the site (no holdout window, backtest, or cross-validation procedure is described - the platform page says only 'robust model validation with every model run' without elaborating), and the one independent signal on this point is negative, a Capterra reviewer citing the lack of cross-validation techniques 'which can lead to overfitting and biased coefficients'. That criticism lands squarely on the genetic-algorithm variable search, which explores thousands of candidate specifications per run and is exactly the design that needs a published overfitting guard. That is the high band above, which is why it scored Strong.

On the record — “Ungated white paper sets out the validation philosophy: experiments replace uncertain modelled coefficients with measured lift, narrow confidence intervals, and become priors for future planning in a repeating Model-Experiment loop. It explicitly cautions that quasi-experimental methods such as synthetic controls and matched markets depend heavily on modelling assumptions, and advocates meta-analytic benchmarking over time rather than a single holdout test.” mass-analytics.com ↗

On the record — “Models are stated to be 'either Frequentist or Bayesian' built on 'sound statistical principles', with analyst guidance through model design and an explicit knowledge-transfer goal. No holdout, backtest, cross-validation or calibration protocol is described on the page.” mass-analytics.com ↗

Evidence: partly checkable — corroborated in part against the sources below; the remainder rests on the agency’s own account.

Strong — 4 of 5 on this rubric’s scale, from Poor (1) to Excellent (5). This dimension carries 20% of the total score.

Benchmark — across the 16 other agencies evaluated in this discipline, this dimension runs Excellent 1 · Strong 6 · Adequate 5 · Weak 4. The typical agency here scores Adequate, and 1 of them score higher than this one.

What this dimension measures: Whether the firm can establish causality rather than only correlation: geo holdout tests, matched-market design, PSA/ghost-ad tests, switchback designs, or reconciliation of modelled results against live experiments.

Scores high — Named experiment designs with described execution score 4-5.

Scores low — Correlation-only modelling with no experimental capability scores 2-3 — legitimate and common, but the buyer should know they are buying a correlational estimate. A firm that presents modelled attribution as proven causality scores 1-2, and the overclaim should be named in the verdict.

What we found — A dedicated incrementality measurement line, not an afterthought. Geo experiments and randomised controlled trials are named as the primary designs, with execution detail that is specific enough to be wrong: test and control regions randomly allocated and assigned BEFORE any media runs, geo holdouts typically running four to six weeks, aggregated regional sales data with no individual identifiers or cookie dependency. It states that post-hoc regional analysis 'is observation, not experimentation and does not establish causality' - a discipline point most vendors in this category skip. Reconciliation into the MMM is described as coefficient anchoring and prior-setting in a repeating Model to Experiment to Model loop. Crucially it does not overclaim: modelled attribution is presented as correlational and experiments as the causal ground truth, which is the opposite of the 1-2 failure mode. Short of Excellent because no executed experiment is attributable to a named client, and the pages never say whether MASS designs and runs the tests as a service or whether the buyer runs them and the platform ingests the results. That is the high band above, which is why it scored Strong.

On the record — “Incrementality practice names geo experiments and randomised controlled trials, requires test and control assignment before media runs, states that post-hoc regional analysis is observation and does not establish causality, uses random allocation of regions, and uses aggregated regional sales data with no individual identifiers or cookie dependency.” mass-analytics.com ↗

attribution — working out which marketing touch actually caused a sale. Good practice names its model and its blind spots; bad practice quotes each ad platform’s self-graded numbers, which overlap and overclaim.

Evidence: vendor stated — the agency’s own claim, recorded as theirs rather than ours.

Strong — 4 of 5 on this rubric’s scale, from Poor (1) to Excellent (5). This dimension carries 15% of the total score.

Benchmark — across the 16 other agencies evaluated in this discipline, this dimension runs Excellent 1 · Strong 8 · Adequate 6 · Weak 1. The typical agency here scores Strong, and 1 of them score higher than this one.

What this dimension measures: Attributable client work at a stated scale and category. Treat any 'we found X% waste' claim as a vendor-stated number unless a client is named and corroborates it.

Scores high — Named clients with described engagements score 4-5.

Scores low — Anonymised case studies score 2-3 — common here for genuine confidentiality reasons, so do not penalise beyond the band, but do not credit unverifiable lift figures either.

What we found — One fully named engagement read first-hand: the Kellogg's case study identifies the client, dates the work 2018-2020, states the scope (three years of weekly sales, price, promotion and distribution data for one leading brand), gives the analytical question, carries a quote attributed to a Kellogg's data science team member, and reports a modelling cycle cut from eight to sixteen weeks down to roughly seven days. The homepage additionally carries client logos for Intel, Monoprix, Publicis, Geant, Spark Foundry, Harlequin, mBank and Publicis Media, none of which have a corresponding case study. Remaining case studies are de-identified by design - the global CPG cost study is explicitly labelled 'DE-IDENTIFIED', and the multi-product CPG study names only 'a major consumer packaged goods company'. Per the rubric the outcome figures are treated as vendor-stated throughout: the 70 percent cost reduction, the 16x speed-up and the 95 percent data-prep saving are the vendor's own numbers with no client attribution, no third-party verification and, in the CPG case, charts showing relative comparisons with no absolute values. Credited here is the existence of a named, described engagement, not the size of the lift. That is the high band above, which is why it scored Strong.

On the record — “The 70-percent-cost-reduction case study is explicitly labelled 'CASE STUDY - GLOBAL CPG - DE-IDENTIFIED'. Scope stated as 10 markets, 15 product groups, 4 brands, about 600 model-unit combinations. No client attribution, no testimonial, no third-party verification; supporting charts show relative comparisons with no absolute numbers.” mass-analytics.com ↗

On the record — “Named client engagement: Kellogg's, 2018-2020, using three years of weekly sales, price, promotion and distribution data for one leading brand to isolate Trade Promotion contribution with Media held constant. Claimed outcomes (vendor-stated): modelling cycle cut from 8-16 weeks to about 7 days, data preparation cut roughly 95 percent. Method named as proprietary log-linear modelling via genetic algorithms over thousands of auto-generated candidate variables. Carries a quote attributed to a Kellogg's data science team member.” mass-analytics.com ↗

attribution — working out which marketing touch actually caused a sale. Good practice names its model and its blind spots; bad practice quotes each ad platform’s self-graded numbers, which overlap and overclaim.

Evidence: partly checkable — corroborated in part against the sources below; the remainder rests on the agency’s own account.

Strong — 4 of 5 on this rubric’s scale, from Poor (1) to Excellent (5). This dimension carries 15% of the total score.

Benchmark — across the 16 other agencies evaluated in this discipline, this dimension runs Strong 10 · Adequate 4 · Weak 2. The typical agency here scores Strong, and none scores higher than this one.

What this dimension measures: What the engagement needs from the buyer (spend, conversion, and channel data at what granularity) and — critically — whether the firm also buys the media it is measuring. A firm that measures media it does not sell is structurally more credible; where the same firm buys and grades its own work, that conflict must be disclosed and should be named in the verdict whether or not the firm names it.

Scores high — Clear data requirements plus independence from media buying scores 4-5.

Scores low — Undisclosed conflict scores 1-2.

What we found — Structurally independent on the axis this rubric cares about most: across every page read, MASS Analytics sells modelling, software and training and does not buy or resell the media it measures, so it is not grading its own placements. Two commercial relationships are worth a buyer's attention even though neither is that conflict - Publicis and Spark Foundry (both Publicis media agencies) appear on the client logo wall, and agencies are named as a target segment, so some engagements are bought by the media agency rather than by the advertiser whose spend is being graded. Data requirements are stated at a workable but not granular level: the platform connects to over 150 sources (ad platforms, CRM, sales, third-party measurement), runs natively on Snowflake and therefore requires data to reside there or be migrated to it, the Kellogg's engagement used weekly sales, price, promotion and distribution history, and geo testing needs only aggregated regional sales. What is never stated is a minimum: no required history length, no minimum spend, market count or market size for a valid model or a valid geo test. That is the high band above, which is why it scored Strong.

On the record — “MASS Analytics does not buy or resell media on any page read, so it does not grade its own media placements. However, media agencies are a named customer segment and Publicis and Spark Foundry (both Publicis media agencies) appear on the homepage client logo wall alongside Intel, Kellogg's, Monoprix, Geant, Harlequin, mBank and Publicis Media.” mass-analytics.com ↗

On the record — “Platform connects to over 150 data sources, runs natively on Snowflake (requiring data to reside there or be migrated), states an initial model build of 6-8 weeks, and describes event-driven automatic refresh rather than a fixed interval. No pricing, tier or engagement structure with numbers appears on the platform page.” mass-analytics.com ↗

CRM — customer relationship management system: the database of record for contacts and deals — HubSpot, Salesforce and kin.

Evidence: partly checkable — corroborated in part against the sources below; the remainder rests on the agency’s own account.

Strong — 4 of 5 on this rubric’s scale, from Poor (1) to Excellent (5). This dimension carries 10% of the total score.

Benchmark — across the 16 other agencies evaluated in this discipline, this dimension runs Excellent 1 · Strong 5 · Adequate 10. The typical agency here scores Adequate, and 1 of them score higher than this one.

What this dimension measures: What arrives and how often: a one-off model, a refreshed quarterly model, a live dashboard, a decision workshop. A model delivered once and never refreshed is a snapshot of a market that has moved, and should be scored as the ceiling it is.

Scores high — Stated deliverables with a stated refresh cadence score 4-5.

Scores low — Undefined deliverables score 2.

What we found — Not a one-off snapshot, which is the failure mode this dimension exists to catch. The stated shape is an initial model build in six to eight weeks, then continuous operation: dashboards refresh automatically, the model re-checks itself as new data arrives, and early campaign reads come in 'days, not months' - the Kellogg's engagement evidences a roughly seven-day refresh cycle in practice. Named deliverables are channel- and campaign-level contribution, MROI, sales decomposition, scenario simulation and budget optimisation, executive recommendations, and training through the MMM Academy. The Walk-Run-Fly structure also states ownership terms unusually plainly: 'The model, the contribution data and the outputs. From day one' belong to the client, and 'No lock-in, ever'. Held at Strong because the refresh is described as event-driven rather than committed to an interval - no page states a contractual weekly, monthly or quarterly refresh a buyer could hold them to, and support obligations are described as SLA-backed without the SLA being published. That is the high band above, which is why it scored Strong.

On the record — “Walk-Run-Fly is a staged services-to-software model. In the Walk phase 'our modellers build your models while coaching your team on the methods'; the Run phase is client-operated with guidance; the Fly phase is fully in-house. Ownership is stated explicitly: the model, the contribution data and the outputs belong to the client 'from day one', with 'No lock-in, ever'. No pricing, minimum term or engagement minimum is disclosed on the page.” mass-analytics.com ↗

On the record — “Platform connects to over 150 data sources, runs natively on Snowflake (requiring data to reside there or be migrated), states an initial model build of 6-8 weeks, and describes event-driven automatic refresh rather than a fixed interval. No pricing, tier or engagement structure with numbers appears on the platform page.” mass-analytics.com ↗

deliverability — deliverability: whether cold email actually lands in the inbox rather than spam. The tell is infrastructure talk — warmed sending domains kept separate from your main domain — because a burned domain outlasts the engagement.

SLA — service-level agreement: a contractual promise about speed or quality of delivery.

Evidence: vendor stated — the agency’s own claim, recorded as theirs rather than ours.

Adequate — 3 of 5 on this rubric’s scale, from Poor (1) to Excellent (5). This dimension carries 10% of the total score.

Benchmark — across the 16 other agencies evaluated in this discipline, this dimension runs Strong 2 · Adequate 2 · Weak 12. The typical agency here scores Weak, and 2 of them score higher than this one.

What this dimension measures: Note any minimum spend and whether the model licence continues after the engagement ends.

Scores high — Published fees, ranges, or a stated engagement structure with numbers score 4-5.

Scores low — A described structure without numbers scores 2-3. Bespoke-only with no anchor scores 1-2.

What we found — Their own site publishes no number anywhere. Every path ends at 'Book a Demo' or 'Talk to an Expert'. The engagement structure is described without figures - Walk-Run-Fly as a staged commitment that starts small and expands, and a blog post that discusses licence cost versus total cost of ownership qualitatively while explicitly declining to give a figure for either MassTer PACE or the open-source alternative. The only anchor located is off-site and vendor-submitted: the Capterra product listing states a starting price of GBP 12,000 per year, flat rate, with a free trial and no free version. AWS Marketplace carries both MassTer Mind and the Managed MMM Consultancy as private offers, which is by definition negotiated enterprise pricing. So a software floor exists for a buyer who goes looking off-site, but the services side - the modelling engagement itself, the part of this firm the rubric is aimed at - has no published anchor at all, and no page states whether the model licence continues after an engagement ends. That is between the two bands, which is why it scored Adequate.

On the record — “Capterra lists MassTer at 4.5 out of 5 from 22 verified reviews, sentiment 95 percent positive / 5 percent neutral / 0 percent negative, with a starting price of GBP 12,000 per year flat rate, a free trial available and no free version. This is the only pricing anchor located anywhere; the vendor's own site publishes none.” capterra.com ↗

On the record — “Licence cost is discussed only qualitatively: open-source MMM 'has no license fee, the cost is in people', while 'commercial platforms like MassTer PACE carry a license cost that decreases relative to total cost of ownership as the team scales'. No figure is given for either option, and no validation methodology is named in the comparison.” mass-analytics.com ↗

Evidence: partly checkable — corroborated in part against the sources below; the remainder rests on the agency’s own account.

Adequate — 3 of 5 on this rubric’s scale, from Poor (1) to Excellent (5). This dimension carries 5% of the total score.

Benchmark — across the 16 other agencies evaluated in this discipline, this dimension runs Adequate 5 · Weak 11. The typical agency here scores Weak, and none scores higher than this one.

What this dimension measures: Independent, verified reviews or checkable references. Low weight deliberately: measurement work sells through procurement and referral, so a thin public review footprint is normal and the method evidence above matters far more.

Scores low — A substantial verified base scores 4-5; a handful scores 2-3; none located scores 2.

What we found — Read first-hand on Capterra: 4.5 out of 5 from exactly 22 verified user reviews, sentiment 95 percent positive, 5 percent neutral, 0 percent negative, from users across marketing, automotive and consulting with tenures from free trial to two-plus years. Substantive rather than promotional in content - praise for the MMM-specific depth, variable transformation and responsive support and consulting hours, alongside pointed criticism of an outdated 'Windows 98 style' interface, excessive clicking, no Mac support, a correlation-matrix ceiling around 300-400 variables, and the missing cross-validation noted above. A G2 listing for MassTer exists but returned HTTP 403 and could not be read, so no G2 rating or count is cited here. Two limits keep this at a handful rather than a substantial base: 22 reviews is modest, and every one of them reviews the SOFTWARE. No independent review of the managed modelling engagement or the consultancy was located on any platform.

On the record — “Independent negative signal on validation: a Capterra reviewer cites the lack of cross-validation techniques, 'which can lead to overfitting and biased coefficients'. Other consistent criticisms are an outdated 'Windows 98 style' interface, too much clicking, no Mac support, and a correlation-matrix ceiling around 300-400 variables.” capterra.com ↗

On the record — “Capterra lists MassTer at 4.5 out of 5 from 22 verified reviews, sentiment 95 percent positive / 5 percent neutral / 0 percent negative, with a starting price of GBP 12,000 per year flat rate, a free trial available and no free version. This is the only pricing anchor located anywhere; the vendor's own site publishes none.” capterra.com ↗

Evidence: verified — checked against a named source you can open; the links under Sources below are where to check it yourself.