How we vet, and who pays
Every number on this page is computed from the library when you load it, not typed in and left to rot.
Who pays
No agency pays us anything. Not for a listing, not for placement, not to change a verdict. That is the whole reason this library can name the agencies it cuts, and it is why no directory funded by the companies it rates ever will. We currently name 37 cuts out of 500 evaluations — the cut list is the part that makes the rest worth reading.
We are not currently charging buyers either. When that changes it will be stated here, and it will never be the rated party paying.
One rubric per discipline, not one rubric
There is no single scorecard. Each of the 22 categories is scored against its own published rubric — 28 exist in the repository — because the things that decide a cold-email engagement are not the things that decide a CRO engagement. Deliverability carries 20% of the outbound score and appears nowhere in the CRO rubric; experimentation rigour carries 25% of CRO and appears nowhere in outbound.
The consequence worth knowing: scores are comparable within a category and not across categories. A 3.4 in video production and a 3.4 in outbound are not the same claim.
What counts as evidence
Every scored dimension carries an evidence tier, and the tiers are visible on the record rather than averaged away:
- verified — 2471 dimensions (66%). Checked against a named source you can open.
- partly checkable — 12. Corroborated in part.
- vendor-stated — 219. The agency's own claim, recorded as theirs and not as ours.
- inferred — 797. Our reading of indirect evidence.
Where we could not establish something, it goes in what we could not verify on the record instead of being quietly filled in. Absence of disclosure is reported as a finding, because at these prices silence usually resolves against the buyer.
Dates, and what we do not claim
Every evaluation shows the date it was last vetted. The set currently spans 2026-07-19 to 2026-08-20. We do not re-check the whole library weekly and will not pretend to — an old date shown honestly is more useful than a fresh one that is decorative.
Three things we do not claim. These are opinions formed from public evidence, not statements of fact. We have not audited anyone's books or spoken to every client. And a verdict is about fit for a described job, not a verdict on a company — several records say explicitly that a buyer shopping for something else should ignore them.
When we are wrong
We publish corrections rather than editing quietly. Records that were re-scored against a different rubric carry a rescored_from note showing the previous verdict and score, so you can see what changed and why. If you believe an evaluation is wrong — including if you are the agency — write to mo@koolav.com. We will re-review and publish the result either way, and there is no paid path to a different answer.