The Recount
The independent standard for whether AI worked.
No hype. No doom. Just the recount. For the people who own the outcome of AI — and will one day be asked to prove it.
The company that builds the AI cannot be the one that certifies it worked. Not because anyone is lying — because “we tested it ourselves and it’s great” is not evidence, it is a press release. Every mature industry eventually separates the builder from the grader. AI is the last high-stakes system still grading its own homework.
The Recount is where that separation happens. We independently, reproducibly verify whether a deployed AI system delivered the outcome it was reported to deliver — and publish the gap between the two. Not a vendor’s dashboard. Not a governance checklist. The independent re-tally.
What makes a number a standard
No figure is published unless it clears all four gates. This is the moat — a standard dies the first time it is debunked.
Every figure traces to a primary source or a knowable answer — exact version and date recorded. No secondhand numbers.
A third party can re-run the method and land on the same result. The method is published, not asserted.
An independent pass tries to break every figure before it ships. If it does not survive scrutiny, it is not published.
We measure the field; we do not attack a vendor. Neutrality is what lets everyone — even those we measure — cite the number.
The Verification Gap
The number we track: what a system was reported to do, versus what it independently verified as doing. The distance between them is where risk lives — and where, until now, no one was standing.
The Independent AI Resolution Benchmark
Across the customer-operations world, vendors publish their own “AI resolution rate.” The industry itself concedes that a resolution rate reported without a fixed definition of “resolved” is closer to a marketing figure than a benchmark.
The independent AI Resolution Benchmark fixes that: a published, locked definition of “resolved,” measured independently and reproducibly from operational data. The verified one, in a field of self-graded ones. Additional instruments — claims-decision accuracy, regulatory-claim accuracy, copilot factual accuracy — follow the same discipline.
There is no single “AI hallucination rate.” The number depends on who measures, how, and when.
If you cannot pin down the accuracy of the models everyone benchmarks in the open, what confidence do you have in a vendor’s single self-reported figure? Consider the public, well-regarded work of one measurement provider, Vectara — used here not as a target but as an ally in rigor.
- Vectara maintains a public hallucination leaderboard, scored by its HHEM detection model — and the yardstick itself is versioned (HHEM 2.1 → 2.3).
- The same provider also maintains FaithJudge, a stricter judge method that surfaces materially higher hallucination rates than the summarization leaderboard — two credible methods, different numbers.
- The rankings move: as of the leaderboard’s May 11, 2026 update (HHEM-2.3), the lowest-hallucination model was measured at 3.3% — a snapshot, not a fixed truth.
The implication:
If even the referee’s own number is a moving target — two methods, a versioned yardstick, a shifting board — then a deployed system’s single self-reported accuracy figure is close to meaningless without a fixed, named, dated method behind it. That method is what a standard provides. That is the entire reason The Recount exists.
Cite: doi.org/10.5281/zenodo.21424795 · Download: PDF · source
Sources verified against primary publications on 2026-07-18. Vectara HHEM leaderboard (HHEM-2.3, last updated 2026-05-11) and FaithJudge (arXiv:2505.04847). Figures are point-in-time snapshots of a living leaderboard and will change; that is precisely the finding. References do not imply endorsement or affiliation.
Case files
Independent Verification Reports — individual recounts, each traced to primary sources, each a fixed and citable record.
A note on how this is built
The Recount is a standing body of work, published on a cadence and versioned over time. Each study and each instrument carries a locked, published method so that anyone can reproduce the result. We do not invent the underlying measurement concepts; we compute and verify them independently, and we hold our own numbers to the same four gates we would hold anyone’s. The attribution methods behind our operating signals are the subject of U.S. provisional patent filings (patent-pending).
We don’t guess. We verify.