In Action · The bigger jobs
Make the numbers in a client deliverable defensible
A client-facing document is about to carry real numbers that will get quoted back to us — a prospect report, a QBR, a sales deck.
The failure that sinks a client-facing report is rarely a wrong calculation — it's a number that was correct once, in one file, and then got copied into three later documents without anyone re-checking it against where it actually came from. This is a discipline, not a tool: a running register of every claim a document makes, tied to its exact source, its cohort and its time window, checked directly against the raw data rather than against the document that's already asserting it — so a stale or copy-forward error gets caught before a client repeats the wrong number back to you.
The failure mode is not a wrong calculation — it’s a number that was correct once, in one file, and then got copied into three deliverables without anyone re-checking it against the source at each stop. A review that reads the document and nods is not the same thing as a review that recomputes the document.
- Build the register before the document is called final. One row per client-facing claim: an id, the exact final wording, the source file it comes from, the cohort/window qualifier it’s only true for, and a status. A number with no row is a number nobody has committed to defending.
- Put the cohort and window in the row, not just in someone’s head. “37% of X” means nothing until it says which patients, over which dates, measured against what. The qualifier is part of the claim — omit it from the row and the number silently drifts to mean whatever the reader assumes.
- Run the adversarial verify pass against raw sources only. Recompute every registered claim from the raw data or source file that should support it — never from the analysis, the summary, or the document that is already asserting it. Verifying against the thing you’re trying to check proves nothing; this is the step that actually catches a stale filter or a copy-forward error.
- Fix at the source, then propagate — never fix the sentence in isolation. When the verify pass finds a wrong number, correct the findings file that produced it first. Then find every deliverable carrying that claim and update each one from the corrected register row. A fix applied only to the document someone happened to be reading leaves the same wrong number live everywhere else it was copied.
- Use the register as the regression gate on every rebuild. Before a re-themed or rebuilt version of the document ships, diff its claim wordings against the register. A number that changed without a matching register update is a regression that slipped in during formatting, not a refresh of the data.
- [manual] Decide which claims are load-bearing enough to register. A scene-setting statistic pulled from a public source is a different risk from a number computed from the client’s own data and destined for a sales meeting. Drawing that line is a judgment call for whoever owns the draft, not a rule this procedure can state in advance.
- Hand the finished register to the data-QA lens of a full document review, if one is being run —
review-a-client-deliverable’s data-QA lens consumes exactly this register rather than re-deriving it.
Regression-tested is not the same as reviewed
A person reading a document end to end catches typos and awkward phrasing. It does not catch a number that was right in the source file and wrong in the sentence quoting it, because reading only checks that the sentence is plausible, never that it matches what produced it. The register turns “someone reviewed this” into “every claim was recomputed from raw data and will be checked again the next time this ships” — a materially different guarantee, and the only one worth making about a number a client will repeat back to us.
What this does not cover
Whether the underlying analysis is right. This discipline proves that a number in the document matches its source, for the cohort and window the document claims — it does not judge whether the methodology that produced the source number was sound. It also is not automated anywhere in the toolkit: every step below is a discipline applied by hand to whatever documents and source files already exist, not a tool that runs against them.