Formula Verification Coverage

What this page reports

Every calculator on this site is backed by a test file that runs its actual shipped calculation engine against expected results. Those files run as one gate: if one fails, or if a published calculator has no mapped test file at all, the gate fails and this report is not written. This page publishes what that gate found, per calculator, so the claim can be checked rather than believed.

Coverage — 630/630 calculators have a suite — is the claim this site is willing to defend. The sitewide assertion total is reported further down for completeness, but it is a poor summary statistic and it is not the headline here: a handful of generated suites dominate it, so quoting it alone would flatter the thin pages and mislead you about the well-covered ones.

Built and maintained by Jay Sudha. See Methodology for how a calculator gets built in the first place.

How to read a row

Each case is classified by what kind of evidence it actually supplies, weakest to strongest. Where a case could plausibly sit in two categories, it is recorded as the weaker one — understating the evidence is safe, overstating it is not.

• Independent cross-check — The expected value does not come from our own engine. It is either a figure published elsewhere — an official table, a worked example, a statute, a standard textbook formula — or the same quantity recomputed by an independent route, written into the test file next to the case.
• Golden — The expected value was worked out by hand, with the arithmetic written into the test file so a reader can re-derive it. A number produced by calling our own engine does not count as golden.
• Boundary — Zero, negative, empty, very large, or degenerate inputs, and the rounding or threshold points where a formula changes behaviour.
• Property — An identity, reconciliation, or monotonic relationship that must hold for any admissible input — for example that a schedule closes to a zero balance, or that a bigger deposit never lowers the result.
• Structural — Output shape and type sanity: the field exists, the number is finite, no NaN, the array is the right length. Real checks, but the weakest evidence here.

A high total made mostly of structural and property cases is weaker evidence than a small total made of golden and independent ones, which is exactly why the table breaks the count out by kind instead of showing a single number.

The distribution, stated plainly

Across 630 calculators and 305 test suites there are 77,192 distinct passing assertions. Counted per calculator — a suite that legitimately covers several closely-related calculators counts toward each of them — that comes to 234,505. Neither figure is evenly distributed:

• Median: 76 cases per calculator. 295 of 630 calculators sit below it. The range runs from 14 (UAE Gratuity Calculator) to 6,730 (Average Cost Basis Calculator).
• The top 5 calculators hold 13.1% of the per-calculator total (30,747 of 234,505). Any average computed over this data is dominated by them.
• 2 suites are property-based and account for 6.4% of all distinct assertions (4,941 of 77,192): 401(k) engine, Retirement engine. These generate thousands of checks from loops over input ranges. That is genuinely useful — it explores far more of the input space than hand-written cases can — but one generated assertion is not equivalent in weight to one hand-derived expected value, and lumping them into a single sitewide number pretends otherwise.
• Case mix sitewide: 6,868 independent cross-check, 16,991 golden, 9,125 boundary, 187,130 property, 14,391 structural.
• 527 of 630 calculators carry at least one case whose expected value does not come from our own engine: it is either published elsewhere or recomputed by an independent route.

Every category, with its coverage

One row per category. Each links to that category's page, which lists every calculator in it, largest suite first, with its passing cases and case mix; math is split by area the same way the math library is. Every calculator's row carries a permanent anchor taken from its slug, so the evidence for one calculator can be linked and cited on its own — see Citing this page. A calculator counts as independently cross-checked when at least one of its cases takes its expected value from outside the engine — a figure published elsewhere, or the same quantity recomputed by another route.

The test file behind every row lives in the project's scripts/ directory under the filename given in the machine-readable report; the repository itself is private, so the filename is a stable identifier you can cite or ask about through contact rather than a public source link.

The largest suites

The 10 biggest test files, so you can see directly where the sitewide total comes from. A suite listed as generated produces its cases from loops over input ranges rather than from individually written scenarios.

What this does not prove

• It is not an independent audit. The tests were written by the same person who wrote the calculators, and no outside auditor, accountant or licensed specialist has reviewed them.
• Outside references carry more weight than internal arithmetic. A case checked against a published authority is stronger evidence than one checked against the calculator’s own arithmetic, which is why those are counted separately above.
• A passing suite only proves the formula matches the cases that were chosen. It cannot prove the right formula was chosen for your situation, and it says nothing about inputs nobody thought to test.
• It is not a guarantee of a result. A lender, tax authority, or supplier applies your exact rate, fees, rounding, and local rules. Every calculator here is a planning estimate under the assumptions printed on its own page. See the Disclaimer.
• Counts are not comparable across calculators. A two-input formula can be exhaustively covered in a couple of dozen cases; an amortisation engine cannot. Read a row against what that calculator actually has to get right, not against the row above it.
• If you find a result that is wrong, the corrections process is how it gets reproduced, fixed at the formula, and re-verified.

Machine-readable version

The same data, generated by the same run, is published as JSON at /formula-verification.json so it can be cited, diffed between releases, and checked without scraping this page. It carries the generation date, the per-calculator counts and case mix, and the per-suite breakdown. It is free to cite and republish with attribution.

Both this page and that file are rendered from one generated source file. There is deliberately no second derivation path, because a trust page whose numbers drift from its own source data is worse than no page.

What a passing case is worth

The kinds of case are not equal, and a large total of weak checks is worth less than a small number of strong ones. How to read this report sets out what each kind proves, what none of them prove, and how to re-derive any figure here yourself.

Citing this page

The report and any single row can both be cited. Each row's anchor is its calculator slug on its category's page — the Average Cost Basis Calculator row is at https://calculatormatters.com/formula-verification/tax/#average-cost-basis-calculator— so a citation can point at the evidence for one calculator rather than at the whole report, and opening a row anchor there adds that row to the citation box in both formats. Older links to /formula-verification/#<slug> still arrive at the row.

The date in both formats is the date the run above was produced, read from the generated data — not the date this page was last deployed, and not a date typed in here. It moves only when the suites are re-run. The same plain-text citation is carried in the JSON file, so a machine reading the data cites it the same way a person does.

How the numbers get here

A single gate discovers every test file, bundles each calculator's shipped engine from source, runs the file, and reads a structured result the file writes as it goes: the calculator slug it covers, every case name, that case's kind, and whether it passed. Nothing on this page is typed in by hand, and the aggregation never re-counts or re-interprets a suite's output.

A file that runs cleanly but produces no structured result is treated as a failure and blocks the report outright, rather than quietly contributing zero. Two further cross-checks run on every regeneration: every calculator in the catalog, released or parked, must have a suite, and each suite's engine must match the separate, hand-maintained verification manifest that maps each calculator to its engine. Either failure stops the report from being written at all.

This page and the JSON file count released calculators only. Parked calculators, which are built but not on the site, keep their suites and run with every regeneration, but they are not counted here. The date at the top of this page is the date the run above was produced, not the date this page was last deployed.