A verification programme can be wrong in a way its own results never show. Every check here was written by the author of the code it checks, against data the same author invented — so a green suite is partly evidence about the software and partly evidence that one person is internally consistent. This page is the attempt to tell those apart.
Seeded faults #
The catalog contains 42 deliberate defects. Each one is an exact source edit that breaks a rule the whitepaper publishes: drop the spillover term from GII-M, let an unmeasured fact pass a materiality gate, take the first candidate when an identifier resolves to two entities. The audit applies one at a time to a scratch copy of the repository and runs every check.
A fault that dies is evidence the rule is defended. A fault that survives is the finding — some rule GII-US publishes is enforced by nothing, and the audit names which rule and which file. Two rules govern what may be added: every fault is a plausible implementation mistake rather than a random character swap, and every fault must be silently wrong rather than loudly broken. A mutant that throws on the first call tests nothing; real defects return a plausible number.
What the faults attack
A control fault leaves every number correct and disables the thing whose job is to object. Both of the audit's survivors were that kind.
- Disable a control3071%
- Change a number1229%
42 faults in total
true for any data at all — and nothing failed, because every assertion in the repository checked that it returned true and none checked that it could return false. Separately, the production acceptance threshold could be lowered from 0.80 to 0.60 with no test noticing, because every consumer read the constant rather than the published number. A gate is only a gate if something proves it can say no, and a published value is only published if something outside the code records it. Both are now fixed, and both faults stay in the catalog permanently.The audit is not part of the ordinary build — it re-runs whole suites once per fault. It is executed with npm run audit:faults. What does run on every change is the catalog's grip on the source: a fault whose anchor text has moved fails the suite, because an audit reporting kills it never attempted would be worse than no audit at all.
The faults aimed at the checks #
A second catalog of 20 faults attacks the verification gate itself rather than the methodology — the audit scripts that guard the repository's own conventions. For each one, a drift fault creates the exact thing it exists to catch, and an inertness fault makes one of its parsers match nothing, so a check that would pass on absence is caught passing on absence. Every check was once proven this way by hand; the proof lived in a commit message, and nothing re-proved it after a refactor. This catalog is executed by npm run check:audits on every pull request, and a check with no fault against it fails the suite.
Metamorphic relations #
Every other harness needs to know what correct output looks like. GII-US has no oracle: no real decision has been scored, so nobody can say "the answer is 61.4" and be checked. Metamorphic testing is the technique for exactly that situation — transform the input in a way whose effect on the output is known in advance, and check the two outputs against each other rather than against a truth nobody has.
Renaming an indicator must change nothing. Tripling every exposure must triple the pressure. Weakening evidence must never publish a headline that was withheld. Each holds with no idea what the right score is, and each catches a class of defect that single-input property testing structurally cannot: a bug that shifts every score by the same amount satisfies every bound and every range check. These are the properties that will still mean something on the first real pilot lineage.
The full set of relations is not published. Each states, precisely, a property the index must satisfy — which makes the list a specification of the method written from the outside. The verdict is the part worth publishing, and it is here.
12 of 12 held when this page was built.
The defect record #
The review packet asks reviewers to weigh the verification programme by its catches rather than its green checkmarks, and lists seven. That list was prose — a claim about the repository's history that a reader had to take on trust, and that nothing prevented from quietly becoming false. Each entry below now carries a check that reconstructs the original failing condition, so it fails for the original reason and not a coincidence.
Each entry reconstructs the condition that was originally wrong and asserts it stays fixed — and two of them additionally assert the fix did not overshoot. The individual defects are not published: a list of the ways this implementation has been wrong is a list of the rules it is required to get right.
7 of 7 historical defects remain locked as of this build.
What this does not establish #
All three parts verify the implementation against the specification. None of them verifies the specification against the world. A seeded fault proves that breaking a published rule is noticed; it says nothing about whether the rule is the right rule. The metamorphic relations would hold just as well for a scorecard whose weights are badly chosen. That gap is what the Phase-1 expert review and the historical pilot are for, and no amount of harness closes it.
The honest summary is narrower than a kill rate: for 42 specific published rules, something in this repository would object if the implementation stopped obeying them — and for two rules, nothing would have, until the audit was run.