The dress rehearsal proved the pipeline survives thirty lineages of authored records. It said nothing about thirty lineages of ingested ones, because it starts downstream of retrieval. This page is that missing seam: payloads shaped like the documented API envelopes go through the real connector specs, the real pagination, and the real raw store — and then through the step nothing else here performs, turning a retrieved payload into a registered decision with reconciled identifiers.
Run at build time from @gii/intake · digest ae792dfcf7235420db7516fa… — the same haul always registers the same state.
The haul#
Six sources across three pagination styles and four credential classes. Every page is checksummed and persisted before it is parsed, so nothing enters the registry that is not already held as evidence.
| Source | Walk | Pages | Items | Registered | Reconciled | Quarantined |
|---|---|---|---|---|---|---|
| congress-gov Library of Congress | offset · registration-key | 2 | 6 | 6 | 0 | 0 |
| federal-register Office of the Federal Register (NARA) / GPO | page · none | 4 | 13 | 13 | 0 | 0 |
| regulations-gov U.S. General Services Administration | page · api-key | 4 | 15 | 4 | 9 | 2 |
| courtlistener Free Law Project (nonprofit) | cursor · token | 2 | 4 | 4 | 0 | 0 |
| govinfo U.S. Government Publishing Office | cursor · registration-key | 2 | 8 | 0 | 8 | 0 |
| usaspending U.S. Department of the Treasury (Bureau of the Fiscal Service) | page · none | 2 | 5 | 5 | 0 | 0 |
Decisions by origin
Each registered decision has exactly one decision-origin code, so this ring partitions the registry. Intergovernmental implementation records are award-level, not decisions in their own right.
- Executive agency928%
- Congress619%
- Intergovernmental implementation516%
- President413%
- Independent regulator413%
- Other (2)413%
32 decisions in total · The registries
A decision retrieved from two sources reconciles to one entity. The second sighting adds identifiers; it does not add a duplicate.
Where two sources meet#
17 cross-source joins fired, on 3 identifier systems. A Regulations.gov document quoting its Federal Register document number is not a similar record — it is the same one, and the crosswalk is what makes that conclusion checkable rather than assumed.
Cross-source joins by identifier system
Bars, not a ring: one decision can join on more than one system, so these count joins rather than partitioning them.
17 joins in total · How the crosswalk works
What the haul found#
Three findings the fixtures could not have been written to produce, because they are properties of the sources rather than of the fixtures.
| Finding | What it means for the pilot |
|---|---|
| Identity reconciles; lineage does not arrive at all. Not one of the six envelopes carries a structured reference to a different decision. | Every lineage edge will come from analyst work or full-text parsing, not from the APIs. Budget for it. |
| Awards cannot be linked to the decisions that created them. The USAspending award envelope carries no authorizing-statute reference. | Linking spending to the decision that caused it needs the appropriation-account layer, not the award search API. |
| 3 lineages cannot be retrieved at all. The Federal Reserve has no documented-API source in the inventory. | Monetary-policy lineages enter by another route or not at all. The haul names them rather than quietly running on 27. |
Controls that fired#
The fixtures include deliberately bad items, because a control that has never fired is a claim rather than a control. 2 items were quarantined with a stated reason and 1 identity conflict was held for review — never merged, never dropped.
| Source | Reason |
|---|---|
| regulations-gov | document SYN-HQ-2018-04-9998 carries no title |
| regulations-gov | identifier conflict: resolves to 2 entities (GII-DEC-00bf28d8e646, GII-DEC-a1eff7cbf281) — held for review, never merged |
Intake invariants#
What must hold of a haul, whatever it retrieved. Each reports what was observed, whether or not it held.
| Invariant | Observed | Held |
|---|---|---|
| raw-persisted-before-parse | 49 normalized records over 16 raw payloads. | held |
| every-decision-has-provenance | 32 decisions, each carrying 1.9 identifiers on average. | held |
| issued-identifiers-are-declared | 9 systems across 62 preserved identifiers. | held |
| cross-source-identity-reconciles | 17 second sightings across 3 identifier systems produced 0 duplicates. | held |
| relations-never-reconcile | no relation reference appeared in any envelope; 6 context identifiers were recorded on their records and kept out of the crosswalk. | held |
| conflicts-surface-for-review | 1 conflict(s) surfaced and quarantined. | held |
| quarantine-not-silence | 2 quarantined of 51 items. | held |
| content-addressing-dedupes | 16 distinct payloads from 16 retrieved pages. | held |
| no-credential-leak | 4 injected credentials, 0 found in 21,817 surfaced characters. | held |
| intake-produces-no-scores | 0 score records, 0 outcome definitions, 0 analyses — intake writes none of them. | held |
| coverage-gaps-are-named | 27/30 corpus lineages arrived as evidence; 3 named as unreachable (federal-reserve). | held |
| re-running-the-haul-is-safe | replayed 6 sources: 0 new payloads, 0 new entities. | held |
| every-extractor-exercised | congress-gov 6 · federal-register 13 · regulations-gov 13 · courtlistener 4 · govinfo 8 · usaspending 5 | held |
Run it#
npm run gii -- intake # the summary above, in a terminal npm run gii -- intake --json # the full report GET /api/v1/intake npm run gii -- export intake-report.json