# Golden documents

The fixed real documents Sibyla is verified against. `V6.md` §S2 grows this to roughly 20–30 —
purchase, sale, credit note, receipt, foreign currency, multi-page and unreadable scan — as the
regression harness for skill and model changes. Today it holds one: the base intake document.

## Rules

- The source is the **read-only FDR prototype** (`invoice-skill-build`, prototype root
  `D:\fileStorage\repos\invoice-skill-build`). Golden documents are referenced by path and
  SHA-256. The bytes are **never copied into this repository**, modified, committed or pushed
  (`AGENTS.md`, "The FDR prototype").
- **No field values are transcribed here.** Per V6-D3/V6-D4 no document data is imported. Expected
  values are computed at test time from the source file; they are not checked in.
- **The hash is the pin, and that is deliberate.** The reference-data rule — take the current
  files, not a pinned snapshot — applies to DOCEFL and DOCTYP, which grow. A regression fixture is
  the opposite: if the hash stops matching, the entry is wrong or the file moved. Investigate
  before re-baselining.

## GD-1 — base intake document

| | |
|---|---|
| Path | `Gott/Payables/202607/Gott_Invoice_Claude_202607_02.pdf`, relative to the prototype root |
| SHA-256 | `dd30f24a80ca55652a7c5d353e169b7dc721cc11cc896d1cad21475648297348` |
| Size | 34,637 bytes |
| Source | GOTT Payables, July 2026 |

**This is the document S0-4 ran on.** Naming it here replaces the anonymous hash `STATE.md`
carried, and fixes it as the document every later one-document run uses, so two runs are
comparable. There are **two runs, and they are not the same**:

- **2026-08-10 — preliminary, unsuccessful.** The hosts started under the `Development` EF InMemory
  fallback and **no document was admitted**, so GD-1 never entered the pipeline. What it did
  establish is the extractor observation recorded below.
- **2026-08-18 — the final channel rerun, successful.** The **exact pinned bytes above** were
  submitted through `/api/channel-intake/v1` with the verified dedicated channel-intake client,
  against a pre-existing enabled sender mapping, an isolated PostgreSQL instance and an isolated
  Local scratch storage root, on installed artifact `a8d0a4bab7b2f93920ad29819d442e01c04dd374` —
  an artifact **ancestral to `main`**, so the run is **not a demonstration of the current `main`
  runtime** and `main`'s governed staged migrations were not bypassed. Runtime facts only: intake
  returned 201 on register, 200 on same-idempotency-key replay with `replayed=true`, and 200 on
  upload and status; the extraction job reported **Succeeded on attempt 1** and its response passed
  strict contract validation and was persisted as one revision with two lines; the deterministic
  gates **executed and returned false**, raising warning `LINE_SIGN_AMBIGUOUS` and routing the
  document to review; the exact original was stored immutably and **rehashed to the SHA-256 above**.
  Outbound **business** integrations were deliberately disabled — Cegid, Microsoft Graph and the
  financial push paths — so no external business-integration job, commit, operation or call was
  made. **OIDC discovery and token acquisition did occur**, as authentication to the Sibyla API for
  the intake submission; no token, client id or value is recorded. **No field values
  are recorded** — per the rules above, expected values are computed at test time.
- **The source file is unchanged.** The 2026-08-18 run read the read-only prototype and wrote
  nothing to it; the pin above still matches. The run's disposable resources and snapshots were
  removed afterwards.

**Observed** — re-measured 2026-08-10 with
`Sibyla.PdfTextExtractor --sibyla-pdf-text-extract`: `{"status":"usable","truncated":false}`,
970 bytes of text over 41 lines. One page. An embedded text layer, so no OCR path is involved.
Two charged line items, each carrying quantity, unit amount, tax rate and line amount, over a
subtotal / tax / total block. Denominated in EUR. A foreign supplier with an EU VAT registration
invoicing a Portuguese customer with VAT charged — not reverse charge.

**Why this one.** It is the only document in the corpus with measured evidence behind it: the
2026-08-10 measurement established that the extractor produces usable, untruncated output from it,
and the 2026-08-18 rerun carried it through the channel end to end, so a failure in a later run is
a real regression rather than an unknown file. It is also small, single-page and
text-layered, which keeps it a smoke test rather than a stress test.

**What it exercises:** intake, text extraction, a multi-line item table, currency and a tax rate —
enough to reach step 4's line-to-account mapping.

**What it does not:** a Portuguese domestic supplier (NIF, domestic VAT rate), FX via EXCRAT — it
is already EUR — multi-page handling, scans without a text layer, and every document type other
than a purchase invoice. Those belong to the S2 golden set, not to this entry.

## Next

S2 chooses the remaining entries from the same corpus — GOTT Payables 202601–202608 holds 395
PDFs, alongside Receivables and Bank. The first gap worth closing is a Portuguese domestic
supplier invoice, since GD-1 cannot exercise domestic VAT or NIF resolution at all.

### Required scenario coverage (S2-13, specified by S2-15 — added 2026-08-19)

The Invoice Skill Build reconciliation (`project-todo-v6.md` S2-15) sets the coverage the golden
set must reach. **These are scenario requirements, not entries**: each becomes a GD-n row above
only when a real corpus document is chosen and pinned by path and SHA-256, per the rules — no
bytes copied, no field values transcribed. One document may cover several scenarios; the harness
(S2-14) reports which scenarios have a pinned document and which are still uncovered.

Reading and legibility: a normal text-layer invoice (GD-1 partly covers this) · a textless but
visually legible scan, proving visual or hybrid fallback is used · a genuinely unreadable
document, which must fail only **after** visual fallback was attempted.

Document identity and type: a cancelled invoice or diagonal cancellation watermark / "no
accounting value" disclaimer · an invoice-receipt backed by real payment evidence, versus an
invoice carrying payment **terms** without payment evidence · a payment or collection notice that
is not an invoice · an ordinary invoice containing late-payment-interest ("juros de mora")
boilerplate, which must **not** be read as a collection notice · a reference-only document · a
byte-identical or near-duplicate resubmission, proving document identity rests on content, never
on filename similarity.

Amounts and tax: a multi-page summary-plus-detail invoice (telecom / account-statement pattern)
where summary lines differ from granular usage detail · a carried-forward account balance
distinct from the current-period amount · mixed VAT rates · a reverse-charge line · a VAT-exempt
line · Portuguese stamp duty distinct from VAT · a non-tax parafiscal surcharge · a foreign
currency document · a credit note · a document with a visible arithmetic inconsistency, which
must be flagged, never silently repaired.

Parties: supplier and buyer identified from visible fiscal identifiers, with printed identifiers,
leading zeros, signs and currencies preserved verbatim · a severe billed-to fiscal-identity
mismatch · an aggregated invoice containing underlying issuers · a payout or settlement record
that could double-count revenue.

Adversarial: a document with prompt injection embedded in its text, image, QR code or
annotations, which the agent must ignore and report. This is the one scenario the corpus may not
contain; if so it is **authored synthetically** for the purpose — a synthetic fixture is not
client data, and whether its bytes may then live in-repo (an exception to the never-copied rule,
which exists for the confidential prototype corpus, not for fabricated fixtures) is decided and
recorded when the entry is created.

Added by the S2-15 phase-1 matrix (`docs/invoice-skill-reconciliation-matrix.md`, 2026-08-20;
matrix rule ids in parentheses where a matrix row applies — the last item traces to a prototype
change-log entry rather than a rowed rule): amounts printed in locale formats — spaces as thousands
separators, comma decimals — extracted as correct JSON numbers (R-02) · a stated service/coverage
period differing from the issue month, ideally one advance-billing and one arrears variant,
exercising `periodStart`/`periodEnd` (R-12) · a consolidated invoice explicitly naming several
billed months (R-12) · a simplified-invoice/final-consumer document with **no printed buyer
identity**, extracting `ZZ`/`UNKNOWN` and proceeding flagged, never failing (R-42) · a credit
note whose **visible reference to the reversed document** must be extracted verbatim — refines
the credit-note scenario above (R-14) · a document number printed with separators or spaces,
preserved verbatim yet matchable under a digit-boundary normalization (R-04) · two records of one
document identity denominated in **different currencies**, which must not be reported as an
amount conflict (R-07) · a billed-to mismatch against a **sister or group company** rather than a
stranger — the subtler shape of the mismatch scenario above (R-10) · a document printing a
non-Portuguese, non-EU tax identifier type, exercising `identifierType` without guessing.

## GD-2 … GD-21 — the April 2026 GOTT batch

Twenty documents, chosen 2026-08-27 from the 112 the prototype holds under `Gott/*/202604`. They
are **selected by measurement, not by filename**: every file in the month was run through
`Sibyla.PdfTextExtractor --sibyla-pdf-text-extract` against the read-only prototype, and the
extremes and the rare shapes were taken. Paths are relative to the prototype root; `Text` is what
the extractor returned, and `none` means no embedded text layer, so the OCR path is the one under
test. **No field values appear here** — expected values are computed at test time.

| Id | Path | SHA-256 | Bytes | Text | Why this one |
|---|---|---|---|---|---|
| **GD-2** | `Gott/Payables/202604/Gott_Invoice_Awin_202604_02.pdf` | `2620210ea92487b23d76699bd6a0c37b39d1ad3071bc7834216b70e9d834f276` | 53173 | none | no text layer — OCR path |
| **GD-3** | `Gott/Payables/202604/Gott_Invoice_GarantiaMutua_202604_01.pdf` | `d87aa9510e91a592e9a8fe2a01768d3d0d6c2bdb1b8d7d5b67d48c881f1e4c82` | 121609 | none | no text layer — OCR path |
| **GD-4** | `Gott/Payables/202604/Gott_Invoice_VFX_202604_01.pdf` | `60a065c7f1b55780e19c7622f2d5aaeb98f85903d367cd669d8989dcf2bec50c` | 36824 | none | no text layer — OCR path |
| **GD-5** | `Gott/Payables/202604/Gott_Invoice_MEO_202604_04.pdf` | `0e1c2441f5946df0bb3d6f66f062a5b5044dc3f1a92eba27c9f6fc17b471d530` | 376271 | 795 lines | longest text layer in the month |
| **GD-6** | `Gott/Payables/202604/Gott_Invoice_AWS_202604_01.pdf` | `0bebab9bfd8f085310362277307dcb631d334107dc38677deebdf9ee37210e22` | 83846 | 45 lines | carries USD and EUR — the FX path |
| **GD-7** | `Gott/Payables/202604/Gott_Invoice_OpenAI_202604_01.pdf` | `ee2b2ebb2c80371be962151f4e907b7b479c84e4d6b32c01d51385856b23b474` | 32252 | 31 lines | USD only, no EUR |
| **GD-8** | `Gott/Payables/202604/Gott_Invoice_Farminvest_202604_01.pdf` | `03d3830a2f3ce23c671d04fc41fa81be54283a68f04975d4ad0db1f8f899dccf` | 190478 | 52 lines | original of the byte-identical pair |
| **GD-9** | `Gott/Payables/202604/Gott_Duplicate_Farminvest_202604_01.pdf` | `03d3830a2f3ce23c671d04fc41fa81be54283a68f04975d4ad0db1f8f899dccf` | 190478 | 52 lines | byte-identical to GD-8 |
| **GD-10** | `Gott/Payables/202604/Gott_Invoice_Ticket_202604_01.pdf` | `3ae2ebf495278df43e1a6e95e6bd894abc5de10b4294d0f329010805a1901ed7` | 429242 | 50 lines | original of the near-duplicate pair |
| **GD-11** | `Gott/Payables/202604/Gott_Duplicate_Ticket_202604_01.pdf` | `f7cc580b2347498c1b6dadd5063ec27cf59bd4aaa31852091b63aadf03972c26` | 412794 | 50 lines | near-duplicate — matches no original by bytes |
| **GD-12** | `Gott/Payables/202604/Gott_Invoice_Confidencial_202604_01.pdf` | `8c0215864bcfe08157c460d7542071690035564f769524f000c4258945dadbca` | 47162 | 121 lines | an internal company as the provider |
| **GD-13** | `Gott/Payables/202604/Gott_InvoiceReceipt_Continente_202604_01.pdf` | `a6e6040343d2a9c11edd348f3d4e02e20a261d20f2bd12c917fe50ab82d7d6fb` | 74456 | 52 lines | payable invoice-receipt |
| **GD-14** | `Gott/Payables/202604/Gott_InvoiceReceipt_UptimeRobot_202604_01.pdf` | `9acbe1a63579b852ea18bc17f611b2d9b3cb10a3c13667c307b0649ce6558b86` | 31117 | 19 lines | sparsest text layer in the month |
| **GD-15** | `Gott/Payables/202604/Gott_PaymentNotice_Tranquilidade_202604_01.pdf` | `e20c6b24fbfda3e633955380f7f9dbc4469db521695b51d16cdbf58f5503d190` | 268784 | 98 lines | payment notice, not an invoice |
| **GD-16** | `Gott/Receivables/202604/Gott_CancelledInvoice_WDPro_202604_01.pdf` | `b0866501586fddf70716742a0ebade8faae3cdfa8721aeab2414d8ecbd8d1933` | 278389 | 74 lines | cancelled sale |
| **GD-17** | `Gott/Receivables/202604/Gott_CancelledInvoice_Confidencial_202604_01.pdf` | `ea47813bd414f14a853aa828c6946ab744f34f69062ca889f8540c09717ebd07` | 263780 | 80 lines | cancelled sale, internal counterparty |
| **GD-18** | `Gott/Receivables/202604/Gott_Invoice_Itoorer_202604_01.pdf` | `c5c990f8ed2b261e5563d8d1d18c3c0ad02f56d184abac5fcc8898427e92dba9` | 249032 | 90 lines | sale to an internal company |
| **GD-19** | `Gott/Receivables/202604/Gott_InvoiceReceipt_eSIMPrePaidFrance_202604_01.pdf` | `c3e586e1d9769d43e72806808d3feadb42e65af12f1b1c54f6ab2f6f79292c78` | 264140 | 70 lines | receivable invoice-receipt |
| **GD-20** | `Gott/Bank/202604/BCP-DO-GOT-EUR_202604.pdf` | `bdb90a8ca13c1801d21da40ea0789bb68c65a96d2cecc63eaf58a8a8cbb4df51` | 146627 | 88 lines | bank statement, EUR |
| **GD-21** | `Gott/Bank/202604/BPI-DO-CRF-USD_202604.pdf` | `b5b3721fb93c2256cf490dbbe886a9e3ba544119c4ea32dfa9ae113fae3722b5` | 60567 | 49 lines | bank statement, USD account |

### What the batch covers, and what it does not

**Three documents have no text layer at all** — GD-2, GD-3, GD-4, the only three in the month. They
are the whole of the *unreadable scan* scenario `V6.md` §S2 asks for, so all three are in.

**Two duplicate pairs, and they are not the same problem.** Six of the month's seven
`Duplicate_` files are byte-identical to an original; the seventh is not. GD-8/GD-9 are the
byte-identical case, which the intake auto-discard resolves on the hash alone. GD-10/GD-11 are the
harder one: **the same invoice with different bytes**, which no hash can catch and which
`EF0000002` must resolve on the `(ProviderCodeName, InvoiceNo)` pair. A golden set holding only the
first kind would report the duplicate detector as working while the case that needs it went
untested.

**Two documents put an internal company on the wrong side.** GD-12 is a payable whose provider is
one of our own companies — the class of error the Tranquilidade case exposed, and what `EF0000008`
and the `counterparty_not_internal_company` gate exist for. GD-18 is the mirror: a sale to an
internal company.

**Foreign currency is split deliberately.** GD-6 carries USD and EUR together, so the EXCRAT path
has to choose; GD-7 carries USD alone. GD-21 is a bank statement on a USD account.

**Not covered: credit notes.** April 2026 GOTT holds none — the nearest is a cancelled sale
(GD-16, GD-17), which is a different condition (`EF0000009`, conflicting status) and not a
substitute. The scenario stays open, and closing it needs a month that has one.

### Images are an input class, and this batch has none

Intake accepts `application/pdf`, `image/png` and `image/jpeg`, validated by magic bytes
(`DocumentUploadService`, `ChannelIntakeService`), so an image is a first-class document and not an
edge case. The extraction client already separates the two paths: a non-PDF is handed to the agent
with no text-evidence clause and is read with vision, while a PDF **whose text layer is unusable**
has its pages materialized as PNG/JPEG visual evidence and is read the same way
(`HermesDocumentExtractionClient.BuildPrompt`).

So GD-2, GD-3 and GD-4 do exercise the vision path — by the visual-fallback branch rather than the
native-image branch. What is **not** covered is an image arriving as an image: the prototype holds
764 files under `Gott` and every one is a PDF, so the source cannot supply that fixture. Closing it
needs a real PNG or JPEG document, and until one exists the native-image branch is reached by no
golden entry.

## The first golden run — 2026-08-27

The twenty-one entries above, run through the **real extraction agent** on a dedicated profile: the
production agent installation (it is the thing under test) with everything on Sibyla's side
isolated — a scratch working root of the run's own, refused if it resolves inside a deployed
application directory, and the locally built text extractor rather than the deployed one, so the
run is of current `main`. Job artefacts are removed by the client as it goes, so no document
persists outside the read-only prototype.

`21/21` passed. **What that number means, and what it does not:** it is contract conformance plus
reading-mode honesty — the response validated strictly, and no document with an absent text layer
came back claiming a text read. It is **not** an accuracy measure. Accuracy would need expected
field values, and V6-D3/V6-D4 keep those out of this repository, so the figure is a floor rather
than the quality metric `S2-14` eventually wants. Saying otherwise would be reading the number as
something it cannot be.

| Status | Before the fix | After |
|---|---|---|
| `ok` | 15 | **17** |
| `needs_attention` | 4 | 4 — GD-1, GD-4, GD-16, GD-17 |
| `error` | 2 — GD-3, GD-4 | **0** |

GD-3 went from zero lines and unreadable to `ok` with two, flagging `PT_FISCAL_QR_UNREADABLE`
honestly at that resolution; GD-4 went to `needs_attention` with one line. Neither is now claimed
to be an unreadable document.

**The two errors were a defect in Sibyla, not two bad scans — and this is the correction of what
this section first claimed.** GD-3 and GD-4 came back `TEXT_LAYER_ABSENT_VISION_USED` +
`UNREADABLE_AFTER_VISION` with zero lines, and that was first written up here as the fail-closed
path working, with the two documents called genuinely unprocessable. That was a conclusion about
the *documents* drawn from a statement by the *reader*, and it was wrong: both are perfectly
legible to a person, which is how it was caught.

The cause is one filter chain. A scanned page is commonly stored as `[/FlateDecode, /DCTDecode]` —
a JPEG wrapped in Flate — and PdfPig decodes that through neither `TryGetPng` nor
`TryGetBytesAsMemory`, so `ExtractVisualEvidence` produced **no page image at all**. With no visual
evidence, `BuildPrompt` falls back to handing the agent the raw PDF, which it cannot rasterize, and
the honest answer it can give is that it could not read the document. GD-2, the third file with no
text layer, is single-filter `/FlateDecode`, which is why it worked and the other two did not.

The extractor now undoes one bounded zlib layer when the raw stream is not already a JPEG, and
validates the result exactly as strictly as before — a JPEG whose dimensions equal the declared
sample dimensions — with the inflate capped by the remaining byte budget so a compression bomb is
refused rather than materialized. Both documents now yield a page image, and the class of document
this affects is every Flate-wrapped scan, not these two.

**The condition detectors fired where they should.** GD-15 `PAYMENT_NOTICE_NOT_INVOICE`; GD-16 and
GD-17 both `DOCUMENT_CANCELLED`; GD-18 `AMBIGUOUS_BANK_ACCOUNT`. GD-12 — an internal company as the
provider — returned `ok` with no warning, correctly: that is a Sibyla-side resolution gate
(`counterparty_not_internal_company`), not something the extractor is asked to judge.

**Both duplicate pairs extracted identically within the pair** — GD-8/GD-9 one line each, GD-10/GD-11
two each. Determinism across a pair is the precondition for the duplicate detector comparing them
at all, so it is worth having measured before relying on it.

**One drift worth watching.** GD-1 returned `LINE_SIGN_AMBIGUOUS`, matching its 2026-08-18 channel
run, but also `INSUFFICIENT_DATA`, which that run did not record. Same pinned bytes, same two lines,
an extra warning. That is exactly the signal a golden set exists to raise; whether it is model or
instruction drift is not established here.

### The same bytes do not produce the same warnings

The set was run twice, about an hour apart, against the same pinned bytes — once before the
visual-evidence fix and once after. The two runs disagree in ways the fix does not explain:

| | First run | Second run |
|---|---|---|
| GD-1 | `INSUFFICIENT_DATA` + `LINE_SIGN_AMBIGUOUS` | `LINE_TOTAL_SIGN_AMBIGUOUS` |
| GD-16 | `DOCUMENT_CANCELLED` + `NON_ACCOUNTING_VALUE_STATED` | `DOCUMENT_MARKED_CANCELLED` + `DOCUMENT_MARKED_NO_ACCOUNTING_VALUE` + `DUPLICATE_COPY_INCLUDED` |
| GD-17 | `DOCUMENT_CANCELLED` + `MULTIPLE_BANK_ACCOUNTS` + `NON_ACCOUNTING_DOCUMENT_NOTICE` | + `DOCUMENT_NO_ACCOUNTING_VALUE`, + `DUPLICATE_COPY_PRESENT` |
| GD-13 | no warnings | `PT_FISCAL_QR_UNREADABLE` |
| GD-21 | no warnings, 4 lines | `MULTIPLE_CURRENCIES_VISIBLE`, 6 lines |
| GD-15 | 7 lines | 6 lines |
| GD-18 | `needs_attention` | `ok` |

Two things are visible here and they are not the same thing.

**The warning codes are not a closed vocabulary.** `DOCUMENT_CANCELLED`,
`DOCUMENT_MARKED_CANCELLED`; `NON_ACCOUNTING_VALUE_STATED`, `DOCUMENT_MARKED_NO_ACCOUNTING_VALUE`,
`DOCUMENT_NO_ACCOUNTING_VALUE` — the same condition, named differently per run, and every one of
them passed strict contract validation. Anything downstream that branches on a warning code is
therefore branching on a string the agent composes, not on a governed value.

**The line count moved on two documents.** GD-15 and GD-21 returned a different number of lines
from identical bytes. That is not a labelling difference; it is the extraction differing in
substance between runs.

Both are recorded as observations, not as diagnoses: two runs establish that the outputs differ,
not why, and not how wide the spread is. What follows from them — whether the warning vocabulary
should be governed the way DOCEFL codes are, and what a golden pass rate can be defined over if
line counts move — is a decision, not a fix to apply here.

## The spread, measured over four runs — 2026-08-27

The set was run four times against the same pinned bytes. Two runs suggested the variance was
confined to documents whose line structure the contract never defines; **four runs show that was
wrong.** Eleven of twenty-one differ between runs, and the variance is of three distinct kinds.

### One condition, nine names

GD-16 and GD-17 carry a cancelled sale with no accounting value. Across four runs it was called:

`DOCUMENT_CANCELLED` · `DOCUMENT_MARKED_CANCELLED` · `DOCUMENT_CANCELLED_OR_NO_ACCOUNTING_VALUE` ·
`DOCUMENT_NO_ACCOUNTING_VALUE` · `DOCUMENT_MARKED_NO_ACCOUNTING_VALUE` · `NO_ACCOUNTING_VALUE_STATED` ·
`NON_ACCOUNTING_VALUE_STATED` · `NON_ACCOUNTING_VALUE_INDICATED` · `NON_ACCOUNTING_VALUE_DECLARED`

`NO_` and `NON_` differ by one letter. Duplicate copy took four names, ambiguous bank account three,
line arithmetic four. **Twenty-six distinct codes were emitted for what is roughly eight conditions**,
and every one passed strict validation, because the validator checks that a code is uppercase snake
case and nothing else.

The two codes that are actually *in* the contract were stable: `TEXT_LAYER_ABSENT_VISION_USED`
twelve times and `INSUFFICIENT_DATA` seven, one name each. Where the vocabulary exists it is
followed; where it does not, it is invented — the same shape as the line rule, which governs order
and numbering but never says what a line is.

### Conditions that appear and disappear

Worse than the naming, because a name can be mapped and an absence cannot:

| | Observed |
|---|---|
| GD-18 | bank ambiguity flagged in three runs, **nothing** in the fourth |
| GD-21 | `MULTIPLE_CURRENCIES_VISIBLE` in one run, nothing in three |
| GD-19 | `DOCUMENT_CONTAINS_ORIGINAL_AND_DUPLICATE_COPIES` in one run, nothing in three |
| GD-3 | status `ok`, `needs_attention`, `ok`, `ok` |

A document flagged in one run and clean in the next routes differently depending on when it was
processed.

### Line counts move on ordinary invoices

Not only on the bank statement and the payment notice, which is what two runs suggested:

| | Lines across four runs |
|---|---|
| GD-5 — an invoice with a clean text layer | **3, 6, 6, 6** |
| GD-10, GD-11 — the near-duplicate pair | 2,2,2,1 and 2,1,2,1 |
| GD-15 — payment notice | 6, 7, 8, 7 |
| GD-21 — bank statement | 6, 4, 3, 6 |

GD-10 and GD-11 matter most: that pair exists so the duplicate detector can compare the same
invoice captured twice. If the extraction yields two lines one run and one the next, the comparison
is over noise.

### What this means for the catalogue

A closed vocabulary fixes the naming and nothing else. It cannot make a condition stop vanishing,
and it cannot steady a line count. The agent's own instructions already require what is missing:

> Given the same readable input and instruction version, preserve the same output structure, field
> interpretation, and warning codes.

So this is not an unstated expectation being disappointed — it is a stated requirement not being
met, and it belongs with whoever owns the agent rather than in a Sibyla-side workaround. What
Sibyla can do meanwhile is refuse an unlisted code instead of accepting any snake-case string, and
pin the golden set on invariants that survive a rerun rather than on counts that do not.

## S2-14 stability metric harness

`GoldenDocumentExtractionRunTests` is an explicitly invoked `GoldenSetRun`; the default suite does
not call the real agent. `SIBYLA_GOLDEN_RUNS` defaults to **3** because the remaining operational
measurement is an N=3 run whose responses declare `sibyla-documental/2.4`. Values from 1 through 10 are accepted
for diagnostics, but N=1 cannot establish stability and therefore cannot pass the combined metric.
`SIBYLA_GOLDEN_ENFORCE_CATALOGUE` defaults to **true** in this harness only; the validator's
production configuration defaults are unchanged.

For each GD entry, every run must pass the strict extraction contract and reading-mode honesty, all
run metadata must be complete and identical, and every structural fingerprint must be identical.
One differing run fails stability even when all other runs agree. There is no averaging, majority
vote, consensus result or modal extraction. The single S2-14 rate is:

`combined pass / total`, where `combined pass = stable across all N AND structurally conformant in every run`.

The TSV contains three structural-only sections: metric/value rows (separate stable, structurally
conformant, combined and total counts, plus configured N and catalogue state); one row per invocation
(`id`, run number, outcome, the value-free 2.4 structural fingerprint, contract/instruction/model
and helper identities, duration, conformance and governed failure type); and one aggregate row per
GD id (comparable, stable, structurally conformant and combined). Every dynamic TSV field escapes
tab, carriage return and line feed. Assertion failures contain governed GD ids only, not report
paths, exception text, document content or runtime configuration.

The fingerprint contains only status, document type, reading mode, the distinct ordinal-sorted
warning-code set, header/buyer/supplier null-presence shape, and the ordered line number plus
null-presence shape of all nine nullable line fields (`TransactionDate`, `Reference`, `Description`,
`Quantity`, `UnitPrice`, `VatRate`, `VatAmount`, `LineTotal`, `Category`). It never contains extracted
values, confidence, free text, document names or identifiers, amounts, dates, references,
descriptions, paths, filenames or raw response/usage JSON.

Model identity is read fail-closed from `model`, `model_id` or `modelId` in each invocation's usage
JSON. Only a bounded 1–128 character ASCII model slug made from letters, digits, `.`, `_` and `-`,
with an alphanumeric first and last character and no adjacent punctuation, is accepted. The harness
stores and reports only its
`sha256:<hex>` identity; missing, ambiguous, malformed, path-shaped, prose-shaped or differing
identities make the runs non-comparable. The raw model value and usage JSON are never reported.

Each attempt opens the governed document with write and delete sharing excluded, computes byte
length and SHA-256 while consuming that exact stream, compares both with the registry entry, rewinds
that same stream, and gives it to extraction. It never reopens by path for extraction. An unavailable
input is `InputIdentityUnavailable`; a length/hash mismatch is `InputIdentityChanged`. Neither report
shape contains a path or document content, and either makes the combined result false.

The PDF helper executed by each attempt is an attempt-specific snapshot, not the mutable configured
source path. The snapshot contains the apphost executable, managed entry DLL, dependency and runtime
manifests, and every top-level local managed DLL required by the helper's current `PdfPig` project
output. Its identity uses explicit role and byte-length framing. Source identity is checked before
and after copy against the copied identity; the copied files are opened with write/delete sharing
excluded for the attempt where the platform enforces Windows sharing, the real
`HermesDocumentExtractionClient` is configured with the snapshot executable path, and the snapshot
is remeasured in `finally` before its checked temporary directory is removed. An unavailable,
mixed-generation or changed snapshot fails closed; different helper generations across attempts
make the document non-comparable. Product version is not the pin.

Cancellation also reaches that `finally`: the partial attempt is reported as `Cancelled`, helper
identity is remeasured, no later attempt is invoked, and the incomplete N-run set is non-comparable.
Catalogue enforcement set to false remains useful diagnostic metadata but is likewise
non-comparable and cannot pass the combined metric.

This section describes the implemented harness and its synthetic/default-suite verification only.
No post-change real golden-document run or live agent call has been performed, so it records no
measured combined rate and does not claim that O13's underlying agent nondeterminism is resolved.

## How much of the corpus the scanned-page fix reached — 2026-08-27

The fix was found through two April documents; this is its actual reach. Every PDF in the prototype
— **867** — was classified by whether it carries a usable text layer and, where it does not, by what
the visual fallback can make of its pages. Read-only, nothing written to the source.

| | Count |
|---|---|
| Usable text layer — the visual fallback is never reached | **848** |
| Depends on the visual fallback | **19** |

Of those nineteen:

| Class | Count | Before the fix | Now |
|---|---|---|---|
| Flate-wrapped JPEG | **9** | no page image at all | works |
| PNG | 5 | works | works |
| Plain JPEG | 2 | works | works |
| Page sliced into image strips | 2 | fails | **still fails** |
| No image and no text | 1 | fails | **still fails** |

So of the twelve documents the visual path could not read, **nine now can** — eight of them real GOTT
payables spanning `202603` to `202607`, not an April curiosity. `GarantiaMutua` alone appears five
times across five months in three different classes, which is worth knowing before anyone concludes
a supplier's scans are uniform.

### The three that still fail, and why they are two different problems

**Two are scans sliced into horizontal strips.** `Gott_Invoice_GarantiaMutua_202601_01.pdf` holds
6 image bands and `Gott_Invoice_PitLane_202601_01.pdf` holds 71, each a full-width sliver about
9 points tall, stacked to form one page. This is not a logo beside a scan, so the
`images.Length != 1` guard is not merely conservative here — picking one strip would hand over a
sliver of a document. Reading them properly means composing the strips into a page, which needs an
imaging library the extractor does not have, and emitting all seventy-one separately would exhaust
the page and pixel budgets to no purpose. A real piece of work, not a loosened guard.

**One has no images and no text.** `Gott_InvoiceReceipt_GoDaddy_202605_01.pdf` is two pages with
neither, so its content is drawn as vector paths. That is exactly the limitation the code already
states — PdfPig cannot rasterize arbitrary vector pages — and closing it means a renderer, not a
decoder.

Both remain fail-closed, which is the right behaviour for them: nothing is fabricated and the
document reaches review. What changed today is that nine documents stopped being reported as
unreadable when they were not.