# P1-0 draft — Discard and Purge lifecycle Status: accepted P1-0 field-level draft. Prototype behavior/governance reference: `invoice-skill-build` detached at `9359c67c4ef0101218d7e0ffff1986114ba5cc7a`. No prototype data is reproduced here. ## 1. States and invariants The registry disposition is a separate closed database enum: `Posted`, `ReferenceOnly`, `Discarded`, `Purged`. It does not replace or overload the current `Document.Status` (`DocumentStatus`), which remains the operational processing state used by the existing state machine; job/intake processing states are not added to the disposition enum. The disposition is nullable while operational review is in progress. `NULL` is not a fifth disposition: it means no final business disposition has yet been chosen, and every query that requires a final disposition excludes it. - `Posted`: accepted and represented in a fiscal table; bytes and audit are retained. - `ReferenceOnly`: legitimate searchable evidence, archived with its full extracted payload, but deliberately has no EntryCode and is structurally barred from FDCHDR, PAYCTR/RCVCTR, ENTBLC, matchers, and aging (D5). - `Discarded`: reviewer-confirmed junk. It is excluded from operational views, exports, business-copy archiving, matching, and financial processing. Original bytes, extraction evidence, hash, comments, and audit remain. - `Purged`: bytes are permanently absent. A never-delete tombstone retains identity, safe metadata, SHA-256, disposition reason, actors, and timestamps. DOCLOG is never deleted. Purge changes the byte-retention state; it does not erase the capture event or its audit history. Purge is capture-scoped storage hygiene. It is not RGPD erasure and must never be named, documented, logged, or presented as erasure, deletion of personal data, or a right-to-erasure remedy. Content-level erasure—deleting every copy of a hash across every capture in a company—is a named future capability, outside P1-1a scope and not implemented. When it is designed, the unchanged §3 eligibility rules govern its conflict with fiscal retention, and retention wins. For every count and acceptance check, an orphan DOCLOG row has exactly one definition: an `EntryCode` that resolves to no document in either FDCHDR or the bank-generated document set, measured at the stated pin. Any other count is a different derived metric and must name its population. ## 2. Flow 1. A schema-valid `(unsupported, NotADocument)` response with `NOT_A_DOCUMENT` is persisted through the normal valid-response path: ExtractionRevision, extraction-derived DOCLOG/LGCode, DOCFAI, and a Decision-class review item. It is not discarded automatically. A malformed or structurally invalid Claude response is different: it creates no ExtractionRevision or DOCLOG and routes durably through ExtractionAttempt plus DOCFAI/review with a NULL LGCode. 2. A reviewer may discard a record that has no final disposition, with a mandatory reason. One transaction sets `Discarded`, closes/supersedes actionable review items, records actor/time/correlation, and makes every downstream query exclude the record by construction. Bytes remain in immutable original storage; no business copy is created. A `Posted` or `ReferenceOnly` record cannot be discarded by one reviewer: the first authenticated actor submits a `DispositionOverrideRequest`, and a different actor holding the `DispositionCounterSign` authority must approve it after seeing the fiscal/archive consequences. Self-counter-signature and role accumulation in one command are rejected. The approved override adds a compensating disposition event; it never deletes or rewrites the fiscal, archive, export, integration, or audit records that made the prior disposition true. 3. Purge is a separate explicit command available only to the configured purge role. It requires a reason, a fresh authorization check, and an eligibility check under row lock. It is never a queue side effect, maintenance action, retention timer, or AI decision. 4. The byte store deletes this capture's own byte objects only after a durable purge request/audit record exists. Byte storage is per capture: each byte object belongs to exactly one capture event and is never shared, even between byte-identical captures. Completion records the storage deletion receipts and tombstone. A retry is idempotent: already-absent bytes are accepted only when their expected SHA-256 and asset identity match the request. 5. Failed or ambiguous deletion leaves state `Discarded`, records the failure, and is retryable. It never reports `Purged` merely because a delete call was attempted. The legal entry transitions are explicit: - a fresh capture starts with disposition `NULL` and `DocumentStatus.Registered`; - a malformed-response capture remains disposition `NULL` and moves to `DocumentStatus.RetryScheduled` while retryable, or `DocumentStatus.NeedsAttention` when retry is not scheduled or has been exhausted; it has no extraction-derived DOCLOG row; - a schema-valid `NOT_A_DOCUMENT` remains disposition `NULL` and moves to `DocumentStatus.AwaitingReview`; - approved fiscal projection enters `Posted`; approved non-posting evidence enters `ReferenceOnly`; reviewer-confirmed junk enters `Discarded`; - `Discarded -> Purged` is the only irreversible disposition transition. These are existing `DocumentStatus` values, not new generic operational statuses. `RestoreForReview` has one target: disposition `NULL` plus `DocumentStatus.AwaitingReview`. It is allowed only from `Discarded`, while bytes exist, through an audited reviewer action. It does not restore the former `Posted` or `ReferenceOnly` disposition automatically; the review must make a new authenticated disposition decision. The disposition-event history retains both the prior state and the restoration. `Purged` is irreversible. Posted and ReferenceOnly remain permanent purge-denial facts even if a counter-signed override later changes the current disposition. ## 3. Purge eligibility This section is unchanged by B1 and is the second, independent barrier: safety rests first on the per-capture storage invariant, not on this predicate remembering to cover a duplicate relationship. Purge is refused if the capture, any revision, or any duplicate relationship has ever: - received an EntryCode; - created or referenced FDCHDR/FDCDTL, PAYCTR/PAYDTL, RCVCTR/RCVDTL, BNKMOV, BNKREC, BNKMAT, DOCARC, or an archive/business-copy record; - reached `Posted` or `ReferenceOnly`; - been used as supporting evidence for an integration, reconciliation, approved decision, or another retained record. Eligibility is historical (“ever reached”), not based only on current rows. It is enforced by a database predicate over append-only audit/link records and rechecked immediately before byte deletion. An administrator cannot override it. ## 4. Tombstone and retained hash The tombstone retains only what is necessary for audit and duplicate control: - stable capture/document ID and LGCode when issued; - company/tenant ID, source channel class, safe original filename, MIME type, byte length; - SHA-256 (lowercase 64-hex), capture timestamp, discard/purge timestamps; - discard reason, purge reason, reviewer and purge actor IDs; - prior state, correlation IDs, and storage asset IDs/deletion receipts; - proof that fiscal/archive eligibility checks returned false. It does not retain document text, line values, bank identifiers, fiscal numbers, thumbnails, page images, extraction payloads, or free-form content beyond the controlled reasons. The hash is not secret but remains company-scoped to prevent cross-company disclosure through duplicate probing. ## 5. Byte-identical resubmission After intake hashing and company resolution, a retained discarded/purged hash match registers a new capture event linked to the canonical hash record and automatically sets that event to `Discarded`. Suppression occurs before extraction, so it creates no ExtractionRevision and no DOCLOG row. The new bytes are retained under the normal soft-discard rule unless an authorized user later purges them. The capture event receives an `ItemClass=Annotation`, `Status=Recorded`, Information, Non-Blocking finding referring to the prior tombstone/capture. This is permanent provenance, not a condition or human decision, so it is never `Status` or `Decision` and never Open. It never creates an EntryCode, archive copy, or financial row. Hash matching is byte-exact SHA-256 plus byte length; similar filenames or semantic similarity never qualify. ## 6. Proposed PostgreSQL support Use `DocumentDisposition` on the document aggregate plus append-only `DocumentDispositionEvents` and `PurgeTombstones`. Repeated byte-identical captures are distinct events and must all be registrable. Therefore neither `Documents` nor dispositions may have a unique company/hash index. Use a canonical `RetainedContentHash` row keyed uniquely by `(CompanyId, Sha256, ByteLength)` and a many-to-one append-only `DocumentCaptureHash` ledger keyed by `DocumentId` with an FK to that retained hash. `RetainedContentHash` is a duplicate-detection index and is never a storage pointer: it has no storage-pointer column, and no storage path or asset identity is derived from the hash. Storage asset identity is per capture; no two `Document` rows may resolve to the same storage asset ID. Reference counting is not implemented because byte objects are never shared. Lookup uses the canonical hash row only for duplicate detection; every capture event has its own Document identity, ledger link, and byte objects, but only extraction-derived captures have a DOCLOG identity. The current global unique index on `Documents.OriginalContentHashSha256` must be replaced in the eventual reviewed P1-1 migration; the hash-index swap is its own reviewed P1-1b migration and is never folded into P1-1a's initial train. This draft does not perform that migration. `PurgeTombstones.DocumentId` is both PK/FK to the never-deleted document row. All business queries use explicit eligible-state scopes; there is no fragile convention that callers remember to filter. The FDR-side equivalent can keep DOCLOG as the never-delete capture record and add the same disposition/event/tombstone concepts beside it. Residual or superseded DOCLOG rows are therefore evidence to classify, not candidates for blanket deletion. This is the D2-safe mechanism to share with Luís. ## 7. Acceptance cases - A schema-valid `NOT_A_DOCUMENT` reaches ExtractionRevision, DOCLOG, DOCFAI, and `DocumentStatus.AwaitingReview`; malformed output reaches ExtractionAttempt and DOCFAI without an ExtractionRevision or DOCLOG, using `DocumentStatus.RetryScheduled` while retryable and `DocumentStatus.NeedsAttention` when retry is not scheduled or has been exhausted. - Discard removes an item from every operational/export/archive selection while bytes and audit remain readable to authorized reviewers. - Purge is denied after any fiscal/archive/reference-only history, including after links are later superseded. - Two byte-identical captures exist in one company, one `Discarded` and one `Posted`. Purge of the discarded capture is denied by §3. With the eligibility check disabled in the test harness, the posted capture's bytes remain intact because the captures never shared byte objects. - Authorized purge of a never-posted discard removes all bytes and leaves one auditable tombstone. - Retry after an ambiguous delete produces exactly one tombstone and no state lie. - Exact company-scoped resubmission auto-discards at Information level; a one-byte difference does not. - Two or more byte-identical capture events coexist and link to one canonical retained-hash row; concurrent registration creates one hash row and all capture links without rejecting an event, and suppressed resubmissions create no DOCLOG row. - DOCLOG counts never fall as a consequence of discard or purge. - A single reviewer cannot discard a Posted or ReferenceOnly record; a distinct authorized counter-signer can approve the compensating override, and no prior business/audit record is removed. - Fresh, malformed-response, and valid `NOT_A_DOCUMENT` captures enter the exact `DocumentStatus` values above; `RestoreForReview` always targets disposition `NULL` plus `DocumentStatus.AwaitingReview`.