kjh2064
3e6f609dda
Source governance schema: append-only source_approval table enforcing approval before ingestion. Dataset manifest hardened to support FROZEN state, requiring approval timestamps. Boundaries tested (6/6 passing). Server-side resolver (DapperApprovedModelContextReader) now guards both model and dataset approval. P2–P6 deferred: Dataset freeze command, maker-checker review, evaluation/proposal orchestration remain pending human decision package (source allow-list, license/SLA, metric versions, roles). No source/model seeded per CLAUDE.md governance. Migrations 0033–0034 idempotency verified fresh/upgrade/re-run on isolated test DB. AGENTS.md: Maturity (contract-first); Necessity (governance prerequisite). Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
90 lines
3.9 KiB
Markdown
90 lines
3.9 KiB
Markdown
# ADR-DATA-001: Governed Source Approval and Dataset Freeze Pipeline
|
|
|
|
## Status
|
|
|
|
`APPROVED` — approved by the repository owner on 2026-08-06 for the Source Approval contract slice. Implementation remains limited to append-only governance records; model activation, orders, and KIS submission remain forbidden.
|
|
|
|
## WBS / contract traceability
|
|
|
|
- WBS: `AEG-X-009`
|
|
- Requirement: `REQ-DATA-SOURCE`
|
|
- Existing contracts: `contracts/schedules/model-operations.v3.json`, `contracts/schedules/execution-assurance.v1.json`
|
|
- Related proposal: `docs/CURRENT/AEG-X-009_AUTOMATION_PROPOSAL.md`
|
|
- Policy boundary: `EVALUATION_ONLY` / `PROPOSAL_ONLY` / `DRILL_ONLY`
|
|
|
|
## Context
|
|
|
|
The live database contains the model-operations schemas, but no approved/frozen `dataset_manifest`, model registry, EvidenceSnapshot, or release bundle records. The source catalog previously claimed operational approval without preserving the required owner, license, SLA, timezone, unit, and approval evidence. This prevents a compliant Phase 1 VersionSet from being resolved.
|
|
|
|
## Decision proposal
|
|
|
|
Introduce a governed, append-only approval boundary before ingestion or evaluation:
|
|
|
|
```text
|
|
SourceCandidate
|
|
-> SourceApproval (human owner/steward + contract evidence)
|
|
-> DatasetManifest (immutable content/lineage hash)
|
|
-> DatasetFreeze (human approval or approved governance command)
|
|
-> ServerSideVersionSetResolver
|
|
-> EvaluationOnly / ProposalOnly operation
|
|
```
|
|
|
|
The resolver must reject any source or dataset that is not approved and frozen. The client cannot supply authoritative evidence, hashes, model/config/code versions, or contract versions.
|
|
|
|
## Proposed data boundary
|
|
|
|
The implementation may add normalized append-only records only after this ADR is approved. Candidate records must include:
|
|
|
|
```text
|
|
source_id, source_version, owner, steward, license_reference,
|
|
availability_sla, freshness_sla, timezone, calendar, unit, currency,
|
|
schema_contract_version, status, approved_by, approved_at,
|
|
published_at, revision, content_hash, lineage_hash
|
|
```
|
|
|
|
No update/delete is permitted for approval, evidence, or freeze history. Corrections are new records/events.
|
|
|
|
## Automation boundary
|
|
|
|
Allowed:
|
|
|
|
- source contract drift checks;
|
|
- data-quality evaluation;
|
|
- immutable manifest creation;
|
|
- deterministic dataset freeze proposal;
|
|
- EvidenceSnapshot creation;
|
|
- proposal packet and maker/checker notification.
|
|
|
|
Forbidden:
|
|
|
|
- automatic model activation/promotion;
|
|
- automatic rollback;
|
|
- threshold/config/policy/code mutation;
|
|
- client publication;
|
|
- broker order or KIS submission.
|
|
|
|
## Acceptance evidence required before implementation is complete
|
|
|
|
1. Unapproved source cannot enter ingestion.
|
|
2. Approved source with missing license/SLA/timezone/unit is quarantined.
|
|
3. Dataset freeze is append-only and content-addressed.
|
|
4. Same input and VersionSet produce the same manifest/evaluation hash.
|
|
5. Client-supplied VersionSet/evidence is ignored or rejected.
|
|
6. Replay with the same scope/idempotency/watermark produces no duplicate side effect.
|
|
7. Proposal approval is maker/checker and does not activate a model.
|
|
8. Failure, alert, runbook, retention, and rollback/stop evidence are preserved.
|
|
|
|
## Alternatives rejected
|
|
|
|
- Trusting `source-catalog.md` as approval: no immutable approval evidence.
|
|
- Creating synthetic DatasetId/ModelVersion values to unblock Shadow Run: violates evidence and reproducibility rules.
|
|
- Reusing existing model-operation tables without an approval boundary: permits ambiguous ownership and incomplete lineage.
|
|
- Adding a scheduler that activates models: forbidden by AGENTS.md v12.4.
|
|
|
|
## Approval record
|
|
|
|
- Decision: APPROVED for the first Source Approval contract slice.
|
|
- Scope: append-only source approval record and validation boundary only.
|
|
- Explicit exclusions: dataset freeze execution, model activation, automatic promotion/rollback, threshold mutation, client publication, broker order, and KIS submission.
|
|
- Follow-up: Dataset Freeze requires a separate reviewed slice and evidence package.
|