kjh2064
3e6f609dda
Source governance schema: append-only source_approval table enforcing approval before ingestion. Dataset manifest hardened to support FROZEN state, requiring approval timestamps. Boundaries tested (6/6 passing). Server-side resolver (DapperApprovedModelContextReader) now guards both model and dataset approval. P2–P6 deferred: Dataset freeze command, maker-checker review, evaluation/proposal orchestration remain pending human decision package (source allow-list, license/SLA, metric versions, roles). No source/model seeded per CLAUDE.md governance. Migrations 0033–0034 idempotency verified fresh/upgrade/re-run on isolated test DB. AGENTS.md: Maturity (contract-first); Necessity (governance prerequisite). Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
3.9 KiB
ADR-DATA-001: Governed Source Approval and Dataset Freeze Pipeline
Status
APPROVED — approved by the repository owner on 2026-08-06 for the Source Approval contract slice. Implementation remains limited to append-only governance records; model activation, orders, and KIS submission remain forbidden.
WBS / contract traceability
- WBS:
AEG-X-009 - Requirement:
REQ-DATA-SOURCE - Existing contracts:
contracts/schedules/model-operations.v3.json,contracts/schedules/execution-assurance.v1.json - Related proposal:
docs/CURRENT/AEG-X-009_AUTOMATION_PROPOSAL.md - Policy boundary:
EVALUATION_ONLY/PROPOSAL_ONLY/DRILL_ONLY
Context
The live database contains the model-operations schemas, but no approved/frozen dataset_manifest, model registry, EvidenceSnapshot, or release bundle records. The source catalog previously claimed operational approval without preserving the required owner, license, SLA, timezone, unit, and approval evidence. This prevents a compliant Phase 1 VersionSet from being resolved.
Decision proposal
Introduce a governed, append-only approval boundary before ingestion or evaluation:
SourceCandidate
-> SourceApproval (human owner/steward + contract evidence)
-> DatasetManifest (immutable content/lineage hash)
-> DatasetFreeze (human approval or approved governance command)
-> ServerSideVersionSetResolver
-> EvaluationOnly / ProposalOnly operation
The resolver must reject any source or dataset that is not approved and frozen. The client cannot supply authoritative evidence, hashes, model/config/code versions, or contract versions.
Proposed data boundary
The implementation may add normalized append-only records only after this ADR is approved. Candidate records must include:
source_id, source_version, owner, steward, license_reference,
availability_sla, freshness_sla, timezone, calendar, unit, currency,
schema_contract_version, status, approved_by, approved_at,
published_at, revision, content_hash, lineage_hash
No update/delete is permitted for approval, evidence, or freeze history. Corrections are new records/events.
Automation boundary
Allowed:
- source contract drift checks;
- data-quality evaluation;
- immutable manifest creation;
- deterministic dataset freeze proposal;
- EvidenceSnapshot creation;
- proposal packet and maker/checker notification.
Forbidden:
- automatic model activation/promotion;
- automatic rollback;
- threshold/config/policy/code mutation;
- client publication;
- broker order or KIS submission.
Acceptance evidence required before implementation is complete
- Unapproved source cannot enter ingestion.
- Approved source with missing license/SLA/timezone/unit is quarantined.
- Dataset freeze is append-only and content-addressed.
- Same input and VersionSet produce the same manifest/evaluation hash.
- Client-supplied VersionSet/evidence is ignored or rejected.
- Replay with the same scope/idempotency/watermark produces no duplicate side effect.
- Proposal approval is maker/checker and does not activate a model.
- Failure, alert, runbook, retention, and rollback/stop evidence are preserved.
Alternatives rejected
- Trusting
source-catalog.mdas approval: no immutable approval evidence. - Creating synthetic DatasetId/ModelVersion values to unblock Shadow Run: violates evidence and reproducibility rules.
- Reusing existing model-operation tables without an approval boundary: permits ambiguous ownership and incomplete lineage.
- Adding a scheduler that activates models: forbidden by AGENTS.md v12.4.
Approval record
- Decision: APPROVED for the first Source Approval contract slice.
- Scope: append-only source approval record and validation boundary only.
- Explicit exclusions: dataset freeze execution, model activation, automatic promotion/rollback, threshold mutation, client publication, broker order, and KIS submission.
- Follow-up: Dataset Freeze requires a separate reviewed slice and evidence package.