kjh2064/KArtSell.Aegis
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(governance): add source-approval + dataset-freeze contract schema (AEG-X-009, gated)

Browse Source 
Source governance schema: append-only source_approval table enforcing approval
before ingestion. Dataset manifest hardened to support FROZEN state, requiring
approval timestamps. Boundaries tested (6/6 passing). Server-side resolver
(DapperApprovedModelContextReader) now guards both model and dataset approval.

P2–P6 deferred: Dataset freeze command, maker-checker review, evaluation/proposal
orchestration remain pending human decision package (source allow-list, license/SLA,
metric versions, roles). No source/model seeded per CLAUDE.md governance.

Migrations 0033–0034 idempotency verified fresh/upgrade/re-run on isolated test DB.

AGENTS.md: Maturity (contract-first); Necessity (governance prerequisite).

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
This commit is contained in:
김재현
2026-08-07 13:31:34 +09:00
parent 67274cbdb6
commit 3e6f609dda
11 changed files with 651 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/CURRENT/AEG-X-009_AUTOMATION_PROPOSAL.md
+176
View File
@@ -0,0 +1,176 @@
# AEG-X-009 Data/Model Proposal Automation — Design Proposal
## Status and traceability
- WBS: `AEG-X-009`
- Requirement: `REQ-DATA-SOURCE`
- Evidence class: `SOURCE+DESIGN_PROPOSAL`
- Status: `DESIGN_PROPOSAL`; not approved implementation
- Source: `contracts/schedules/model-operations.v3.json`, `contracts/schedules/execution-assurance.v1.json`, `contracts/model-governance/evaluation-promotion.v2.json`, `src/KArtSell.BuildingBlocks/Versioning/VersionSet.cs`, live read-only schema inspection on 2026-08-06
- Assumption: source ingestion and model evaluation are allowed to create immutable proposal/evidence records when their mode is `EVALUATION_ONLY` or `PROPOSAL_ONLY`.
- Unknown: approved source owners, source licenses/SLA values, model training implementation, retention period, and operator/secondary assignments.
- Decision Required: approve the proposal schema, job ownership, source allow-list, promotion review roles, and retention/alert contracts before implementation.
## Non-negotiable boundary
Automation may:
1. discover and validate an approved source;
2. ingest immutable raw records and create a content-addressed dataset manifest;
3. run deterministic evaluation against a frozen server-side VersionSet;
4. create EvidenceSnapshot and a human-review proposal;
5. notify the maker/checker queue and expose status/metrics.
Automation must never:
- activate or promote a model;
- mutate thresholds, policy, configuration, or source code;
- rollback a model automatically;
- publish to clients;
- submit an order or KIS request.
## Required state flow
```text
SOURCE_CANDIDATE
-> SOURCE_APPROVED (human owner + license/SLA/timezone/unit)
-> INGESTION_EVALUATION_ONLY
-> DATASET_QUARANTINED | DATASET_FROZEN
-> MODEL_EVALUATION_ONLY
-> EVIDENCE_SNAPSHOT_CREATED
-> PROPOSAL_ONLY_REVIEW
-> HUMAN_APPROVED | HUMAN_REJECTED | EXPIRED
-> HUMAN_CHANGE_APPLIED (separate release, never by scheduler)
```
`DATASET_QUARANTINED`, missing evidence, hash mismatch, PIT violation, or VersionSet drift is a terminal hold for that run. It is not a retryable transient failure.
## Required immutable records
### Source catalog entry
```text
source_id
source_version
owner / steward / secondary
license_reference
availability_sla / freshness_sla
timezone / calendar
unit / currency
schema_contract_version
approved_at / approved_by
status: CANDIDATE | APPROVED | SUSPENDED | RETIRED
```
### Dataset manifest
Use the existing `evaluation.dataset_manifest` table. A row is eligible for evaluation only when:
```text
status = FROZEN
dataset_id and content_hash are non-blank
source_catalog_version is approved
lineage_hash is present
frozen_at and approved_at are present
published_at/revision/PIT rules pass
```
### Evaluation VersionSet
Use the existing `VersionSet` contract. It must be loaded server-side and contain:
```text
DatasetId, DataHash, ModelVersion, ConfigVersion, CodeSha, ContractVersion
```
The client may submit scope and requested window only. The client must not submit evidence, hashes, model versions, or configuration versions as authoritative values.
### Proposal packet
The proposal must reference, without copying or mutating, the EvidenceSnapshot and VersionSet. It must contain:
```text
proposal_id / idempotency_key / scope_key / job_run_id
version_set / evidence_id / dataset_id / input_hash / output_hash
policy_id / policy_trace_schema_version / decision_contract_version
evaluation windows and metric definition versions
PBO / DSR / frozen OOS / double-cost / false-exit-reentry evidence
maker / checker / expiry / disposition
```
## Existing schedule mapping
Do not add a new schedule until ADR/Issue approval. Use the existing contract entries as follows:
| Existing job | Mode | Automated responsibility | Forbidden result |
|---|---|---|---|
| J25 SourceContractDriftCheck | EVALUATION_ONLY | detect source contract/license/SLA drift | no source activation |
| J26 MarketCalendarCompletenessCheck | EVALUATION_ONLY | detect calendar/timezone/unit gaps | no threshold mutation |
| J27 EvidenceChainAudit | EVALUATION_ONLY | validate lineage/hash/PIT chain | no evidence repair by overwrite |
| J28 ProjectionFreshnessCheck | EVALUATION_ONLY | validate read-model freshness | no client publication |
| J30 ReleaseEvidenceAssemble | PROPOSAL_ONLY | assemble a review packet | no release or activation |
The missing business flow is not a new automatic promotion job. It is the contract and application boundary that creates a frozen dataset and proposal packet for the existing review process.
## Repository catalog mapping
The following mapping is grounded in the current catalog and data contracts. It is a design mapping, not an authorization to ingest.
| Domain | Current catalog/source | Current logical tables/contracts | Automation entry condition | Current status |
|---|---|---|---|---|
| Market data | KRX OpenAPI | `market_data.prices`, `VS-03_DATA_CONTRACT.md` | source approval + calendar/unit/SLA + PIT/hash checks | CANDIDATE |
| Corporate/fundamental data | OpenDart API | `model_operations.disclosures`, `VS-05_DATA_CONTRACT.md` | license/redistribution approval + filing schema/DQ | CANDIDATE |
| Portfolio | User input | `portfolio.holdings`, `VS-04_DATA_CONTRACT.md` | authenticated owner input + audit + PIT | CANDIDATE |
| Model operations | computed/evaluation output | `evaluation.dataset_manifest`, `governance.model_version_registry`, `signal_engine.evidence_snapshot` | frozen dataset and approved model/config/code contract | BLOCKED until seed/approval |
| Shadow evaluation | Hangfire/shadow run | `model_operations.shadow_run`, result/evidence contracts | server-side VersionSet + EVALUATION_ONLY capability | BLOCKED until VersionSet |
The source catalog's logical table descriptions must be reconciled with active runtime SQL and the live schema before a migration or ingestion implementation. The catalog itself is not a substitute for runtime schema evidence.
## Existing debt and decision linkage
This proposal directly addresses, but does not close, the following open items:
- `TD-044`: approved Dataset Manifest and Model Registry initial data absent;
- `TD-063`: total-return/delisting/corporate-action golden data incomplete;
- `TD-099` / `TD-105`: market calendar/timezone source and SLA not approved;
- `TD-132`: current total-return source not approved;
- `DEC-037`, `DEC-038`, `DEC-079`: source/license/SLA and calendar ownership decisions required.
These items remain OPEN/DECISION_REQUIRED until their evidence is attached. No automation job may treat the catalog row as approved merely because the row exists.
## Proposed WBS decomposition (proposal only)
These rows must be approved before being added to `WBS_MASTER.csv`:
| Proposed ID | Scope | Acceptance evidence |
|---|---|---|
| AEG-X-009-P1 | Source allow-list and approval record | unapproved source cannot enter ingestion |
| AEG-X-009-P2 | Dataset manifest freeze command | same input produces same dataset/content hash; append-only |
| AEG-X-009-P3 | Server-side VersionSet resolver | client-supplied evidence/version values ignored |
| AEG-X-009-P4 | Evaluation/Proposal orchestration | idempotent JobRun/Watermark; modes fail closed |
| AEG-X-009-P5 | Human review packet/API/UI | maker-checker, expiry, reject, audit trail |
| AEG-X-009-P6 | Replay/failure/observability evidence | quarantine, replay hash, alert, runbook, rollback/stop evidence |
## Gate progression
| Gate | Required before next gate |
|---|---|
| G0 | contract, source owner, data semantics, WBS approval |
| G1 | approved source catalog + isolated fresh/upgrade/re-run rehearsal |
| G2 | frozen dataset + VersionSet resolver + golden/replay evidence |
| G3 | evaluation-only execution and EvidenceSnapshot proof |
| G4 | proposal packet + maker/checker review evidence |
| G5 | separate human change approval; no scheduler activation |
## Immediate decision package
Before code or migration work, approve these six values explicitly:
1. source allow-list and owner/steward;
2. license, SLA, timezone, calendar, unit, and currency contracts;
3. dataset freeze status and retention policy;
4. model evaluation metric definition versions and population/window rules;
5. maker/checker roles and proposal expiry;
6. alert, stop, runbook, and secondary owner.
Until these are approved, the correct behavior is `BLOCKED`/`QUARANTINED`, not synthetic data/model creation.