docs(phase1): add comprehensive readiness validation checklist

PHASE-1_READINESS_VALIDATION_CHECKLIST.md provides structured pre-execution validation across 6 sections: A. Governance & Approvals (DEC-037/038/079, VersionSet) - Validates law/compliance, calendar SLA, business sign-off B. Infrastructure & Environment (PostgreSQL, Host, Frontend) - Database connectivity, migration 0032, Host startup, Hangfire C. Tools & Scripts Validation (freeze, generate, runbook) - Script syntax, dry-run test, error handling, execution procedure D. Data Quality & State Validation (Model/Dataset, PIT queries) - Model card, dataset manifest, market data completeness, audit trail E. Monitoring & Observability (Logging, metrics, alerts) - Structured logging, Grafana dashboard, on-call setup (recommended) F. Final Readiness Sign-offs - Go/No-Go decision matrix with stakeholder approvals - Launch window, emergency contacts, expected completion timeline Features: - 40+ detailed check items across governance + technical + operations - Sign-off blanks for traceability - Error handling matrix for common blockers - Reference links to supporting docs AGENTS.md: Necessity (real validation gap), Maturity (checklist before execution), Traceability (approval audit trail), Right-Way (documented procedure vs ad-hoc). Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
2026-08-07 14:24:37 +09:00
parent 0395ad8ddc
commit 627e7397b4
1 changed files with 557 additions and 0 deletions
@@ -0,0 +1,557 @@
+# Phase 1 Readiness Validation Checklist
+
+**Date:** 2026-08-07  
+**Purpose:** Pre-execution validation of all prerequisites before Phase 1 shadow run activation  
+**Audience:** SRE, Platform Lead, Business Owner  
+**Status:** TEMPLATE (ready to execute)
+
+---
+
+## 🎯 Overview
+
+**Phase 1 Shadow Run:** 252+ trading days autonomous market simulation with auditable evidence  
+**Setup Time:** ~2 hours (pre-checks + tool validation)  
+**Execution Time:** 50-90 calendar days (automatic, no manual intervention)  
+**Success Criteria:** All checks PASS before proceeding to activation
+
+---
+
+## 📋 SECTION A: Governance & Approvals
+
+### A.1 — DEC-037: Source/License/SLA Approved
+
+**Owner:** Law + Data Governance  
+**Deadline:** 2026-08-10  
+**Blocking:** YES (blocks P2-P6 automation)
+
+- [ ] **Source Approved:** KRX/OpenDart/Consensus data sources confirmed
+  - Evidence: `docs/CURRENT/AEG-X-009_DECISION_PACKAGE.md` signed-off
+  - Confirm: Which sources are approved for ingestion?
+
+- [ ] **License Verified:** All sources have compliant license terms
+  - Evidence: License agreement file path: ___________
+  - Confirm: No GPL/AGPL (incompatible with commercial products)?
+
+- [ ] **Retention SLA Confirmed:** Data retention period defined (1yr/3yr/perpetual)
+  - Evidence: SLA document: ___________
+  - Confirm: Complies with GDPR/PCI-DSS?
+
+- [ ] **Update Freshness SLA Confirmed:** Daily/weekly/monthly refresh rate
+  - Evidence: SLA document: ___________
+  - Confirm: Shadow run can consume data at this frequency?
+
+**Sign-off:** ___________  (Law Lead)  /  ___________ (DataGov Lead)
+
+---
+
+### A.2 — DEC-038: Market Calendar Source & Operator Assigned
+
+**Owner:** Data Governance + Ops Lead  
+**Deadline:** 2026-08-12  
+**Blocking:** YES (blocks market simulation accuracy)
+
+- [ ] **Calendar Source Approved:** KRX official holidays/trading calendar
+  - Evidence: Data source URI: ___________
+  - Confirm: 3rd-party aggregator or direct KRX API?
+
+- [ ] **Owner Assigned:** Named operator responsible for calendar data
+  - Owner Name: ___________
+  - Email: ___________
+  - Confirm: On-call rotation configured?
+
+- [ ] **Secondary Assigned:** Backup operator for calendar updates
+  - Secondary Name: ___________
+  - Email: ___________
+  - Confirm: Escalation path defined?
+
+- [ ] **Timezone Standardized:** Asia/Seoul or UTC chosen globally
+  - Timezone: ___________
+  - Evidence: Config location: ___________
+  - Confirm: All shadow run calculations use same timezone?
+
+**Sign-off:** ___________ (DataGov Lead)  /  ___________ (Ops Lead)
+
+---
+
+### A.3 — DEC-079: Holiday Correction SLA & Policy
+
+**Owner:** Data Architecture + Ops + Legal  
+**Deadline:** 2026-08-12  
+**Blocking:** YES (blocks ad-hoc holiday handling)
+
+- [ ] **Timezone Standard Confirmed:** Asia/Seoul official timezone
+  - Standard: ___________
+  - Evidence: appsettings.json: ___________
+
+- [ ] **Holiday Corrections Procedure Defined:** Request → Approve → Reflect
+  - Request mechanism: ___________
+  - Approver(s): ___________
+  - SLA (e.g., T+0, T+1, EOM): ___________
+  - Evidence: Runbook path: ___________
+
+- [ ] **Correction Authority Assigned:** Who can request/approve corrections?
+  - Request Authority: ___________
+  - Approval Authority: ___________
+  - Emergency escalation: ___________
+
+**Sign-off:** ___________ (Ops Lead)  /  ___________ (Compliance)
+
+---
+
+### A.4 — VersionSet Approved by Business
+
+**Owner:** Business Owner / Portfolio Manager  
+**Deadline:** TBD (Phase 1 start signal)  
+**Blocking:** YES (gates entire Phase 1)
+
+- [ ] **Model ID Confirmed:** UUID of model to shadow-run
+  - Model ID: ___________
+  - Model Name: ___________
+  - Model Version: ___________
+  - Evidence: governance.model_version_registry query result
+
+- [ ] **Dataset ID Confirmed:** UUID of dataset for backtest period
+  - Dataset ID: ___________
+  - Dataset Name: ___________
+  - Coverage: ___________ to ___________
+  - Evidence: evaluation.dataset_manifest query result
+
+- [ ] **Approval Signed:** Model approved for production shadow run
+  - Approved By (email): ___________
+  - Approval Date: ___________
+  - Confidence Level (High/Medium/Low): ___________
+  - Evidence: Approval document path: ___________
+
+- [ ] **Risk Sign-off:** Risk team has signed off on model usage
+  - Risk Lead: ___________
+  - Approval Date: ___________
+  - Known Risks Documented: YES / NO
+  - Risk Mitigation Plan: ___________
+
+**Sign-off:** ___________ (Business Owner)  /  ___________ (Risk Lead)
+
+---
+
+## 🏗️ SECTION B: Infrastructure & Environment
+
+### B.1 — PostgreSQL Database (Remote)
+
+**Owner:** DBA / Database Team  
+**Blocking:** YES (core persistence)
+
+- [ ] **Remote Host Accessible:** 178.104.200.7 responding to SSH
+  ```bash
+  ssh -v kjh2064@178.104.200.7 "exit"
+  ```
+  - Result: ✅ / ❌
+  - Latency (ms): ___________
+
+- [ ] **SSH Port Forwarding Works:** localhost:5432 → remote PostgreSQL
+  ```bash
+  ssh -L 5432:127.0.0.1:5432 kjh2064@178.104.200.7 &
+  psql -h localhost -U kartsell -d kartsell -c "SELECT NOW()"
+  ```
+  - Result: ✅ / ❌
+  - Connection Time (ms): ___________
+
+- [ ] **Database Connectivity:** kartsell DB accessible with test query
+  - Query: `SELECT COUNT(*) FROM governance.model_version_registry`
+  - Result: ✅ (row count: _______) / ❌
+  - Last Backup: ___________
+
+- [ ] **Migration 0032 Deployed:** Queued status contract present
+  - Query: `SELECT schema_version FROM schema_version_history WHERE script_name LIKE '0032_%'`
+  - Result: ✅ (version: _______) / ❌
+  - Evidence: DbMigrator log timestamp: ___________
+
+- [ ] **Tables Pre-checked:**
+  ```sql
+  SELECT COUNT(*) FROM governance.model_version_registry;
+  SELECT COUNT(*) FROM evaluation.dataset_manifest;
+  SELECT COUNT(*) FROM model_operations.shadow_runs;
+  ```
+  - model_version_registry rows: _______
+  - dataset_manifest rows: _______
+  - shadow_runs rows: _______
+
+**Sign-off:** ___________ (DBA)
+
+---
+
+### B.2 — Host Application (.NET)
+
+**Owner:** Backend Lead / Platform SRE  
+**Blocking:** YES (API endpoint required)
+
+- [ ] **Build Successful:** dotnet build -c Release produces artifact
+  ```bash
+  dotnet build KArtSell.sln -c Release
+  ```
+  - Result: ✅ (warnings: _______) / ❌
+  - Build Time: _______s
+  - Build Date: ___________
+
+- [ ] **Host Startup (DEVELOPMENT mode):** App listens on http://127.0.0.1:5002
+  ```bash
+  dotnet run --project src/KArtSell.Host -c Debug --no-build
+  ```
+  - Result: ✅ / ❌
+  - Startup Time: _______s
+  - Expected Log: "Now listening on: http://127.0.0.1:5002"
+
+- [ ] **DevelopmentHeaderAuthenticationHandler Active:**
+  - Log output contains: "DevelopmentHeaderAuthenticationHandler" ✅ / ❌
+  - Confirm: Debug mode enables X-KArtSell-User header acceptance
+  - NOT Release mode (which uses FailClosedAuthenticationHandler) ✅ / ❌
+
+- [ ] **Hangfire Scheduler Initialized:**
+  - Log output contains: "Hangfire: JobStorage initialized" ✅ / ❌
+  - Dashboard available: http://127.0.0.1:5002/admin/dashboard ✅ / ❌
+  - Job queues visible: q-customer-sla, q-research ✅ / ❌
+
+- [ ] **API Health Check:**
+  ```bash
+  curl -H "X-KArtSell-User: admin" -H "X-KArtSell-Role: Admin" \
+       http://127.0.0.1:5002/health
+  ```
+  - Result: HTTP 200 ✅ / ❌
+
+- [ ] **Shadow Run Endpoint Accessible:**
+  ```bash
+  curl -X POST \
+       -H "X-KArtSell-User: admin" \
+       -H "X-KArtSell-Role: Admin" \
+       -H "Content-Type: application/json" \
+       -d '{"modelId":"","datasetId":"","windowStart":"2024-01-02","windowEnd":"2024-09-10","phaseFilter":"All"}' \
+       http://127.0.0.1:5002/api/shadow-runs
+  ```
+  - Result: HTTP 202 Accepted ✅ / HTTP 422 Validation Error ❌ / HTTP 5xx Server Error ❌
+  - Response Job ID: ___________
+
+**Sign-off:** ___________ (Backend Lead)
+
+---
+
+### B.3 — Frontend Build & Distribution
+
+**Owner:** Frontend Lead  
+**Blocking:** NO (Phase 1 is backend-only, but validates deployment)
+
+- [ ] **Frontend Build Successful:** pnpm build produces dist/
+  ```bash
+  cd frontend && pnpm build
+  ```
+  - Result: ✅ / ❌
+  - Build Time: _______s
+  - Bundle Size (gzip): _______kb
+
+- [ ] **Static Assets Copied to Host:** dist → src/KArtSell.Host/wwwroot/
+  - Confirm: `ls -lh src/KArtSell.Host/wwwroot/index.html`
+  - Result: ✅ / ❌
+  - File Size: _______kb
+  - Modification Time: ___________
+
+- [ ] **UI Contract Markers Present:**
+  ```bash
+  grep -r "app-version" dist/ && grep -r "UI contract 4.0" dist/
+  ```
+  - Result: ✅ (found) / ❌ (missing)
+
+**Sign-off:** ___________ (Frontend Lead)
+
+---
+
+## 🔧 SECTION C: Tools & Scripts Validation
+
+### C.1 — freeze-versionset.ps1 Validation
+
+**Owner:** SRE  
+**Blocking:** YES (mandatory for VersionSet freeze)
+
+- [ ] **Script Syntax Valid:** PowerShell parse-check succeeds
+  ```powershell
+  pwsh -NoProfile -Command ". scripts/freeze-versionset.ps1 -Help" -ErrorAction Stop
+  ```
+  - Result: ✅ / ❌
+
+- [ ] **Parameters Documented:** Help shows all 5 required params
+  ```powershell
+  Get-Help scripts/freeze-versionset.ps1 -Full
+  ```
+  - Params found: ModelId ✅, DatasetId ✅, ApprovedBy ✅, ConfigVersion ✅, CodeSha ✅
+
+- [ ] **Dry-run Test:** Script validates input without DB modification
+  ```powershell
+  scripts/freeze-versionset.ps1 `
+    -ModelId "00000000-0000-0000-0000-000000000001" `
+    -DatasetId "00000000-0000-0000-0000-000000000002" `
+    -ApprovedBy "test@example.com" `
+    -ConfigVersion "v1.0.0" `
+    -CodeSha "aaaaaaaaaa"
+  ```
+  - Pre-flight Check: ✅ Passed / ❌ Failed
+  - Migration 0032: ✅ Found / ❌ Not deployed
+  - Database Insert: ✅ Success / ❌ Failed
+  - Correlation ID: ___________
+
+- [ ] **Error Handling:** Script fails safely if parameter missing
+  ```powershell
+  scripts/freeze-versionset.ps1 -ModelId "..." -DatasetId "..."
+  # Missing: -ApprovedBy, -ConfigVersion, -CodeSha
+  ```
+  - Result: ✅ (fails immediately) / ❌ (proceeds incorrectly)
+
+**Sign-off:** ___________ (SRE)
+
+---
+
+### C.2 — generate-shadow-run-identifiers.ps1 Validation
+
+**Owner:** SRE  
+**Blocking:** NO (utility; can be run anytime)
+
+- [ ] **Script Syntax Valid:**
+  ```powershell
+  pwsh -NoProfile -Command ". scripts/generate-shadow-run-identifiers.ps1 -Help" -ErrorAction Stop
+  ```
+  - Result: ✅ / ❌
+
+- [ ] **UUID Generation Works:**
+  ```powershell
+  scripts/generate-shadow-run-identifiers.ps1 -OutputPath ./test-versionset.json
+  ```
+  - Result: ✅ / ❌
+  - JSON Valid: ✅ / ❌
+  - IDs Generated: RunId ✅, JobId ✅, CorrelationId ✅
+  - File Size: _______bytes
+
+- [ ] **Output Format Correct:**
+  ```bash
+  jq '.phase1_run | keys' test-versionset.json
+  ```
+  - Keys present: runId ✅, jobId ✅, jobRunId ✅, correlationId ✅, idempotencyKey ✅
+
+**Sign-off:** ___________ (SRE)
+
+---
+
+### C.3 — PHASE-1_ACTIVATION_RUNBOOK.md Validation
+
+**Owner:** SRE / Platform Lead  
+**Blocking:** YES (execution procedure)
+
+- [ ] **Pre-flight Checklist Complete:**
+  - [ ] Migration 0032 deployed ✅
+  - [ ] Host running in DEVELOPMENT mode ✅
+  - [ ] PostgreSQL accessible via SSH tunnel ✅
+  - [ ] Hangfire scheduler running ✅
+  - [ ] Scripts available in ./scripts/ ✅
+
+- [ ] **3-Step Procedure Verified:**
+  - [ ] STEP 1: FREEZE VersionSet (2 min) — ready to execute
+  - [ ] STEP 2: GENERATE identifiers (1 min) — ready to execute
+  - [ ] STEP 3: ENQUEUE Job 893 (1 min) — ready to execute
+
+- [ ] **Troubleshooting Matrix Present:**
+  - Common errors documented ✅
+  - Recovery procedures clear ✅
+
+- [ ] **Monitoring Instructions Clear:**
+  - Log tailing command: ✅
+  - Grafana dashboard: ✅
+  - Alert setup: ✅
+  - Emergency rollback: ✅
+
+**Sign-off:** ___________ (SRE Lead)
+
+---
+
+## 📊 SECTION D: Data Quality & State Validation
+
+### D.1 — Model & Dataset State
+
+**Owner:** Data Governance / Quant Lead  
+**Blocking:** YES (ensures reproducibility)
+
+- [ ] **Model Card Complete:**
+  - [ ] Model ID: ___________
+  - [ ] Model Name: ___________
+  - [ ] Algorithm: ___________
+  - [ ] Training Data Window: ___________ to ___________
+  - [ ] Last Validated: ___________
+  - [ ] Known Limitations: ___________
+
+- [ ] **Dataset Manifest Complete:**
+  - [ ] Dataset ID: ___________
+  - [ ] Dataset Name: ___________
+  - [ ] Features: ___________
+  - [ ] Data Quality Score: ___________
+  - [ ] Last Refreshed: ___________
+  - [ ] Completeness: _______% (target: ≥95%)
+
+- [ ] **Market Data Available:**
+  - [ ] KRX price data: 2024-01-02 to 2024-09-10 ✅ / ❌ (gaps: _________)
+  - [ ] Index data: KOSPI/KOSDAQ ✅ / ❌
+  - [ ] Volume data: Available ✅ / ❌
+  - [ ] Corporate actions: Splits/dividends integrated ✅ / ❌
+
+- [ ] **No Data Quality Anomalies:**
+  ```sql
+  SELECT COUNT(*) FROM market_data WHERE price_close <= 0 OR volume = 0;
+  ```
+  - Bad rows: _______ (target: 0)
+
+**Sign-off:** ___________ (Quant Lead)
+
+---
+
+### D.2 — PIT (Point-in-Time) Query Validation
+
+**Owner:** Data Architect  
+**Blocking:** YES (ensures audit trail)
+
+- [ ] **Correlation IDs Trackable:**
+  - Sample query passes ✅ / ❌
+  - `SELECT COUNT(*) FROM outbox WHERE correlation_id = ?`
+  - Result: _______rows
+
+- [ ] **Revision History Preserved:**
+  - Append-only tables confirmed ✅
+  - No UPDATE/DELETE allowed ✅
+  - Soft deletes only ✅
+
+- [ ] **Published_at Timestamp Correct:**
+  ```sql
+  SELECT COUNT(*) FROM governance.model_version_registry
+  WHERE published_at > NOW();
+  ```
+  - Result: 0 rows (no future dates) ✅ / ❌
+
+**Sign-off:** ___________ (Data Architect)
+
+---
+
+## 📈 SECTION E: Monitoring & Observability Setup
+
+### E.1 — Logging Configured
+
+**Owner:** SRE / Observability Lead  
+**Blocking:** NO (but strongly recommended)
+
+- [ ] **Structured Logging Active:**
+  - Log file: `/app/kartsell/logs/phase-1-execution.log` ✅
+  - Format: JSON with CorrelationId ✅
+  - Retention: _______ days
+
+- [ ] **Serilog PII Redaction Active:**
+  - SSN redaction: ✅
+  - Credit card redaction: ✅
+  - API key redaction: ✅
+
+- [ ] **Log Aggregation Ready:**
+  - ELK / Splunk / Datadog connected: ✅ / ❌
+  - Search by CorrelationId functional: ✅ / ❌
+
+**Sign-off:** ___________ (Observability Lead)
+
+---
+
+### E.2 — Metrics & Alerting
+
+**Owner:** SRE / Observability  
+**Blocking:** NO (but recommended for incident response)
+
+- [ ] **Grafana Dashboard:**
+  - Phase 1 dashboard available: https://grafana.internal/d/phase1-shadow-run ✅ / ❌
+  - Key metrics: Job status, trading days elapsed, data quality, cost simulation ✅
+  - Real-time refresh: 5-minute interval ✅
+
+- [ ] **Alert Thresholds Configured:**
+  - Job failure alert: ✅
+  - Data quality anomaly (>5% bad rows): ✅
+  - Processing latency >30min: ✅
+
+- [ ] **On-Call Escalation Path:**
+  - Primary: ___________
+  - Secondary: ___________
+  - Escalation delay: _______ minutes
+
+**Sign-off:** ___________ (SRE Lead)
+
+---
+
+## 🚀 SECTION F: Final Readiness Sign-offs
+
+### F.1 — Technical Readiness
+
+**All sections B, C, D must be PASS before proceeding**
+
+| Section | Status | Signed Off By | Date |
+|---------|--------|---------------|------|
+| B.1 Database | ✅ / ❌ | ___________ | _______ |
+| B.2 Host App | ✅ / ❌ | ___________ | _______ |
+| B.3 Frontend | ✅ / ❌ | ___________ | _______ |
+| C.1 freeze-versionset | ✅ / ❌ | ___________ | _______ |
+| C.2 generate-identifiers | ✅ / ❌ | ___________ | _______ |
+| C.3 Runbook | ✅ / ❌ | ___________ | _______ |
+| D.1 Data State | ✅ / ❌ | ___________ | _______ |
+| D.2 PIT Queries | ✅ / ❌ | ___________ | _______ |
+
+---
+
+### F.2 — Business Readiness
+
+**All sections A must be PASS before proceeding**
+
+| Gate | Status | Signed Off By | Date |
+|------|--------|---------------|------|
+| A.1 DEC-037 (Source/License) | ✅ / ❌ | ___________ | _______ |
+| A.2 DEC-038 (Calendar/Owner) | ✅ / ❌ | ___________ | _______ |
+| A.3 DEC-079 (Timezone/Correction) | ✅ / ❌ | ___________ | _______ |
+| A.4 VersionSet Approved | ✅ / ❌ | ___________ | _______ |
+
+---
+
+### F.3 — Final Go/No-Go Decision
+
+**OVERALL READINESS:**
+
+**GO CRITERIA:**
+- ✅ All Section A gates APPROVED (governance)
+- ✅ All Section B-D checks PASS (technical)
+- ✅ Emergency rollback procedure validated
+- ✅ On-call team briefed & ready
+
+**NO-GO CRITERIA:**
+- ❌ Any governance approval pending (A.1-A.4)
+- ❌ Technical blocker unresolved (B.1-D.2)
+- ❌ Critical data quality issue (>10% bad rows)
+- ❌ Insufficient monitoring coverage
+
+**FINAL DECISION:**
+
+```
+Phase 1 Execution: ☐ GO (proceed to activation) / ☐ NO-GO (defer)
+
+Date: ___________
+Approved By: ___________ (Platform Lead)
+Emergency Contact: ___________
+Backup Lead: ___________
+```
+
+**Launch Window:** ___________ to ___________ (UTC)  
+**Expected Completion:** 2026-10-27 to 2026-11-26 (50-90 days)  
+**Evidence Preservation:** Phase 1 logs → evidence/PHASE-1/logs/
+
+---
+
+## 📚 Supporting Documents
+
+- **Pre-flight Reference:** `docs/CURRENT/PHASE-1_PRODUCTION_PREFLIGHT_20260806.md`
+- **Activation Procedure:** `docs/CURRENT/PHASE-1_ACTIVATION_RUNBOOK.md`
+- **Evidence Plan:** `docs/CURRENT/PHASE-1_EXECUTION_EVIDENCE_PLAN.md`
+- **Tech Decision Log:** `docs/DECISIONS/ADR-*.md` (authentication, data contract, etc.)
+
+---
+
+**Co-Authored-By:** Claude Haiku 4.5 <noreply@anthropic.com>