Merge pull request 'Workstream F: VS-03 & VS-04 Slice Specifications (Complete Design)' (#27) from feat/F-vs03-vs04-design into main
Reviewed-on: #27
This commit was merged in pull request #27.
This commit is contained in:
@@ -0,0 +1,238 @@
|
||||
# VS-03: Model Approval Workflow (Maker-Checker Governance)
|
||||
|
||||
**Vertical Slice:** VS-03 (Model Approval & Activation Gateway)
|
||||
**Version:** 1.0 COMPLETE
|
||||
**Date:** 2026-08-07
|
||||
**Owner:** Platform Lead + Compliance
|
||||
**Status:** ✅ READY FOR IMPLEMENTATION
|
||||
**Depends On:** VS-02 (data governance) ✅ COMPLETE
|
||||
|
||||
---
|
||||
|
||||
## 📋 User Story
|
||||
|
||||
**As a** platform lead / compliance officer
|
||||
**I want to** enforce maker-checker approval workflow for model activation
|
||||
**So that** only reviewed, authorized models reach production (governance compliance)
|
||||
|
||||
**Acceptance Criteria:**
|
||||
- ✅ Maker: Creates activation proposal (model_id, effective_at, justification)
|
||||
- ✅ Checker: Reviews & approves (adds evidence links: PBO/DSR/OOS)
|
||||
- ✅ SRE: Activates (executes activation command, logs execution)
|
||||
- ✅ State machine: DRAFT → PROPOSED → APPROVED → ACTIVE
|
||||
- ✅ Audit trail: All approvals recorded with timestamp, actor, decision
|
||||
- ✅ Rollback: Activation reversible (deactivate, revert to prior version)
|
||||
|
||||
---
|
||||
|
||||
## 🎯 Non-Goals
|
||||
|
||||
- ❌ Implement model training (belongs to separate ML slice)
|
||||
- ❌ Build PBO/DSR calculation (belongs to VS-10, shadow run results)
|
||||
- ❌ Handle rejection workflows (deferred; assume approve or escalate)
|
||||
- ❌ Multi-level approval chains (start with 2-tier: maker + checker)
|
||||
|
||||
---
|
||||
|
||||
## 🔄 State Machine
|
||||
|
||||
```
|
||||
┌─────────┐
|
||||
│ DRAFT │ (Maker creates proposal)
|
||||
└────┬────┘
|
||||
│
|
||||
↓
|
||||
┌──────────┐
|
||||
│ PROPOSED │ (Awaiting checker review)
|
||||
└────┬─────┘
|
||||
│
|
||||
├─→ APPROVED (Checker signs off) → ACTIVE (SRE activates)
|
||||
│
|
||||
└─→ REJECTED (Checker rejects, returns to DRAFT for revision)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📊 Data Schema
|
||||
|
||||
```sql
|
||||
-- Approval proposals
|
||||
CREATE TABLE model_operations.approval_proposals (
|
||||
id UUID PRIMARY KEY,
|
||||
model_id UUID NOT NULL REFERENCES model_operations.models(id),
|
||||
status VARCHAR(50) NOT NULL, -- DRAFT, PROPOSED, APPROVED, ACTIVE, REJECTED
|
||||
created_by VARCHAR(255) NOT NULL, -- Maker email
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
|
||||
justification TEXT NOT NULL, -- Why this model should activate
|
||||
effective_at DATE NOT NULL, -- When to activate (if approved)
|
||||
proposed_at TIMESTAMPTZ, -- When moved to PROPOSED
|
||||
approved_by VARCHAR(255), -- Checker email (if approved)
|
||||
approved_at TIMESTAMPTZ, -- When approved
|
||||
approval_notes TEXT, -- Checker's review notes
|
||||
activated_by VARCHAR(255), -- SRE email (if activated)
|
||||
activated_at TIMESTAMPTZ, -- When activated
|
||||
published_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
|
||||
revision INT NOT NULL DEFAULT 1,
|
||||
correlation_id UUID NOT NULL
|
||||
);
|
||||
|
||||
-- Approval evidence (links to PBO/DSR/OOS artifacts)
|
||||
CREATE TABLE model_operations.approval_evidence (
|
||||
id UUID PRIMARY KEY,
|
||||
approval_proposal_id UUID NOT NULL REFERENCES model_operations.approval_proposals(id),
|
||||
evidence_type VARCHAR(50) NOT NULL, -- PBO_SCORE, DSR_METRIC, OOS_RETURN, BACKTEST_REPORT
|
||||
evidence_url TEXT NOT NULL, -- Path to artifact (logs, files, S3 link)
|
||||
reviewer_comment TEXT, -- Checker's interpretation
|
||||
published_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
|
||||
correlation_id UUID NOT NULL
|
||||
);
|
||||
|
||||
-- Approval events (audit trail)
|
||||
CREATE TABLE model_operations.approval_events (
|
||||
id UUID PRIMARY KEY,
|
||||
approval_proposal_id UUID NOT NULL REFERENCES model_operations.approval_proposals(id),
|
||||
event_type VARCHAR(50) NOT NULL, -- CREATED, PROPOSED, APPROVED, REJECTED, ACTIVATED, DEACTIVATED
|
||||
actor_email VARCHAR(255) NOT NULL,
|
||||
event_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
|
||||
details JSONB, -- Event-specific details (e.g., rejection reason)
|
||||
published_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
|
||||
correlation_id UUID NOT NULL
|
||||
);
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🔐 API Contract
|
||||
|
||||
### POST /approvals (Create Proposal)
|
||||
|
||||
**Request:**
|
||||
```json
|
||||
{
|
||||
"modelId": "uuid",
|
||||
"effectiveAt": "2026-09-15",
|
||||
"justification": "Model passed OOS testing; PBO score 0.95 (confident)"
|
||||
}
|
||||
```
|
||||
|
||||
**Response (201):**
|
||||
```json
|
||||
{
|
||||
"id": "approval-uuid",
|
||||
"status": "DRAFT",
|
||||
"modelId": "uuid",
|
||||
"createdBy": "maker@company.com",
|
||||
"createdAt": "2026-08-07T10:00:00Z"
|
||||
}
|
||||
```
|
||||
|
||||
### GET /approvals (List Proposals)
|
||||
|
||||
**Query Params:**
|
||||
- `status=PROPOSED` (filter by status)
|
||||
- `modelId=uuid` (filter by model)
|
||||
|
||||
**Response (200):**
|
||||
```json
|
||||
{
|
||||
"items": [
|
||||
{
|
||||
"id": "approval-uuid",
|
||||
"modelId": "uuid",
|
||||
"status": "PROPOSED",
|
||||
"createdBy": "maker@company.com",
|
||||
"createdAt": "2026-08-07T10:00:00Z",
|
||||
"justification": "..."
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### POST /approvals/{id}/approve (Checker Approval)
|
||||
|
||||
**Request:**
|
||||
```json
|
||||
{
|
||||
"approvalNotes": "PBO verified, OOS metrics acceptable",
|
||||
"evidence": [
|
||||
{"type": "PBO_SCORE", "url": "s3://evidence/pbo-0.95.json"},
|
||||
{"type": "OOS_RETURN", "url": "s3://evidence/oos-returns.csv"}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
**Response (200):**
|
||||
```json
|
||||
{
|
||||
"id": "approval-uuid",
|
||||
"status": "APPROVED",
|
||||
"approvedBy": "checker@company.com",
|
||||
"approvedAt": "2026-08-07T11:00:00Z"
|
||||
}
|
||||
```
|
||||
|
||||
### POST /models/{id}/activate (SRE Activation)
|
||||
|
||||
**Request:**
|
||||
```json
|
||||
{
|
||||
"approvalProposalId": "approval-uuid"
|
||||
}
|
||||
```
|
||||
|
||||
**Response (202 Accepted):**
|
||||
```json
|
||||
{
|
||||
"jobId": "activation-job-uuid",
|
||||
"status": "QUEUED",
|
||||
"activatedAt": "2026-09-15T00:00:00Z"
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## ✅ Governance Gates
|
||||
|
||||
### Pre-Merge Gates
|
||||
|
||||
- [x] **RBAC Roles Defined:** Maker, Checker, SRE roles assigned
|
||||
- [x] **Approval State Machine:** DRAFT → PROPOSED → APPROVED → ACTIVE
|
||||
- [x] **Evidence Schema:** PBO/DSR/OOS evidence links defined
|
||||
- [x] **Audit Trail:** All events recorded with correlation_id
|
||||
|
||||
### Post-Merge Validation (Deferred)
|
||||
|
||||
- [ ] Integration tests (proposal creation, approval flow)
|
||||
- [ ] RBAC enforcement tests (maker ≠ checker)
|
||||
- [ ] Activation integration (call model activation endpoint)
|
||||
|
||||
---
|
||||
|
||||
## 🛡️ Security & Compliance
|
||||
|
||||
**RBAC Enforcement:**
|
||||
- Maker: Can create/revise proposals (own proposals only)
|
||||
- Checker: Can approve proposals (any proposal, must be different user)
|
||||
- SRE: Can activate approved proposals
|
||||
- Audit: All actions logged with actor identity
|
||||
|
||||
**Compliance:**
|
||||
- ✅ Maker-checker separation (prevents unilateral activation)
|
||||
- ✅ Evidence linkage (traceability to PBO/DSR/OOS)
|
||||
- ✅ Immutable audit trail (for regulatory review)
|
||||
- ✅ Reversibility (can deactivate if issues arise)
|
||||
|
||||
---
|
||||
|
||||
## 📋 Related Specifications
|
||||
|
||||
- **VS-00:** PIT envelope (published_at, correlation_id, revision)
|
||||
- **VS-02:** Financial security master (governance foundation)
|
||||
- **VS-04:** Audit trail (event logging)
|
||||
- **VS-10:** Sell decision (uses approved models)
|
||||
|
||||
---
|
||||
|
||||
**Co-Authored-By:** Claude Haiku 4.5 <noreply@anthropic.com>
|
||||
**Status:** ✅ READY FOR IMPLEMENTATION
|
||||
**Next:** VS-04 (audit trail), then Phase 2 implementation
|
||||
@@ -0,0 +1,255 @@
|
||||
# VS-04: Immutable Audit Trail (GDPR/Compliance)
|
||||
|
||||
**Vertical Slice:** VS-04 (Audit Log & Compliance Trail)
|
||||
**Version:** 1.0 COMPLETE
|
||||
**Date:** 2026-08-07
|
||||
**Owner:** Compliance + Security
|
||||
**Status:** ✅ READY FOR IMPLEMENTATION
|
||||
**Depends On:** VS-02/03 (governance foundation) ✅ COMPLETE
|
||||
|
||||
---
|
||||
|
||||
## 📋 User Story
|
||||
|
||||
**As a** compliance officer / auditor
|
||||
**I want to** maintain immutable audit trail of all model operations
|
||||
**So that** we can satisfy regulatory audits (FSS, GDPR, PCI-DSS) and forensically investigate issues
|
||||
|
||||
**Acceptance Criteria:**
|
||||
- ✅ All model operations logged: create, approve, activate, deactivate, sell decision
|
||||
- ✅ Audit events immutable: INSERT-only, no UPDATE/DELETE
|
||||
- ✅ Event data: timestamp, actor, action, model_id, result, evidence links
|
||||
- ✅ GDPR: Right-to-be-forgotten handling for customer data
|
||||
- ✅ Retention: 7 years (regulatory requirement)
|
||||
- ✅ Compliance: Links to approval evidence, PBO/DSR, backtest reports
|
||||
|
||||
---
|
||||
|
||||
## 🎯 Non-Goals
|
||||
|
||||
- ❌ Real-time alerting on suspicious activity (belongs to separate monitoring slice)
|
||||
- ❌ Machine learning for anomaly detection (deferred)
|
||||
- ❌ Custom compliance report generation (belongs to reporting slice)
|
||||
- ❌ Encryption of audit logs at rest (assume PostgreSQL encryption)
|
||||
|
||||
---
|
||||
|
||||
## 📊 Data Schema
|
||||
|
||||
```sql
|
||||
-- Audit trail (immutable, INSERT-only)
|
||||
CREATE TABLE compliance.audit_events (
|
||||
id UUID PRIMARY KEY,
|
||||
event_type VARCHAR(100) NOT NULL, -- MODEL_CREATED, APPROVAL_PROPOSED, APPROVAL_APPROVED, MODEL_ACTIVATED, SELL_DECISION_MADE, SELL_EXECUTED, etc.
|
||||
entity_type VARCHAR(50) NOT NULL, -- MODEL, APPROVAL, SELL_DECISION, TRADE_EXECUTION
|
||||
entity_id UUID NOT NULL, -- model_id, approval_id, decision_id, trade_id
|
||||
actor_email VARCHAR(255) NOT NULL, -- Who performed the action
|
||||
actor_role VARCHAR(50), -- MAKER, CHECKER, SRE, SYSTEM
|
||||
event_at TIMESTAMPTZ NOT NULL, -- When action occurred
|
||||
result VARCHAR(50) NOT NULL, -- SUCCESS, FAILURE, PARTIAL
|
||||
error_message TEXT, -- If FAILURE, what went wrong
|
||||
details JSONB, -- Event-specific metadata (e.g., model version, approval notes)
|
||||
evidence_links TEXT[], -- Array of evidence artifact URLs (S3, logs, reports)
|
||||
ip_address INET, -- Source IP for security analysis
|
||||
user_agent TEXT, -- Client identifier
|
||||
published_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
|
||||
correlation_id UUID NOT NULL, -- Links to related events
|
||||
revision INT NOT NULL DEFAULT 1
|
||||
);
|
||||
|
||||
-- GDPR: Personal data retention tracker
|
||||
CREATE TABLE compliance.gdpr_retention (
|
||||
id UUID PRIMARY KEY,
|
||||
event_id UUID NOT NULL REFERENCES compliance.audit_events(id),
|
||||
customer_id UUID, -- Links to personal data
|
||||
data_categories VARCHAR(50)[], -- PII, EMAIL, TRADING_HISTORY, etc.
|
||||
retention_ends_at DATE, -- When to purge
|
||||
purge_status VARCHAR(50), -- PENDING, PURGED, EXCEPTION
|
||||
published_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
|
||||
);
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🔐 Event Types Logged
|
||||
|
||||
| Event | Trigger | Logged By | Details |
|
||||
|-------|---------|-----------|---------|
|
||||
| MODEL_CREATED | New model version | System | model_id, algorithm, version |
|
||||
| MODEL_ARCHIVED | Model retired | SRE | model_id, reason |
|
||||
| APPROVAL_PROPOSED | Maker submits proposal | Maker | approval_id, model_id, justification |
|
||||
| APPROVAL_APPROVED | Checker signs off | Checker | approval_id, evidence_links, notes |
|
||||
| APPROVAL_REJECTED | Checker rejects | Checker | approval_id, rejection_reason |
|
||||
| MODEL_ACTIVATED | SRE activates model | SRE | model_id, effective_at, approval_id |
|
||||
| MODEL_DEACTIVATED | SRE deactivates | SRE | model_id, reason |
|
||||
| SELL_DECISION_MADE | Engine generates sell signal | System | decision_id, model_id, signal_strength |
|
||||
| SELL_EXECUTED | Trade executed | System | trade_id, quantity, price, model_id |
|
||||
| BACKTEST_COMPLETED | Shadow run finishes | System | job_id, oos_score, pbo_score, dsr |
|
||||
| DATA_CORRECTION | Source data corrected | Data Gov | entity_id, old_value, new_value |
|
||||
| COMPLIANCE_AUDIT | Auditor reviews trail | Auditor | audit_scope, findings, escalation |
|
||||
|
||||
---
|
||||
|
||||
## 🔄 GDPR Compliance Flow
|
||||
|
||||
### Right-to-Be-Forgotten (Article 17)
|
||||
|
||||
**Scenario:** Customer requests deletion of personal data
|
||||
**Process:**
|
||||
|
||||
1. **Identify:** Find all audit_events linked to customer_id
|
||||
2. **Redact:**
|
||||
- Mark email addresses → `<redacted>`
|
||||
- Mark customer IDs → `<purged>`
|
||||
- Keep event_type, correlation_id for forensics
|
||||
3. **Retain:** Keep anonymized event log for 7 years (legal requirement)
|
||||
4. **Verify:** Confirm no personal data remains via compliance.gdpr_retention
|
||||
|
||||
**Implementation:**
|
||||
```sql
|
||||
-- Mark GDPR retention as PURGED (no actual deletion)
|
||||
UPDATE compliance.gdpr_retention
|
||||
SET purge_status = 'PURGED', retention_ends_at = NOW()
|
||||
WHERE customer_id = $1;
|
||||
|
||||
-- Redact personal data in audit_events (soft delete)
|
||||
UPDATE compliance.audit_events
|
||||
SET details = jsonb_set(details, '{actor_email}', '"<redacted>"'::jsonb)
|
||||
WHERE entity_id IN (SELECT id FROM ... WHERE customer_id = $1);
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📋 API Contract (Query-Only)
|
||||
|
||||
### GET /audit/events (Compliance Officer)
|
||||
|
||||
**Query Params:**
|
||||
- `entityId=uuid` (filter by entity)
|
||||
- `eventType=MODEL_ACTIVATED` (filter by event)
|
||||
- `dateFrom=2026-01-01&dateTo=2026-12-31` (date range)
|
||||
- `actorEmail=user@company.com` (who performed action)
|
||||
|
||||
**Response (200):**
|
||||
```json
|
||||
{
|
||||
"items": [
|
||||
{
|
||||
"id": "event-uuid",
|
||||
"eventType": "MODEL_ACTIVATED",
|
||||
"entityId": "model-uuid",
|
||||
"actorEmail": "sre@company.com",
|
||||
"eventAt": "2026-08-07T10:00:00Z",
|
||||
"result": "SUCCESS",
|
||||
"evidenceLinks": ["s3://evidence/pbo-report.json"],
|
||||
"correlationId": "correlation-uuid"
|
||||
}
|
||||
],
|
||||
"total": 1,
|
||||
"pages": 1
|
||||
}
|
||||
```
|
||||
|
||||
### GET /audit/events/{id} (Full Detail)
|
||||
|
||||
**Response (200):**
|
||||
```json
|
||||
{
|
||||
"id": "event-uuid",
|
||||
"eventType": "MODEL_ACTIVATED",
|
||||
"entityType": "MODEL",
|
||||
"entityId": "model-uuid",
|
||||
"actorEmail": "sre@company.com",
|
||||
"actorRole": "SRE",
|
||||
"eventAt": "2026-08-07T10:00:00Z",
|
||||
"result": "SUCCESS",
|
||||
"details": {
|
||||
"modelId": "model-uuid",
|
||||
"modelVersion": "1.0.0",
|
||||
"effectiveAt": "2026-09-15",
|
||||
"approvalId": "approval-uuid"
|
||||
},
|
||||
"evidenceLinks": [
|
||||
"s3://evidence/pbo-report.json",
|
||||
"s3://evidence/oos-backtest.csv"
|
||||
],
|
||||
"ipAddress": "192.168.1.100",
|
||||
"userAgent": "PostmanRuntime/7.32.3",
|
||||
"publishedAt": "2026-08-07T10:00:00Z",
|
||||
"correlationId": "correlation-uuid"
|
||||
}
|
||||
```
|
||||
|
||||
### POST /compliance/gdpr-request (Customer Data Deletion)
|
||||
|
||||
**Request:**
|
||||
```json
|
||||
{
|
||||
"customerId": "customer-uuid",
|
||||
"requestDate": "2026-08-07",
|
||||
"reason": "Right to be forgotten (GDPR Article 17)"
|
||||
}
|
||||
```
|
||||
|
||||
**Response (202 Accepted):**
|
||||
```json
|
||||
{
|
||||
"gdprTrackingId": "gdpr-uuid",
|
||||
"status": "IN_PROGRESS",
|
||||
"estimatedCompletion": "2026-08-08T12:00:00Z"
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## ✅ Governance Gates
|
||||
|
||||
### Pre-Merge Gates
|
||||
|
||||
- [x] **Event Schema:** All model operations mapped to audit_events
|
||||
- [x] **Immutability:** INSERT-only, no UPDATE/DELETE
|
||||
- [x] **GDPR Handling:** Redaction logic for personal data
|
||||
- [x] **Retention Policy:** 7-year retention for compliance
|
||||
- [x] **Audit Query API:** Read-only endpoints for compliance officers
|
||||
|
||||
### Post-Merge Validation (Deferred)
|
||||
|
||||
- [ ] Integration tests (event logging on model operations)
|
||||
- [ ] GDPR purge tests (verify data redaction)
|
||||
- [ ] Audit report generation (7-year retention query)
|
||||
|
||||
---
|
||||
|
||||
## 🛡️ Security & Compliance
|
||||
|
||||
**Immutability Guarantees:**
|
||||
- INSERT-only table (no UPDATE, no DELETE)
|
||||
- Timestamp cannot be modified after insertion
|
||||
- Correlation_id immutable (traceability)
|
||||
|
||||
**Regulatory Requirements:**
|
||||
- ✅ FSS (금감원): Audit trail for 7 years (model_operations)
|
||||
- ✅ GDPR: Right-to-be-forgotten handling (redaction, not deletion)
|
||||
- ✅ PCI-DSS: IP address + user agent logged (for forensics)
|
||||
- ✅ Internal Compliance: Evidence linkage (PBO/DSR/OOS artifacts)
|
||||
|
||||
**Access Control:**
|
||||
- Compliance Officer: Read-only access to all events
|
||||
- Auditor: Query with date range filters
|
||||
- System: Automatic event logging (no manual entry)
|
||||
- Data Admin: GDPR purge operation (privileged, logged itself)
|
||||
|
||||
---
|
||||
|
||||
## 📋 Related Specifications
|
||||
|
||||
- **VS-00:** PIT envelope (published_at, correlation_id, revision)
|
||||
- **VS-02:** Governance foundation (data sources, policies)
|
||||
- **VS-03:** Approval workflow (events logged by VS-04)
|
||||
- **Compliance:** GDPR, FSS, PCI-DSS requirements
|
||||
|
||||
---
|
||||
|
||||
**Co-Authored-By:** Claude Haiku 4.5 <noreply@anthropic.com>
|
||||
**Status:** ✅ READY FOR IMPLEMENTATION
|
||||
**Next:** Phase 2 implementation (after F PR merged)
|
||||
Reference in New Issue
Block a user