diff --git a/PHASE_3_IMPLEMENTATION_PLAN.md b/PHASE_3_IMPLEMENTATION_PLAN.md new file mode 100644 index 00000000..dc9c8877 --- /dev/null +++ b/PHASE_3_IMPLEMENTATION_PLAN.md @@ -0,0 +1,569 @@ +# Phase 3 Implementation Plan: Sell Decision + Trade Execution + +**Date:** 2026-08-07 +**Status:** šŸ“‹ PLANNING (Ready for execution) +**Execution Model:** WBS Optimization (Parallel + Phase 1 concurrent) +**Compliance:** AGENTS.md v16.0 13/13 criteria + +--- + +## šŸ“Š PHASE 3 OVERVIEW + +### Context +``` +Phase 1: šŸš€ Shadow Run (autonomous, 50-90 days, data generating) +Phase 2: āœ… Complete (10 PRs merged, code integrated) +Phase 3: šŸ“‹ Ready to plan (use Phase 1 data → decisions → execution) +Phase 4: šŸ”® Advanced (post-Phase 1, Gate 2+ prerequisites) +``` + +### Phase 3 Goals +``` +1ļøāƒ£ Sell Decision Engine + → Generate sell signals based on model recommendations + → Implement approval workflow integration + → Enforce PBO/DSR validation gates + +2ļøāƒ£ Trade Execution System + → Execute approved sell decisions + → Handle KIS API integration + → Track execution lifecycle + +3ļøāƒ£ Portfolio Reconciliation + → Verify execution vs. approval + → Update holdings & cost basis + → Generate reconciliation reports +``` + +### Key Dependencies +``` +Blockers: Phase 1 must provide OOS/PBO/DSR evidence āœ… (autonomous) +Ready Now: Phase 2 infrastructure (approval/audit) āœ… (merged) +New Work: VS-10 (Sell Decision), VS-05+ (advanced features) +``` + +--- + +## šŸŽÆ PHASE 3 WORKSTREAMS + +### **WORKSTREAM J: VS-10 Sell Decision Engine** + +**Owner:** Quant Lead + PM +**Duration:** 4-5 weeks +**Start:** 2026-09-05 (after Phase 1 reaches 50% progress) +**Blocks:** VS-12, VS-13 (downstream) + +#### Deliverables + +**J1: Data Contract & Slice Spec** +- **Document:** `VS-10-SLICE_SPEC.md` (300-400 lines) +- **Inputs:** Model recommendations, PBO/DSR scores, OOS validation +- **Outputs:** Sell decision (quantity, timing, exit strategy) +- **State Machine:** + ``` + PENDING (awaiting Phase 1 evidence) + ↓ + SIGNAL_GENERATED (model consensus) + ↓ + PBO_VALIDATED (score check ≄ threshold) + ↓ + DSR_VALIDATED (ratio check ≄ threshold) + ↓ + OOS_APPROVED (out-of-sample performance confirmed) + ↓ + READY_FOR_APPROVAL (meets governance gates) + ↓ + APPROVED (maker-checker approval from VS-03) + ↓ + EXECUTED (trade sent to KIS) + ↓ + CONFIRMED (settlement confirmed) + ``` + +**J2: Sell Priority Logic** +- **Immutable Sell Priority:** `HARD_IMPAIRMENT → PORTFOLIO_SURVIVAL → DYNAMIC_PROFIT_FLOOR → CONCENTRATION/LIQUIDITY → OPPORTUNITY_COST → REENTRY_OPTION` +- **Algorithm:** Score-based ranking (fairness + compliance) +- **Output:** Ordered list of candidates for execution + +**J3: API Endpoints (3)** +``` +POST /sell-decisions + Input: model_id, threshold_pbo, threshold_dsr + Output: 201 Created with decision_id + +GET /sell-decisions + Query: status, model_id, execution_date + Output: Paginated list + +POST /sell-decisions/{id}/execute + Input: approval_id (from VS-03) + Output: 202 Accepted (job queued) +``` + +**J4: Database Schema** +```sql +CREATE TABLE sell_decisions ( + id UUID PRIMARY KEY, + model_id UUID REFERENCES models(id), + status VARCHAR(50), -- PENDING, SIGNAL_GENERATED, PBO_VALIDATED, ..., CONFIRMED + pbo_score DECIMAL(5,4), + dsr_metric DECIMAL(5,4), + oos_performance JSONB, + sell_priority INT, + target_quantity INT, + target_price DECIMAL(15,2), + approval_id UUID REFERENCES approval_proposals(id), + execution_id UUID, -- Reference to KIS trade + published_at TIMESTAMPTZ, + correlation_id UUID, + revision INT +); + +CREATE TABLE sell_decision_evidence ( + id UUID PRIMARY KEY, + decision_id UUID REFERENCES sell_decisions(id), + evidence_type VARCHAR(50), -- PBO_REPORT, OOS_BACKTEST, DSR_METRIC + evidence_url TEXT, + validated_at TIMESTAMPTZ, + published_at TIMESTAMPTZ, + correlation_id UUID +); +``` + +**J5: Handlers & Jobs** +- `GenerateSellDecisionHandler` — Orchestrates scoring + validation +- `ValidatePboHandler` — PBO score gate (≄ 0.65 recommended) +- `ValidateDsrHandler` — DSR ratio gate (≄ 0.015 recommended) +- `ValidateOosHandler` — OOS performance gate (pass/fail) +- `ExecuteSellDecisionJob` — Queues trade via KIS API + +**J6: Tests** +- 15+ unit tests (scoring logic, validation gates, priority ranking) +- 8+ integration tests (E2E from signal to approval) +- 3+ contract tests (approval/audit integration) + +**J7: Compliance** +- āœ… AGENTS.md 13/13 (SOLID, complexity, audit, necessity, etc.) +- āœ… PIT tracking (published_at, correlation_id, revision) +- āœ… Immutable decisions (INSERT-only, no UPDATE) +- āœ… Evidence linkage (S3 artifacts) + +--- + +### **WORKSTREAM K: VS-12 Trade Execution** + +**Owner:** Backend Lead + Trading Ops +**Duration:** 3-4 weeks +**Start:** 2026-09-10 (parallel with J, overlapping) +**Depends On:** J (sell decision approval) + +#### Deliverables + +**K1: KIS API Integration** +- **Service:** `KisTradeExecutionService.cs` +- **Methods:** + ```csharp + ExecuteTradeAsync(tradeRequest, correlationId) + GetOrderStatusAsync(orderId) + CancelOrderAsync(orderId, reason) + ConfirmSettlementAsync(orderId) + ``` +- **Features:** + - Connection pooling + retry logic (exponential backoff) + - Order validation (quantity, price, liquidity checks) + - Failure classification (transient/permanent/liquidity) + +**K2: Trade Lifecycle States** +``` +PENDING (awaiting execution) +↓ +SUBMITTED (sent to KIS) +↓ +ACCEPTED (KIS confirmed receipt) +↓ +PARTIAL_FILLED / FILLED (execution progress) +↓ +CONFIRMED (settlement confirmed) +↓ +RECONCILED (cost basis updated) +``` + +**K3: API Endpoints (2)** +``` +POST /trades + Input: sell_decision_id, quantity, limit_price + Output: 202 Accepted with trade_id + +GET /trades + Query: status, decision_id, execution_date + Output: Paginated list with execution details +``` + +**K4: Database Schema** +```sql +CREATE TABLE trades ( + id UUID PRIMARY KEY, + sell_decision_id UUID REFERENCES sell_decisions(id), + kis_order_id VARCHAR(50), -- KIS-assigned order ID + status VARCHAR(50), -- PENDING, SUBMITTED, ACCEPTED, FILLED, CONFIRMED, RECONCILED + quantity INT, + executed_quantity INT, + unit_price DECIMAL(15,2), + total_amount DECIMAL(18,2), + commission DECIMAL(15,2), + net_proceeds DECIMAL(18,2), + execution_timestamp TIMESTAMPTZ, + settlement_timestamp TIMESTAMPTZ, + error_message TEXT, + kis_response JSONB, + published_at TIMESTAMPTZ, + correlation_id UUID, + revision INT +); +``` + +**K5: Handlers & Jobs** +- `SubmitTradeHandler` — Submit to KIS +- `PollTradeStatusJob` — Hangfire polling (q-evaluation queue) +- `ConfirmSettlementHandler` — Mark settlement complete +- `ReconcileTradeHandler` — Update cost basis + +**K6: Tests** +- 12+ unit tests (validation, state transitions) +- 8+ integration tests (KIS mock + real DB) +- 3+ failure scenario tests (transient/permanent errors) + +**K7: Compliance** +- āœ… AGENTS.md 13/13 +- āœ… Idempotent execution (no duplicate trades) +- āœ… Audit trail (all state changes logged) +- āœ… Error classification + +--- + +### **WORKSTREAM L: VS-14 Portfolio Reconciliation** + +**Owner:** Data Architecture + Finance +**Duration:** 2-3 weeks +**Start:** 2026-09-15 (parallel with K, uses K output) +**Depends On:** K (trade execution) + +#### Deliverables + +**L1: Reconciliation Engine** +- **Algorithm:** Compare approved decisions vs. executed trades +- **Inputs:** + - Sell decision (approved, PBO/DSR/OOS validated) + - Trade execution (settled, cost basis confirmed) + - Holdings (before execution) +- **Outputs:** + - Holdings updated + - Cost basis adjusted + - Reconciliation report (matches/mismatches) + +**L2: Mismatch Detection** +- Quantity mismatch (approved vs. executed) +- Price variance (approved limit vs. actual) +- Timing variance (decision date vs. execution date) +- Settlement delay (execution vs. confirmation) + +**L3: Cost Basis Update** +- Weighted average cost tracking +- Lot tracking (FIFO/LIFO methods) +- Gain/loss calculation +- Tax lot reporting + +**L4: API Endpoints (2)** +``` +GET /reconciliation/holdings + Response: Current portfolio state (updated after trade) + +GET /reconciliation/mismatches + Query: date_range, severity + Response: Flagged discrepancies for manual review +``` + +**L5: Database Schema** +```sql +CREATE TABLE holdings ( + id UUID PRIMARY KEY, + security_id UUID REFERENCES financial_security_master.securities(id), + quantity INT, + weighted_avg_cost DECIMAL(15,2), + total_cost_basis DECIMAL(18,2), + market_value DECIMAL(18,2), + unrealized_gain_loss DECIMAL(18,2), + updated_at TIMESTAMPTZ, + published_at TIMESTAMPTZ, + correlation_id UUID, + revision INT +); + +CREATE TABLE reconciliation_logs ( + id UUID PRIMARY KEY, + trade_id UUID REFERENCES trades(id), + holding_id UUID REFERENCES holdings(id), + quantity_before INT, + quantity_after INT, + cost_basis_delta DECIMAL(18,2), + mismatch_detected BOOLEAN, + mismatch_reason TEXT, + reconciled_at TIMESTAMPTZ, + published_at TIMESTAMPTZ, + correlation_id UUID +); +``` + +**L6: Tests** +- 10+ unit tests (cost basis, gain/loss calculation) +- 6+ integration tests (reconciliation workflow) +- 3+ scenario tests (edge cases: splits, dividends) + +--- + +## šŸ“ˆ EXECUTION TIMELINE + +### Week 1-2 (2026-09-05 ~ 2026-09-18) +``` +J1: VS-10 Spec & Contract Design (parallel) +K1: VS-12 API & KIS Integration (parallel) +L1: VS-14 Design & Algorithm (parallel) + +Status: D/E/F design docs, ready for implementation +Phase 1: 50%-75% progress +``` + +### Week 3-4 (2026-09-19 ~ 2026-10-02) +``` +J2-J7: VS-10 Implementation & Tests +K2-K6: VS-12 Implementation & Tests +L2-L5: VS-14 Implementation & Tests + +Status: All 3 slices in parallel, 50% code complete +Phase 1: 75%-90% progress +``` + +### Week 5-6 (2026-10-03 ~ 2026-10-16) +``` +J/K/L: Integration testing (cross-slice) + Phase 1 final results available + Gate 2 validation begins + +Status: All code complete, integration verified +Phase 1: 90-100% (completion), results ready +``` + +### Week 7+ (2026-10-17+) +``` +Phase 1 Complete → Gate 2 Execution +Phase 3 Implementation → Production Deployment (~November) +``` + +--- + +## šŸŽÆ WBS OPTIMIZATION STRATEGY + +### Parallel Execution (J + K + L Simultaneous) +``` +Sequential (Baseline): J(4w) → K(3w) → L(2w) = 9 weeks +Parallel (Actual): All 3 simultaneous = 5 weeks +──────────────────────────────────────────────── +TIME SAVED: 4 weeks ā±ļø + +Dependencies: + J outputs → K inputs (sell decision → trade execution) + K outputs → L inputs (trade execution → reconciliation) + +Overlap Strategy: + Week 1-2: J design, K design, L design (PARALLEL) + Week 2-3: J → 50%, K start (J unblocks K) + Week 3-4: J → 100%, K → 50%, L start (K unblocks L) + Week 4-5: All 3 at 75-100% (overlapping) + Week 5-6: Integration testing (all done) +``` + +### Phase 1 Concurrent Execution +``` +Phase 1: šŸš€ Autonomous (50-90 days, data generating) +Phase 3: šŸ“‹ Implementation in parallel (uses accumulated data) + +Benefit: + • No waiting for Phase 1 to complete + • Infrastructure ready when Phase 1 evidence available + • Gate 2 validation can begin on Day 75+ (mid-way through Phase 1) + • Production deployment by November 2026 +``` + +--- + +## āœ… AGENTS.md v16.0 COMPLIANCE PLAN + +### Verification Framework (Apply to J/K/L) + +| Criterion | J (Sell Decision) | K (Trade Execution) | L (Reconciliation) | +|-----------|------------------|---------------------|-------------------| +| 1. SOLID | 3 services (scoring, validation, approval) | KIS service + handlers | Reconciliation + reports | +| 2. Complexity | Each <300 lines, readable | Connection pool, retry logic | Calc engine, mismatch detection | +| 3. Audit | correlation_id, PIT tracking | All state changes logged | Cost basis trail | +| 4. Necessity | Grounded in Phase 1 evidence | Spec-before-code āœ… | Portfolio integrity | +| 5. Normalization | 3NF schema, append-only | PIT tracked decisions | Versioned holdings | +| 6. Simplicity | State machine clear | No magic numbers | Algorithm transparent | +| 7. Pattern | Vertical Slice (Services/Handlers/Endpoints/Sql) | Contract-driven | Domain-driven design | +| 8. Guardrails | Validation gates (PBO/DSR/OOS) | Error classification | Mismatch alerts | +| 9. Traceability | Evidence links to S3 | CorrelationId throughout | Audit trail immutable | +| 10. Safety | Idempotent operations | Rollback-safe state | No partial reconciliation | +| 11. Maturity | Spec-before-code āœ… | Data contracts āœ… | Design docs āœ… | +| 12. Right-Way | Formal gates, no shortcuts | KIS official API | Regulatory compliance | +| 13. Debt | No new tech debt | Enables Phase 4 | Tech debt registry | + +--- + +## šŸ“Š RESOURCE ALLOCATION + +### Team Assignment (Recommended) + +**Workstream J (Sell Decision)** — 3 people, 5 weeks +``` +Lead: Quant Lead (decision logic, PBO/DSR validation) +Backend: 2 engineers (API, database, handlers, tests) +Effort: ~200 hours +``` + +**Workstream K (Trade Execution)** — 3 people, 4 weeks +``` +Lead: Backend Lead (KIS integration, error handling) +Trading: 1 operations engineer (KIS API knowledge) +Backend: 1 engineer (handlers, jobs, reconciliation) +Effort: ~150 hours +``` + +**Workstream L (Portfolio Reconciliation)** — 2 people, 3 weeks +``` +Lead: Data Architect (reconciliation algorithm) +Finance: 1 engineer (cost basis, gain/loss, reporting) +Effort: ~100 hours +``` + +**Total Phase 3 Effort:** ~450 hours (~11 weeks serial, 5 weeks parallel) + +--- + +## šŸ“‹ MILESTONE CHECKLIST + +### Phase 3 Gates (Pre-Merge) + +**J (Sell Decision):** +- [ ] VS-10 SLICE_SPEC complete (Spec-before-code) +- [ ] PBO/DSR/OOS validation gates designed +- [ ] API contracts finalized +- [ ] Database migration validated (fresh/upgrade/re-run) +- [ ] Unit tests: 15/15 PASS +- [ ] Integration tests: 8/8 PASS +- [ ] Architecture tests: SOLID compliance verified +- [ ] No SELECT *, schema-qualified SQL +- [ ] Immutable decisions (INSERT-only) +- [ ] Correlation_id traceability + +**K (Trade Execution):** +- [ ] VS-12 SLICE_SPEC complete +- [ ] KIS API contract finalized +- [ ] Error classification (transient/permanent/liquidity) +- [ ] Idempotency key strategy +- [ ] Unit tests: 12/12 PASS +- [ ] Integration tests: 8/8 PASS +- [ ] State machine transitions verified +- [ ] Rollback-safe design confirmed + +**L (Portfolio Reconciliation):** +- [ ] VS-14 SLICE_SPEC complete +- [ ] Reconciliation algorithm validated +- [ ] Cost basis calculations verified +- [ ] Unit tests: 10/10 PASS +- [ ] Integration tests: 6/6 PASS +- [ ] Edge cases (splits, dividends) handled +- [ ] Tax lot tracking verified + +**Cross-Slice Integration:** +- [ ] J → K flow verified (decision → execution) +- [ ] K → L flow verified (execution → reconciliation) +- [ ] Audit trail (VS-04) integration complete +- [ ] Approval workflow (VS-03) integration complete +- [ ] E2E tests: PASS +- [ ] Gate 2 prerequisite data ready (Phase 1 evidence) + +--- + +## šŸŽÆ SUCCESS CRITERIA + +### Code Quality +``` +Tests: 48+ (unit/integration/E2E) +Coverage: ≄80% code coverage +Complexity: All classes <300 lines +Compliance: AGENTS.md 13/13 āœ… +Tech Debt: No new unbounded debt +``` + +### Business Metrics +``` +Sell Decision Accuracy: PBO/DSR/OOS validation pass rate ≄95% +Trade Execution Rate: Approved decisions → executed ≄99% +Reconciliation Success: Mismatches ≤0.1% (normal variance) +SLA Compliance: Execution latency <1 hour (from approval) +``` + +### Timeline +``` +Week 5-6: All code merged to main +Week 6-7: Integration testing & bug fixes +Week 7+: Production deployment (Gate 2+ validation) +November: Production live (full automation) +``` + +--- + +## šŸ“ˆ PHASE 3 ROADMAP DIAGRAM + +``` +Phase 1 (Autonomous) Phase 2 (Merged) Phase 3 (Parallel) +───────────────── ─────────────── ────────────────── +50-90 days āœ… Complete J: Sell Decision +(Data generating) 10 PRs merged K: Trade Exec (Parallel) + L: Reconciliation + + VS-03 Approval ──→ J→K (flow) + VS-04 Audit ──→ all J/K/L logged + + ↓ (Week 6) + Integration tests + + ↓ (Week 7) + Gate 2 validation + (Phase 1 evidence) + + ↓ (Week 8+) + Production +``` + +--- + +## āœ… APPROVAL & SIGN-OFF + +**Phase 3 Plan Status:** šŸ“‹ Ready for review and team assignment +**Dependencies:** Phase 1 autonomous (no manual action needed) āœ… +**Readiness:** Phase 2 infrastructure (approval/audit) merged āœ… +**AGENTS.md Compliance:** 13/13 criteria framework āœ… + +**Next Steps:** +1. Team review Phase 3 plan +2. Assign teams to J/K/L workstreams +3. Start Phase 3 implementation (2026-09-05) +4. Monitor Phase 1 progress (autonomous) +5. Execute Phase 3 in parallel with Phase 1 completion + +--- + +**Generated:** 2026-08-07 +**Prepared By:** Claude Haiku 4.5 +**Framework:** WBS Optimization + AGENTS.md v16.0 +**Status:** āœ… READY FOR EXECUTION