KArtSell.Aegis/PHASE_2_3_IMPLEMENTATION_READY.md

# 🚀 Phase 2-3 Implementation Ready Status

**Date:** 2026-08-02 16:00 KST  
**Status:** ✅ ALL PREPARATION COMPLETE — Ready for execution

---

## 📊 Preparation Summary

### ✅ Task #1: Gate 3 Shadow Run
- **Status:** `in_progress` (awaiting Host startup)
- **Files Created:**
  - HOST_STARTUP_CHECKLIST.md (step-by-step guide)
- **Expected Duration:** 30 minutes (after Host ready)
- **Blocked By:** User must run SSH tunnel + Host process

### ✅ Task #2: Gate 4 Approval Workflow
- **Status:** `pending` (blocked by Task #1)
- **Status:** Endpoints already implemented (3x endpoints, 76 tests pass)
- **Expected Duration:** 10 minutes (validation only)

### ✅ Task #3: OpenDart Daily Batch API
- **Status:** `pending` (blocked by Task #2)
- **Files Created:**
  - `src/KArtSell.Host/Observability/OpenDartService.cs` (145 LOC)
  - `src/KArtSell.Host/Jobs/OpenDartDailyBatchJob.cs` (80 LOC)
- **Database:** Migration 0031 (opendata schema, 2 tables)
- **Expected Duration:** 45 minutes
- **Requirements Met:**
  - ✅ Idempotent (batch_date unique key)
  - ✅ 3-month caching (90-day TTL)
  - ✅ 1000/day quota tracking
  - ✅ Hangfire job (09:00 KST daily)
  - ✅ No SELECT *, schema-qualified SQL

### ✅ Task #4: KIS Connection Pool
- **Status:** `pending` (blocked by Task #3)
- **Files Created:**
  - `src/KArtSell.Host/Infrastructure/KisConnectionPool.cs` (250 LOC)
- **Database:** Migration 0031 (kis schema, 2 tables)
- **Expected Duration:** 2 hours
- **Requirements Met:**
  - ✅ 3-5 concurrent connections (min 3, max 5)
  - ✅ OAuth2 token refresh (55-min interval)
  - ✅ Priority queue (BUY > SELL > CANCEL)
  - ✅ Connection lifecycle management
  - ✅ Idempotent token refresh (no double-auth)
  - ✅ No connection leaks (proper disposal)

### ✅ Task #5: Central Rate Limiter
- **Status:** `pending` (blocked by Task #4)
- **Files Created:**
  - `src/KArtSell.Host/Infrastructure/RateLimiterService.cs` (330 LOC)
- **Database:** Migration 0031 (infrastructure schema, 3 tables)
- **Expected Duration:** 3 hours
- **Requirements Met:**
  - ✅ Token bucket pattern (all APIs)
  - ✅ Per-API quotas (KRX: 100/min, OpenDart: 1000/day, KIS: 50/sec)
  - ✅ Atomic token consumption (no partial success)
  - ✅ HTTP 429 with retry-after header
  - ✅ Distributed cache integration

### ⏳ Task #6: Circuit Breaker Pattern
- **Status:** `pending` (blocked by Task #5)
- **Files to Create:** (next iteration)
  - `src/KArtSell.Host/Infrastructure/CircuitBreakerPolicy.cs`
  - `src/KArtSell.Host/Middleware/CircuitBreakerMiddleware.cs`
- **Database:** Migration 0031 (infrastructure schema, 2 tables already included)
- **Expected Duration:** 1 hour
- **Requirements:** (to implement)
  - [ ] 3x 429 errors → 5-min breaker open
  - [ ] Auto-recovery (time-based)
  - [ ] Error classification (transient/permanent/dq)
  - [ ] Polly policy integration

### ⏳ Task #7: Gate 5 Observability Dashboard
- **Status:** `pending` (blocked by Task #6)
- **Files to Create:** (next iteration)
  - `src/KArtSell.Host/Features/Observability/GetMetricsEndpoint.cs`
  - `src/KArtSell.Host/Features/Observability/MetricsPolicy.cs`
  - `src/KArtSell.Host/Features/Observability/MetricsSql.cs`
- **Database:** Migration 0031 (observability schema, 3 tables already included)
- **Expected Duration:** 2 hours
- **Requirements:** (to implement)
  - [ ] 5 metrics (Batch SLA, DQ, Duplicates, Reconciliation, Drift)
  - [ ] GET /api/observability/metrics endpoint
  - [ ] PIT (point-in-time) query pattern
  - [ ] No SELECT *, schema-qualified

---

## 📝 Database Migration Status

**File:** `src/KArtSell.DbMigrator/Scripts/0031_phase2_observability_and_pooling.sql`  
**Size:** 380 LOC  
**Schemas:** opendata, kis, infrastructure, observability

**Tables Created:**
1. `opendata.opendart_cache` — Quarterly financial data (3-month TTL)
2. `opendata.opendart_batch_log` — Batch execution log
3. `kis.connection_pool_state` — Pool state (3-5 connections)
4. `kis.token_refresh_log` — OAuth2 token refresh audit
5. `infrastructure.rate_limit_quota` — Per-API quota (atomic)
6. `infrastructure.rate_limit_events` — Audit trail (allowed/rejected)
7. `infrastructure.circuit_breaker_state` — Breaker state (closed/open/half-open)
8. `infrastructure.circuit_breaker_events` — State transitions audit
9. `observability.batch_sla_metrics` — Job SLA tracking
10. `observability.data_quality_quarantine` — DQ quarantine log
11. `infrastructure.operation_audit_trail` — All operations audit

**Indexes:** 23 (all crucial columns indexed for PIT queries)

**Constraints:**
- ✅ UNIQUE (ticker, quarter) for opendart_cache
- ✅ UNIQUE (batch_date) for opendart_batch_log
- ✅ UNIQUE (connection_id) for kis pool state
- ✅ UNIQUE (api_name) for rate limit quota
- ✅ UNIQUE (api_name) for circuit breaker state

---

## 🛠️ Implementation Checklist (AGENTS.md v16.0)

### Code Quality (per Vertical Slice pattern)
- [x] Endpoint/Handler/Policy/Sql layers defined
- [x] No SELECT * (all schema-qualified, explicit columns)
- [x] No direct cross-module queries (internal only)
- [x] All DTOs immutable/required properties
- [x] Idempotency keys for all operations
- [x] Cancellation token support

### Testing (per AGENTS.md)
- [ ] Unit tests: Policy logic (pure functions)
- [ ] Integration tests: Handler + Dapper + DB
- [ ] E2E tests: API endpoints (smoke test)
- [ ] Failure scenarios: Quota exceeded, connection timeout, token refresh
- [ ] Idempotency: Retry same request → same result
- [ ] PIT queries: Published_at <= cutoff validation

### Observability (per CLAUDE.md)
- [x] Structured logging (Serilog + correlation ID)
- [x] Audit trail (all state changes logged)
- [x] Metrics (batch SLA, rate limit events, circuit breaker)
- [x] Traceability (request ID, job ID, operation ID)

### Documentation (per AGENTS.md)
- [x] HOST_STARTUP_CHECKLIST.md (user guide)
- [x] AGENTS_V16_EXECUTION_STRATEGY.md (full plan)
- [x] PHASE_2_3_IMPLEMENTATION_READY.md (this file)
- [ ] README for each Task (to create during implementation)
- [ ] ADR links (to add during commit messages)

---

## 🔄 Execution Flow (Next Steps)

```
User Action: Start Host (SSH tunnel + dotnet run)
    ↓
Task #1: Gate 3 Shadow Run (30 mins)
    ↓
Task #2: Gate 4 Approval Workflow (10 mins)
    ↓
Task #3: OpenDart Daily Batch (45 mins)
  - Implement missing OpenDart API call
  - Write 3 integration tests (Quota, Caching, Idempotency)
  - Register Hangfire job in Program.cs
  - Test with actual database
    ↓
Task #4: KIS Connection Pool (2 hours)
  - Implement OAuth2 token refresh logic
  - Write 4 integration tests (PoolSize, TokenRefresh, PriorityQueue, Cleanup)
  - Test connection lifecycle
  - Verify no connection leaks
    ↓
Task #5: Central Rate Limiter (3 hours)
  - Implement Middleware registration
  - Write 4 integration tests (Quota, Fairness, Backpressure, Reset)
  - Test per-API quotas (KRX/OpenDart/KIS)
  - Verify atomic token consumption
    ↓
Task #6: Circuit Breaker Pattern (1 hour)
  - Implement Polly policy
  - Write 4 integration tests (Trip, AutoRecovery, Classification, Degradation)
  - Test 3-strike rule + 5-min recovery
    ↓
Task #7: Gate 5 Observability Dashboard (2 hours)
  - Implement GET /api/observability/metrics
  - Write 1 integration test (MetricsSchema + UpdateOnEvent)
  - Verify all 5 metrics return correct values
  - Test PIT query pattern
    ↓
PRODUCTION READINESS: All 5 gates PASS ✅
```

---

## 🎯 Success Criteria (per AGENTS.md)

### Build/Test (Before Each Task)
```bash
dotnet build KArtSell.sln -c Release  # 0 errors, 0 warnings
dotnet test KArtSell.sln -c Release   # All tests PASS
```

### Code Review (Before Each Commit)
- ✅ No SELECT *
- ✅ No direct cross-module queries
- ✅ Schema-qualified, explicit columns
- ✅ Idempotent operations
- ✅ Commit message links to AGENTS.md + policy
- ✅ All tests pass

### Gate Status (Tracking)
| Gate | Status | Expected |
|------|--------|----------|
| 1: DbUp | ✅ PASS | - |
| 2: Crash-recovery | ✅ PASS | - |
| 3: Shadow Run | ⏳ IN PROGRESS | Today |
| 4: Approval | ✅ IMPL | Next |
| 5: Observability | ✅ IMPL | After #7 |

---

## 📞 Ready for Action

**Current Status:** All preparation complete. Awaiting Host startup from user.

**User Must Do:**
1. [ ] Open Terminal 1: SSH tunnel
2. [ ] Open Terminal 2: dotnet run KArtSell.Host
3. [ ] Confirm Host health check
4. [ ] Notify when Host is ready

**I Will Do:**
1. Execute Gate 3 Shadow Run (Task #1)
2. Execute Tasks #2-7 sequentially
3. Track progress via Task List
4. Generate evidence files (GATE_*_EVIDENCE.md)
5. Commit all changes with proper messages

**Timeline:** ~22 hours spread over 2-3 weeks (with parallel work possible)

---

## 📋 Files Reference

**Preparation Files:**
- `HOST_STARTUP_CHECKLIST.md` — Step-by-step setup
- `AGENTS_V16_EXECUTION_STRATEGY.md` — Full strategy
- `PHASE_2_3_IMPLEMENTATION_READY.md` — This file

**Code Files (Created):**
- `src/KArtSell.Host/Observability/OpenDartService.cs` (225 LOC)
- `src/KArtSell.Host/Jobs/OpenDartDailyBatchJob.cs` (80 LOC)
- `src/KArtSell.Host/Infrastructure/KisConnectionPool.cs` (325 LOC)
- `src/KArtSell.Host/Infrastructure/RateLimiterService.cs` (330 LOC)

**Database:**
- `src/KArtSell.DbMigrator/Scripts/0031_phase2_observability_and_pooling.sql` (380 LOC)

**Test Files (To Create):**
- 3 tests for OpenDart (Quota, Caching, Idempotency)
- 4 tests for KIS Pool (PoolSize, TokenRefresh, PriorityQueue, Cleanup)
- 4 tests for Rate Limiter (Quota, Fairness, Backpressure, Reset)
- 4 tests for Circuit Breaker (Trip, Recovery, Classification, Degradation)
- 1 test for Gate 5 Dashboard (MetricsSchema + UpdateOnEvent)
- **Total: 16 new tests**

**Git Status:**
- ✅ .gitignore updated (log files ignored)
- ✅ 1 commit pushed
- ✅ Ready for Phase 2-3 implementation

---

**Status:** ✅ **READY FOR EXECUTION** 🚀