Merge pull request 'Workstream D: AEG-X-009 Source Catalog Consolidation' (#25) from feat/D-aeg-x009-source-catalog into main

Reviewed-on: #25
2026-08-07 17:18:25 +09:00
parent d602c2819b 80d1636107
commit 4e8a7bd021
3 changed files with 344 additions and 11 deletions
@@ -1,21 +1,23 @@
 # Data Source Catalog

-**Purpose:** Master reference for all data sources, APIs, and lineage  
+**Purpose:** Master reference for all data sources, APIs, SLAs, and lineage  
 **Owner:** Data Governance Team  
-**Version:** 1.0  
-**Date:** 2026-08-06
+**Version:** 2.0  
+**Date:** 2026-08-07  
+**Status:** CONSOLIDATED (AEG-X-009)

 ---

 ## 📊 Source Systems Summary

-| Source | Type | Frequency | Availability SLA | Consumers | Retention |
-|--------|------|-----------|------------------|-----------|-----------|
-| **KRX OpenAPI** | External REST | Daily (T+0) | 99.5% | prices, signals, portfolio | 5 years |
-| **OpenDart API** | External REST | T+2 | 99.0% | disclosure, models, recommendations | 7 years |
-| **Portfolio (User Input)** | Internal Form | Real-time | 100% (manual) | rebalance, risk, holdings | 5 years |
-| **Shadow Run Output** | Computed (Hangfire) | 252+ days | 99.9% | evidence, PBO/DSR, activation | 10 years |
-| **Audit Events** | Internal Database | Real-time (write) | 99.99% | compliance, security, tracing | 7 years |
+| Source | Type | Frequency | Availability SLA | Import Delay | Consumers | Retention | Owner |
+|--------|------|-----------|------------------|--------------|-----------|-----------|-------|
+| **KRX OpenAPI** | External REST | Daily (T+0) | 99.5% | <4 hours (EoD) | prices, signals, portfolio | 5 years | KRX |
+| **OpenDart API** | External REST | T+2 business | 99.0% | +2 calendar days | disclosure, models, recommendations | 7 years | FSS |
+| **KIS API** | External REST | Real-time | 99.2% | <1 minute | trading, orders, execution | 3 years | Korea Investment & Securities |
+| **Portfolio (User Input)** | Internal Form | Real-time | 100% (manual) | Immediate | rebalance, risk, holdings | 5 years | Internal |
+| **Shadow Run Output** | Computed (Hangfire) | 252+ days | 99.9% | Async (Job 976) | evidence, PBO/DSR, activation | 10 years | Internal |
+| **Audit Events** | Internal Database | Real-time (write) | 99.99% | Immediate | compliance, security, tracing | 7 years | Internal |

 ---

@@ -306,6 +308,79 @@ Legend: ✅ = Primary consumer, ⚪ = Secondary/Optional

 ---

+---
+
+## 🔑 KIS API (Korea Investment & Securities)
+
+**Service:** Korea Investment & Securities Trading API  
+**Base URL:** `https://openapivts.kbopenplatform.com` (KIS VTS) or `https://openapi.kbopenplatform.com`  
+**Authentication:** `APP_KEY` + `APP_SECRET` (OAuth2, JWT)  
+**Rate Limit:** 5000 req/minute (varies by tier)
+
+**Endpoints Used:**
+
+| Endpoint | Method | Purpose | Frequency |
+|----------|--------|---------|-----------|
+| `/uapi/trading-order` | POST | Place order | Real-time |
+| `/uapi/trading-cancel-order` | POST | Cancel order | Real-time |
+| `/uapi/domestic-stock-cash-daily` | GET | Account balance | Daily EOD |
+
+**Authentication Flow:**
+```
+1. Get OAuth2 token: POST /oauth2/authorize + refresh_token
+2. Call trading endpoint: X-APP-KEY + Authorization: Bearer <token>
+3. Retry on 401: Refresh token if expired
+```
+
+**Fallback Strategy:**
+- **Primary:** Live API
+- **Secondary:** Last Known Good (LKG) state from DB
+- **Tertiary:** Cached execution snapshot from previous day
+
+---
+
+## ⏱️ SLA & Retry Policy
+
+### Service Level Agreements
+
+| Source | Availability | Support Hours | Incident Contact | Escalation |
+|--------|--------------|----------------|------------------|------------|
+| **KRX** | 99.5% | Weekdays 9 AM-5 PM KST | `support@krx.co.kr` | → Operations Manager |
+| **OpenDart** | 99.0% | Business hours only | FSS Helpdesk | → Data Governance Lead |
+| **KIS** | 99.2% | 24/5 (trading hours) | `api-support@kimconsulting.com` | → Backend Lead |
+
+### Error Classification & Retry
+
+| Error | Classification | Retry Delay | Max Attempts | Action |
+|-------|-----------------|------------|--------------|--------|
+| **Network timeout** | Transient | 30s exponential backoff | 5 | Retry immediately |
+| **429 (Rate limit)** | Transient | 60s + random jitter | 3 | Queue to Hangfire |
+| **401 (Auth expired)** | Transient | Refresh token, retry | 2 | Obtain new credentials |
+| **400 (Bad request)** | Permanent | None | 0 | Log error, alert ops |
+| **503 (Service unavailable)** | Transient | 5min + exponential | 10 | Use fallback (cache) |
+| **Data quality rule fail** | Permanent | None | 0 | Quarantine + manual review |
+
+### Fallback & Recovery
+
+**When Primary Source Fails:**
+1. **KRX API down:** Use LKG prices from cache (up to 1 trading day old)
+2. **OpenDart rate limit:** Queue job for retry (Hangfire q-backfill)
+3. **KIS trading timeout:** Use cached balance, resume next market open
+4. **Shadow run interrupted:** Resume from last checkpoint (idempotent)
+
+---
+
+## 📋 Data Retention Policy
+
+| Source | Cold Storage | Archive Retention | Purge Policy |
+|--------|--------------|-------------------|--------------|
+| **KRX Prices** | After 2 years | 5 years (compliance) | After 5 years |
+| **OpenDart** | After 3 years | 7 years (regulatory) | After 7 years |
+| **KIS Trading** | After 1 year | 3 years (audit) | After 3 years |
+| **Shadow Run** | Never | 10 years (evidence) | Never (immutable) |
+
+---
+
 **Owner:** Data Governance  
-**Last Updated:** 2026-08-06  
+**Last Updated:** 2026-08-07 (AEG-X-009 Consolidated)  
 **Status:** ✅ **APPROVED FOR OPERATIONS**