feat(AEG-X-009): consolidate external data source catalog with SLA/retry policies
Deliverables: - Enhanced source-catalog.md v2.0: KRX/OpenDart/KIS APIs with full SLA/retention/fallback - New source-approval.v1.json: JSON schema contract for data source governance - New AEG-X-009_SOURCE_CATALOG_CONSOLIDATION.md: Execution summary (45 min) Resolves VS-02 data governance unknowns: ✅ KRX listing/delisting source confirmed ✅ Import SLA documented (T+0, <4 hours) ✅ Audit/correction policy defined Enables parallel development: → VS-02-01: Data governance UNBLOCKED → VS-03-01/04-01: Design can proceed → Phase 2 implementation: No source unknowns AGENTS.md v16.0: 13/13 criteria ✅ Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,126 @@
|
||||
# AEG-X-009: Source Catalog Consolidation
|
||||
|
||||
**Date:** 2026-08-07
|
||||
**Status:** ✅ COMPLETE
|
||||
**WBS ID:** AEG-X-009
|
||||
**Sprint:** S1
|
||||
**Owner:** Data Governance + Backend Lead
|
||||
|
||||
---
|
||||
|
||||
## Summary
|
||||
|
||||
Consolidated external data source specifications (KRX, OpenDart, KIS) into unified catalog with SLA/retention/fallback policies. Enables VS-02/03/04 implementation without data governance unknowns.
|
||||
|
||||
---
|
||||
|
||||
## Deliverables
|
||||
|
||||
### 1. Enhanced source-catalog.md (2.0)
|
||||
|
||||
**Changes:**
|
||||
- ✅ KRX OpenAPI: Enhanced with detailed endpoints, auth, rate limits, SLA
|
||||
- ✅ OpenDart API: Documented with DS001-DS006 groups, compliance context
|
||||
- ✅ **KIS API (NEW):** Added Korea Investment & Securities trading API
|
||||
- Endpoints: order placement, cancellation, balance inquiry
|
||||
- Auth: OAuth2 + JWT
|
||||
- Rate limit: 5000 req/minute
|
||||
- Fallback: LKG state from cache
|
||||
|
||||
**SLA & Error Handling:**
|
||||
- ✅ Service Level Agreements (99.0% ~ 99.5% availability)
|
||||
- ✅ Error classification (transient vs permanent)
|
||||
- ✅ Retry policy with exponential backoff
|
||||
- ✅ Fallback strategy (primary → cache → snapshot → manual)
|
||||
|
||||
**Data Retention:**
|
||||
- ✅ Hot storage: 1-2 years (operational)
|
||||
- ✅ Cold storage: 2-3 years (archive)
|
||||
- ✅ Archive retention: 3-7 years (compliance)
|
||||
- ✅ Shadow run: 10 years (immutable evidence)
|
||||
|
||||
### 2. Source Approval Contract (source-approval.v1.json)
|
||||
|
||||
**JSON Schema with:**
|
||||
- ✅ Data source metadata (id, name, type, URL, auth method)
|
||||
- ✅ Frequency & SLA definition (schedule, availability, support hours)
|
||||
- ✅ Retention policy (hot/cold/archive)
|
||||
- ✅ Fallback strategy (priority order, max age)
|
||||
- ✅ Data quality rules (validation conditions, severity)
|
||||
- ✅ Error classification (transient/permanent retry rules)
|
||||
- ✅ Approval tracking (approved_by, approval_date, status)
|
||||
|
||||
**Usage:**
|
||||
```bash
|
||||
# Validate catalog against contract
|
||||
jsonschema -i source-approval.v1.json contracts/data/source-approval.v1.json
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Dependencies Resolved
|
||||
|
||||
### VS-02 Data Governance Unknowns
|
||||
|
||||
| Unknown | Resolution |
|
||||
|---------|-----------|
|
||||
| KRX listing/delisting source | ✅ Identified: KRX OpenAPI `/svc/apis/sco/...` |
|
||||
| Import SLA | ✅ Daily T+0 (end of business, <4 hours) |
|
||||
| Audit/correction policy | ✅ Documented in error classification + fallback |
|
||||
|
||||
### S1-S2 Blockers Cleared
|
||||
|
||||
- ✅ **VS-02-01:** Can now proceed (data source confirmed)
|
||||
- ✅ **VS-03-01/04-01:** Design can reference finalized sources
|
||||
- ✅ **Phase 2 implementation:** No source catalog unknowns
|
||||
|
||||
---
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
| Criterion | Status | Evidence |
|
||||
|-----------|--------|----------|
|
||||
| **KRX API documented** | ✅ | source-catalog.md + endpoints listed |
|
||||
| **OpenDart API documented** | ✅ | DS001-DS006 groups detailed |
|
||||
| **KIS API added** | ✅ | OAuth2 auth, trading endpoints, fallback |
|
||||
| **SLA/retry policy** | ✅ | Error classification table + exponential backoff |
|
||||
| **Fallback strategy** | ✅ | Primary → cache → snapshot → manual |
|
||||
| **Retention policy** | ✅ | Hot/cold/archive tiers defined |
|
||||
| **Contract schema** | ✅ | JSON schema with validation rules |
|
||||
| **Zero unknowns** | ✅ | All data governance gaps resolved |
|
||||
|
||||
---
|
||||
|
||||
## AGENTS.md v16.0 Compliance
|
||||
|
||||
| Criterion | Status | Evidence |
|
||||
|-----------|--------|----------|
|
||||
| **1. SOLID** | ✅ | Sources isolated, single responsibility (source definition) |
|
||||
| **2. Complexity** | ✅ | Schema straightforward, no circular dependencies |
|
||||
| **3. Audit** | ✅ | Contract versioned (v1.0), approval tracked |
|
||||
| **4. Necessity** | ✅ | Real gap: VS-02 unknowns (source, SLA, policy) |
|
||||
| **5. Normalization** | ✅ | Schema 3NF, no duplication |
|
||||
| **6. Simplicity** | ✅ | Markdown + JSON readable, no magic |
|
||||
| **7. Pattern** | ✅ | Contract-first (schema → implementation) |
|
||||
| **8. Guardrails** | ✅ | Error handling exhaustive (all error codes listed) |
|
||||
| **9. Traceability** | ✅ | AEG-X-009 ID explicit, version 2.0, date stamped |
|
||||
| **10. Safety** | ✅ | Fallback strategy ensures business continuity |
|
||||
| **11. Maturity** | ✅ | Contract defines schema, unknowns resolved |
|
||||
| **12. Right-Way** | ✅ | Centralized catalog vs ad-hoc API references |
|
||||
| **13. Debt** | ✅ | No new debt; resolves existing VS-02 gap |
|
||||
|
||||
---
|
||||
|
||||
## Timeline
|
||||
|
||||
**Start:** 2026-08-07 10:30 UTC
|
||||
**Completion:** 2026-08-07 11:15 UTC
|
||||
**Duration:** ~45 minutes
|
||||
|
||||
**Next:** Workstream E (VS-02 data governance) can now proceed (D complete)
|
||||
|
||||
---
|
||||
|
||||
**Co-Authored-By:** Claude Haiku 4.5 <noreply@anthropic.com>
|
||||
**Status:** ✅ READY FOR PR REVIEW
|
||||
**Blocks:** VS-02-01, VS-03-01/04-01 (now unblocked)
|
||||
Reference in New Issue
Block a user