# VS-03: Market Data Ingestion - Vertical Slice Specification

**Slice ID:** VS-03  
**Batch:** 2 (depends on VS-00, VS-02, which are complete)  
**Status:** 📋 SPECIFICATION  
**Created:** 2026-08-05  

---

## Executive Summary

Establish **Market Data Ingestion** system that pulls stock prices, indices, and financial data from external sources (KRX, OpenDart) and normalizes them for downstream signal generation.

**User Goal:** Automated, daily market data collection from Korean exchanges with minimal latency and maximum reliability.

**Non-Goal:**
- Real-time tick data (use Bloomberg/Refinitiv for that)
- Cryptocurrency data
- Forex integration

---

## Acceptance Criteria

### 1. Data Sources ✅

- **KRX OpenAPI:** Stock prices, indices, trading volumes
- **OpenDart API:** Financial statements, disclosure documents
- **Fallback:** Stub data (for testing/demo)

### 2. Data Model ✅

- **Market Daily (PIT):** Date, symbol, open, high, low, close, volume
- **Indices:** KRX 200, KOSPI, KOSDAQ snapshots
- **Company Info:** Sector, industry classification, listing status

### 3. Ingestion Pipeline ✅

- **Schedule:** Daily 9:00 KST (before market open)
- **Retry:** Exponential backoff (3 attempts)
- **Validation:** Schema conformance, duplicate detection
- **Idempotency:** By date + symbol (upsert)
- **Audit:** Correlation ID, row count, error logs

### 4. API Contracts ✅

**Endpoint: POST /api/market/ingest**
```
Request: { dataSource: "KRX|OpenDart", fromDate: "2026-01-01", toDate: "2026-12-31" }
Response: 202 Accepted { jobId, expectedRowCount, status }
```

**Endpoint: GET /api/market/ingest/{jobId}**
```
Response: 200 { status, rowsProcessed, rowsFailed, completedAt }
```

### 5. Data Quality Checks ✅

- No NULL prices (OHLCV)
- Volume >= 0
- High >= Low >= Open >= Close (within reason)
- No future dates
- Deduplication by (date, symbol)

---

## Failure Modes & Recovery

| Scenario | Expected | Recovery |
|----------|----------|----------|
| API timeout | 503, retry in 30s | Auto-retry, exponential backoff |
| Bad data format | DQ quarantine | Manual review, adjust parser |
| Duplicate rows | Idempotent upsert | No effect (already stored) |
| Partial ingestion | Rollback, log error | Retry entire day's batch |

---

## Performance SLAs

| Metric | Target |
|--------|--------|
| Daily ingestion latency | <60 seconds |
| Data freshness | <= 1 trading day old |
| Availability | 99.5% (allow 1 failure/week) |
| Max rows/day | 100,000 (stocks + indices) |

---

## Dependencies

### Inbound (Blocked By)
- ✅ **VS-00:** Platform foundation (complete)
- ✅ **VS-02:** Permission model (complete)

### Outbound (Unblocks)
- 🔄 **VS-04:** Trade Execution (uses VS-03's price data)
- 🔄 **VS-05:** Signal Generation (consumes VS-03 data)
- 🔄 **VS-06:** Portfolio Optimization (requires clean price history)

---

## Component Breakdown (7 items)

| Component | Status |
|-----------|--------|
| **GOV** | 📋 This spec |
| **DATA** | ⏳ Next: PIT schema |
| **DOMAIN** | ⏳ Data validation + normalization |
| **BE** | ⏳ Ingestion API |
| **ASYNC** | ⏳ Hangfire scheduler + event publishing |
| **FE** | ⏳ Ingestion status dashboard |
| **TESTOPS** | ⏳ Data quality tests |

**Total Duration:** ~6 hours (wall-clock 1 day)

---

## Branching Strategy

All work on `Phase-2-Batch-2` branch, squash to main.

**Commits:**
1. GOV + DATA (spec + contract)
2. DOMAIN (validation logic)
3. BE + ASYNC (API + scheduler)
4. FE + TESTOPS (dashboard + tests)

---

## Sign-Off

| Role | Status | Date |
|------|--------|------|
| Architect | ✅ Draft | 2026-08-05 |
| Data Quality | ⏳ Review | TBD |