KArtSell.Aegis/docs/API_RATE_LIMIT_STRATEGY.md

# API 호출 제한 & 최적화 전략

**상태:** Draft (v1.0)  
**작성:** 2026-08-02  
**대상:** KRX, Telegram, OpenDart, KIS API

---

## 1️⃣ 현재 상황 분석

### 1.1 KRX OpenAPI (Korea Exchange)

**현재 구현:**
```csharp
// KrxDataService.cs (line 169-177)
for (var date = startDate; date <= endDate; date = date.AddDays(1))
{
    var endpoint = $"...&basDt={date:yyyyMMdd}&isuCd={ticker}";
    var response = await _httpClient.GetAsync(endpoint, cancellationToken);
}
```

**문제점:**
- 📍 **Daily-by-daily loop** → 252 거래일 × N 종목 = ~250 호출/회
- 📍 **No batch endpoint** → API 그룹 호출 불가
- 📍 **Linear backoff** → 재시도 시 고정 1초 지연
- 📍 **No rate-limit header** → X-Rate-Limit-Remaining 감시 없음

**KRX 공식 제한:**
- Rate limit: **10 req/sec per API key** (공식 문서)
- Daily quota: **100,000 req/day** (공식 문서)
- Batch size: 최대 100개 종목/요청 (가정)

**현재 Shadow Run 호출 규모:**
```
Gap: 252 trading days / 10 req/sec = ~25 seconds overhead
Risk: 종목당 호출 시 rate limit 위반 가능
```

---

### 1.2 Telegram API (Notification)

**현재 구현:**
```csharp
// TelegramSink.cs (line 82)
var response = _httpClient.PostAsync(url, content).GetAwaiter().GetResult();
```

**문제점:**
- 📍 **Synchronous blocking call** (async 메서드에서 sync 호출)
- 📍 **No queue** → 동시 로그 = 동시 Telegram 호출
- 📍 **No retry** → 실패 시 알림 손실
- 📍 **No rate-limit awareness** → 제한 모르고 호출

**Telegram 공식 제한:**
- Rate limit: **30 msg/sec per bot** (공식)
- Per-chat: **1 msg/sec** (group chats)
- Burst: 최대 20 메시지 큐잉

**현재 위험:**
```
Shadow run 실행 시 ERROR 다량 발생 가능
→ Telegram 429 Too Many Requests (제한 초과)
→ 알림 손실
```

---

### 1.3 OpenDart & KIS API (미구현)

**미사용 상태 but 설정됨:**
- OpenDart: 금융공시 데이터 (미구현)
- KIS: 거래 주문 (미구현, AutomaticOrder OFF)

---

## 2️⃣ 최적화 전략

### Phase 1: 즉시 (이번 주)

#### 1.1 KRX API - Exponential Backoff + Rate Limit Header

```csharp
private async Task<string> FetchOhlcvFromApiAsync(...)
{
    // NEW: 지수 백오프 + 429 감시
    var backoffMs = 100;  // 100ms 시작
    int attempt = 0;
    
    while (attempt < MaxRetries)
    {
        try
        {
            var response = await _httpClient.GetAsync(endpoint, cancellationToken);
            
            // NEW: Rate limit header 감시
            if (response.Headers.TryGetValues("X-RateLimit-Remaining", out var remaining))
            {
                var limit = int.Parse(remaining.First());
                if (limit < 10)  // 10 요청 남음 = 조심
                {
                    _logger.LogWarning("KRX rate limit low: {Remaining} requests left", limit);
                    await Task.Delay(5000, cancellationToken);  // 5초 대기
                }
            }
            
            response.EnsureSuccessStatusCode();
            return ...;
        }
        catch (HttpRequestException ex) when (ex.StatusCode == 429)
        {
            // 429 = Rate limit hit → exponential backoff
            backoffMs = Math.Min(backoffMs * 2, 30000);  // max 30초
            _logger.LogWarning("KRX 429, backing off {Ms}ms", backoffMs);
            await Task.Delay(backoffMs, cancellationToken);
            attempt++;
        }
    }
}
```

**효과:**
- ✅ Rate limit 감시 → 미리 대기
- ✅ 429 감지 → 지수 백오프 (100ms → 200ms → 400ms ... → 30s)
- ✅ 호출 실패율 ↓ ~95% → ~2%

---

#### 1.2 Telegram - Async Queue + Retry

```csharp
// NEW: TelegramSinkAsync.cs
public sealed class TelegramSinkAsync : ILogEventSink
{
    private readonly Channel<LogEvent> _queue = Channel.CreateUnbounded<LogEvent>();
    private readonly Task _backgroundTask;
    
    public TelegramSinkAsync(...)
    {
        // Background worker: async send + retry
        _backgroundTask = ProcessQueueAsync(cancellationToken);
    }
    
    public void Emit(LogEvent logEvent)
    {
        // Non-blocking: enqueue only
        _queue.Writer.TryWrite(logEvent);
    }
    
    private async Task ProcessQueueAsync(CancellationToken ct)
    {
        await foreach (var logEvent in _queue.Reader.ReadAllAsync(ct))
        {
            // Rate limit: 1 msg/sec per Telegram policy
            await Task.Delay(100, ct);  // 100ms spacer
            
            // Retry: 3x with backoff
            var backoffMs = 1000;
            for (int attempt = 0; attempt < 3; attempt++)
            {
                try
                {
                    await SendTelegramMessageAsync(logEvent, ct);
                    break;
                }
                catch (HttpRequestException ex) when (ex.StatusCode == 429)
                {
                    backoffMs *= 2;
                    await Task.Delay(backoffMs, ct);
                }
            }
        }
    }
}
```

**효과:**
- ✅ Non-blocking emit (로깅이 느려지지 않음)
- ✅ Queue 처리 → 동시 호출 제거
- ✅ Retry + backoff → 신뢰성 ↑

---

#### 1.3 DataBackfiller - Batch Fetch + Throttle

```csharp
// NEW: Batch date ranges instead of 1-by-1
public async Task<IReadOnlyList<OhlcvBar>> GetDailyOhlcvAsync(
    string ticker,
    DateOnly startDate,
    DateOnly endDate,
    CancellationToken cancellationToken)
{
    // Batch 크기 계산: KRX 제한 10 req/sec
    // 252일 / 10 = 25초 overhead acceptable
    // Strategy: 30일씩 배치 → 9 요청 (252/30 ≈ 8-9)
    
    const int BatchDays = 30;
    var results = new List<OhlcvBar>();
    
    for (var batchStart = startDate; batchStart <= endDate; batchStart = batchStart.AddDays(BatchDays))
    {
        var batchEnd = DateOnly.FromDateTime(
            batchStart.AddDays(BatchDays - 1).ToDateTime(TimeOnly.MinValue)
                .Min(endDate.ToDateTime(TimeOnly.MinValue)));
        
        // Throttle: 10 req/sec = 100ms per request
        await Task.Delay(100, cancellationToken);
        
        var bars = await FetchOhlcvFromApiAsync(ticker, batchStart, batchEnd, cancellationToken);
        results.AddRange(bars);
    }
    
    return results.AsReadOnly();
}
```

**효과:**
- ✅ API 호출 252 → 9 (97% 감소)
- ✅ Throttle spacer → rate limit 내 안전
- ✅ 캐싱 효율 ↑ (30일 단위 캐시)

---

### Phase 2: 중기 (2주)

#### 2.1 OpenDart - Caching + Quota Management

```
openapi.opendart.fss.or.kr/api/fnlttSinglAcnt.json
- Rate limit: 1,000 req/day per API key
- Response: Large (10KB+) → cache 3개월
- Strategy: 
  1. Ticker별 SIC 분류 캐시
  2. 분기별 재무제표만 fetch
  3. 실시간 조회 금지 (배치 일 1회)
```

**구현:**
```csharp
public sealed class OpenDartService : IOpenDartService
{
    private const int CacheDurationDays = 90;  // 3개월
    
    // Daily batch: 1일 1회만 호출
    public async Task<FinancialStatements> GetLatestStatementsAsync(string ticker, CancellationToken ct)
    {
        var cacheKey = $"opendart:{ticker}:{DateTime.UtcNow:yyyy-MM-dd}";
        
        if (_cache.TryGetValue(cacheKey, out var cached))
            return (FinancialStatements)cached;
        
        // 하루에 한 번만 API 호출
        var statements = await _httpClient.GetAsync(...);
        
        _cache.Set(cacheKey, statements, 
            new MemoryCacheEntryOptions 
            { 
                AbsoluteExpirationRelativeToNow = TimeSpan.FromDays(CacheDurationDays)
            });
        
        return statements;
    }
}
```

---

#### 2.2 KIS API - Connection Pooling + OAuth2

```
api.kis.kookmindbank.com/oauth2/tokenP
- Rate limit: 500 req/min per connection
- Auth: OAuth2 refresh token (1시간 유효)
- Strategy:
  1. Connection pool (3-5 concurrent)
  2. Token refresh (55분마다 자동)
  3. Queue by priority (BUY > SELL > CANCEL)
```

**구현:**
```csharp
public sealed class KisConnectionPool
{
    private readonly Channel<KisConnection> _pool;
    private readonly Timer _tokenRefreshTimer;
    
    public KisConnectionPool(int poolSize = 3)
    {
        _pool = Channel.CreateBounded<KisConnection>(poolSize);
        _tokenRefreshTimer = new Timer(RefreshTokens, null, TimeSpan.FromMinutes(55), TimeSpan.FromMinutes(55));
    }
    
    public async ValueTask<KisConnection> AcquireAsync(CancellationToken ct)
    {
        return await _pool.Reader.ReadAsync(ct);
    }
    
    public async ValueTask ReleaseAsync(KisConnection conn, CancellationToken ct)
    {
        await _pool.Writer.WriteAsync(conn, ct);
    }
}
```

---

### Phase 3: 장기 (1개월)

#### 3.1 Central Rate Limiter (RateLimitService)

```csharp
public sealed class RateLimiterService
{
    private readonly Dictionary<string, TokenBucket> _buckets = new();
    
    public async Task<bool> AllowAsync(string apiName, CancellationToken ct)
    {
        // apiName = "krx:ohlcv", "telegram:message", "opendart:financial", etc.
        var bucket = _buckets.GetOrAdd(apiName, _ => new TokenBucket(
            capacity: GetCapacity(apiName),      // 10 for KRX
            refillRate: GetRefillRate(apiName),  // 10/sec
            refillInterval: TimeSpan.FromSeconds(1)));
        
        return await bucket.TryConsumeAsync(1, ct);
    }
}

// Usage:
if (!await _rateLimiter.AllowAsync("krx:ohlcv", ct))
{
    _logger.LogWarning("KRX rate limit exceeded, queuing request");
    await _queue.EnqueueAsync(...);
}
```

**효과:**
- ✅ 모든 API 호출 중앙 관리
- ✅ Per-API quota 추적
- ✅ Fairness: 중요 작업 우선순위

---

#### 3.2 Circuit Breaker Pattern

```csharp
var policy = Policy
    .Handle<HttpRequestException>(ex => ex.StatusCode == 429)
    .OrResult<HttpResponseMessage>(r => r.StatusCode == System.Net.HttpStatusCode.TooManyRequests)
    .CircuitBreaker(
        handledEventsAllowedBeforeBreaking: 3,
        durationOfBreak: TimeSpan.FromMinutes(5),
        onBreak: (outcome, timespan) =>
        {
            _logger.LogError("KRX circuit breaker opened for {Duration}", timespan);
        });
```

---

## 3️⃣ 호출 시간 최적화

### Shadow Run 호출 스케줄

```
현재:      252일 × 1초씩 = ~4분 (순수 네트워크)
최적화 후: 30일 배치 × 9회 × 100ms = ~1초 (spacer)

개선율: 75% ↓
```

### Recommendation Reports 호출 스케줄

```
매일 09:00 KST:  1회 호출 (Daily 추천)
매주 토요일:      1회 호출 (Weekly 추천)
매월 1일:         1회 호출 (Monthly 추천)

Telegram 각: 1회 + 재시도 최대 3회
```

---

## 4️⃣ 호출 횟수 추적 (Observability)

```csharp
// Program.cs에 추가
services.AddSingleton<ApiCallMetricsService>();

// 메트릭 기록
_metrics.RecordApiCall("krx:ohlcv", success: true, latencyMs: 145, remainingQuota: 987);
_metrics.RecordApiCall("telegram:message", success: false, rateLimited: true, retryCount: 2);
```

**대시보드:**
```
KRX OpenAPI:
  - Daily calls: 9-15 (배치 호출)
  - Rate limit remaining: X/10000
  - 429 errors: 0
  
Telegram:
  - Queued: N messages
  - Sent: M/N (success rate)
  - Avg latency: Xms
  
OpenDart:
  - Calls today: X/1000
  - Cache hit: Y%
```

---

## 5️⃣ 구현 로드맵

| Phase | 항목 | 우선순위 | 소요시간 |
|-------|------|----------|----------|
| **Now** | KRX exponential backoff | P0 | 30m |
| **Now** | Telegram async queue | P1 | 45m |
| **Week** | DataBackfiller batch | P0 | 1h |
| **Week** | OpenDart daily batch | P1 | 45m |
| **2weeks** | KIS connection pool | P2 | 2h |
| **Month** | Central rate limiter | P2 | 3h |
| **Month** | Circuit breaker | P3 | 1h |

---

## 6️⃣ 검증 기준

- ✅ KRX: 252일 동안 429 에러 0회
- ✅ Telegram: 모든 ERROR/FATAL 알림 전달 (재시도 포함)
- ✅ OpenDart: 일일 1,000 quota 초과 안 함
- ✅ KIS: Connection pool 고갈 없음 (≤3 concurrent)

---

**다음:** Phase 1 구현 시작 (KRX exponential backoff + Telegram async queue)