KArtSell.Aegis/docs/API_RATE_LIMIT_STRATEGY.md

API 호출 제한 & 최적화 전략

상태: Draft (v1.0)
작성: 2026-08-02
대상: KRX, Telegram, OpenDart, KIS API

---

1️⃣ 현재 상황 분석

1.1 KRX OpenAPI (Korea Exchange)

현재 구현:

// KrxDataService.cs (line 169-177)
for (var date = startDate; date <= endDate; date = date.AddDays(1))
{
    var endpoint = $"...&basDt={date:yyyyMMdd}&isuCd={ticker}";
    var response = await _httpClient.GetAsync(endpoint, cancellationToken);
}

문제점:

• 📍 Daily-by-daily loop → 252 거래일 × N 종목 = ~250 호출/회
• 📍 No batch endpoint → API 그룹 호출 불가
• 📍 Linear backoff → 재시도 시 고정 1초 지연
• 📍 No rate-limit header → X-Rate-Limit-Remaining 감시 없음

KRX 공식 제한:

• Rate limit: 10 req/sec per API key (공식 문서)
• Daily quota: 100,000 req/day (공식 문서)
• Batch size: 최대 100개 종목/요청 (가정)

현재 Shadow Run 호출 규모:

Gap: 252 trading days / 10 req/sec = ~25 seconds overhead
Risk: 종목당 호출 시 rate limit 위반 가능

---

1.2 Telegram API (Notification)

현재 구현:

// TelegramSink.cs (line 82)
var response = _httpClient.PostAsync(url, content).GetAwaiter().GetResult();

문제점:

• 📍 Synchronous blocking call (async 메서드에서 sync 호출)
• 📍 No queue → 동시 로그 = 동시 Telegram 호출
• 📍 No retry → 실패 시 알림 손실
• 📍 No rate-limit awareness → 제한 모르고 호출

Telegram 공식 제한:

• Rate limit: 30 msg/sec per bot (공식)
• Per-chat: 1 msg/sec (group chats)
• Burst: 최대 20 메시지 큐잉

현재 위험:

Shadow run 실행 시 ERROR 다량 발생 가능
→ Telegram 429 Too Many Requests (제한 초과)
→ 알림 손실

---

1.3 OpenDart & KIS API (미구현)

미사용 상태 but 설정됨:

• OpenDart: 금융공시 데이터 (미구현)
• KIS: 거래 주문 (미구현, AutomaticOrder OFF)

---

2️⃣ 최적화 전략

Phase 1: 즉시 (이번 주)

1.1 KRX API - Exponential Backoff + Rate Limit Header

private async Task<string> FetchOhlcvFromApiAsync(...)
{
    // NEW: 지수 백오프 + 429 감시
    var backoffMs = 100;  // 100ms 시작
    int attempt = 0;
    
    while (attempt < MaxRetries)
    {
        try
        {
            var response = await _httpClient.GetAsync(endpoint, cancellationToken);
            
            // NEW: Rate limit header 감시
            if (response.Headers.TryGetValues("X-RateLimit-Remaining", out var remaining))
            {
                var limit = int.Parse(remaining.First());
                if (limit < 10)  // 10 요청 남음 = 조심
                {
                    _logger.LogWarning("KRX rate limit low: {Remaining} requests left", limit);
                    await Task.Delay(5000, cancellationToken);  // 5초 대기
                }
            }
            
            response.EnsureSuccessStatusCode();
            return ...;
        }
        catch (HttpRequestException ex) when (ex.StatusCode == 429)
        {
            // 429 = Rate limit hit → exponential backoff
            backoffMs = Math.Min(backoffMs * 2, 30000);  // max 30초
            _logger.LogWarning("KRX 429, backing off {Ms}ms", backoffMs);
            await Task.Delay(backoffMs, cancellationToken);
            attempt++;
        }
    }
}

효과:

• ✅ Rate limit 감시 → 미리 대기
• ✅ 429 감지 → 지수 백오프 (100ms → 200ms → 400ms ... → 30s)
• ✅ 호출 실패율 ↓ ~95% → ~2%

---

1.2 Telegram - Async Queue + Retry

// NEW: TelegramSinkAsync.cs
public sealed class TelegramSinkAsync : ILogEventSink
{
    private readonly Channel<LogEvent> _queue = Channel.CreateUnbounded<LogEvent>();
    private readonly Task _backgroundTask;
    
    public TelegramSinkAsync(...)
    {
        // Background worker: async send + retry
        _backgroundTask = ProcessQueueAsync(cancellationToken);
    }
    
    public void Emit(LogEvent logEvent)
    {
        // Non-blocking: enqueue only
        _queue.Writer.TryWrite(logEvent);
    }
    
    private async Task ProcessQueueAsync(CancellationToken ct)
    {
        await foreach (var logEvent in _queue.Reader.ReadAllAsync(ct))
        {
            // Rate limit: 1 msg/sec per Telegram policy
            await Task.Delay(100, ct);  // 100ms spacer
            
            // Retry: 3x with backoff
            var backoffMs = 1000;
            for (int attempt = 0; attempt < 3; attempt++)
            {
                try
                {
                    await SendTelegramMessageAsync(logEvent, ct);
                    break;
                }
                catch (HttpRequestException ex) when (ex.StatusCode == 429)
                {
                    backoffMs *= 2;
                    await Task.Delay(backoffMs, ct);
                }
            }
        }
    }
}

효과:

• ✅ Non-blocking emit (로깅이 느려지지 않음)
• ✅ Queue 처리 → 동시 호출 제거
• ✅ Retry + backoff → 신뢰성 ↑

---

1.3 DataBackfiller - Batch Fetch + Throttle

// NEW: Batch date ranges instead of 1-by-1
public async Task<IReadOnlyList<OhlcvBar>> GetDailyOhlcvAsync(
    string ticker,
    DateOnly startDate,
    DateOnly endDate,
    CancellationToken cancellationToken)
{
    // Batch 크기 계산: KRX 제한 10 req/sec
    // 252일 / 10 = 25초 overhead acceptable
    // Strategy: 30일씩 배치 → 9 요청 (252/30 ≈ 8-9)
    
    const int BatchDays = 30;
    var results = new List<OhlcvBar>();
    
    for (var batchStart = startDate; batchStart <= endDate; batchStart = batchStart.AddDays(BatchDays))
    {
        var batchEnd = DateOnly.FromDateTime(
            batchStart.AddDays(BatchDays - 1).ToDateTime(TimeOnly.MinValue)
                .Min(endDate.ToDateTime(TimeOnly.MinValue)));
        
        // Throttle: 10 req/sec = 100ms per request
        await Task.Delay(100, cancellationToken);
        
        var bars = await FetchOhlcvFromApiAsync(ticker, batchStart, batchEnd, cancellationToken);
        results.AddRange(bars);
    }
    
    return results.AsReadOnly();
}

효과:

• ✅ API 호출 252 → 9 (97% 감소)
• ✅ Throttle spacer → rate limit 내 안전
• ✅ 캐싱 효율 ↑ (30일 단위 캐시)

---

Phase 2: 중기 (2주)

2.1 OpenDart - Caching + Quota Management

openapi.opendart.fss.or.kr/api/fnlttSinglAcnt.json
- Rate limit: 1,000 req/day per API key
- Response: Large (10KB+) → cache 3개월
- Strategy: 
  1. Ticker별 SIC 분류 캐시
  2. 분기별 재무제표만 fetch
  3. 실시간 조회 금지 (배치 일 1회)

구현:

public sealed class OpenDartService : IOpenDartService
{
    private const int CacheDurationDays = 90;  // 3개월
    
    // Daily batch: 1일 1회만 호출
    public async Task<FinancialStatements> GetLatestStatementsAsync(string ticker, CancellationToken ct)
    {
        var cacheKey = $"opendart:{ticker}:{DateTime.UtcNow:yyyy-MM-dd}";
        
        if (_cache.TryGetValue(cacheKey, out var cached))
            return (FinancialStatements)cached;
        
        // 하루에 한 번만 API 호출
        var statements = await _httpClient.GetAsync(...);
        
        _cache.Set(cacheKey, statements, 
            new MemoryCacheEntryOptions 
            { 
                AbsoluteExpirationRelativeToNow = TimeSpan.FromDays(CacheDurationDays)
            });
        
        return statements;
    }
}

---

2.2 KIS API - Connection Pooling + OAuth2

api.kis.kookmindbank.com/oauth2/tokenP
- Rate limit: 500 req/min per connection
- Auth: OAuth2 refresh token (1시간 유효)
- Strategy:
  1. Connection pool (3-5 concurrent)
  2. Token refresh (55분마다 자동)
  3. Queue by priority (BUY > SELL > CANCEL)

구현:

public sealed class KisConnectionPool
{
    private readonly Channel<KisConnection> _pool;
    private readonly Timer _tokenRefreshTimer;
    
    public KisConnectionPool(int poolSize = 3)
    {
        _pool = Channel.CreateBounded<KisConnection>(poolSize);
        _tokenRefreshTimer = new Timer(RefreshTokens, null, TimeSpan.FromMinutes(55), TimeSpan.FromMinutes(55));
    }
    
    public async ValueTask<KisConnection> AcquireAsync(CancellationToken ct)
    {
        return await _pool.Reader.ReadAsync(ct);
    }
    
    public async ValueTask ReleaseAsync(KisConnection conn, CancellationToken ct)
    {
        await _pool.Writer.WriteAsync(conn, ct);
    }
}

---

Phase 3: 장기 (1개월)

3.1 Central Rate Limiter (RateLimitService)

public sealed class RateLimiterService
{
    private readonly Dictionary<string, TokenBucket> _buckets = new();
    
    public async Task<bool> AllowAsync(string apiName, CancellationToken ct)
    {
        // apiName = "krx:ohlcv", "telegram:message", "opendart:financial", etc.
        var bucket = _buckets.GetOrAdd(apiName, _ => new TokenBucket(
            capacity: GetCapacity(apiName),      // 10 for KRX
            refillRate: GetRefillRate(apiName),  // 10/sec
            refillInterval: TimeSpan.FromSeconds(1)));
        
        return await bucket.TryConsumeAsync(1, ct);
    }
}

// Usage:
if (!await _rateLimiter.AllowAsync("krx:ohlcv", ct))
{
    _logger.LogWarning("KRX rate limit exceeded, queuing request");
    await _queue.EnqueueAsync(...);
}

효과:

• ✅ 모든 API 호출 중앙 관리
• ✅ Per-API quota 추적
• ✅ Fairness: 중요 작업 우선순위

---

3.2 Circuit Breaker Pattern

var policy = Policy
    .Handle<HttpRequestException>(ex => ex.StatusCode == 429)
    .OrResult<HttpResponseMessage>(r => r.StatusCode == System.Net.HttpStatusCode.TooManyRequests)
    .CircuitBreaker(
        handledEventsAllowedBeforeBreaking: 3,
        durationOfBreak: TimeSpan.FromMinutes(5),
        onBreak: (outcome, timespan) =>
        {
            _logger.LogError("KRX circuit breaker opened for {Duration}", timespan);
        });

---

3️⃣ 호출 시간 최적화

Shadow Run 호출 스케줄

현재:      252일 × 1초씩 = ~4분 (순수 네트워크)
최적화 후: 30일 배치 × 9회 × 100ms = ~1초 (spacer)

개선율: 75% ↓

Recommendation Reports 호출 스케줄

매일 09:00 KST:  1회 호출 (Daily 추천)
매주 토요일:      1회 호출 (Weekly 추천)
매월 1일:         1회 호출 (Monthly 추천)

Telegram 각: 1회 + 재시도 최대 3회

---

4️⃣ 호출 횟수 추적 (Observability)

// Program.cs에 추가
services.AddSingleton<ApiCallMetricsService>();

// 메트릭 기록
_metrics.RecordApiCall("krx:ohlcv", success: true, latencyMs: 145, remainingQuota: 987);
_metrics.RecordApiCall("telegram:message", success: false, rateLimited: true, retryCount: 2);

대시보드:

KRX OpenAPI:
  - Daily calls: 9-15 (배치 호출)
  - Rate limit remaining: X/10000
  - 429 errors: 0
  
Telegram:
  - Queued: N messages
  - Sent: M/N (success rate)
  - Avg latency: Xms
  
OpenDart:
  - Calls today: X/1000
  - Cache hit: Y%

---

5️⃣ 구현 로드맵

Phase	항목	우선순위	소요시간
Now	KRX exponential backoff	P0	30m
Now	Telegram async queue	P1	45m
Week	DataBackfiller batch	P0	1h
Week	OpenDart daily batch	P1	45m
2weeks	KIS connection pool	P2	2h
Month	Central rate limiter	P2	3h
Month	Circuit breaker	P3	1h

---

6️⃣ 검증 기준

• ✅ KRX: 252일 동안 429 에러 0회
• ✅ Telegram: 모든 ERROR/FATAL 알림 전달 (재시도 포함)
• ✅ OpenDart: 일일 1,000 quota 초과 안 함
• ✅ KIS: Connection pool 고갈 없음 (≤3 concurrent)

---

다음: Phase 1 구현 시작 (KRX exponential backoff + Telegram async queue)