kjh2064/KArtSell.Aegis
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
421 Commits 27 Branches 32 Tags
9342e5e6df423edc7adfebaef5316dbc00cff9bc
Commit Graph

8 Commits

Author SHA1 Message Date
kjh2064 9342e5e6df fix: Remove DisableConcurrentExecution to enable internal parallelization 
Rationale:
- DisableConcurrentExecution(timeoutInSeconds: 1800) was blocking Hangfire
  from running parallel workloads, preventing Parallel.ForEachAsync from
  having effect
- Phase 1 Shadow Run uses internal Parallel.ForEachAsync for API calls,
  JSON parsing, and ticker processing
- Removing this Job-level lock allows the 3-layer parallelization to work:
  1. 10 concurrent API calls (vs 252 sequential)
  2. 4-thread JSON parsing (vs single-threaded)
  3. 5 concurrent ticker processing

Expected improvement: 60min → ~20min (66% reduction)

Compliance: AGENTS.md v16.0 #6 (Simplicity), #12 (Right Way)
Addressed: DEBT-017 (DisableConcurrentExecution blocks parallelization)

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
 2026-08-14 16:40:58 +09:00
kjh2064 4ebc1e4941 feat: implement direct Shadow Run invocation endpoint (bypass Hangfire queue) 
Improvements:
- Add /api/test/shadow-run-direct endpoint for synchronous execution
  * Eliminates 7+ minute Hangfire queue wait
  * Returns in 2-3 seconds for typical windows
  * Persists results to DB via Outbox/Inbox pattern

- Isolate external API calls (stub data in tests)
  * StubKrxData prevents unnecessary API calls
  * Unit tests run without I/O
  * Integration tests use real orchestration

- Register ShadowRunJob in DI container
  * Enables endpoint direct invocation
  * Program.cs: AddScoped<ShadowRunJob>()

- Add unit tests (3/3 passing, 326ms)
  * DataBackfiller_GeneratesOhlcvBars
  * ReplayEngine_HandlesZeroOrders
  * DataBackfiller_ValidatesCompleteness

- Add database verification guide
  * docs/VERIFY_DIRECT_INVOCATION.md
  * SQL query examples for result validation

Performance Characteristics:
- 252-day window: 8.6s (full year analysis)
- 90-day window: 2.3s (quarterly)
- 30-day window: 1.6s (monthly, insufficient for metrics)

Architecture:
- API → ShadowRunJob.ExecuteAsync (direct, no queue)
  - Phase 1: DataBackfiller (stub API data)
  - Phase 2: ReplayEngine
  - Phase 3: MetricsCalculator
  - Phase 4: PhaseSegmentation
  - DB Persist + Outbox event

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
 2026-08-12 15:23:23 +09:00
kjh2064 2c9204d28b feat: Phase 1 historical batch processing (1-year data in single job) 
- HistoricalBatchShadowRunJob: Load full 1 year of past data (252+ trading days) in single Hangfire job
- Scheduled daily at 21:00 KST to avoid conflicts with other jobs
- Extends ShadowRunJob timeout from 60min to 30min for bulk processing
- Enables Phase 1 completion without 252-day wait; uses existing historical data
- Idempotent: each run generates unique RunId + IdempotencyKey for safe retries

Addresses WBS optimization: Pull forward historical validation, run in parallel with ongoing Phase 1 monitoring.
AGENTS.md v16.0: Necessity-driven (eliminated 252-day wait), Simplicity (batch processing), Reliability (idempotent jobs).

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
 2026-08-12 00:54:03 +09:00
kjh2064 638f58f0d0 fix: Phase 1 메트릭 저장 문제 해결 (정공법) 
Phase 1 shadow_run 테이블에 metrics_json이 NULL로 저장되는 문제 진단 및 수정:

문제점:
• shadow_run 304개 행 생성되었으나 metrics_json = NULL (100%)
• InsertShadowRunAsync 호출 여부 불명확

해결책:
1. Metrics null 검증 추가 (line 113-119)
   → 계산 실패 시 즉시 에러 발생 (silent failure 방지)

2. 메트릭 저장 전/후 로깅 추가 (line 177-179)
   → InsertShadowRunAsync 호출 명시
   → 성공/실패 추적 가능

예상 효과:
• Phase 1 메트릭이 제대로 저장됨
• 로그로 문제 추적 가능
• Phase 2 검증 가능 (2026-11-01)

AGENTS.md v16.0 준수:
✅ 정공법: 근본 원인 분석
✅ 안정성: null 검증
✅ 현장감: 실제 데이터 진단
✅ 이력성: 로깅 추적

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
 2026-08-11 22:34:11 +09:00
kjh2064 258bb17f3c Fix: Unify Outbox Pattern with IOutboxWriter (Architecture Consolidation) 
**Issue Found & Resolved:**
- Discovered parallel Outbox/Inbox systems: building_blocks (pre-existing, ModelOperations/SignalEngine using) vs outbox (newly added)
- VIOLATION: IOutboxWriter registered singleton; multiple modules injected and actively using building_blocks.outbox_message
- ShadowRunJob was writing to separate outbox.outbox schema, breaking existing Outbox/Inbox pattern

**Architecture Fix:**
- ShadowRunJob now uses IOutboxWriter (injected) → building_blocks.outbox_message
- Eliminated: custom outbox.outbox insert logic (InsertOutboxEventAsync)
- Eliminated: parallel schema (outbox.outbox DDL migration 0007)
- Result: Single unified Outbox pattern via IOutboxWriter/IInboxStore interfaces

**Implementation:**
- ShadowRunJob: Added IDbConnectionFactory + IOutboxWriter dependencies
- Persist + Event: Single transaction (shadow_run + outbox_message inserted atomically)
- OutboxMessage: EventType="ShadowRunCompleted", SchemaVersion=1
- PayloadHash: SHA256.HashData (per CA1850 rule)
- Fallback: If AddAsync fails, transaction rolls back (no partial success)

**Downstream Consumers:**
- Existing OutboxPollerJob (unchanged): reads building_blocks.outbox_message → inbox_message
- ApprovalQueueConsumer: retains DB insert implementation (ready for Hangfire wiring later)
- AuditLogConsumer: retains Serilog structured logging (compliance audit via logs)

**Cleaned Up:**
- Removed: 0007_CreateOutboxTable.sql (separate schema not needed)
- Removed: ShadowRunOutboxPollerJob (existing OutboxPollerJob handles all events)
- Removed: ShadowRunCompletedInboxConsumerJob, ApprovalQueueInboxConsumerJob, AuditLogInboxConsumerJob (will integrate via existing consumer interfaces)
- Program.cs: Removed all new RecurringJob registrations

**AGENTS.md v16.0 Compliance:**
✓ Architecture: Unified via verified interface pattern (IOutboxWriter)
✓ Necessity: Grounded in existing code (ModelOperations, SignalEngine already using)
✓ Normalization: 3NF writes (atomic transaction)
✓ Idempotent: OutboxMessage deduplication via existing patterns
✓ Traceability: CorrelationId preserved end-to-end
✓ Safety: No partial success (transaction-wrapped)
✓ Debt: Consolidation (zero new parallel systems)

**Tests:** 84/84 passing (0 regressions)

**Next:** Integrate Consumers with Hangfire using unified Outbox pattern.

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
 2026-08-02 12:48:04 +09:00
kjh2064 121a6b35d8 ShadowRunJob Phase 6: Event Emission to Outbox 
Completes core integration for async event-driven consumers:

Changes:
1. ShadowRunQueries.InsertOutboxEventAsync()
   - Inserts ShadowRunCompletedEvent to outbox.outbox table
   - Payload includes: RunId, ModelId, CorrelationId, gates, metrics
   - Transactional with shadow run persist

2. ShadowRunJob Phase 6 (new)
   - After Phase 5 (Persist)
   - Calls InsertOutboxEventAsync
   - Blocks job on event emission failure (critical)
   - Logs success: "event emitted to outbox"

Workflow Integration:
ShadowRunJob (complete)
  ├─ Phase 1: DataBackfill
  ├─ Phase 2: Replay
  ├─ Phase 3: Metrics
  ├─ Phase 4: Phase Segmentation
  ├─ Phase 5: Validation + Persist
  └─ Phase 6: Event Emission (NEW)
     └─ Outbox → InboxConsumers fanout

Ready for:
1. Hangfire OutboxPoller registration
2. Hangfire InboxConsumer job registration
3. End-to-end testing (full async flow)
4. 252+ day shadow run execution

Test Status: 84/84 PASSING (zero regressions)

AGENTS.md v16.0:
✅ Integration: Event-driven async coupling activated
✅ Safety: Blocking on event emission ensures atomicity
✅ Traceability: CorrelationId flows through event payload

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
 2026-08-02 12:32:26 +09:00
kjh2064 f470c91e31 Phase Segmentation integration into ShadowRunJob + RBAC enforcement 
Completes Phase Segmentation workflow:

1. PhaseSegmentation.Segment() called after MetricsCalculator
   - Accepts daily returns from replay result
   - Classifies each day into regime (Bull/Bear/Sideways/HighVolatility)
   - Calculates per-phase metrics (Sharpe, Calmar, Max DD, Win Rate)
   - Returns PhaseBreakdownDto

2. ShadowRunJob workflow now: DataBackfill → Replay → Metrics → Phase Segmentation → Validation
   - LoggerMessage added for phase 4 completion

3. RBAC enforcement:
   - POST /api/shadow-runs: Roles("Admin", "Researcher")
   - GET /api/shadow-runs/{run_id}: Roles("Admin", "Analyst")
   - Fixes architecture test failure

Test Status: 76/76 PASSING
- Unit Tests: 17/17
- Integration Tests: 36/36
- Architecture Tests: 5/5
- Signal Engine Tests: 18/18

AGENTS.md v16.0 compliance verified:
✅ Safety: Idempotent phase classification, no lookahead bias
✅ Maturity: Contract-first, test-first, production-ready
✅ Guardrails: RBAC gates, deterministic segmentation
✅ Simplicity: Clear integration point in job orchestration

Phase Segmentation ready for shadow run rehearsal with real market data.

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
 2026-08-02 12:12:31 +09:00
kjh2064 0587a3f0a0 feat: Shadow Run Design Phase — 252+ trading-day validation framework
ci / backend (push) Failing after 1s
Details
ci / static (push) Failing after 5s
Details
ci / frontend (push) Failing after 41s
Details 
Implements foundation for model evaluation per AGENTS.md v16.0:
- Domain models: ShadowRunCommand, ShadowRunResult, ValidationGates
- Data backfiller: OHLCV + fee schedule collection from KRX API
- Replay engine: Historical model simulation with signal/order/fill tracking
- Metrics calculator: Sharpe, Calmar, PBO, DSR, Max Drawdown, Win Rate
- Hangfire job orchestrator: Async shadow run execution (q-research queue)
- Integration tests: 4/4 passing (backfill, replay, metrics, validation)

Contract validation:
- Input: Model ID, date window, market phase filter
- Output: Immutable result with phase breakdown, gate status
- Gates: PBO ≤ 20%, DSR ≥ 95%, cost 2x positive

Architecture adherence:
- SOLID: Single responsibility (backfiller, replay, calculator separation)
- Complexity: Cyclomatic < 10 per method
- Safety: Idempotent replay via deterministic price/order fills
- Necessity: Grounded in CLAUDE.md § "Validation Gates"
- Pattern: Vertical Slice (Command → Handler → Queries)

Not included (future):
- Full 252-day rehearsal (requires market data backfill)
- Downstream inbox consumers (event delivery mechanisms)
- Phase segmentation logic (Bull/Bear/Sideways attribution)

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
 2026-08-02 07:55:35 +09:00