QuantEngineByItz/CLAUDE.md

# CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

## Project Overview

**QuantEngine v0.1** — A comprehensive quantitative analysis and data collection system for retirement asset portfolio management.

- **Architecture**: .NET 9 + C# (web UI + APIs), Python (legacy data collection/analysis)
- **Web UI**: Blazor Interactive WebAssembly (MudBlazor) + ASP.NET Core Web API (API-First)
- **Database**: PostgreSQL (Npgsql 8.0), single unified database
- **Data Source**: KIS Open API (quotations/ranking read-only), with fallbacks
- **Key Runtimes**: .NET 9, Python 3.9+, Node.js 16+

### Migration Phases Status (2026-07-11)

**Phase 1: Web UI Migration** ✅ 완료 (2026-07-11)
- **새로운 표준**: Razor Pages (Server-Rendered) + Cookie Authentication + Tabler UI
- **폐기 대상**: Blazor Interactive WebAssembly, MudBlazor, SmartAdmin
- **완료 기준 — Phase 1 Success Criteria**:
  - ✅ Cookie 인증 구현 (AuthService + IpLockoutService + BCrypt)
  - ✅ Razor Pages 렌더링 (Admin 레이아웃 + 3개 이상 기본 페이지)
  - ✅ 공용 UI 컴포넌트 (4개 이상 shared partials)
  - ✅ 보안: 백도어 제거, 무솔트 해시 마이그레이션, IP 잠금
  - ✅ 빌드 성공: 0 errors, 0 warnings
  - ✅ CLAUDE.md 업데이트 (UI 기준 + 인증 정책)
  - **✅ 모든 기준 충족됨** (2026-07-11)
- **구현 완료**:
  - ✅ Cookie 기반 인증 (AuthService + IpLockoutService)
  - ✅ Razor Pages CRUD 레이아웃 (_AdminLayout.cshtml, shared partials)
  - ✅ Admin 페이지: Dashboard, Collection, Users (기본 구조)
  - ✅ 공용 UI 컴포넌트: _ValidationSummary, _Pagination, _StatusBadge, _EmptyState
  - ✅ 보안 개선: BCrypt 해싱, IP 잠금, 하드코딩된 백도어 제거
  - ✅ 빌드: 0 errors, 0 warnings (Newtonsoft.Json 보안 경고 제외)
  - ✅ CLAUDE.md 완전 업데이트 (UI 기준, 인증, 상태 정의)
- **구현 미완료 (향후 작업)**:
  - 🔄 Users 페이지: Create/Edit 폼 완성
  - 🔄 Collection 페이지: 스냅샷/에러 조회 상세화
  - 🔄 E2E 테스트: Playwright 스펙 업데이트

**Phase 2: KIS Data Collection Pipeline** ✅ 95% COMPLETE
- ✅ KIS API Client: Full implementation complete
  - IKisApiClient interface (5 quotation methods)
  - KisApiClient with real HTTP implementation + token caching
  - All governance rules enforced (no trading APIs)
  - Windows env var + registry fallback for credentials
  - Build: 0 errors, 0 warnings
- ✅ PostgreSQL Infrastructure: Complete
  - PostgresTokenCache (token management, 10-min skew)
  - CollectionRepository (full CRUD + dashboard aggregations)
  - Auto-creates kis_tokens, kis_collection_runs, kis_collection_snapshots, kis_collection_errors
  - Dapper ORM + parameterized SQL (injection-proof)
- ✅ Web API Endpoints: Complete
  - CollectionEndpoints (6 endpoints: state, runs, snapshots, errors, latest, start)
  - ApiClient for Blazor consumption
- ✅ Blazor UI: Complete
  - Collection.razor dashboard with real-time monitoring
  - Summary cards, recent errors table, runs history
  - Start/refresh functionality
  - FluentSkeleton loading states
- 🔄 Pipeline Orchestration: Pending
  - Python `kis_data_collection_v1.py` → .NET (data fetching + validation)
  - Real KIS API data collection workflow integration
  - E2E test: API → DB → UI validation

**Phase 3: Node.js→.NET CLI Tools** 📋 PLANNED
- Makefile created (npm → make mappings)
- np operations documented

**Phase 4: CI/CD Pipeline Hardening** ✅ 80% COMPLETE (2026-07-11)
- ✅ deploy-prod.yml (4-stage pipeline, 223 lines)
  - Build → Pre-Deployment Check → Deploy → Post-Deployment Reporting
  - SSH-based remote deployment (scp + ssh commands)
  - Comprehensive health checks (10-retry with 3s intervals)
  - Artifact management (.tar.gz)
- ✅ Workflow consolidation (2 active files)
  - ci.yml: PR validation only (maintains 29 validators)
  - deploy-prod.yml: Production deployment
  - Deleted: merge-to-main.yml (non-functional), fast-validation.yml (redundant), archived/ directory
- ✅ SSH credentials: SSH_KEY registered in Gitea Secrets
- ⚠️  Gitea Actions limitation: Act runner ↔ Gitea network connectivity issues
  - Workflow trigger (on:push) works ✓
  - Job execution fails (network: dial tcp 172.18.0.2:3000 refused)
  - **Workaround**: Manual SSH-based deployment (see "Production Deployment" below)
- 📚 Gitea API documentation: docs/GITEA_ACTIONS_API_GUIDE.md

**Phase 5: Admin UI & Deployment Optimization** ✅ COMPLETE (2026-07-11)
- ✅ Admin UI redesign (Tabler framework)
  - Dashboard: stat cards, quick actions, system info
  - Responsive sidebar navigation
  - Professional layout (dark sidebar #2c3e50, white content)
- ✅ Build output: 0 errors, 0 warnings
- ✅ E2E tests: 8/8 passing (Playwright)
- ✅ Production deployment: Active since 2026-07-11 21:00:55 KST
  - Commit: 30fb702
  - HTTP 200 health check
  - Service: active (running)

**Status Summary**:
- Python codebase: Operational (1,140 files)
- .NET 9 coverage: Core (✅), Infrastructure (✅), API (✅), Web UI (✅)
- Database: PostgreSQL fully migrated
- CI/CD: Manual SSH deployment (fully operational), Gitea Actions (limited by infrastructure)
- Release gates: Python gates remain authority until Phase 2 integration testing complete

## Deployment & Operations (Phase 4-5, 2026-07-11)

**Production Server**: Hetzner Cloud `178.104.200.7` (kjh2064@178.104.200.7)

Projects on server:
1. **TaxBaik** (홈페이지) — Nginx location `/taxbaik`
2. **QuantEngine** (데이터 수집/분석) — Nginx location `/quantengine`

### Production Deployment Strategy (Manual SSH-Based)

**Current Status**: Gitea Actions automated deployment limited by infrastructure constraints. Deployed via stable manual SSH pipeline.

**Pre-Deployment Checklist**:
1. ✅ Local build: `dotnet build src/dotnet/QuantEngine.Web/QuantEngine.Web.csproj -c Release`
2. ✅ E2E tests pass: `npx playwright test`
3. ✅ Admin pages verified (200 status, no 500 errors)
4. ✅ Commit to main branch: `git push origin main`

**Deployment Procedure** (Manual SSH):

```powershell
# 1. SSH into production server
ssh kjh2064@178.104.200.7

# 2. Navigate to deployment directory
cd ~/deployments

# 3. Run deployment script (or manual steps below)
./deploy.sh

# OR manual deployment:
# ─────────────────────
# 3a. Build release artifact locally, then SCP to server:
dotnet publish src/dotnet/QuantEngine.Web/QuantEngine.Web.csproj -c Release -o ./publish
tar -czf quantengine-release.tar.gz -C ./publish .
scp quantengine-release.tar.gz kjh2064@178.104.200.7:/tmp/

# 3b. On production server, extract and deploy:
mkdir -p ~/deployments/quantengine_$(date +%Y%m%d_%H%M%S)_$(git rev-parse --short HEAD)
tar -xzf /tmp/quantengine-release.tar.gz -C $DEPLOY_DIR
ln -sfn $DEPLOY_DIR ~/quantengine_active
systemctl restart quantengine

# 4. Verify deployment
curl http://127.0.0.1:5000/Account/Login
systemctl status quantengine
journalctl -u quantengine -n 20
```

**Monitoring Post-Deployment**:
```powershell
# Check service status
systemctl status quantengine.service

# View live logs
journalctl -u quantengine.service -f

# Verify active deployment
readlink ~/quantengine_active

# Health check (HTTP)
curl -I http://127.0.0.1:5000/Account/Login
```

### Rollback Procedure

```bash
# List recent deployments
ls -lht ~/deployments/quantengine_* | head -10

# Rollback to previous deployment
PREV_DEPLOY=$(ls -dt ~/deployments/quantengine_* | head -2 | tail -1)
ln -sfn $PREV_DEPLOY ~/quantengine_active
systemctl restart quantengine

# Verify
systemctl status quantengine
curl http://127.0.0.1:5000/Account/Login
```

### Gitea Actions (Limited - For Reference)

**Status**: Workflow trigger works (on:push detected), but Act runner cannot execute jobs due to Docker network constraints.

**Workaround**: Use manual SSH deployment (above). Gitea Actions configuration is prepared in:
- `.gitea/workflows/deploy-prod.yml` (4-stage pipeline, ready)
- `docs/GITEA_ACTIONS_API_GUIDE.md` (API reference for monitoring)

**API Monitoring** (when Actions are operational):
```powershell
$token = $env:GITEA_TOKEN_TAXBAIK
$response = Invoke-WebRequest `
    -Uri "https://gitea.taxbaik.com/api/v1/repos/kjh2064/QuantEngineByItz/actions/runs?limit=5" `
    -Headers @{ "Authorization" = "token $token" }
($response.Content | ConvertFrom-Json).workflow_runs | ForEach-Object {
    Write-Host "Run #$($_.id): $($_.display_title) [$($_.conclusion)]"
}
```

See `docs/GITEA_ACTIONS_API_GUIDE.md` for complete API documentation.

### Git Repository

**Gitea Server** (동일 호스트):
- **HTTP**: `https://gitea.taxbaik.com/kjh2064/QuantEngineByItz.git`
- **SSH**: `ssh://git@gitea.taxbaik.com:2222/kjh2064/QuantEngineByItz.git`

## UI Design Principles (2026-07-11 — Migrated to Razor Pages)

### Framework & Design System (NEW — 2026-07-11)

- **Primary Framework**: ASP.NET Core Razor Pages + Bootstrap 5 + Tabler UI
- **Design System**: Tabler (Bootstrap 5 기반), 밀집 레이아웃 + 전통 서버 렌더링
- **Render Mode**: **Server-side Razor Pages** — 모든 Admin UI는 서버에서 렌더링, Cookie 기반 인증 (API-First WASM 폐기)
- **Authentication**: Cookie Authentication (HttpOnly) + BCrypt password hashing + IP lockout (3 strikes, 15-min)
- **Deprecation**: **Blazor Interactive WebAssembly 폐기**, **MudBlazor 컴포넌트 폐기** (2026-07-11), **SmartAdmin 폐기**. 기존 WASM 코드는 `/QuantEngine.Web.Client` 폴더에 참고용으로만 보관 (`.sln`에서 제외)

### Component Development Rules (NEW)

1. **All Admin UI Development** (New + Refactored):
   - Use **Razor Pages** (.cshtml + .cshtml.cs PageModel) exclusively for admin
   - UI는 Repository/Service를 생성자 DI로 직접 호출 (API 홉 없음)
   - Bootstrap 5 + Tabler UI CSS classes for styling
   - **Form Validation**: DataAnnotations DTO + FluentValidation IValidator<T> 이중 검증
   - HTML `<form>` + tag helpers (`asp-for`, `asp-action`, `asp-page`)

2. **Authentication & Authorization**:
   - Cookie name: `QuantEngine.Admin.Auth` (HttpOnly, SameSite=Lax)
   - Session duration: 12 hours (sliding expiration)
   - Folder-level `[Authorize]` via `AuthorizeFolder("/Admin")` convention (per-page 반복 금지)
   - Login: `/Account/Login` (Razor Page, NO WASM)
   - Password: BCrypt-hashed (auto-migrates existing SHA-256 hashes on first login)
   - IP Lockout: 3 failed attempts → 15-minute lockout

3. **Data & Form Patterns**:
   - PageModel constructor: `public IndexModel(IWorkspaceRepository repo, ILogger<IndexModel> logger)`
   - Form submission: `OnPostAsync()` / `OnPostDeleteAsync()` (multi-handler pattern)
   - Validation failures: return `Page()` (re-render with ModelState errors)
   - Pagination: `PaginationModel` record (Page, TotalPages, Func<int,string> BuildPageUrl)
   - Empty states: `<PartialView name="_EmptyState" model="message" />`

4. **Component Mapping** (Bootstrap 5 + Tabler):

| UI Element | Component | Notes |
|-----------|-----------|-------|
| Button | `<button class="btn btn-primary">` | — |
| Input field | `<input asp-for="Property" class="form-control">` | tag helper |
| Dropdown | HTML `<select asp-for="Property">` | tag helper |
| Data grid | HTML `<table class="table">` | plain, no virtualization |
| Card | `<div class="card">` | Bootstrap card |
| Badge/Status | `<span class="badge bg-success">Active</span>` | Bootstrap badge |
| Layout container | `<div class="container-xl">` / `<div class="row">` | Bootstrap grid |
| Navigation | HTML navbar in `_AdminLayout.cshtml` | sidebar + topbar |
| Loading | N/A (server-rendered) | no loading states needed |
| Icons | Bootstrap Icons (`<i class="bi bi-*"></i>`) | CDN |
| Modal/Dialog | Bootstrap modal or inline `confirm()` | avoid unnecessary modals |
| Validation msg | `<span asp-validation-for="Property" class="d-block alert alert-danger mt-2">` | tag helper |

## Development Commands (Phase 1 + 2)

### Python / Node.js (Legacy & Release Gates)
```powershell
npm install
npm run ops:validate              # Warn-only validation
npm run full-gate                 # Strict validation (all gates PASS)
npm run ops:data-collect          # KIS collection (Python subprocess)
npm run ops:release               # Full release DAG
```

### .NET (Primary - Phase 1 + 2)
```powershell
cd src/dotnet
dotnet restore
dotnet build                      # Debug build (0 errors, 0 warnings)
dotnet build -c Release           # Release build
dotnet watch run --project QuantEngine.Web  # Hot-reload (http://localhost:5265)
dotnet run --project QuantEngine.Web        # Run API server
```

### Collection Pipeline Testing (Phase 2)
```powershell
# Set KIS credentials (sandbox account)
$env:KIS_APP_Key_TEST = "your_kis_test_key"
$env:KIS_APP_Secret_TEST = "your_kis_test_secret"

# Start web server (http://localhost:5265)
dotnet run --project QuantEngine.Web

# Verify Collection dashboard
# Navigate to http://localhost:5265/collection
# - Click "Start Collection" to trigger async run
# - Backend uses PostgreSQL-backed data storage
# - Dashboard updates with run status, snapshots, errors

# Verify API endpoints
curl http://localhost:5265/api/collection/state
curl http://localhost:5265/api/collection/runs
curl "http://localhost:5265/api/collection/latest/005930"
```

## API Endpoints (Phase 1 + 2)

### Workspace & History (Phase 1)
All endpoints prefixed with `/api/`:

| Route | Purpose |
|-------|---------|
| `GET /state` | Full UI state snapshot |
| `GET /tables` | Browsable tables list |
| `GET /table-rows` | Paginated rows |
| `POST /settings/save` | Save settings |
| `POST /account-snapshot/save` | Save snapshots |
| `POST /bootstrap` | Seed DB from JSON |
| `POST /account-snapshot/import-tsv` | Import TSV |
| `POST /autofix` | Auto-correct data |

### Collection Pipeline (Phase 2)
| Route | Purpose |
|-------|---------|
| `GET /collection/state` | Dashboard summary (runs, snapshots, errors) |
| `GET /collection/runs` | Recent collection runs (paginated) |
| `GET /collection/runs/{runId}/snapshots` | Snapshots from a run |
| `GET /collection/runs/{runId}/errors` | Errors from a run |
| `GET /collection/latest/{ticker}` | Latest snapshots for ticker |
| `POST /collection/run` | Start new collection run (async) |

### Collection Run Status Values
| Status | Meaning | UI Badge | Transitions |
|--------|---------|----------|------------|
| `running` | Collection in progress | <span class="badge bg-warning">진행 중</span> | → completed or failed |
| `completed` | Collection finished (may have errors) | <span class="badge bg-success">완료</span> | (final) |
| `failed` | Collection crashed/aborted | <span class="badge bg-danger">실패</span> | (final) |
| `pending` | Queued, not yet started | <span class="badge bg-secondary">대기 중</span> | → running |

### Collection Run Success Criteria
**Success** is defined as:
- Status = `completed` (not `failed`)
- `TotalSnapshots > 0` (at least one snapshot captured)
- `TotalErrors == 0` OR `TotalErrors < TotalSnapshots * 0.1` (error rate < 10%)

**Partial Success** (warning state):
- Status = `completed`
- `TotalSnapshots > 0` (some data captured)
- `TotalErrors > 0` (has errors, but not total loss)

**Failure**:
- Status = `failed` OR
- Status = `completed` + `TotalSnapshots == 0` (no data captured)

UI: `Pages/Admin/Collection/Index.cshtml` — status 값에 따라 배지 색상 결정, 향후 TotalSnapshots/TotalErrors로 상세 상태 표시

## KIS API Client Security (Phase 2)

### Governance Enforcement
- **Read-Only Mandate**: `AssertReadOnly(path, trId)` blocks all trading-related endpoints
- **Forbidden Paths**: `/trading/` substring triggers 🚫 immediate exception
- **Forbidden TR_IDs**: TTTC* / VTTC* prefixes (buy/sell order codes) blocked
- **Source**: `governance/rules/06_no_direct_api_trading.yaml`

### Token Management
- **ITokenCache** abstraction: PostgreSQL-backed in production
- **Credential Loading**:
  - Windows environment variables: `KIS_APP_Key`, `KIS_APP_Secret`, `KIS_APP_Key_TEST`, `KIS_APP_Secret_TEST`
  - Fallback: `HKCU\Environment` registry (Windows only)
  - Account modes: `"real"` (prod) vs `"mock"` (sandbox)

### Quotation Methods (All Read-Only)
1. **GetCurrentPriceAsync** (FHKST01010100) — Current price inquiry
2. **GetAskingPrice10LevelAsync** (FHKST01010200) — Order book (10-level)
3. **GetDailyShortSaleAsync** (FHPST04830000) — Short-sale trends
4. **GetDailyItemChartPriceAsync** (FHKST03010100) — Daily OHLCV data
5. **GetInvestorTrendAsync** (FHKST01010900) — Investor sentiment (개인/외국인/기관)

## Local Development & Testing (2026-07-11)

### ⚠️ CRITICAL: SSH Tunnel for Remote Database Access

**Never use Docker locally.** Always use SSH tunneling to connect to remote PostgreSQL:

```powershell
# 1. Setup SSH tunnel (Terminal 1) — forwards local 5432 to remote DB
ssh -L 127.0.0.1:5432:localhost:5432 kjh2064@178.104.200.7 -N

# 2. Configure appsettings.Development.json
{
  "ConnectionStrings": {
    "DefaultConnection": "Host=127.0.0.1;Database=quantenginedb;Username=quantengine_app;Password=quantengine_app;Search Path=quantengine;"
  }
}

# 3. Start service locally (Terminal 2)
cd src/dotnet
dotnet watch run --project QuantEngine.Web

# 4. Access locally
http://localhost:5265/Account/Login
```

### Mandatory Pre-Deployment Checklist

**EVERY code change must pass:**

1. ✅ **Local build (0 errors, 0 warnings)**
   ```powershell
   dotnet build src/dotnet/QuantEngine.Web/QuantEngine.Web.csproj -c Release
   ```

2. ✅ **Local service startup with SSH tunnel**
   - Service must start without DB connection errors
   - DbUp migrations must succeed

3. ✅ **Login test (admin/quant123!)**
   - `/Account/Login` must return 200
   - Authentication flow must complete
   - Cookie must be set

4. ✅ **All Admin pages must load**
   - `/Admin/Dashboard` → 200 (NOT 500)
   - `/Admin/Users` → 200 (NOT 500)
   - `/Admin/Collection` → 200 (NOT 500)
   - `/Admin/Monitoring` → 200 (NOT 500)
   - `/Admin/Operations` → 200 (NOT 500)
   - **No 500 errors in response body**

5. ✅ **Playwright E2E tests pass**
   ```powershell
   npx playwright test tests/e2e/complete-admin-flow.spec.ts
   ```

### Deployment Gates

**NEVER deploy without:**
- ❌ Local testing complete
- ❌ All Admin pages verified (200 status, no 500 errors)
- ❌ E2E tests passing
- ❌ Authorization Policy configured (if changes made to Program.cs)

**Deployment failure is better than service outage.** Halt and investigate if local tests fail.

---

## Notes for Contributors (2026-07-11)

- **SQL Safety**: Whitelist-only table access (enum switch in Repository)
- **KIS API**: Read-only quotations/ranking; no order/trade endpoints (governance enforced)
- **Admin UI**: Server-rendered Razor Pages only; no WASM, no APIs between PageModel and Repository
- **Authentication**: Cookie-based only; no Bearer tokens; password reset via API endpoints only (no UI form)
- **Password Policy**: BCrypt hashing (auto-upgrade from SHA-256 on login); IP lockout: 3 strikes = 15 min ban
- **Database**: PostgreSQL contract maintained; Dapper ORM with raw SQL (no EF)
- **Legacy Code**: `QuantEngine.Web.Client` folder kept for reference (not in .sln, not built)
- **Newtonsoft.Json**: Known high-severity vulnerability (GHSA-5crp-9r3c-p9vr); update or replace when feasible
- **Release Authority**: Python gates (`full-gate`, `prepare-upload-zip`) remain authority; .NET Admin fully operational as of 2026-07-11
- **Testing Requirement**: All code changes must pass local testing with SSH tunnel to remote DB before deployment (see "Local Development & Testing" above)