docs(claude): comprehensive CLAUDE.md update for future Claude Code instances
Validators (Pushes and Pull Requests) / Database & Schema Validation (push) Failing after 7s
Validators (Pushes and Pull Requests) / UI & Storage Validation (push) Failing after 13s
Validators (Pushes and Pull Requests) / Security & Secrets (push) Failing after 7s
Validators (Pushes and Pull Requests) / Core Validators & Database Setup (push) Failing after 20s
Validators (Pushes and Pull Requests) / CI Workflow Lint (push) Failing after 5s
Validators (Pushes and Pull Requests) / WBS & Audit Validations (push) Has been skipped
Validators (Pushes and Pull Requests) / .NET Contracts (push) Has been skipped
Validators (Pushes and Pull Requests) / Calibration & Performance (push) Has been skipped
Validators (Pushes and Pull Requests) / Operational Report & Decision Packet (push) Has been skipped
Validators (Pushes and Pull Requests) / Notify PR Results (push) Has been skipped

Major additions (2026-07-24):
- High-level architecture overview with system layers (9 layers from UI to CI/CD)
- Key design decisions (SOLID + domain-driven):
  * Razor Pages server-rendering (no WASM)
  * Repository pattern + Dapper ORM (SQL-first)
  * Read-only KIS governance enforcement
  * PostgreSQL single source of truth
  * Hybrid Python-to-.NET transition strategy
  * Contract-driven validation (Parity, Provenance, etc.)
  * Canonical JSON renderer (.NET Tools)

- Quick reference development commands:
  * Build & restore (.NET + Python)
  * Run services locally (SSH tunnel + dotnet watch)
  * Data collection (KIS, snapshot admin, calibration)
  * Validation & release gates (ops:validate, full-gate, ops:release)
  * Testing (unit + E2E)
  * CI/CD monitoring

- Core workflows for 6 common scenarios:
  1. Day-to-day development (code change cycle)
  2. Data collection setup (KIS API validation)
  3. Admin data editing (snapshot admin web UI)
  4. Release & deployment (multi-stage with checklists)
  5. CI workflow debugging
  6. Database schema changes (with DBML sync requirement)

- Expanded contributor notes:
  * Code standards (SQL safety, KIS API, Auth, DB patterns)
  * Testing & validation requirements
  * Deployment checklist (6-point health checks)
  * Known issues & tech debt
  * Reliability principles (reproducibility, audit trail, contracts)
  * Change-making guidelines

- Troubleshooting table for common issues
- Updated for 2026-07-24 CI refactoring (9 parallel jobs, ~15-20min runtime)

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
This commit is contained in:
2026-07-24 13:42:00 +09:00
parent 60c8e6dbe2
commit b2b5be666a
+382 -15
View File
@@ -2,11 +2,100 @@
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
## High-Level Architecture Overview
**QuantEngine v0.1** — A hybrid quantitative analysis and data collection system for retirement asset portfolio management, transitioning from Python (legacy) to .NET 9 (primary).
### System Architecture Layers
```
┌─────────────────────────────────────────────────────────────┐
│ Web UI Layer (Razor Pages) │
│ ├─ Admin Dashboard, Users, Collection, Monitoring, Ops │
│ └─ Server-side rendering + Cookie auth (no WASM) │
├─────────────────────────────────────────────────────────────┤
│ .NET Web API Layer (FastEndpoints, Minimal APIs) │
│ ├─ Collection API: /api/collection/{state,runs,snapshots} │
│ ├─ Emergency password reset, auth endpoints │
│ └─ Portfolio/settings endpoints │
├─────────────────────────────────────────────────────────────┤
│ Repository Layer (Dapper ORM, SQL-driven) │
│ ├─ IWorkspaceRepository: workspace/settings CRUD │
│ ├─ ICollectionRepository: KIS runs, snapshots, errors │
│ ├─ IPortfolioRepository: asset/position data │
│ └─ Whitelist-only table access (enum-based security) │
├─────────────────────────────────────────────────────────────┤
│ Infrastructure Layer (.NET) │
│ ├─ DbUp Migrations: src/dotnet/QuantEngine.Infrastructure/ │
│ ├─ PostgreSQL connection pooling, Npgsql 8.0 │
│ ├─ ITokenCache (KIS token management) │
│ ├─ PostProcessing: JSON normalization, schema validation │
│ └─ Contracts: Parity, Provenance, Scheduling, Normalization│
├─────────────────────────────────────────────────────────────┤
│ KIS API Client Layer (.NET) │
│ ├─ IKisApiClient: GetCurrentPrice, AskingPrice, DailyChart │
│ ├─ Read-only mandate: quotations + ranking only │
│ ├─ Governance enforcement: AssertReadOnly(path, trId) │
│ ├─ Fallback chain: KIS → Naver → Yahoo → OpenDART │
│ └─ Token caching: PostgreSQL-backed with 10-min skew │
├─────────────────────────────────────────────────────────────┤
│ Data Collection Pipeline (Python legacy + .NET new) │
│ ├─ Python (ops:data-collect): SQLite accumulator │
│ │ └─ tools/run_kis_data_collection_v1.py │
│ ├─ .NET: Async collection runner, PostgreSQL persistence │
│ ├─ Snapshot Admin (web UI): settings/account_snapshot edit │
│ └─ Calibration Backlog: priority + change ledger │
├─────────────────────────────────────────────────────────────┤
│ Decision Engine & Reporting (.NET Canonical Renderer) │
│ ├─ QuantEngine.Tools: Final decision packet + report render │
│ ├─ JSON schema: schemas/operational_report.schema.json │
│ ├─ Outputs: Temp/operational_report.{json,md} │
│ └─ Contract validation via canonical JSON │
├─────────────────────────────────────────────────────────────┤
│ Database Layer (PostgreSQL) │
│ ├─ Unified schema: quantenginedb / quantengine schema │
│ ├─ Key tables: kis_tokens, kis_collection_runs/snapshots │
│ │ workspaces, account_snapshots, settings │
│ ├─ Auto-migrations via DbUp (V001.sql, V002.sql, ...) │
│ └─ Documented in: docs/db/quantengine.dbml (DBML sync) │
├─────────────────────────────────────────────────────────────┤
│ Validation & Quality Gates (Python) │
│ ├─ 30+ validators: specs, contracts, WBS, audits │
│ ├─ CI pipeline: 9 parallel jobs (core + 8 independent) │
│ └─ Release gates: full-gate (strict), ops:release (warn-ok) │
├─────────────────────────────────────────────────────────────┤
│ CI/CD Orchestration (Gitea Actions) │
│ ├─ ci.yml: 10 jobs, core + contracts + reports + notify │
│ ├─ prepare-release.yml: tag + artifact + checksum │
│ ├─ deploy-prod.yml: SSH deploy + health checks │
│ ├─ kis_data_collection.yml: cron (00:30 KST) validation │
│ └─ Other workflows: snapshot admin, qualitative strategy │
└─────────────────────────────────────────────────────────────┘
```
### Key Design Decisions (SOLID + Domain-Driven)
1. **Razor Pages (Server-Rendered)** — No WASM; Admin UI uses server-side rendering with cookie auth. Why? Simpler security, faster initial load, team familiarity.
2. **Repository Pattern + Dapper ORM** — SQL-first, no EF. Why? Direct control over queries, performance, audit trail (raw SQL = explicit intent).
3. **Read-Only KIS Governance**`AssertReadOnly()` blocks all /trading/ paths + TTTC*/VTTC* TR_IDs. Why? Hard mandate: data collection only, no order placement.
4. **PostgreSQL Single Source of Truth** — All collection runs, snapshots, tokens centralized. Why? Audit, reproducibility, real-time coordination.
5. **Hybrid Python-to-.NET Transition** — Python handles legacy ops:data-collect; .NET does web, API, orchestration. Why? Gradual migration reduces risk; Python stays until .NET collection fully validated.
6. **Contract-Driven Validation** — Parity, Provenance, Scheduler, Normalization contracts in CI. Why? Catch schema drift, data quality regressions before deployment.
7. **Canonical JSON Renderer** (.NET Tools) — Single source of operational_report.json schema. Why? One contract, all tools emit to it (no format wars).
## Project Overview
**QuantEngine v0.1** — A comprehensive quantitative analysis and data collection system for retirement asset portfolio management.
- **Architecture**: .NET 9 + C# (web UI + APIs), Python (legacy data collection/analysis)
- **Architecture**: .NET 9 + C# (web UI + APIs, primary); Python (legacy data collection, gradual deprecation)
- **Critical Path**: KIS API → PostgreSQL → Admin Dashboard (read-only quotations only, no trading APIs)
- **Validation**: 30+ contract validators + 9 parallel CI jobs + manual release gates
- **Web UI**: Blazor Interactive WebAssembly (MudBlazor) + ASP.NET Core Web API (API-First)
- **Database**: PostgreSQL (Npgsql 8.0), single unified database
- **Data Source**: KIS Open API (quotations/ranking read-only), with fallbacks
@@ -676,21 +765,299 @@ See `docs/GITEA_ACTIONS_API_GUIDE.md` for complete API reference.
---
## Notes for Contributors (2026-07-11)
## Core Workflows & Common Scenarios
- **SQL Safety**: Whitelist-only table access (enum switch in Repository)
- **KIS API**: Read-only quotations/ranking; no order/trade endpoints (governance enforced)
- **Admin UI**: Server-rendered Razor Pages only; no WASM, no APIs between PageModel and Repository
- **Authentication**: Cookie-based only; no Bearer tokens; password reset via API endpoints only (no UI form)
- **Password Policy**: BCrypt hashing (auto-upgrade from SHA-256 on login); IP lockout: 3 strikes = 15 min ban
- **Database**: PostgreSQL contract maintained; Dapper ORM with raw SQL (no EF)
- **Legacy Code**: `QuantEngine.Web.Client` folder kept for reference (not in .sln, not built)
- **Newtonsoft.Json**: Known high-severity vulnerability (GHSA-5crp-9r3c-p9vr); update or replace when feasible
- **Release Authority**: Python gates (`full-gate`, `prepare-upload-zip`) remain authority; .NET Admin fully operational as of 2026-07-11
- **Testing Requirement**: All code changes must pass local testing with SSH tunnel to remote DB before deployment (see "Local Development & Testing" above)
- **DBML Schema Sync (2026-07-12)**: DbUp 마이그레이션(`src/dotnet/QuantEngine.Infrastructure/Migrations/V*.sql`)으로 관리되는 모든 테이블은 **반드시 `docs/db/quantengine.dbml`에도 동기화**되어야 하며, 개발 시 스키마 참조는 이 DBML 파일을 기준으로 한다. 새 마이그레이션 추가 시 같은 커밋에서 DBML 갱신 필수.
- **Diagrams**: 상태전이/플로우차트/시퀀스 다이어그램은 Mermaid로 `docs/diagrams/`에 작성해 코딩 참조로 활용 (수집 파이프라인: `docs/diagrams/collection-pipeline.md`)
- **WBS Evidence Gate (2026-07-12)**: 퀀트 엔진 로드맵/WBS는 `spec/60_quant_engine_wbs.yaml`(기계 판정)로 관리. 작업 완료는 `npm run verify:task -- <TASK_ID>` 게이트 PASS로만 인정 (BE=PG쿼리/로그/JSON, FE=Playwright+스크린샷). 전체 게이트: `npm run verify:wbs`
### Scenario 1: Day-to-Day Development (Code Change)
1. **Make code changes** (C# Razor Pages / .NET API / Python tools)
2. **Local validation**:
```powershell
dotnet build src/dotnet/QuantEngine.Web/QuantEngine.Web.csproj -c Release
dotnet test src/dotnet/QuantEngine.Core.Tests -c Release
```
3. **Test admin pages locally** (with SSH tunnel):
```powershell
ssh -L 127.0.0.1:5432:localhost:5432 kjh2064@178.104.200.7 -N &
dotnet watch run --project QuantEngine.Web
# Verify: /Admin/Dashboard, /Admin/Users, /Admin/Collection, etc. all return 200
```
4. **Commit & push**: Changes automatically trigger ci.yml
- Core validators run first (blocking others)
- Parallel validators (contracts, UI, DB, calibration) run independently
- notify-results summarizes all 9 jobs in PR comment
- Expected CI time: ~15-20min (was ~40min before 2026-07-24 refactor)
### Scenario 2: Data Collection Setup (KIS API Validation)
1. **Obtain KIS credentials** (real or mock account)
2. **Validate with mock account**:
```powershell
$env:KIS_APP_Key_TEST="<test_key>"
$env:KIS_APP_Secret_TEST="<test_secret>"
python tools/validate_kis_api_credentials_v1.py --account mock --ticker 005930 --dry-run
```
3. **Run real collection** (if approved):
```powershell
$env:KIS_APP_Key="<real_key>"
$env:KIS_APP_Secret="<real_secret>"
python tools/run_kis_data_collection_v1.py --input-json GatherTradingData.json --sqlite-db src/quant_engine/kis_data_collection.db --output-json Temp/kis_data_collection_v1.json --kis-account real
```
4. **Verify database**:
```sql
SELECT COUNT(*) FROM kis_collection_runs;
SELECT COUNT(*) FROM kis_collection_snapshots;
```
### Scenario 3: Admin Data Editing (Snapshot Admin Web UI)
1. **Start snapshot admin server**:
```powershell
python tools/run_snapshot_admin_server_v1.py --host 127.0.0.1 --port 8787 --db src/quant_engine/snapshot_admin.db --seed GatherTradingData.json
```
2. **Access web UI**: http://127.0.0.1:8787
3. **Edit settings / account_snapshot** in browser (like Excel)
4. **Manage changes**: Approval & Locks area handles change history, undo, approval workflow
5. **Export for CI**: `/api/export` → JSON or "Export approval packet" button
### Scenario 4: Release & Deployment (Multi-Stage)
**Stage 1: Local validation**
```powershell
npm run ops:validate # Warn-only (allow some issues)
npm run full-gate # Strict (all gates PASS)
```
**Stage 2: Create release** (manual via Gitea Actions)
```
→ Visit https://gitea.taxbaik.com/kjh2064/QuantEngineByItz/actions
→ Run "prepare-release.yml" workflow_dispatch
- Builds and publishes .NET
- Creates git tag (e.g., quant_20260724.0.abc1234)
- Generates Gitea Release with artifact + checksums
- Packages as .tar.gz
```
**Stage 3: Deploy** (manual, only after release exists)
```
→ Run "deploy-prod.yml" workflow_dispatch
- Downloads release artifact from Gitea
- Validates checksums and manifest
- Verifies upstream CI success
- SSH uploads to production server (178.104.200.7)
- Extracts and symlinks
- Restarts systemd service
- 6-point health checks (HTTP, login page, CSS, service, release tag, DB auth)
- Reports final status
```
**Pre-deployment checklist** (MANDATORY):
- ✅ Local build: 0 errors, 0 warnings
- ✅ E2E tests pass: `npx playwright test`
- ✅ All admin pages tested locally (200 status, no 500)
- ✅ `git status` clean (no uncommitted changes)
- ✅ Commit pushed to main
### Scenario 5: CI Workflow Debugging
**Problem**: A specific validation fails in CI
1. Identify failing job from PR comment (notify-results output)
2. Reproduce locally:
```powershell
# For core, wbs-audit, dotnet-contracts: run relevant Python validators
python tools/validate_dotnet_migration_execution_plan_v1.py
python tools/validate_dotnet_parity_contract_v1.py
# etc.
```
3. Fix and re-push (triggers ci.yml again)
4. Monitor in Gitea Actions dashboard
**Problem**: Workflow syntax error
1. Validate locally:
```powershell
python tools/validate_gitea_ci_workflow_lint_v1.py --workflow .gitea/workflows/ci.yml
```
2. Fix YAML and test again
### Scenario 6: Database Schema Changes
1. **Create migration**: `src/dotnet/QuantEngine.Infrastructure/Migrations/V003.sql`
2. **Update DBML**: `docs/db/quantengine.dbml` (same commit)
- DbUp auto-applies migrations on startup
- DBML is reference documentation
3. **Test locally** (with SSH tunnel): Migrations must apply cleanly
4. **Commit both** (SQL + DBML) together
5. **CI validates**: ci.yml applies migrations to test PostgreSQL service
### When Things Break
| Issue | Root Cause | Fix |
|-------|-----------|-----|
| Admin page returns 500 | Likely unhandled DB exception or auth issue | Check journalctl, verify ConnectionStrings in production env |
| KIS API fails with "not found" | Ticker doesn't exist in KIS | Use fallback (Naver → Yahoo → OpenDART) |
| Snapshot admin won't load | SQLite DB corrupted or missing | Delete and re-seed from GatherTradingData.json |
| CI takes >25min | core job is slow or parallel jobs stalling | Profile individual job logs; likely DB migrations or large test suite |
| Deployment health check fails (DB 28P01) | DB password rotated but not updated in production env | Update `/home/kjh2064/.config/quantengine.env` on server only (not in repo) |
---
## Notes for Contributors (2026-07-11 — Updated 2026-07-24)
### Code Standards
- **SQL Safety**: Whitelist-only table access (enum switch in Repository). Never raw concatenation.
- **KIS API**: Read-only quotations/ranking; no order/trade endpoints. Enforced via `AssertReadOnly()` in client.
- **Admin UI**: Server-rendered Razor Pages only; no WASM. PageModel constructor: DI repos directly, no API hops.
- **Authentication**: Cookie-based (HttpOnly, SameSite=Lax). No Bearer tokens. Password reset via API endpoints only (no UI form).
- **Password Hashing**: BCrypt (auto-migrates from SHA-256 on login). IP lockout: 3 failed attempts → 15 min ban.
- **Database**: PostgreSQL as single source of truth. Dapper ORM + raw SQL (no Entity Framework). Auto-migrations via DbUp.
- **Error Handling**: Validate only at system boundaries (user input, external APIs). Trust internal guarantees.
- **Comments**: Minimal; only explain WHY for non-obvious constraints. Removed comments on refactor; code names explain WHAT.
### Repository Structure Rules
- **Legacy Code**: `QuantEngine.Web.Client` (WASM) kept for reference; excluded from .sln, not built.
- **DBML Sync (2026-07-12)**: All DbUp migrations (`V*.sql`) must sync with `docs/db/quantengine.dbml` in same commit. Future schema reads use DBML.
- **Diagrams**: Mermaid diagrams in `docs/diagrams/` for state machines, flows, sequences (e.g., collection-pipeline.md).
- **WBS Evidence (2026-07-12)**: Task completion = `npm run verify:task -- <TASK_ID>` PASS (BE=DB queries/logs/JSON, FE=Playwright+screenshot). Full check: `npm run verify:wbs`.
### Testing & Validation
- **Local pre-deployment**: Build (0 errors), E2E tests, all admin pages return 200 (no 500), git status clean.
- **CI auto-validation**: 9 parallel jobs (core + 8 independent). Expected time: ~15-20min (was ~40min, parallelized 2026-07-24).
- core: CRITICAL tests + DB setup
- wbs-audit, dotnet-contracts, ui-storage, database-schema, calibration, operational-reporting, security-validation, workflow-lint: parallel
- notify-results: final PR summary
- **Release gates**: `npm run ops:validate` (warn-ok), `npm run full-gate` (strict), `npm run ops:release` (warn-ok + full DAG).
### Deployment
- **Mandatory checklist**: Local build, E2E pass, admin pages 200, git clean, pushed to main.
- **Release creation**: Manual workflow_dispatch → prepare-release.yml (tag + artifact).
- **Deployment**: Manual workflow_dispatch → deploy-prod.yml (SSH upload + health checks).
- **DB secrets**: Never hardcode `ConnectionStrings` in artifacts. Runtime injection via systemd EnvironmentFile (production only).
- **Health checks (6-point)**: HTTP 200, login page, CSS load, service active, release tag match, DB auth (no 28P01 errors).
### Known Issues & Tech Debt
- **Newtonsoft.Json**: High-severity vuln (GHSA-5crp-9r3c-p9vr); update or replace when feasible.
- **Python-to-.NET transition**: Legacy `ops:data-collect` remains until .NET collection validated in production (est. Q3 2026).
- **Gitea Actions limitation**: Act runner ↔ Gitea network issues (workaround: manual SSH deployment active).
### Reliability & Data Quality
- **Reproducibility**: All validation gates, WBS tasks, and CI runs must be reproducible. Use deterministic inputs (GatherTradingData.json seed).
- **Audit trail**: PostgreSQL is source of truth for all collection runs/snapshots. Raw SQL = explicit intent (no ORM magic).
- **Contract validation**: Parity, Provenance, Scheduler, Normalization contracts catch schema drift before deployment.
- **Canonical rendering**: .NET Tools (QuantEngine.Tools) is single source for operational_report.json schema. No format divergence.
### When Making Changes
1. **Change type determines scope**:
- Bug fix: No cleanup beyond fix itself.
- Feature: Add only what's requested; no hypothetical abstractions.
- Refactor: Break down large methods/classes; extract reusable patterns only if used 3+ times.
- Architecture change: Must update CLAUDE.md and CI contracts simultaneously.
2. **Locality-first**: Prefer editing in place. Create new files only when genuinely new concept or isolation needed.
3. **Testing locality**: Unit tests verify code correctness; UI testing (Playwright) verifies feature correctness. If UI can't be tested, say so.
4. **Git commits**: New commits preferred over amend. Include co-author: `Claude Haiku 4.5 <noreply@anthropic.com>`.
## Development Commands (Quick Reference)
### Build & Restore
```powershell
# .NET
cd src/dotnet
dotnet restore
dotnet build -c Release
dotnet build src/dotnet/QuantEngine.Web/QuantEngine.Web.csproj -c Release
# Python (no explicit build needed, but validate setup)
python3 --version
python3 -m pip install --quiet requests pyyaml openpyxl pytest psycopg[binary]
```
### Run Services Locally
```powershell
# 1. SSH tunnel for remote PostgreSQL (Terminal 1)
ssh -L 127.0.0.1:5432:localhost:5432 kjh2064@178.104.200.7 -N
# 2. .NET service with hot-reload (Terminal 2)
cd src/dotnet
dotnet watch run --project QuantEngine.Web
# 3. Access locally
http://localhost:5265/Account/Login # Admin UI
http://localhost:5265/api/collection/state # API
```
### Data Collection & Snapshot Admin
```powershell
# KIS data collection to SQLite
$env:KIS_APP_Key="<your_kis_key>"
$env:KIS_APP_Secret="<your_kis_secret>"
python tools/run_kis_data_collection_v1.py --input-json GatherTradingData.json --sqlite-db src/quant_engine/kis_data_collection.db --output-json Temp/kis_data_collection_v1.json --kis-account real
# Snapshot admin web UI (for editing settings/account_snapshot)
python tools/run_snapshot_admin_server_v1.py --host 127.0.0.1 --port 8787 --db src/quant_engine/snapshot_admin.db --seed GatherTradingData.json
# With hot reload
npm run ops:snapshot-web-watch
# Validate snapshot admin web UI
python tools/validate_snapshot_admin_web_v1.py
```
### Validation & Release Gates
```powershell
# Validation (warn-only mode)
npm run ops:validate
# Strict validation (all gates PASS)
npm run full-gate
# Release DAG (includes warn-only gates)
npm run ops:release
# Package for distribution
npm run prepare-upload-zip
# Calibration backlog rebuild
python tools/build_calibration_priority_v1.py
python tools/build_calibration_change_ledger_v4.py
python tools/build_calibration_review_report_v1.py
python tools/validate_calibration_change_ledger_v1.py
```
### CI/CD Monitoring
```powershell
# Monitor Gitea Actions
# Open: https://gitea.taxbaik.com/kjh2064/QuantEngineByItz/actions
# Validate workflows locally
python3 tools/validate_gitea_ci_workflow_lint_v1.py
# Check recent deployment runs
# UI: https://gitea.taxbaik.com/kjh2064/QuantEngineByItz/actions
```
### Testing
```powershell
# .NET unit tests (Core tests)
dotnet test src/dotnet/QuantEngine.Core.Tests/QuantEngine.Core.Tests.csproj -c Release
# All .NET tests
dotnet test src/dotnet/QuantEngine.sln --configuration Release
# Python unit tests (storage, KIS, snapshot admin)
python3 -m pytest tests/unit/test_*.py -v
# E2E tests (Playwright)
npx playwright test
npx playwright test tests/e2e/complete-admin-flow.spec.ts
```
## Gitea Actions Workflow Structure (2026-07-24)