Major additions (2026-07-24): - High-level architecture overview with system layers (9 layers from UI to CI/CD) - Key design decisions (SOLID + domain-driven): * Razor Pages server-rendering (no WASM) * Repository pattern + Dapper ORM (SQL-first) * Read-only KIS governance enforcement * PostgreSQL single source of truth * Hybrid Python-to-.NET transition strategy * Contract-driven validation (Parity, Provenance, etc.) * Canonical JSON renderer (.NET Tools) - Quick reference development commands: * Build & restore (.NET + Python) * Run services locally (SSH tunnel + dotnet watch) * Data collection (KIS, snapshot admin, calibration) * Validation & release gates (ops:validate, full-gate, ops:release) * Testing (unit + E2E) * CI/CD monitoring - Core workflows for 6 common scenarios: 1. Day-to-day development (code change cycle) 2. Data collection setup (KIS API validation) 3. Admin data editing (snapshot admin web UI) 4. Release & deployment (multi-stage with checklists) 5. CI workflow debugging 6. Database schema changes (with DBML sync requirement) - Expanded contributor notes: * Code standards (SQL safety, KIS API, Auth, DB patterns) * Testing & validation requirements * Deployment checklist (6-point health checks) * Known issues & tech debt * Reliability principles (reproducibility, audit trail, contracts) * Change-making guidelines - Troubleshooting table for common issues - Updated for 2026-07-24 CI refactoring (9 parallel jobs, ~15-20min runtime) Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
52 KiB
CLAUDE.md
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
High-Level Architecture Overview
QuantEngine v0.1 — A hybrid quantitative analysis and data collection system for retirement asset portfolio management, transitioning from Python (legacy) to .NET 9 (primary).
System Architecture Layers
┌─────────────────────────────────────────────────────────────┐
│ Web UI Layer (Razor Pages) │
│ ├─ Admin Dashboard, Users, Collection, Monitoring, Ops │
│ └─ Server-side rendering + Cookie auth (no WASM) │
├─────────────────────────────────────────────────────────────┤
│ .NET Web API Layer (FastEndpoints, Minimal APIs) │
│ ├─ Collection API: /api/collection/{state,runs,snapshots} │
│ ├─ Emergency password reset, auth endpoints │
│ └─ Portfolio/settings endpoints │
├─────────────────────────────────────────────────────────────┤
│ Repository Layer (Dapper ORM, SQL-driven) │
│ ├─ IWorkspaceRepository: workspace/settings CRUD │
│ ├─ ICollectionRepository: KIS runs, snapshots, errors │
│ ├─ IPortfolioRepository: asset/position data │
│ └─ Whitelist-only table access (enum-based security) │
├─────────────────────────────────────────────────────────────┤
│ Infrastructure Layer (.NET) │
│ ├─ DbUp Migrations: src/dotnet/QuantEngine.Infrastructure/ │
│ ├─ PostgreSQL connection pooling, Npgsql 8.0 │
│ ├─ ITokenCache (KIS token management) │
│ ├─ PostProcessing: JSON normalization, schema validation │
│ └─ Contracts: Parity, Provenance, Scheduling, Normalization│
├─────────────────────────────────────────────────────────────┤
│ KIS API Client Layer (.NET) │
│ ├─ IKisApiClient: GetCurrentPrice, AskingPrice, DailyChart │
│ ├─ Read-only mandate: quotations + ranking only │
│ ├─ Governance enforcement: AssertReadOnly(path, trId) │
│ ├─ Fallback chain: KIS → Naver → Yahoo → OpenDART │
│ └─ Token caching: PostgreSQL-backed with 10-min skew │
├─────────────────────────────────────────────────────────────┤
│ Data Collection Pipeline (Python legacy + .NET new) │
│ ├─ Python (ops:data-collect): SQLite accumulator │
│ │ └─ tools/run_kis_data_collection_v1.py │
│ ├─ .NET: Async collection runner, PostgreSQL persistence │
│ ├─ Snapshot Admin (web UI): settings/account_snapshot edit │
│ └─ Calibration Backlog: priority + change ledger │
├─────────────────────────────────────────────────────────────┤
│ Decision Engine & Reporting (.NET Canonical Renderer) │
│ ├─ QuantEngine.Tools: Final decision packet + report render │
│ ├─ JSON schema: schemas/operational_report.schema.json │
│ ├─ Outputs: Temp/operational_report.{json,md} │
│ └─ Contract validation via canonical JSON │
├─────────────────────────────────────────────────────────────┤
│ Database Layer (PostgreSQL) │
│ ├─ Unified schema: quantenginedb / quantengine schema │
│ ├─ Key tables: kis_tokens, kis_collection_runs/snapshots │
│ │ workspaces, account_snapshots, settings │
│ ├─ Auto-migrations via DbUp (V001.sql, V002.sql, ...) │
│ └─ Documented in: docs/db/quantengine.dbml (DBML sync) │
├─────────────────────────────────────────────────────────────┤
│ Validation & Quality Gates (Python) │
│ ├─ 30+ validators: specs, contracts, WBS, audits │
│ ├─ CI pipeline: 9 parallel jobs (core + 8 independent) │
│ └─ Release gates: full-gate (strict), ops:release (warn-ok) │
├─────────────────────────────────────────────────────────────┤
│ CI/CD Orchestration (Gitea Actions) │
│ ├─ ci.yml: 10 jobs, core + contracts + reports + notify │
│ ├─ prepare-release.yml: tag + artifact + checksum │
│ ├─ deploy-prod.yml: SSH deploy + health checks │
│ ├─ kis_data_collection.yml: cron (00:30 KST) validation │
│ └─ Other workflows: snapshot admin, qualitative strategy │
└─────────────────────────────────────────────────────────────┘
Key Design Decisions (SOLID + Domain-Driven)
-
Razor Pages (Server-Rendered) — No WASM; Admin UI uses server-side rendering with cookie auth. Why? Simpler security, faster initial load, team familiarity.
-
Repository Pattern + Dapper ORM — SQL-first, no EF. Why? Direct control over queries, performance, audit trail (raw SQL = explicit intent).
-
Read-Only KIS Governance —
AssertReadOnly()blocks all /trading/ paths + TTTC*/VTTC* TR_IDs. Why? Hard mandate: data collection only, no order placement. -
PostgreSQL Single Source of Truth — All collection runs, snapshots, tokens centralized. Why? Audit, reproducibility, real-time coordination.
-
Hybrid Python-to-.NET Transition — Python handles legacy ops:data-collect; .NET does web, API, orchestration. Why? Gradual migration reduces risk; Python stays until .NET collection fully validated.
-
Contract-Driven Validation — Parity, Provenance, Scheduler, Normalization contracts in CI. Why? Catch schema drift, data quality regressions before deployment.
-
Canonical JSON Renderer (.NET Tools) — Single source of operational_report.json schema. Why? One contract, all tools emit to it (no format wars).
Project Overview
QuantEngine v0.1 — A comprehensive quantitative analysis and data collection system for retirement asset portfolio management.
- Architecture: .NET 9 + C# (web UI + APIs, primary); Python (legacy data collection, gradual deprecation)
- Critical Path: KIS API → PostgreSQL → Admin Dashboard (read-only quotations only, no trading APIs)
- Validation: 30+ contract validators + 9 parallel CI jobs + manual release gates
- Web UI: Blazor Interactive WebAssembly (MudBlazor) + ASP.NET Core Web API (API-First)
- Database: PostgreSQL (Npgsql 8.0), single unified database
- Data Source: KIS Open API (quotations/ranking read-only), with fallbacks
- Key Runtimes: .NET 9, Python 3.9+, Node.js 16+
Migration Phases Status (2026-07-11)
Phase 1: Web UI Migration ✅ 완료 (2026-07-11)
- 새로운 표준: Razor Pages (Server-Rendered) + Cookie Authentication + Tabler UI
- 폐기 대상: Blazor Interactive WebAssembly, MudBlazor, SmartAdmin
- 완료 기준 — Phase 1 Success Criteria:
- ✅ Cookie 인증 구현 (AuthService + IpLockoutService + BCrypt)
- ✅ Razor Pages 렌더링 (Admin 레이아웃 + 3개 이상 기본 페이지)
- ✅ 공용 UI 컴포넌트 (4개 이상 shared partials)
- ✅ 보안: 백도어 제거, 무솔트 해시 마이그레이션, IP 잠금
- ✅ 빌드 성공: 0 errors, 0 warnings
- ✅ CLAUDE.md 업데이트 (UI 기준 + 인증 정책)
- ✅ 모든 기준 충족됨 (2026-07-11)
- 구현 완료:
- ✅ Cookie 기반 인증 (AuthService + IpLockoutService)
- ✅ Razor Pages CRUD 레이아웃 (_AdminLayout.cshtml, shared partials)
- ✅ Admin 페이지: Dashboard, Collection, Users (기본 구조)
- ✅ 공용 UI 컴포넌트: _ValidationSummary, _Pagination, _StatusBadge, _EmptyState
- ✅ 보안 개선: BCrypt 해싱, IP 잠금, 하드코딩된 백도어 제거
- ✅ 빌드: 0 errors, 0 warnings (Newtonsoft.Json 보안 경고 제외)
- ✅ CLAUDE.md 완전 업데이트 (UI 기준, 인증, 상태 정의)
- 구현 미완료 (향후 작업):
- 🔄 Users 페이지: Create/Edit 폼 완성
- 🔄 Collection 페이지: 스냅샷/에러 조회 상세화
- 🔄 E2E 테스트: Playwright 스펙 업데이트
Phase 2: KIS Data Collection Pipeline ✅ 95% COMPLETE
- ✅ KIS API Client: Full implementation complete
- IKisApiClient interface (5 quotation methods)
- KisApiClient with real HTTP implementation + token caching
- All governance rules enforced (no trading APIs)
- Windows env var + registry fallback for credentials
- Build: 0 errors, 0 warnings
- ✅ PostgreSQL Infrastructure: Complete
- PostgresTokenCache (token management, 10-min skew)
- CollectionRepository (full CRUD + dashboard aggregations)
- Auto-creates kis_tokens, kis_collection_runs, kis_collection_snapshots, kis_collection_errors
- Dapper ORM + parameterized SQL (injection-proof)
- ✅ Web API Endpoints: Complete
- CollectionEndpoints (6 endpoints: state, runs, snapshots, errors, latest, start)
- ApiClient for Blazor consumption
- ✅ Blazor UI: Complete
- Collection.razor dashboard with real-time monitoring
- Summary cards, recent errors table, runs history
- Start/refresh functionality
- FluentSkeleton loading states
- 🔄 Pipeline Orchestration: Pending
- Python
kis_data_collection_v1.py→ .NET (data fetching + validation) - Real KIS API data collection workflow integration
- E2E test: API → DB → UI validation
- Python
Phase 3: Node.js→.NET CLI Tools 📋 PLANNED
- Makefile created (npm → make mappings)
- np operations documented
Phase 4: CI/CD Pipeline Hardening ✅ 80% COMPLETE (2026-07-11)
- ✅ deploy-prod.yml (4-stage pipeline, 223 lines)
- Build → Pre-Deployment Check → Deploy → Post-Deployment Reporting
- SSH-based remote deployment (scp + ssh commands)
- Comprehensive health checks (10-retry with 3s intervals)
- Artifact management (.tar.gz)
- ✅ Workflow consolidation (2 active files)
- ci.yml: PR validation only (maintains 29 validators)
- deploy-prod.yml: Production deployment
- Deleted: merge-to-main.yml (non-functional), fast-validation.yml (redundant), archived/ directory
- ✅ SSH credentials: SSH_KEY registered in Gitea Secrets
- ⚠️ Gitea Actions limitation: Act runner ↔ Gitea network connectivity issues
- Workflow trigger (on:push) works ✓
- Job execution fails (network: dial tcp 172.18.0.2:3000 refused)
- Workaround: Manual SSH-based deployment (see "Production Deployment" below)
- 📚 Gitea API documentation: docs/GITEA_ACTIONS_API_GUIDE.md
Phase 5: Admin UI & Deployment Optimization ✅ COMPLETE (2026-07-11)
- ✅ Admin UI redesign (Tabler framework)
- Dashboard: stat cards, quick actions, system info
- Responsive sidebar navigation
- Professional layout (dark sidebar #2c3e50, white content)
- ✅ Build output: 0 errors, 0 warnings
- ✅ E2E tests: 8/8 passing (Playwright)
- ✅ Production deployment: Active since 2026-07-11 21:00:55 KST
- Commit:
30fb702 - HTTP 200 health check
- Service: active (running)
- Commit:
Status Summary:
- Python codebase: Operational (1,140 files)
- .NET 9 coverage: Core (✅), Infrastructure (✅), API (✅), Web UI (✅)
- Database: PostgreSQL fully migrated
- CI/CD: Manual SSH deployment (fully operational), Gitea Actions (limited by infrastructure)
- Release gates: Python gates remain authority until Phase 2 integration testing complete
Deployment & Operations (Phase 4-5, 2026-07-11)
Production Server: Hetzner Cloud 178.104.200.7 (kjh2064@178.104.200.7)
Projects on server:
- TaxBaik (홈페이지) — Nginx location
/taxbaik - QuantEngine (데이터 수집/분석) — Nginx location
/quantengine
⚠️ CRITICAL: CI/CD-Only Deployment Mandate
Rule: ALL production deployments MUST go through Gitea Actions CI/CD. Manual SSH deployments are FORBIDDEN.
Why:
- Automatic validation (build, health checks, version verification)
- Audit trail (all deployments logged in Gitea Actions)
- Consistent process (no manual errors)
- Rollback safety (deployment history retained)
- Release traceability (version control via git tags)
⚠️ CRITICAL: DB Secret Management (Incident 2026-07-12)
Incident: quant.taxbaik.com/login이 28P01 password authentication failed로 장애 발생.
원인: appsettings.Production.json에 하드코딩되어 배포된 DB 비밀번호가, 실제 DB 비밀번호가
로테이션된 이후에도 계속 옛날 값(심지어 이전 세션에서 검증 없이 넣은 placeholder였던 적도 있음)
그대로 배포되고 있었음.
Rule: DB 접속 문자열(ConnectionStrings)은 절대 appsettings.Production.json이나
워크플로우 파일에 하드코딩하지 않는다. prepare-release.yml이 생성하는
appsettings.Production.json에는 Logging 설정만 있고 ConnectionStrings는 없다 —
이는 의도된 설계다 (Gitea Release는 누구나 다운로드 가능한 아티팩트이므로 시크릿을
담으면 안 됨).
실제 DB 비밀번호의 출처: 프로덕션 서버의 /home/kjh2064/.config/quantengine.env
파일 (ConnectionStrings__DefaultConnection=... 형식) 하나뿐이며,
quantengine.service.d/env.conf drop-in의 EnvironmentFile= 지시자로 systemd가
이 값을 환경변수로 주입한다. ASP.NET Core 설정 우선순위상 환경변수가
appsettings.Production.json을 오버라이드하므로, 배포되는 아티팩트 자체에는
DB 정보가 없어도 서비스는 정상 동작한다.
DB 비밀번호가 바뀌면 (로테이션 등): /home/kjh2064/.config/quantengine.env 파일만
갱신하고 sudo systemctl restart quantengine. 워크플로우 파일이나 Gitea Secrets는
건드릴 필요 없음 (배포 파이프라인은 DB 비밀번호를 모른 채로 동작해야 정상).
배포 전 체크리스트에 추가:
- ✅ 새 릴리즈 배포 후 반드시
/Account/Login실제 HTTP 응답 +journalctl -u quantengine에서28P01/password authentication failed부재 확인 (단순 프로세스active상태만으로는 DB 연결 실패를 못 잡음 — ASP.NET Core는 DB 없이도 기동은 되고 로그인 요청 시점에야 실패함) - ✅
.config/quantengine.env의 존재와quantengine.service.d/env.conf의EnvironmentFile=배선이 서버에 유지되고 있는지 (systemd unit 자체를 재생성/덮어쓰는 배포 방식으로 전환할 경우 이 drop-in이 날아가지 않는지 확인 필요)
Production Deployment Strategy (Release-Based)
Architecture: Two-Workflow System (Release Creation → Deployment)
Workflow 1: prepare-release.yml (Release Creation)
Purpose: Create a release with built artifact
Trigger: Manual (workflow_dispatch)
# Visit Gitea Actions and select prepare-release.yml
# Input version: v0.1.20260711 (or any semantic version)
What it does:
- ✓ Build (restore, build, publish)
- ✓ Generate
appsettings.Production.json - ✓ Package artifact:
.tar.gz - ✓ Create git tag:
v0.1.20260711 - ✓ Create Gitea Release with artifact attached
- ✓ Notify: Release ready for deployment
Output: Gitea Release with downloadable artifact
Workflow 2: deploy-prod.yml (Deployment)
Purpose: Deploy a release to production
Trigger: Manual (workflow_dispatch)
# Visit Gitea Actions and select deploy-prod.yml
# Input release: v0.1.20260711 (optional — uses latest if empty)
What it does:
- ✓ Fetch Release (from Gitea Releases)
- ✓ Download artifact
- ✓ Verify SSH credentials
- ✓ Upload to production server
- ✓ Extract and symlink
- ✓ Restart service
- ✓ 6-point health checks
- ✓ Report deployment status
Deployment Pipeline (5 Stages):
| Stage | Purpose | Timeout |
|---|---|---|
| 1. Fetch Release | Query Gitea Releases, download artifact | 10min |
| 2. Pre-Check | Verify SSH keys, secrets, release | 5min |
| 3. Deploy | Upload, extract, symlink, restart service | 30min |
| 4. Health Check | 6-point verification (HTTP, CSS, login, service, release, DB auth) | 10min |
| 5. Report | Final deployment status | Auto |
Health Checks (Automatic):
- ✓ HTTP 200 on
/Account/Login - ✓ Login page content verification
- ✓ CSS file loads (
/css/admin.css) - ✓ Service status (systemctl active)
- ✓ Release verification (deployed release tag matches)
- ✓ DB authentication check (
journalctl에서28P01/password authentication failed부재 확인 — GET/Account/Login은 DB가 끊겨도 200을 반환하므로 이 체크가 없으면 DB 장애를 배포 파이프라인이 놓친다. 2026-07-12 사고 이후 추가됨)
Complete Deployment Flow:
1. Code committed to main branch
2. Create release: prepare-release.yml workflow_dispatch (manual)
→ Builds code
→ Creates Gitea Release with artifact
→ Tags repository
3. Deploy release: deploy-prod.yml workflow_dispatch (manual)
→ Selects release version
→ Downloads artifact from Gitea Release
→ Deploys to production server
→ Runs health checks
→ Reports status
Pre-Deployment Checklist
Before creating a release, verify:
- ✅ Local build:
dotnet build src/dotnet/QuantEngine.Web/QuantEngine.Web.csproj -c Release(0 errors, 0 warnings) - ✅ E2E tests pass:
npx playwright test - ✅ Admin pages verified (200 status, no 500 errors)
- ✅ All changes committed and pushed to main branch
- ✅ No uncommitted changes:
git status
Release & Deployment Workflow
Step 1: Create Release (prepare-release.yml)
# Visit Gitea Actions
# https://gitea.taxbaik.com/kjh2064/QuantEngineByItz/actions
# Run prepare-release.yml workflow
# Input: version = v0.1.20260711
# Workflow will:
# - Build and publish
# - Package artifact
# - Create git tag
# - Create Gitea Release
# - Attach artifact
Step 2: Deploy Release (deploy-prod.yml)
# Visit Gitea Actions (same page)
# Run deploy-prod.yml workflow
# Input: release = v0.1.20260711 (leave empty for latest)
# Workflow will:
# - Download artifact from release
# - Deploy to production server
# - Run health checks
# - Report status
SSH Key Configuration (Required)
Setup (One-time):
-
Generate ED25519 key locally (or reuse existing):
ssh-keygen -t ed25519 -f ~/.ssh/quantengine_deploy -C "QuantEngine CI/CD" -
Add public key to production server:
ssh-copy-id -i ~/.ssh/quantengine_deploy.pub kjh2064@178.104.200.7 -
Get private key in base64 format:
# macOS/Linux base64 -w 0 ~/.ssh/quantengine_deploy > /tmp/key_b64.txt cat /tmp/key_b64.txt | pbcopy # Or Windows PowerShell $key = Get-Content ~/.ssh/quantengine_deploy -Raw [Convert]::ToBase64String([Text.Encoding]::UTF8.GetBytes($key)) | Set-Clipboard -
Configure in Gitea:
- URL: https://gitea.taxbaik.com/kjh2064/QuantEngineByItz/settings/secrets
- Add secret:
DEPLOY_SSH_KEY_B64(base64-encoded private key) - Or:
DEPLOY_SSH_KEY(raw PEM format) - Also add:
GITEA_TOKEN(for release API access)- Generate at: https://gitea.taxbaik.com/user/settings/applications
- Required permissions:
repo+read:actions
Deployment Monitoring
During Deployment:
- Watch live in Gitea Actions UI
- Jobs complete in order: Build → Pre-Check → Deploy → Health Check → Report
After Deployment:
# SSH into server
ssh kjh2064@178.104.200.7
# Check active deployment
readlink ~/quantengine_active
# View service status
systemctl status quantengine
# Tail live logs
journalctl -u quantengine -f
# Health check
curl -I http://127.0.0.1:5000/Account/Login
Automatic Rollback (if health check fails)
If health check fails, deployment stops automatically:
- Service restart may fail
- Symlink update reverts to previous deployment
- Gitea Actions marks deployment as FAILED
- Logs include failure details
Manual rollback (if needed):
# List deployments
ls -lht ~/deployments/quantengine_*
# Revert symlink to previous version
ln -sfn /home/kjh2064/deployments/quantengine_YYYYMMDD_HHMMSS_COMMIT ~/quantengine_active
# Restart service
sudo systemctl restart quantengine
# Verify
curl http://127.0.0.1:5000/Account/Login
Troubleshooting Deployment Failures
Issue: Build fails
- Check:
dotnet buildlocally first - Ensure: No compilation errors, 0 warnings
Issue: Health check timeout
- Check: Service logs:
journalctl -u quantengine -n 50 - Check: Port 5000 listening:
ss -tlnp | grep 5000 - Check: DB connectivity in appsettings.Production.json
Issue: SSH key error
- Verify:
DEPLOY_SSH_KEY_B64orDEPLOY_SSH_KEYin Gitea Secrets - Check: Public key added to
~/.ssh/authorized_keyson server - Test:
ssh -i ~/.ssh/key_file kjh2064@178.104.200.7 echo OK
Git Repository
Gitea Server (동일 호스트):
- HTTP:
https://gitea.taxbaik.com/kjh2064/QuantEngineByItz.git - SSH:
ssh://git@gitea.taxbaik.com:2222/kjh2064/QuantEngineByItz.git
UI Design Principles (2026-07-11 — Migrated to Razor Pages)
Framework & Design System (NEW — 2026-07-11)
- Primary Framework: ASP.NET Core Razor Pages + Bootstrap 5 + Tabler UI
- Design System: Tabler (Bootstrap 5 기반), 밀집 레이아웃 + 전통 서버 렌더링
- Render Mode: Server-side Razor Pages — 모든 Admin UI는 서버에서 렌더링, Cookie 기반 인증 (API-First WASM 폐기)
- Authentication: Cookie Authentication (HttpOnly) + BCrypt password hashing + IP lockout (3 strikes, 15-min)
- Deprecation: Blazor Interactive WebAssembly 폐기, MudBlazor 컴포넌트 폐기 (2026-07-11), SmartAdmin 폐기. 기존 WASM 코드는
/QuantEngine.Web.Client폴더에 참고용으로만 보관 (.sln에서 제외)
Component Development Rules (NEW)
-
All Admin UI Development (New + Refactored):
- Use Razor Pages (.cshtml + .cshtml.cs PageModel) exclusively for admin
- UI는 Repository/Service를 생성자 DI로 직접 호출 (API 홉 없음)
- Bootstrap 5 + Tabler UI CSS classes for styling
- Form Validation: DataAnnotations DTO + FluentValidation IValidator 이중 검증
- HTML
<form>+ tag helpers (asp-for,asp-action,asp-page)
-
Authentication & Authorization:
- Cookie name:
QuantEngine.Admin.Auth(HttpOnly, SameSite=Lax) - Session duration: 12 hours (sliding expiration)
- Folder-level
[Authorize]viaAuthorizeFolder("/Admin")convention (per-page 반복 금지) - Login:
/Account/Login(Razor Page, NO WASM) - Password: BCrypt-hashed (auto-migrates existing SHA-256 hashes on first login)
- IP Lockout: 3 failed attempts → 15-minute lockout
- Cookie name:
-
Data & Form Patterns:
- PageModel constructor:
public IndexModel(IWorkspaceRepository repo, ILogger<IndexModel> logger) - Form submission:
OnPostAsync()/OnPostDeleteAsync()(multi-handler pattern) - Validation failures: return
Page()(re-render with ModelState errors) - Pagination:
PaginationModelrecord (Page, TotalPages, Func<int,string> BuildPageUrl) - Empty states:
<PartialView name="_EmptyState" model="message" />
- PageModel constructor:
-
Component Mapping (Bootstrap 5 + Tabler):
| UI Element | Component | Notes |
|---|---|---|
| Button | <button class="btn btn-primary"> |
— |
| Input field | <input asp-for="Property" class="form-control"> |
tag helper |
| Dropdown | HTML <select asp-for="Property"> |
tag helper |
| Data grid | HTML <table class="table"> |
plain, no virtualization |
| Card | <div class="card"> |
Bootstrap card |
| Badge/Status | <span class="badge bg-success">Active</span> |
Bootstrap badge |
| Layout container | <div class="container-xl"> / <div class="row"> |
Bootstrap grid |
| Navigation | HTML navbar in _AdminLayout.cshtml |
sidebar + topbar |
| Loading | N/A (server-rendered) | no loading states needed |
| Icons | Bootstrap Icons (<i class="bi bi-*"></i>) |
CDN |
| Modal/Dialog | Bootstrap modal or inline confirm() |
avoid unnecessary modals |
| Validation msg | <span asp-validation-for="Property" class="d-block alert alert-danger mt-2"> |
tag helper |
Development Commands (Phase 1 + 2)
Python / Node.js (Legacy & Release Gates)
npm install
npm run ops:validate # Warn-only validation
npm run full-gate # Strict validation (all gates PASS)
npm run ops:data-collect # KIS collection (Python subprocess)
npm run ops:release # Full release DAG
.NET (Primary - Phase 1 + 2)
cd src/dotnet
dotnet restore
dotnet build # Debug build (0 errors, 0 warnings)
dotnet build -c Release # Release build
dotnet watch run --project QuantEngine.Web # Hot-reload (http://localhost:5265)
dotnet run --project QuantEngine.Web # Run API server
Collection Pipeline Testing (Phase 2)
# Set KIS credentials (sandbox account)
$env:KIS_APP_Key_TEST = "your_kis_test_key"
$env:KIS_APP_Secret_TEST = "your_kis_test_secret"
# Start web server (http://localhost:5265)
dotnet run --project QuantEngine.Web
# Verify Collection dashboard
# Navigate to http://localhost:5265/collection
# - Click "Start Collection" to trigger async run
# - Backend uses PostgreSQL-backed data storage
# - Dashboard updates with run status, snapshots, errors
# Verify API endpoints
curl http://localhost:5265/api/collection/state
curl http://localhost:5265/api/collection/runs
curl "http://localhost:5265/api/collection/latest/005930"
API Endpoints (Phase 1 + 2)
Workspace & History (Phase 1)
All endpoints prefixed with /api/:
| Route | Purpose |
|---|---|
GET /state |
Full UI state snapshot |
GET /tables |
Browsable tables list |
GET /table-rows |
Paginated rows |
POST /settings/save |
Save settings |
POST /account-snapshot/save |
Save snapshots |
POST /bootstrap |
Seed DB from JSON |
POST /account-snapshot/import-tsv |
Import TSV |
POST /autofix |
Auto-correct data |
Collection Pipeline (Phase 2)
| Route | Purpose |
|---|---|
GET /collection/state |
Dashboard summary (runs, snapshots, errors) |
GET /collection/runs |
Recent collection runs (paginated) |
GET /collection/runs/{runId}/snapshots |
Snapshots from a run |
GET /collection/runs/{runId}/errors |
Errors from a run |
GET /collection/latest/{ticker} |
Latest snapshots for ticker |
POST /collection/run |
Start new collection run (async) |
Collection Run Status Values
| Status | Meaning | UI Badge | Transitions |
|---|---|---|---|
running |
Collection in progress | 진행 중 | → completed or failed |
completed |
Collection finished (may have errors) | 완료 | (final) |
failed |
Collection crashed/aborted | 실패 | (final) |
pending |
Queued, not yet started | 대기 중 | → running |
Collection Run Success Criteria
Success is defined as:
- Status =
completed(notfailed) TotalSnapshots > 0(at least one snapshot captured)TotalErrors == 0ORTotalErrors < TotalSnapshots * 0.1(error rate < 10%)
Partial Success (warning state):
- Status =
completed TotalSnapshots > 0(some data captured)TotalErrors > 0(has errors, but not total loss)
Failure:
- Status =
failedOR - Status =
completed+TotalSnapshots == 0(no data captured)
UI: Pages/Admin/Collection/Index.cshtml — status 값에 따라 배지 색상 결정, 향후 TotalSnapshots/TotalErrors로 상세 상태 표시
KIS API Client Security (Phase 2)
Governance Enforcement
- Read-Only Mandate:
AssertReadOnly(path, trId)blocks all trading-related endpoints - Forbidden Paths:
/trading/substring triggers 🚫 immediate exception - Forbidden TR_IDs: TTTC* / VTTC* prefixes (buy/sell order codes) blocked
- Source:
governance/rules/06_no_direct_api_trading.yaml
Token Management
- ITokenCache abstraction: PostgreSQL-backed in production
- Credential Loading:
- Windows environment variables:
KIS_APP_Key,KIS_APP_Secret,KIS_APP_Key_TEST,KIS_APP_Secret_TEST - Fallback:
HKCU\Environmentregistry (Windows only) - Account modes:
"real"(prod) vs"mock"(sandbox)
- Windows environment variables:
Quotation Methods (All Read-Only)
- GetCurrentPriceAsync (FHKST01010100) — Current price inquiry
- GetAskingPrice10LevelAsync (FHKST01010200) — Order book (10-level)
- GetDailyShortSaleAsync (FHPST04830000) — Short-sale trends
- GetDailyItemChartPriceAsync (FHKST03010100) — Daily OHLCV data
- GetInvestorTrendAsync (FHKST01010900) — Investor sentiment (개인/외국인/기관)
Local Development & Testing (2026-07-11)
⚠️ CRITICAL: SSH Tunnel for Remote Database Access
Never use Docker locally. Always use SSH tunneling to connect to remote PostgreSQL:
# 1. Setup SSH tunnel (Terminal 1) — forwards local 5432 to remote DB
ssh -L 127.0.0.1:5432:localhost:5432 kjh2064@178.104.200.7 -N
# 2. Configure appsettings.Development.json
{
"ConnectionStrings": {
"DefaultConnection": "Host=127.0.0.1;Database=quantenginedb;Username=quantengine_app;Password=quantengine_app;Search Path=quantengine;"
}
}
# 3. Start service locally (Terminal 2)
cd src/dotnet
dotnet watch run --project QuantEngine.Web
# 4. Access locally
http://localhost:5265/Account/Login
Mandatory Pre-Deployment Checklist
EVERY code change must pass:
-
✅ Local build (0 errors, 0 warnings)
dotnet build src/dotnet/QuantEngine.Web/QuantEngine.Web.csproj -c Release -
✅ Local service startup with SSH tunnel
- Service must start without DB connection errors
- DbUp migrations must succeed
-
✅ Login test (admin/quant123!)
/Account/Loginmust return 200- Authentication flow must complete
- Cookie must be set
-
✅ All Admin pages must load
/Admin/Dashboard→ 200 (NOT 500)/Admin/Users→ 200 (NOT 500)/Admin/Collection→ 200 (NOT 500)/Admin/Monitoring→ 200 (NOT 500)/Admin/Operations→ 200 (NOT 500)- No 500 errors in response body
-
✅ Playwright E2E tests pass
npx playwright test tests/e2e/complete-admin-flow.spec.ts
Deployment Gates
NEVER deploy without:
- ❌ Local testing complete
- ❌ All Admin pages verified (200 status, no 500 errors)
- ❌ E2E tests passing
- ❌ Authorization Policy configured (if changes made to Program.cs)
Deployment failure is better than service outage. Halt and investigate if local tests fail.
Gitea Actions Workflows
Active Workflows:
-
prepare-release.yml — Release creation (workflow_dispatch only)
- Build → Publish → Package → Tag → Gitea Release
- Does NOT write ConnectionStrings into the artifact (see "DB Secret
Management" above) — only
Loggingconfig ships inappsettings.Production.json
-
deploy-prod.yml — Production deployment (workflow_dispatch only, takes a release tag)
- 5 stages: Fetch Release → Pre-Check → Deploy → Health Check → Report
- 6-point health checks (HTTP, login page, CSS, service, release, DB auth)
- SSH-based deployment with artifact validation
-
ci.yml — PR validation (on:pull_request)
- 29 validators for code quality
- Runs on every pull request
Accessing Gitea Actions:
- Web UI: https://gitea.taxbaik.com/kjh2064/QuantEngineByItz/actions
- Runs API: https://gitea.taxbaik.com/api/v1/repos/kjh2064/QuantEngineByItz/actions/runs
API Monitoring (CLI)
Monitor deployment status from command line:
# Setup (one-time)
$env:GITEA_TOKEN_TAXBAIK = "your_gitea_personal_token"
# List recent deployment runs
$token = $env:GITEA_TOKEN_TAXBAIK
$response = Invoke-WebRequest `
-Uri "https://gitea.taxbaik.com/api/v1/repos/kjh2064/QuantEngineByItz/actions/runs?limit=5" `
-Headers @{ "Authorization" = "token $token" }
($response.Content | ConvertFrom-Json).workflow_runs | ForEach-Object {
Write-Host "Run #$($_.id): $($_.display_title) [$($_.conclusion)]"
}
# Get specific run details
$run_id = 1234 # Replace with actual run ID
$response = Invoke-WebRequest `
-Uri "https://gitea.taxbaik.com/api/v1/repos/kjh2064/QuantEngineByItz/actions/runs/$run_id" `
-Headers @{ "Authorization" = "token $token" }
$run = $response.Content | ConvertFrom-Json
Write-Host "Commit: $($run.head_sha)"
Write-Host "Status: $($run.status) / $($run.conclusion)"
See docs/GITEA_ACTIONS_API_GUIDE.md for complete API reference.
Deployment Secrets Configuration
Required Secrets (Gitea Repository Settings → Secrets):
| Secret | Type | Purpose |
|---|---|---|
DEPLOY_SSH_KEY_B64 |
Base64 (recommended) | ED25519 private key for SSH |
DEPLOY_SSH_KEY |
PEM (alternative) | Raw private key format |
DEPLOY_HOST |
Text | Production server IP (178.104.200.7) |
DEPLOY_USER |
Text | SSH username (kjh2064) |
How to add secrets:
- Go to: https://gitea.taxbaik.com/kjh2064/QuantEngineByItz/settings/secrets
- Click "Add Secret"
- Name:
DEPLOY_SSH_KEY_B64 - Value:
base64 -w 0 ~/.ssh/deploy_key | pbcopy(macOS) orcertutil -encode deploy_key deploy_key.b64(Windows) - Save
Core Workflows & Common Scenarios
Scenario 1: Day-to-Day Development (Code Change)
- Make code changes (C# Razor Pages / .NET API / Python tools)
- Local validation:
dotnet build src/dotnet/QuantEngine.Web/QuantEngine.Web.csproj -c Release dotnet test src/dotnet/QuantEngine.Core.Tests -c Release - Test admin pages locally (with SSH tunnel):
ssh -L 127.0.0.1:5432:localhost:5432 kjh2064@178.104.200.7 -N & dotnet watch run --project QuantEngine.Web # Verify: /Admin/Dashboard, /Admin/Users, /Admin/Collection, etc. all return 200 - Commit & push: Changes automatically trigger ci.yml
- Core validators run first (blocking others)
- Parallel validators (contracts, UI, DB, calibration) run independently
- notify-results summarizes all 9 jobs in PR comment
- Expected CI time: ~15-20min (was ~40min before 2026-07-24 refactor)
Scenario 2: Data Collection Setup (KIS API Validation)
- Obtain KIS credentials (real or mock account)
- Validate with mock account:
$env:KIS_APP_Key_TEST="<test_key>" $env:KIS_APP_Secret_TEST="<test_secret>" python tools/validate_kis_api_credentials_v1.py --account mock --ticker 005930 --dry-run - Run real collection (if approved):
$env:KIS_APP_Key="<real_key>" $env:KIS_APP_Secret="<real_secret>" python tools/run_kis_data_collection_v1.py --input-json GatherTradingData.json --sqlite-db src/quant_engine/kis_data_collection.db --output-json Temp/kis_data_collection_v1.json --kis-account real - Verify database:
SELECT COUNT(*) FROM kis_collection_runs; SELECT COUNT(*) FROM kis_collection_snapshots;
Scenario 3: Admin Data Editing (Snapshot Admin Web UI)
- Start snapshot admin server:
python tools/run_snapshot_admin_server_v1.py --host 127.0.0.1 --port 8787 --db src/quant_engine/snapshot_admin.db --seed GatherTradingData.json - Access web UI: http://127.0.0.1:8787
- Edit settings / account_snapshot in browser (like Excel)
- Manage changes: Approval & Locks area handles change history, undo, approval workflow
- Export for CI:
/api/export→ JSON or "Export approval packet" button
Scenario 4: Release & Deployment (Multi-Stage)
Stage 1: Local validation
npm run ops:validate # Warn-only (allow some issues)
npm run full-gate # Strict (all gates PASS)
Stage 2: Create release (manual via Gitea Actions)
→ Visit https://gitea.taxbaik.com/kjh2064/QuantEngineByItz/actions
→ Run "prepare-release.yml" workflow_dispatch
- Builds and publishes .NET
- Creates git tag (e.g., quant_20260724.0.abc1234)
- Generates Gitea Release with artifact + checksums
- Packages as .tar.gz
Stage 3: Deploy (manual, only after release exists)
→ Run "deploy-prod.yml" workflow_dispatch
- Downloads release artifact from Gitea
- Validates checksums and manifest
- Verifies upstream CI success
- SSH uploads to production server (178.104.200.7)
- Extracts and symlinks
- Restarts systemd service
- 6-point health checks (HTTP, login page, CSS, service, release tag, DB auth)
- Reports final status
Pre-deployment checklist (MANDATORY):
- ✅ Local build: 0 errors, 0 warnings
- ✅ E2E tests pass:
npx playwright test - ✅ All admin pages tested locally (200 status, no 500)
- ✅
git statusclean (no uncommitted changes) - ✅ Commit pushed to main
Scenario 5: CI Workflow Debugging
Problem: A specific validation fails in CI
- Identify failing job from PR comment (notify-results output)
- Reproduce locally:
# For core, wbs-audit, dotnet-contracts: run relevant Python validators python tools/validate_dotnet_migration_execution_plan_v1.py python tools/validate_dotnet_parity_contract_v1.py # etc. - Fix and re-push (triggers ci.yml again)
- Monitor in Gitea Actions dashboard
Problem: Workflow syntax error
- Validate locally:
python tools/validate_gitea_ci_workflow_lint_v1.py --workflow .gitea/workflows/ci.yml - Fix YAML and test again
Scenario 6: Database Schema Changes
- Create migration:
src/dotnet/QuantEngine.Infrastructure/Migrations/V003.sql - Update DBML:
docs/db/quantengine.dbml(same commit)- DbUp auto-applies migrations on startup
- DBML is reference documentation
- Test locally (with SSH tunnel): Migrations must apply cleanly
- Commit both (SQL + DBML) together
- CI validates: ci.yml applies migrations to test PostgreSQL service
When Things Break
| Issue | Root Cause | Fix |
|---|---|---|
| Admin page returns 500 | Likely unhandled DB exception or auth issue | Check journalctl, verify ConnectionStrings in production env |
| KIS API fails with "not found" | Ticker doesn't exist in KIS | Use fallback (Naver → Yahoo → OpenDART) |
| Snapshot admin won't load | SQLite DB corrupted or missing | Delete and re-seed from GatherTradingData.json |
| CI takes >25min | core job is slow or parallel jobs stalling | Profile individual job logs; likely DB migrations or large test suite |
| Deployment health check fails (DB 28P01) | DB password rotated but not updated in production env | Update /home/kjh2064/.config/quantengine.env on server only (not in repo) |
Notes for Contributors (2026-07-11 — Updated 2026-07-24)
Code Standards
- SQL Safety: Whitelist-only table access (enum switch in Repository). Never raw concatenation.
- KIS API: Read-only quotations/ranking; no order/trade endpoints. Enforced via
AssertReadOnly()in client. - Admin UI: Server-rendered Razor Pages only; no WASM. PageModel constructor: DI repos directly, no API hops.
- Authentication: Cookie-based (HttpOnly, SameSite=Lax). No Bearer tokens. Password reset via API endpoints only (no UI form).
- Password Hashing: BCrypt (auto-migrates from SHA-256 on login). IP lockout: 3 failed attempts → 15 min ban.
- Database: PostgreSQL as single source of truth. Dapper ORM + raw SQL (no Entity Framework). Auto-migrations via DbUp.
- Error Handling: Validate only at system boundaries (user input, external APIs). Trust internal guarantees.
- Comments: Minimal; only explain WHY for non-obvious constraints. Removed comments on refactor; code names explain WHAT.
Repository Structure Rules
- Legacy Code:
QuantEngine.Web.Client(WASM) kept for reference; excluded from .sln, not built. - DBML Sync (2026-07-12): All DbUp migrations (
V*.sql) must sync withdocs/db/quantengine.dbmlin same commit. Future schema reads use DBML. - Diagrams: Mermaid diagrams in
docs/diagrams/for state machines, flows, sequences (e.g., collection-pipeline.md). - WBS Evidence (2026-07-12): Task completion =
npm run verify:task -- <TASK_ID>PASS (BE=DB queries/logs/JSON, FE=Playwright+screenshot). Full check:npm run verify:wbs.
Testing & Validation
- Local pre-deployment: Build (0 errors), E2E tests, all admin pages return 200 (no 500), git status clean.
- CI auto-validation: 9 parallel jobs (core + 8 independent). Expected time: ~15-20min (was ~40min, parallelized 2026-07-24).
- core: CRITICAL tests + DB setup
- wbs-audit, dotnet-contracts, ui-storage, database-schema, calibration, operational-reporting, security-validation, workflow-lint: parallel
- notify-results: final PR summary
- Release gates:
npm run ops:validate(warn-ok),npm run full-gate(strict),npm run ops:release(warn-ok + full DAG).
Deployment
- Mandatory checklist: Local build, E2E pass, admin pages 200, git clean, pushed to main.
- Release creation: Manual workflow_dispatch → prepare-release.yml (tag + artifact).
- Deployment: Manual workflow_dispatch → deploy-prod.yml (SSH upload + health checks).
- DB secrets: Never hardcode
ConnectionStringsin artifacts. Runtime injection via systemd EnvironmentFile (production only). - Health checks (6-point): HTTP 200, login page, CSS load, service active, release tag match, DB auth (no 28P01 errors).
Known Issues & Tech Debt
- Newtonsoft.Json: High-severity vuln (GHSA-5crp-9r3c-p9vr); update or replace when feasible.
- Python-to-.NET transition: Legacy
ops:data-collectremains until .NET collection validated in production (est. Q3 2026). - Gitea Actions limitation: Act runner ↔ Gitea network issues (workaround: manual SSH deployment active).
Reliability & Data Quality
- Reproducibility: All validation gates, WBS tasks, and CI runs must be reproducible. Use deterministic inputs (GatherTradingData.json seed).
- Audit trail: PostgreSQL is source of truth for all collection runs/snapshots. Raw SQL = explicit intent (no ORM magic).
- Contract validation: Parity, Provenance, Scheduler, Normalization contracts catch schema drift before deployment.
- Canonical rendering: .NET Tools (QuantEngine.Tools) is single source for operational_report.json schema. No format divergence.
When Making Changes
-
Change type determines scope:
- Bug fix: No cleanup beyond fix itself.
- Feature: Add only what's requested; no hypothetical abstractions.
- Refactor: Break down large methods/classes; extract reusable patterns only if used 3+ times.
- Architecture change: Must update CLAUDE.md and CI contracts simultaneously.
-
Locality-first: Prefer editing in place. Create new files only when genuinely new concept or isolation needed.
-
Testing locality: Unit tests verify code correctness; UI testing (Playwright) verifies feature correctness. If UI can't be tested, say so.
-
Git commits: New commits preferred over amend. Include co-author:
Claude Haiku 4.5 <noreply@anthropic.com>.
Development Commands (Quick Reference)
Build & Restore
# .NET
cd src/dotnet
dotnet restore
dotnet build -c Release
dotnet build src/dotnet/QuantEngine.Web/QuantEngine.Web.csproj -c Release
# Python (no explicit build needed, but validate setup)
python3 --version
python3 -m pip install --quiet requests pyyaml openpyxl pytest psycopg[binary]
Run Services Locally
# 1. SSH tunnel for remote PostgreSQL (Terminal 1)
ssh -L 127.0.0.1:5432:localhost:5432 kjh2064@178.104.200.7 -N
# 2. .NET service with hot-reload (Terminal 2)
cd src/dotnet
dotnet watch run --project QuantEngine.Web
# 3. Access locally
http://localhost:5265/Account/Login # Admin UI
http://localhost:5265/api/collection/state # API
Data Collection & Snapshot Admin
# KIS data collection to SQLite
$env:KIS_APP_Key="<your_kis_key>"
$env:KIS_APP_Secret="<your_kis_secret>"
python tools/run_kis_data_collection_v1.py --input-json GatherTradingData.json --sqlite-db src/quant_engine/kis_data_collection.db --output-json Temp/kis_data_collection_v1.json --kis-account real
# Snapshot admin web UI (for editing settings/account_snapshot)
python tools/run_snapshot_admin_server_v1.py --host 127.0.0.1 --port 8787 --db src/quant_engine/snapshot_admin.db --seed GatherTradingData.json
# With hot reload
npm run ops:snapshot-web-watch
# Validate snapshot admin web UI
python tools/validate_snapshot_admin_web_v1.py
Validation & Release Gates
# Validation (warn-only mode)
npm run ops:validate
# Strict validation (all gates PASS)
npm run full-gate
# Release DAG (includes warn-only gates)
npm run ops:release
# Package for distribution
npm run prepare-upload-zip
# Calibration backlog rebuild
python tools/build_calibration_priority_v1.py
python tools/build_calibration_change_ledger_v4.py
python tools/build_calibration_review_report_v1.py
python tools/validate_calibration_change_ledger_v1.py
CI/CD Monitoring
# Monitor Gitea Actions
# Open: https://gitea.taxbaik.com/kjh2064/QuantEngineByItz/actions
# Validate workflows locally
python3 tools/validate_gitea_ci_workflow_lint_v1.py
# Check recent deployment runs
# UI: https://gitea.taxbaik.com/kjh2064/QuantEngineByItz/actions
Testing
# .NET unit tests (Core tests)
dotnet test src/dotnet/QuantEngine.Core.Tests/QuantEngine.Core.Tests.csproj -c Release
# All .NET tests
dotnet test src/dotnet/QuantEngine.sln --configuration Release
# Python unit tests (storage, KIS, snapshot admin)
python3 -m pytest tests/unit/test_*.py -v
# E2E tests (Playwright)
npx playwright test
npx playwright test tests/e2e/complete-admin-flow.spec.ts
Gitea Actions Workflow Structure (2026-07-24)
Workflow Architecture Refactoring
2026-07-24 refactoring: Single-job ci.yml (30+ steps, ~40min runtime) → 9-job parallel pipeline (~15-20min runtime).
CI Pipeline Jobs (ci.yml)
| Job | Dependencies | Purpose | Parallelizable |
|---|---|---|---|
| core | — | CRITICAL: .NET tests, API trading gate, KIS creds, DB migrations | ✗ (blocks others) |
| wbs-audit | core | WBS validation, platform migration, coverage audits | ✓ |
| dotnet-contracts | core | .NET parity, provenance, scheduler, normalization contracts | ✓ |
| ui-storage | — | Admin UI, storage backend, integration tests | ✓ |
| database-schema | — | DB pipeline, PostgreSQL schema, history contracts | ✓ |
| calibration-pipeline | core | Calibration priority, change ledger, qualitative sell strategy | ✓ |
| operational-reporting | calibration | Decision packet, operational report, performance metrics | ✗ (depends on calibration) |
| security-validation | — | Secrets contract, workflow validation | ✓ |
| workflow-lint | — | CI workflow structure, secrets contract | ✓ |
| notify-results | ALL | PR notification with job status summary | — |
Dependency Graph:
core ─┬─> wbs-audit ─────────────────────┐
├─> dotnet-contracts ─────────────┤
└─> calibration-pipeline ────────┤
└─> operational-reporting ─┤
└─> notify-results
ui-storage ────────────────────────────────┘
database-schema ──────────────────────────┘
security-validation ───────────────────────┘
workflow-lint ─────────────────────────────┘
Other Workflow Files
| File | Trigger | Purpose | Status |
|---|---|---|---|
| kis_data_collection.yml | cron (00:30 KST M-F) + dispatch | Validate KIS credentials & PostgreSQL pipeline | ✓ 2026-07-24 |
| qualitative_sell_strategy.yml | cron (00:15 KST M-F) + push + dispatch | Validate sell strategy pipeline & store | ✓ 2026-07-24 |
| workflow_lint.yml (was ci_lint.yml) | push (.gitea/workflows/) + dispatch | Lint all workflow files, validate job dependencies, secrets contract | ✓ 2026-07-24 |
| snapshot_admin.yml | push (snapshot_admin_*) + dispatch | Validate snapshot admin workflow & UI (2 jobs) | ✓ 2026-07-24 |
| prepare-release.yml | workflow_run (ci.yml success) + dispatch | Build, tag, create Gitea Release with artifact + checksums | — |
| deploy-prod.yml | dispatch | Deploy release, run health checks, report status (3 jobs) | — |
Performance Improvements (2026-07-24)
ci.yml refactoring results:
- Before: 1 job, 30+ sequential steps, ~40min runtime
- After: 9 jobs, 7 in parallel, ~15-20min total runtime
- Speedup: ~2-2.5x faster CI feedback (core branch blocks only downstream, others parallel)
- Fault isolation: Single validation failure no longer blocks unrelated checks
Key changes:
- Setup consolidation: Database migrations, Python, .NET setup in
corejob only - Parallel validation groups: 7 jobs run independently from core (ui-storage, database-schema, security-validation, workflow-lint, etc.)
- Dependency clarity:
needs:explicitly defines blocking relationships - Error reporting:
notify-resultssummarizes all 9 job statuses in PR comment
Workflow Maintenance Checklist
When modifying workflows (.gitea/workflows/*.yml):
- ✅ Update
workflow_lint.ymlif adding new triggers or job dependencies - ✅ Test locally with
python3 tools/validate_gitea_ci_workflow_lint_v1.py - ✅ Verify all
needs:references point to existing jobs - ✅ Document new jobs in this section above
- ✅ Validate YAML syntax:
python3 -m yaml < .gitea/workflows/new.yml - ✅ Ensure no hardcoded secrets in workflow files (env vars only)
Troubleshooting Workflows
Symptom: CI job timeout
- Check: Does your job need PostgreSQL? Only
coreprovides it; others must be independent. - Fix: Add
services: postgres:block or restructure to parallel-safe job.
Symptom: Cascading failure (multiple jobs fail)
- Check: Does your job have missing dependencies? Review
needs:and dependency graph above. - Fix: Add explicit
needs: [job_name]if job depends on another's output.
Symptom: "job not found" error in notify-results
- Check: Job name typo in
notify-results.needslist. - Fix: Match job name exactly (case-sensitive).
Workflow Trigger Schedule (2026-07-24)
| Time (KST) | Workflow | Trigger | Purpose |
|---|---|---|---|
| 00:15 | qualitative_sell_strategy.yml | cron (M-F) | Validate sell strategy before daily operations |
| 00:30 | kis_data_collection.yml | cron (M-F) | Validate KIS API & DB pipeline before data collection |
| Push | ci.yml | on:push (main) | Validate code on every push to main |
| PR | ci.yml | on:pull_request | Gate PR merges with full validation suite |
| Manual | prepare-release.yml | workflow_dispatch | Create release tag & artifact |
| Manual | deploy-prod.yml | workflow_dispatch | Deploy release to production |
Dependencies:
- Release creation (prepare-release.yml) is gated by ci.yml success (workflow_run trigger)
- Deployment (deploy-prod.yml) is manual — only after release artifact exists