QuantEngineByItz/docs/DEV_WORKFLOWS.md

QuantEngine Development Workflows & Common Scenarios

Full day-to-day workflow walkthroughs, extracted from CLAUDE.md (2026-07-30) to keep the main file within the character budget.

Scenario 1: Day-to-Day Development (Code Change)

1. Make code changes (C# Razor Pages / .NET API / Python tools)
2. Local validation:

   dotnet build src/dotnet/QuantEngine.Web/QuantEngine.Web.csproj -c Release
   dotnet test src/dotnet/QuantEngine.Core.Tests -c Release
   

3. Test admin pages locally (with SSH tunnel):

   ssh -L 127.0.0.1:5432:localhost:5432 kjh2064@178.104.200.7 -N &
   dotnet watch run --project QuantEngine.Web
   # Verify: /Admin/Dashboard, /Admin/Users, /Admin/Collection, etc. all return 200
   

4. Commit & push: Changes automatically trigger ci.yml
   • Core validators run first (blocking others)
   • Parallel validators (contracts, UI, DB, calibration) run independently
   • notify-results summarizes all 9 jobs in PR comment
   • Expected CI time: ~15-20min (was ~40min before 2026-07-24 refactor)

Scenario 2: Data Collection Setup (KIS API Validation)

1. Obtain KIS credentials (real or mock account)
2. Validate with mock account:

   $env:KIS_APP_Key_TEST="<test_key>"
   $env:KIS_APP_Secret_TEST="<test_secret>"
   python tools/validate_kis_api_credentials_v1.py --account mock --ticker 005930 --dry-run
   

3. Run real collection (if approved):

   $env:KIS_APP_Key="<real_key>"
   $env:KIS_APP_Secret="<real_secret>"
   python tools/run_kis_data_collection_v1.py --input-json GatherTradingData.json --sqlite-db src/quant_engine/kis_data_collection.db --output-json Temp/kis_data_collection_v1.json --kis-account real
   

4. Verify database:

   SELECT COUNT(*) FROM kis_collection_runs;
   SELECT COUNT(*) FROM kis_collection_snapshots;
   

Scenario 3: Admin Data Editing (Snapshot Admin Web UI)

1. Start snapshot admin server:

   python tools/run_snapshot_admin_server_v1.py --host 127.0.0.1 --port 8787 --db src/quant_engine/snapshot_admin.db --seed GatherTradingData.json
   

2. Access web UI: http://127.0.0.1:8787
3. Edit settings / account_snapshot in browser (like Excel)
4. Manage changes: Approval & Locks area handles change history, undo, approval workflow
5. Export for CI: /api/export → JSON or "Export approval packet" button

Scenario 4: Release & Deployment (Multi-Stage)

Stage 1: Local validation

npm run ops:validate              # Warn-only (allow some issues)
npm run full-gate                 # Strict (all gates PASS)

Stage 2: Create release (manual via Gitea Actions)

→ Visit https://gitea.taxbaik.com/kjh2064/QuantEngineByItz/actions
→ Run "prepare-release.yml" workflow_dispatch
  - Builds and publishes .NET
  - Creates git tag (e.g., quant_20260724.0.abc1234)
  - Generates Gitea Release with artifact + checksums
  - Packages as .tar.gz

Stage 3: Deploy (manual, only after release exists)

→ Run "deploy-prod.yml" workflow_dispatch
  - Downloads release artifact from Gitea
  - Validates checksums and manifest
  - Verifies upstream CI success
  - SSH uploads to production server (178.104.200.7)
  - Extracts and symlinks
  - Restarts systemd service
  - 6-point health checks (HTTP, login page, CSS, service, release tag, DB auth)
  - Reports final status

Pre-deployment checklist (MANDATORY):

• ✅ Local build: 0 errors, 0 warnings
• ✅ E2E tests pass: npx playwright test
• ✅ All admin pages tested locally (200 status, no 500)
• ✅ git status clean (no uncommitted changes)
• ✅ Commit pushed to main

Full runbook: DEPLOYMENT_RUNBOOK.md

Scenario 5: CI Workflow Debugging

Problem: A specific validation fails in CI

1. Identify failing job from PR comment (notify-results output)
2. Reproduce locally:

   # For core, wbs-audit, dotnet-contracts: run relevant Python validators
   python tools/validate_dotnet_migration_execution_plan_v1.py
   python tools/validate_dotnet_parity_contract_v1.py
   # etc.
   

3. Fix and re-push (triggers ci.yml again)
4. Monitor in Gitea Actions dashboard

Problem: Workflow syntax error

1. Validate locally:

   python tools/validate_gitea_ci_workflow_lint_v1.py --workflow .gitea/workflows/ci.yml
   

2. Fix YAML and test again

Scenario 6: Database Schema Changes

1. Create migration: src/dotnet/QuantEngine.Infrastructure/Migrations/V003.sql
2. Update DBML: docs/db/quantengine.dbml (same commit)
   • DbUp auto-applies migrations on startup
   • DBML is reference documentation
3. Test locally (with SSH tunnel): Migrations must apply cleanly
4. Commit both (SQL + DBML) together
5. CI validates: ci.yml applies migrations to test PostgreSQL service

When Things Break

Issue	Root Cause	Fix
Admin page returns 500	Likely unhandled DB exception or auth issue	Check journalctl, verify ConnectionStrings in production env
KIS API fails with "not found"	Ticker doesn't exist in KIS	Use fallback (Naver → Yahoo → OpenDART)
Snapshot admin won't load	SQLite DB corrupted or missing	Delete and re-seed from GatherTradingData.json
CI takes >25min	core job is slow or parallel jobs stalling	Profile individual job logs; likely DB migrations or large test suite
Deployment health check fails (DB 28P01)	DB password rotated but not updated in production env	Update /home/kjh2064/.config/quantengine.env on server only (not in repo)