QuantEngineByItz/docs/CICD_PIPELINE.md

QuantEngine CI/CD Pipeline Structure

Full Gitea Actions workflow structure, extracted from CLAUDE.md (2026-07-30) to keep the main file within the character budget.

Workflow Architecture Refactoring (2026-07-24)

2026-07-24 refactoring: Single-job ci.yml (30+ steps, ~40min runtime) → 9-job parallel pipeline (~15-20min runtime).

CI Pipeline Jobs (ci.yml)

Job	Dependencies	Purpose	Parallelizable
core	—	CRITICAL: .NET tests, API trading gate, KIS creds, DB migrations	✗ (blocks others)
wbs-audit	core	WBS validation, platform migration, coverage audits	✓
dotnet-contracts	core	.NET parity, provenance, scheduler, normalization contracts	✓
ui-storage	—	Admin UI, storage backend, integration tests	✓
database-schema	—	DB pipeline, PostgreSQL schema, history contracts	✓
calibration-pipeline	core	Calibration priority, change ledger, qualitative sell strategy	✓
operational-reporting	calibration	Decision packet, operational report, performance metrics	✗ (depends on calibration)
security-validation	—	Secrets contract, workflow validation	✓
workflow-lint	—	CI workflow structure, secrets contract	✓
notify-results	ALL	PR notification with job status summary	—

Dependency Graph:

core ─┬─> wbs-audit ─────────────────────┐
      ├─> dotnet-contracts ─────────────┤
      └─> calibration-pipeline ────────┤
              └─> operational-reporting ─┤
                                         └─> notify-results
ui-storage ────────────────────────────────┘
database-schema ──────────────────────────┘
security-validation ───────────────────────┘
workflow-lint ─────────────────────────────┘

Other Workflow Files

File	Trigger	Purpose	Status
kis_data_collection.yml	cron (00:30 KST M-F) + dispatch	Validate KIS credentials & PostgreSQL pipeline	✓ 2026-07-24
qualitative_sell_strategy.yml	cron (00:15 KST M-F) + push + dispatch	Validate sell strategy pipeline & store	✓ 2026-07-24
ci_lint.yml	push (.gitea/workflows/) + dispatch	Lint all workflow files, validate job dependencies, secrets contract	✓ 2026-07-24
snapshot_admin.yml	push (snapshot_admin_*) + dispatch	Validate snapshot admin workflow & UI (2 jobs)	✓ 2026-07-24
ci-frontend.yml	push (main/master/feature/**) + PR	8-step src/frontend/ pipeline: install, typecheck, import-boundary lint, unit test, enterprise CRUD contract parity, Vite build, Playwright E2E, npm audit	✓ (undocumented until 2026-07-30)
t20_ledger.yml	cron (17:00 KST M-F) + dispatch	Build tools/build_operational_t20_outcome_ledger_v1.py daily T+20 outcome ledger	✓ (undocumented until 2026-07-30)
prepare-release.yml	workflow_run (ci.yml success) + dispatch	Build, tag, create Gitea Release with artifact + checksums	—
deploy-prod.yml	dispatch	Deploy release, run health checks, report status (3 jobs)	—

Note (2026-07-30): An earlier version of this table claimed ci_lint.yml had been renamed to workflow_lint.yml. That rename was never actually carried out — the file on disk is still ci_lint.yml. Corrected here after direct verification against .gitea/workflows/.

Performance Improvements (2026-07-24)

ci.yml refactoring results:

• Before: 1 job, 30+ sequential steps, ~40min runtime
• After: 9 jobs, 7 in parallel, ~15-20min total runtime
• Speedup: ~2-2.5x faster CI feedback (core branch blocks only downstream, others parallel)
• Fault isolation: Single validation failure no longer blocks unrelated checks

Key changes:

1. Setup consolidation: Database migrations, Python, .NET setup in core job only
2. Parallel validation groups: 7 jobs run independently from core (ui-storage, database-schema, security-validation, workflow-lint, etc.)
3. Dependency clarity: needs: explicitly defines blocking relationships
4. Error reporting: notify-results summarizes all 9 job statuses in PR comment

Workflow Maintenance Checklist

When modifying workflows (.gitea/workflows/*.yml):

1. ✅ Update ci_lint.yml if adding new triggers or job dependencies
2. ✅ Test locally with python3 tools/validate_gitea_ci_workflow_lint_v1.py
3. ✅ Verify all needs: references point to existing jobs
4. ✅ Document new jobs in this section above
5. ✅ Validate YAML syntax: python3 -m yaml < .gitea/workflows/new.yml
6. ✅ Ensure no hardcoded secrets in workflow files (env vars only)

Troubleshooting Workflows

Symptom: CI job timeout

• Check: Does your job need PostgreSQL? Only core provides it; others must be independent.
• Fix: Add services: postgres: block or restructure to parallel-safe job.

Symptom: Cascading failure (multiple jobs fail)

• Check: Does your job have missing dependencies? Review needs: and dependency graph above.
• Fix: Add explicit needs: [job_name] if job depends on another's output.

Symptom: "job not found" error in notify-results

• Check: Job name typo in notify-results.needs list.
• Fix: Match job name exactly (case-sensitive).

Workflow Trigger Schedule (2026-07-24)

Time (KST)	Workflow	Trigger	Purpose
00:15	qualitative_sell_strategy.yml	cron (M-F)	Validate sell strategy before daily operations
00:30	kis_data_collection.yml	cron (M-F)	Validate KIS API & DB pipeline before data collection
Push	ci.yml	on:push (main)	Validate code on every push to main
PR	ci.yml	on:pull_request	Gate PR merges with full validation suite
Manual	prepare-release.yml	workflow_dispatch	Create release tag & artifact
Manual	deploy-prod.yml	workflow_dispatch	Deploy release to production

Dependencies:

• Release creation (prepare-release.yml) is gated by ci.yml success (workflow_run trigger)
• Deployment (deploy-prod.yml) is manual — only after release artifact exists