kjh2064
5589a0432b
Critical re-review of the QuantEngine WBS evidence system found several
regressions of the "no fake gates" discipline established by M0, plus a
still-unwired M1 collection path. This closes 10 more WBS tasks
(QE-M1-01..06, QE-M2-01/02/04/05/06 — see spec/60_quant_engine_wbs.yaml)
with real, gate-verified evidence (18/34 total).
M1 — real KIS data now lands in PostgreSQL end-to-end:
- SchedulerService: load ticker universe from GatherTradingData.json instead
of a hardcoded array; fix a Hangfire scoped-service resolution bug.
- KisDataCollectionOrchestrator: restore logging on the lineage-event write
path (was a bare `catch {}` swallowing all failures silently); persist
daily OHLCV bars into quantengine.price_history_daily per run.
- Verified live: POST /api/collection/run -> Hangfire -> orchestrator ->
KIS mock API -> PostgreSQL, with Playwright DOM/API parity evidence.
M2 — historical price-history pipeline:
- CollectionRepository: SavePriceHistoryDailyAsync (idempotent upsert),
GetPriceHistorySummaryAsync (per-ticker aggregation) + a new
DateOnlyTypeHandler registered globally, since Dapper has no built-in
System.DateOnly support in either direction (write threw
NotSupportedException, read threw a constructor-mismatch
InvalidOperationException — found by exercising both paths live).
- tools/validate_price_history_integrity_v1.py: gap-freeness (vs KIS
trading calendar) + price-sanity gate over collected history.
- Admin Collection page: new "히스토리 현황" summary table +
GET /api/collection/history-summary, with Playwright evidence.
Governance/gate fixes:
- validate_market_time_series_schema_v1.py mislabeled its own output
"runtime_database_query": "DATA_GATED" despite never opening a DB
connection (pure file/regex check) — relabeled "check_scope":
"STATIC_STRUCTURAL_ONLY" and wired the node into the release DAG so it
isn't only reachable from ci.yml, matching every other validator.
Live-data authority for the same claim stays with QE-M2-01's pg_query
gate (spec/60), documented in spec/64.
- Fixed a WBS log_pattern check (QE-M1-06) that couldn't match its own
multi-line target; loosened two depends_on edges (QE-M1-05/06,
QE-M2-04/05) that encoded "needs X verified" when the real requirement
was only "needs X's code merged."
- Discovered and fixed admin-pages.spec.ts logging in with the wrong
seeded password (admin/admin instead of admin/quant123!, per CLAUDE.md)
— every test in that suite had been silently failing at the login step.
Deferred: QE-M2-03 (2-year backfill) — the KIS mock/VTS token endpoint
started returning 403 after the first successful call this session; looks
like a token-issuance rate limit or credential issue on KIS's side, not a
code defect. Backfilling at scale right now would just generate more 403s,
so left QE-M2-03 PENDING pending KIS account/console verification.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
42 lines
1.9 KiB
Python
42 lines
1.9 KiB
Python
from __future__ import annotations
|
|
|
|
import json
|
|
import re
|
|
from pathlib import Path
|
|
|
|
ROOT = Path(__file__).resolve().parents[1]
|
|
MIGRATION = ROOT / "src/dotnet/QuantEngine.Infrastructure/Migrations/V6__Add_Market_Time_Series.sql"
|
|
DBML = ROOT / "docs/db/quantengine.dbml"
|
|
REPORT = ROOT / "Temp/market_time_series_schema_v1.json"
|
|
TABLES = ("quantengine.price_history_daily", "quantengine.macro_history_daily")
|
|
|
|
|
|
def main() -> int:
|
|
migration = MIGRATION.read_text(encoding="utf-8") if MIGRATION.exists() else ""
|
|
dbml = DBML.read_text(encoding="utf-8") if DBML.exists() else ""
|
|
checks = {
|
|
"migration_exists": MIGRATION.exists(),
|
|
"dbml_exists": DBML.exists(),
|
|
"migration_tables": all(re.search(rf"CREATE TABLE IF NOT EXISTS {re.escape(t)}\b", migration) for t in TABLES),
|
|
"dbml_tables": all(re.search(rf"Table {re.escape(t)}\s*\{{", dbml) for t in TABLES),
|
|
"price_primary_key": "PRIMARY KEY (ticker, trade_date)" in migration,
|
|
"macro_primary_key": "PRIMARY KEY (symbol, trade_date)" in migration,
|
|
"dbup_embedded_glob": "Migrations/**/*.sql" in (ROOT / "src/dotnet/QuantEngine.Infrastructure/QuantEngine.Infrastructure.csproj").read_text(encoding="utf-8"),
|
|
}
|
|
payload = {
|
|
"formula_id": "MARKET_TIME_SERIES_SCHEMA_V1",
|
|
"gate": "PASS" if all(checks.values()) else "FAIL",
|
|
"expected_tables": list(TABLES),
|
|
"checks": checks,
|
|
"check_scope": "STATIC_STRUCTURAL_ONLY",
|
|
"check_scope_note": "No database connection - verifies migration SQL + DBML text only. Live-data verification is QE-M2-01's pg_query evidence gate in spec/60_quant_engine_wbs.yaml.",
|
|
}
|
|
REPORT.parent.mkdir(parents=True, exist_ok=True)
|
|
REPORT.write_text(json.dumps(payload, ensure_ascii=False, indent=2), encoding="utf-8")
|
|
print(json.dumps(payload, ensure_ascii=False, indent=2))
|
|
return 0 if payload["gate"] == "PASS" else 1
|
|
|
|
|
|
if __name__ == "__main__":
|
|
raise SystemExit(main())
|