feat(collection): wire KIS collection end-to-end, add price-history pipeline (WBS QE-M0/M1/M2)
Critical re-review of the QuantEngine WBS evidence system found several
regressions of the "no fake gates" discipline established by M0, plus a
still-unwired M1 collection path. This closes 10 more WBS tasks
(QE-M1-01..06, QE-M2-01/02/04/05/06 — see spec/60_quant_engine_wbs.yaml)
with real, gate-verified evidence (18/34 total).
M1 — real KIS data now lands in PostgreSQL end-to-end:
- SchedulerService: load ticker universe from GatherTradingData.json instead
of a hardcoded array; fix a Hangfire scoped-service resolution bug.
- KisDataCollectionOrchestrator: restore logging on the lineage-event write
path (was a bare `catch {}` swallowing all failures silently); persist
daily OHLCV bars into quantengine.price_history_daily per run.
- Verified live: POST /api/collection/run -> Hangfire -> orchestrator ->
KIS mock API -> PostgreSQL, with Playwright DOM/API parity evidence.
M2 — historical price-history pipeline:
- CollectionRepository: SavePriceHistoryDailyAsync (idempotent upsert),
GetPriceHistorySummaryAsync (per-ticker aggregation) + a new
DateOnlyTypeHandler registered globally, since Dapper has no built-in
System.DateOnly support in either direction (write threw
NotSupportedException, read threw a constructor-mismatch
InvalidOperationException — found by exercising both paths live).
- tools/validate_price_history_integrity_v1.py: gap-freeness (vs KIS
trading calendar) + price-sanity gate over collected history.
- Admin Collection page: new "히스토리 현황" summary table +
GET /api/collection/history-summary, with Playwright evidence.
Governance/gate fixes:
- validate_market_time_series_schema_v1.py mislabeled its own output
"runtime_database_query": "DATA_GATED" despite never opening a DB
connection (pure file/regex check) — relabeled "check_scope":
"STATIC_STRUCTURAL_ONLY" and wired the node into the release DAG so it
isn't only reachable from ci.yml, matching every other validator.
Live-data authority for the same claim stays with QE-M2-01's pg_query
gate (spec/60), documented in spec/64.
- Fixed a WBS log_pattern check (QE-M1-06) that couldn't match its own
multi-line target; loosened two depends_on edges (QE-M1-05/06,
QE-M2-04/05) that encoded "needs X verified" when the real requirement
was only "needs X's code merged."
- Discovered and fixed admin-pages.spec.ts logging in with the wrong
seeded password (admin/admin instead of admin/quant123!, per CLAUDE.md)
— every test in that suite had been silently failing at the login step.
Deferred: QE-M2-03 (2-year backfill) — the KIS mock/VTS token endpoint
started returning 403 after the first successful call this session; looks
like a token-issuance rate limit or credential issue on KIS's side, not a
code defect. Backfilling at scale right now would just generate more 403s,
so left QE-M2-03 PENDING pending KIS account/console verification.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
@@ -6,13 +6,14 @@ test.describe('관리자 페이지 플로우 테스트', () => {
|
||||
await page.goto('/Account/Login');
|
||||
await page.waitForLoadState('domcontentloaded');
|
||||
|
||||
// 로그인 수행 (admin/admin 자격증명)
|
||||
// 로그인 수행 (admin/quant123! 자격증명 — CLAUDE.md "Mandatory Pre-Deployment Checklist" 참조.
|
||||
// "admin/admin"은 실제 시드 비밀번호(V4 마이그레이션)와 불일치해 로그인 실패를 유발하던 버그였음.
|
||||
const usernameInput = page.locator('#username');
|
||||
const passwordInput = page.locator('#password');
|
||||
const loginButton = page.locator('#loginBtn');
|
||||
|
||||
await usernameInput.fill('admin');
|
||||
await passwordInput.fill('admin');
|
||||
await passwordInput.fill('quant123!');
|
||||
await loginButton.click();
|
||||
|
||||
// 로그인 후 페이지 로드 대기
|
||||
@@ -58,9 +59,9 @@ test.describe('관리자 페이지 플로우 테스트', () => {
|
||||
await page.goto('/Admin/Users/Create');
|
||||
await page.waitForLoadState('domcontentloaded');
|
||||
|
||||
// 페이지 타이틀 확인
|
||||
// 페이지 타이틀 확인 (실제 렌더링 타이틀: "새 사용자 추가 - QuantEngine")
|
||||
const title = await page.title();
|
||||
expect(title).toContain('생성');
|
||||
expect(title).toContain('사용자 추가');
|
||||
|
||||
// 폼 필드 확인
|
||||
const usernameField = page.locator('input[name="username"]');
|
||||
|
||||
@@ -8,13 +8,15 @@ test.describe('QE-M1-02: Collection Run List & Detail Verification', () => {
|
||||
await page.goto('/Account/Login');
|
||||
await page.waitForLoadState('domcontentloaded');
|
||||
|
||||
// Fill login form with credentials (admin/admin)
|
||||
// Fill login form with credentials (admin/quant123! — see CLAUDE.md
|
||||
// "Mandatory Pre-Deployment Checklist"; "admin/admin" does not match the
|
||||
// seeded password from V4 migration and silently fails login)
|
||||
const usernameInput = page.locator('#username');
|
||||
const passwordInput = page.locator('#password');
|
||||
const loginButton = page.locator('#loginBtn');
|
||||
|
||||
await usernameInput.fill('admin');
|
||||
await passwordInput.fill('admin');
|
||||
await passwordInput.fill('quant123!');
|
||||
await loginButton.click();
|
||||
|
||||
// Wait for login to complete
|
||||
@@ -107,8 +109,9 @@ test.describe('QE-M1-02: Collection Run List & Detail Verification', () => {
|
||||
console.log(`✓ Screenshot saved: 01-collection-page.png`);
|
||||
|
||||
// Step 7: Navigate to the run detail page
|
||||
// The detail page route is /Admin/Collection/{runId}
|
||||
await page.goto(`/Admin/Collection/${expectedRunId}`);
|
||||
// Detail.cshtml declares @page "{runId?}" under Pages/Admin/Collection/, so the
|
||||
// route is /Admin/Collection/Detail/{runId} (Razor Pages route = folder + page name + template)
|
||||
await page.goto(`/Admin/Collection/Detail/${expectedRunId}`);
|
||||
await page.waitForLoadState('domcontentloaded');
|
||||
|
||||
// Step 8: Verify detail page title contains the runId
|
||||
|
||||
@@ -0,0 +1,112 @@
|
||||
import { test, expect } from '@playwright/test';
|
||||
import * as fs from 'fs';
|
||||
import * as path from 'path';
|
||||
|
||||
test.describe('QE-M2-05: Price History Summary Tab Verification', () => {
|
||||
// Login before each test
|
||||
test.beforeEach(async ({ page }) => {
|
||||
await page.goto('/Account/Login');
|
||||
await page.waitForLoadState('domcontentloaded');
|
||||
|
||||
// Fill login form with credentials (admin/quant123! — see CLAUDE.md)
|
||||
const usernameInput = page.locator('#username');
|
||||
const passwordInput = page.locator('#password');
|
||||
const loginButton = page.locator('#loginBtn');
|
||||
|
||||
await usernameInput.fill('admin');
|
||||
await passwordInput.fill('quant123!');
|
||||
await loginButton.click();
|
||||
|
||||
// Wait for login to complete
|
||||
await page.waitForLoadState('domcontentloaded');
|
||||
});
|
||||
|
||||
test('QE-M2-05: History tab renders with API-derived ticker data', async ({ page }) => {
|
||||
// Step 1: Fetch expected values from API (source of truth)
|
||||
const apiResponse = await page.request.get('/api/collection/history-summary');
|
||||
expect(apiResponse.ok()).toBeTruthy();
|
||||
|
||||
const responseJson = await apiResponse.json();
|
||||
const tickers = (responseJson as any).tickers || [];
|
||||
|
||||
// Fail if no price history data exists
|
||||
if (tickers.length === 0) {
|
||||
throw new Error(
|
||||
'No price_history_daily rows in DB — run collection with price-history persistence first. ' +
|
||||
'Expected at least 1 ticker in price_history_daily table.'
|
||||
);
|
||||
}
|
||||
|
||||
// Extract expected values from first ticker
|
||||
const expectedTicker = tickers[0];
|
||||
const expectedTickerValue = expectedTicker.ticker;
|
||||
const expectedRowCount = expectedTicker.rowCount;
|
||||
const expectedFirstDate = expectedTicker.firstDate;
|
||||
const expectedLastDate = expectedTicker.lastDate;
|
||||
|
||||
console.log(
|
||||
`\n=== QE-M2-05 Test Started ===\n` +
|
||||
`Expected Ticker: ${expectedTickerValue}\n` +
|
||||
`Expected RowCount: ${expectedRowCount}\n` +
|
||||
`Expected FirstDate: ${expectedFirstDate}\n` +
|
||||
`Expected LastDate: ${expectedLastDate}\n`
|
||||
);
|
||||
|
||||
// Step 2: Navigate to Collection admin page
|
||||
await page.goto('/Admin/Collection');
|
||||
await page.waitForLoadState('domcontentloaded');
|
||||
|
||||
// Step 3: Verify page title contains "데이터 수집" (collection)
|
||||
const pageTitle = await page.title();
|
||||
expect(pageTitle).toContain('데이터 수집');
|
||||
|
||||
// Step 4: Verify history section is visible
|
||||
const historyCard = page.locator('h3.card-title:has-text("히스토리 현황")');
|
||||
await expect(historyCard).toBeVisible();
|
||||
console.log('✓ History card section found and visible');
|
||||
|
||||
// Step 5: Verify the table contains a row with expected ticker
|
||||
const tickerCell = page.locator(`td:has-text("${expectedTickerValue}")`).first();
|
||||
await expect(tickerCell).toBeVisible();
|
||||
console.log(`✓ Ticker row found: ${expectedTickerValue}`);
|
||||
|
||||
// Step 6: Find the row and verify row count matches
|
||||
const tableRow = tickerCell.locator('xpath=ancestor::tr');
|
||||
const cells = tableRow.locator('td');
|
||||
const cellCount = await cells.count();
|
||||
expect(cellCount).toBeGreaterThanOrEqual(4); // At least 4 columns (ticker, row count, first date, last date)
|
||||
|
||||
// Cell 1 (index 1) is "데이터 수" (row count)
|
||||
const rowCountCell = cells.nth(1);
|
||||
const rowCountText = await rowCountCell.textContent();
|
||||
expect(rowCountText?.trim()).toBe(String(expectedRowCount));
|
||||
console.log(`✓ Row count matches: ${rowCountText?.trim()} == ${expectedRowCount}`);
|
||||
|
||||
// Cell 2 (index 2) is "시작일" (first date)
|
||||
const firstDateCell = cells.nth(2);
|
||||
const firstDateText = await firstDateCell.textContent();
|
||||
expect(firstDateText?.trim()).toContain(expectedFirstDate);
|
||||
console.log(`✓ First date matches: ${firstDateText?.trim()} contains ${expectedFirstDate}`);
|
||||
|
||||
// Cell 3 (index 3) is "종료일" (last date)
|
||||
const lastDateCell = cells.nth(3);
|
||||
const lastDateText = await lastDateCell.textContent();
|
||||
expect(lastDateText?.trim()).toContain(expectedLastDate);
|
||||
console.log(`✓ Last date matches: ${lastDateText?.trim()} contains ${expectedLastDate}`);
|
||||
|
||||
// Step 7: Create screenshot directory and take screenshot
|
||||
const screenshotDir = path.join(process.cwd(), 'Temp', 'evidence', 'QE-M2-05', 'screenshots');
|
||||
fs.mkdirSync(screenshotDir, { recursive: true });
|
||||
|
||||
await page.screenshot({
|
||||
path: path.join(screenshotDir, '01-history-tab.png'),
|
||||
fullPage: true,
|
||||
});
|
||||
console.log(`✓ Screenshot saved: 01-history-tab.png`);
|
||||
|
||||
console.log(
|
||||
`\n=== QE-M2-05 Test Completed Successfully ===\n` +
|
||||
`Evidence files saved to: ${screenshotDir}\n`
|
||||
);
|
||||
});
|
||||
});
|
||||
Reference in New Issue
Block a user