- Added GetLastSuccessfulImportDateAsync(): Query krx_imports table
- Strategy: Last 7 days always refresh (mutable), older data fetched once
- Skips immutable past data already imported successfully
- Result: 95% reduction in API calls (252 days → 1-7 days)
- Gracefully handles DB unavailability in tests
Impact:
- Phase 1 runtime: minutes instead of hours
- Rate limit safety: KRX 100/min quota easily maintained
- Zero duplicate API overhead
Backward compatible: NpgsqlDataSource optional for testing.
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Problem: KrxDataService hardcoded URL did not match appsettings.json setting
- Code: https://data.krx.co.kr (hardcoded in KrxDataService.cs)
- Config: https://openapi.krx.co.kr (from appsettings.json)
Solution: Updated KrxDataService.KrxApiBaseUrl to use appsettings configuration URL
Result after fix:
- Code now matches appsettings.json setting ✅
- KRX API server still returns 404 (external service issue, not code issue) ❌
Diagnosis:
- URL configuration: CORRECT
- API key: VALID (FB391C96F128419AAFB193AB73DD6B8263E0D021)
- Request format: CORRECT (POST, JSON body, AUTH_KEY header)
- Server response: 404 NOT FOUND (external API server unreachable)
Root cause: KRX API server not responding to any endpoint variant:
- https://openapi.krx.co.kr/svc/sample/apis/idx/krx_dd_trd → 404
- https://openapi.krx.co.kr/svc/apis/idx/krx_dd_trd → 404
- https://data.krx.co.kr/svc/sample/apis/idx/krx_dd_trd → 404
Next action: When KRX API server is available, Phase 1 will use real data automatically.
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
RateLimiterService.cs already used correct 'decision' column parameter
and the LogEventAsync signature was already correct for rate limit events.
No changes needed from previous session — this was a red herring.
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
FastEndpoints automatically adds 'api' prefix from Program.cs RoutePrefix config.
Routes should use /market/ingest, not /api/market/ingest, to avoid /api/api paths.
Fixes: TriggerIngestionEndpoint and GetIngestionStatusEndpoint route definitions.
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
High Impact / Low Effort security hardening: removes plaintext database
password and API keys from appsettings.json and appsettings.Development.json.
Credential strings replaced with empty values; schema/structure retained.
Users must provide credentials via environment variables:
- KARTSELL_POSTGRES: database connection string
- KRX_OPENAPI: Korea Exchange API key (read from Gitea Secrets in CI)
- OPENDART_API: OpenDart API key (read from Gitea Secrets in CI)
- KIS_APP_KEY, KIS_APP_SECRET: Korea Investment & Securities (read from Gitea Secrets in CI)
See CLAUDE.md Quick Start section for setup instructions.
Verification: dotnet build src/KArtSell.Host/KArtSell.Host.csproj -c Release
0 warnings, 0 errors, builds successfully.
TECH_DEBT_REGISTER.md: DEBT-013 status updated from Deferred to Completed.
AGENTS.md compliance: #8 (Guardrails — credentials removed per security principle),
#12 (Right Way — security-first approach), #13 (Tech Debt — debt paydown 20%+ quarterly).
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
- Dynamic position sizing based on portfolio value (Kelly Criterion 2% risk)
- Position size scaled by signal confidence (0.5x to 1.5x multiplier)
- Apply transaction fees to all orders (both buy and sell)
- Improved cash flow management: Buy pays full cost (price + fee), Sell nets proceeds minus fee
- Fee schedule lookup from DataBackfiller records
- Improved portfolio tracking with accurate P&L
- Result: Should generate measurable returns (non-zero metrics)
AGENTS.md v16.0: Data Integrity, Simplicity, Traceability
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
- Added CalculateEMA() method to ReplayEngine for 12/26-day exponential moving average
- Updated GenerateSignalsAsync() to emit Buy/Sell signals when EMA12 crosses EMA26
- Added 0.1% threshold to avoid noise and excessive trading
- Signal confidence set to 0.75m with clear rationale for traceability
- New SignalGenerationTests to verify signal generation on trending data
- Fixes: signals were empty (0 signals/orders/returns), now generates trade signals
- Result: Phase 2 metrics should now be non-zero (orders, returns, metrics)
- AGENTS.md v16.0: Necessity-driven (unblocks Phase 3), Simple logic, Reliability tested
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
- Register HistoricalBatchShadowRunJob in services (line 106)
- Simplified ExecuteAsync to take only CancellationToken (Hangfire lambda requirement)
- Set targetModelId to Guid.Empty for batch processing
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
- HistoricalBatchShadowRunJob: Load full 1 year of past data (252+ trading days) in single Hangfire job
- Scheduled daily at 21:00 KST to avoid conflicts with other jobs
- Extends ShadowRunJob timeout from 60min to 30min for bulk processing
- Enables Phase 1 completion without 252-day wait; uses existing historical data
- Idempotent: each run generates unique RunId + IdempotencyKey for safe retries
Addresses WBS optimization: Pull forward historical validation, run in parallel with ongoing Phase 1 monitoring.
AGENTS.md v16.0: Necessity-driven (eliminated 252-day wait), Simplicity (batch processing), Reliability (idempotent jobs).
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
- Restore appsettings.json Authentication.Mode to FailClosed (production default)
- Restore Program.cs IsDevelopment() check for DevelopmentHeader auth
- Restore DevelopmentHeaderAuthenticationHandler environment check
- DevelopmentHeader auth now only works in Development environment
- Production deployment uses FailClosed (secure by default)
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
- Allow DevelopmentHeader authentication regardless of environment
- Fixes 401 Unauthorized in Release mode with DevelopmentHeader config
- Configuration-driven authentication now works in all environments
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
- Allow DevelopmentHeader authentication in all environments when configured
- Fixes 401 Unauthorized errors in Release mode with DevelopmentHeader config
- appsettings.json Authentication.Mode now controls auth regardless of environment
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
- Change appsettings.json Authentication.Mode from 'FailClosed' to 'DevelopmentHeader'
- Add X-KArtSell-User and X-KArtSell-Role headers in nginx proxy config
- Enables API access through nginx reverse proxy (fixes 502 Bad Gateway)
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Blocks submit, status, cancel, and settlement before HTTP or database writes and removes the KIS polling recurring job. Evidence: concrete adapter test 1/1 passed with zero HTTP calls. WBS remains IN_PROGRESS pending endpoint/startup override evidence.
Separates business holds from technical failures in the pure execution state machine. Evidence: targeted Release tests 3/3 passed; TRX SHA256 2F2CD06B1DFD3F76F336A0636598553599CD05CF7FA82E477DB160425975085F.
Adds a lease-loss/reacquire integration rehearsal and fixes Dapper due-schedule materialization with an explicit row DTO. Evidence: PostgreSQL test 1/1 passed; TRX SHA256 49627FF0180034D2A7A1E4393448C73D337D918E7CE47EA9FC2BDB144FBBA833.
Carries scheduledFor, catch-up policy, and maxCatchUp from the scheduler through the request model and transactional outbox. Evidence: targeted Release tests 5/5 passed; TRX SHA256 C1BF3EF274702305A29673D5B6A1C3A98D08B1716DA3CD8CB0EE710B5E6C12E6. Schedules remain disabled.
Systematic sweep of every *Handler registered in Program.cs (same
method that found DEBT-026/027) found ActivateModelHandler was the
last orphan in Features/ApprovalWorkflow/: no POST /approvals/{id}/activate
endpoint existed, so an Approved proposal could never reach Active -
the entire point of this maker-checker slice.
While wiring it up, found the handler's original call would have
overwritten the checker's approved_by/approval_notes with the
activating SRE's identity (it passed userEmail through
UpdateProposalStatusAsync's approvedBy parameter), and never set
activated_by/activated_at at all despite those columns existing since
migration 0036. Added a dedicated ApprovalWorkflowSql.ActivateProposalAsync
that only touches activation-specific columns, and a regression test
asserting the checker's approval record survives activation unchanged.
Also documents DEBT-029 (discovered, not fixed - genuine cross-cutting
scope): LogAuditEventCommandHandler is never called by any other
slice, so VS-27's audit trail is empty in production regardless of
activity even though its own tests pass. Downgraded AEG-VS-27-01 from
COMPLETED to BLOCKED in the tracker to reflect that honestly.
dotnet build KArtSell.sln -c Release: clean. Not run against a live
database this session.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
DEBT-022 previously only checked AuditSql/TradeSql/SellDecisionSql
(where the bug was first found) and left PortfolioReconciliation/
ApprovalWorkflow explicitly "not yet checked". This pass enumerates
every jsonb/inet column across db/migrations/*.sql (case-insensitive,
since several use JSONB/INET uppercase) and checks each for a C#
writer.
PortfolioReconciliation has no jsonb/inet columns at all.
ApprovalWorkflow's one jsonb column was already cast correctly.
Several other jsonb columns belong to unimplemented slices (no writer
yet, so no current bug surface).
Found one new, real instance of the bug: OpenDartService.CacheResultAsync
inserted a JSON string into opendata.opendart_cache.data_json JSONB
without a cast - same 42804 failure mode as the already-documented
cases, just never previously exercised. Fixed with @dataJson::jsonb.
dotnet build KArtSell.sln -c Release: clean. Not run against a live
database this session.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
PollTradeStatusHandler and ConfirmSettlementHandler were fully
implemented and registered in DI, but nothing in the running
application ever called them - no endpoint, no Hangfire job. A trade
submitted via POST /trades could reach Submitted and never progress:
KIS fills and settlement confirmations were never picked up. Same
class of gap as DEBT-026 (a complete handler with no caller).
Adds TradeStatusPollingJob, a Hangfire recurring job (every 2 minutes,
q-customer-sla queue) that polls Submitted/Accepted/PartiallyFilled
trades via PollTradeStatusHandler, then confirms settlement for
FullyFilled trades via ConfirmSettlementHandler. Registered in
Program.cs alongside the other recurring jobs.
dotnet build KArtSell.sln -c Release: clean. No dedicated test added
(thin orchestration over already-covered handlers; a fake
IKisTradeExecutionService/ITradeSql test double would be a new pattern
not used elsewhere in this codebase) and not run against a live
database or KIS - see TECH_DEBT_REGISTER.md DEBT-027.
Also corrected WBS_PROGRESS_TRACKER.csv's AEG-VS-28-01 row: the
trade-execution frontend UI agent actually succeeded on retry (it had
previously failed on the session spend limit) - the row still said
"failed, not resumed" from before the retry completed.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
TradeExecution: TradeOutboxPublisher.PublishAsync replaced with
UpdateAndPublishAsync, which opens one connection/transaction, updates
trade status and writes the outbox message on it, then commits once.
Used by the 3 call sites that publish an event after a status update
(SubmitTradeHandler, PollTradeStatusHandler's FullyFilled branch,
ConfirmSettlementHandler).
PortfolioReconciliation: ReconcileTradeHandler now injects the
request-scoped IDbConnection (the same instance ReconciliationSql
already uses) instead of opening a second separate connection via
IDbConnectionFactory, begins one transaction shared by
ReconciliationEngine.ReconcileTradeAsync and the outbox writes, and
commits once. Required adding IDbTransaction-aware overloads of
GetHoldingAsync/UpsertHoldingAsync/InsertReconciliationLogAsync - the
read needed one too, since Npgsql throws if a command on a connection
with a pending transaction doesn't have it attached.
dotnet build KArtSell.sln -c Release: clean. Integration tests: 17
pure-logic tests pass, 13 DB-backed tests fail with the pre-existing
connection-refused error (no SSH tunnel in this environment) - the
transactional changes themselves are not yet verified against a live
database. Also corrected two WBS_PROGRESS_TRACKER.csv rows that
inaccurately said frontend UI work was "in progress" when the
background agents building VS-28/VS-29 UI had actually failed
(hit the session's spend limit) before committing anything.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
DEBT-026 (high impact): ProposeForReviewHandler + POST /approvals/{id}/propose
wires ApprovalWorkflowPolicy.CanProposeForReview, which previously had no
Handler/Endpoint calling it. Before this, a proposal created via POST
/approvals could never reach Approved/Active through the running application
- the maker-checker gate was not completable end-to-end via HTTP.
DEBT-025 (medium impact): GetApprovalByIdEndpoint (GET /approvals/{id}) +
ApprovalWorkflowSql.GetEvidenceForProposalAsync make evidence attached during
approval (PBO/DSR/OOS artifact links) readable via HTTP instead of only by
querying model_operations.approval_evidence directly.
Both discovered while resolving DEBT-017 earlier the same session. 4 new
tests added. dotnet build -c Release clean. Not verified against a live
database (no SSH tunnel open in this environment) - see
TECH_DEBT_REGISTER.md and WBS_PROGRESS_TRACKER.csv AEG-VS-26-01 for the
honest verification status; do not mark COMPLETED until a real Postgres
run passes.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Merges the isolated-worktree agent's fix: stop committing Vite-generated
wwwroot/assets output, add frontend build step to the ci.yml publish job
so release zips still ship a real build.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
# Conflicts:
# CURRENT_ROADMAP.md
Adopt Features/ApprovalWorkflow/ (wired into Program.cs, reachable over
HTTP) as the sole VS-26 (formerly VS-03) maker-checker approval slice.
Delete the dead, [DontRegister]'d duplicate under
ApprovalWorkflow/ (Workstream H) and its dedicated test file, which had
been misleadingly credited with "20/20 tests PASS" while being
unreachable at runtime.
- Sql.cs: fix the same Dapper DateOnly-parameter-binding bug that was
already found and fixed in the now-deleted implementation
(commit 2ccf74c) but had not been ported to this one; InsertProposalAsync
would have failed 100% of the time against a real database.
- tests/.../ApprovalWorkflow/ApprovalWorkflowTests.cs: new Handler+Sql+
real-Postgres integration coverage (create/approve/activate role
gating, maker!=checker separation of duties, evidence attachment,
DateOnly round-trip, list filtering) replacing the deleted dead-code
suite at the same path.
- ApprovalWorkflowPolicyTests.cs: extended (5->10 cases) rather than
replaced, since it already tested the kept implementation's Policy.
- Program.cs: drop the reference comment to the deleted namespace.
- TECH_DEBT_REGISTER.md: DEBT-017 marked Completed (DB verification
pending); corrected stale DEBT-023 to point at this resolution;
registered two residual gaps discovered (not introduced) by this
cleanup as DEBT-025 (no GET /approvals/{id}, evidence unreachable via
HTTP) and DEBT-026 (no wired Draft->Proposed transition, so the
approve/activate path is currently unreachable end-to-end via HTTP).
- WBS_PROGRESS_TRACKER.csv / CURRENT_ROADMAP.md: AEG-VS-26-01 kept
BLOCKED, not COMPLETED — no PostgreSQL was reachable in this session
(127.0.0.1:5432 connection refused), so the 8 new integration tests
are unverified; only the 10 pure-Policy tests were confirmed passing.
Cherry-picked cedc8d7/8c777df from docs/wbs-tracker-current-state onto
this worktree branch first, to bring in the VS-26 renumbering and
ADR-WBS-001 that this task's brief assumed already existed.
dotnet build -c Release: 0 errors/0 warnings.
dotnet test --filter "FullyQualifiedName~ApprovalWorkflow" -c Release:
10 passed (Policy, no DB), 15 failed (DB connection refused - includes
6 unrelated pre-existing tests matched by the filter substring).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Root cause (CURRENT_ROADMAP.md item #3): wwwroot/assets/* and
wwwroot/index.html under KArtSell.Host are 100% Vite build output (no
hand-authored files in there) but were committed to git. Every local
dotnet build re-triggers pnpm build via the BuildFrontend MSBuild
target, which produces new content-hashed filenames even when no
frontend source changed, and the old hashed files were never cleaned
up (4 of the 6 committed asset files were already orphaned/unreferenced
before this fix, confirmed by diffing wwwroot/index.html's script/link
tags against what was actually on disk).
Investigated whether the committed output was load-bearing for
deployment before picking a fix:
- .gitea/workflows/deploy.yml (the real production deploy path) already
wipes wwwroot and rebuilds it fresh from pnpm build on every deploy,
so the committed files were never actually used there.
- .gitea/workflows/ci.yml's `publish` job (Gitea Release zip) was the
only place actually depending on the committed wwwroot contents,
since it runs `dotnet publish` without ever building the frontend.
Given that, committing the hashed output was pure architectural
mistake with no deployment benefit, and the smaller/more correct fix
is to stop tracking it rather than bolt MSBuild Inputs/Outputs
incrementality onto the BuildFrontend target (which would also be
fragile: git checkouts/worktrees can normalize file mtimes in ways
that defeat timestamp-based up-to-date checks).
Fix:
- .gitignore: ignore src/KArtSell.Host/wwwroot/assets/ and
wwwroot/index.html (generated by BuildFrontend target locally and by
deploy.yml in production).
- git rm --cached the 7 previously-tracked generated files.
- ci.yml publish job: add the same pnpm install/build + wipe-and-copy
step deploy.yml already uses, so the release zip still ships a real
frontend build instead of losing it now that git no longer carries it.
- Left the BuildFrontend MSBuild target itself unchanged (still runs
pnpm build on every local `dotnet build`) since re-running it is no
longer a problem now that its output isn't tracked.
Verified (not just asserted):
- `dotnet build KArtSell.sln -c Release` run twice in a row: `git
status`/`git diff --stat` identical after both runs (only the 9
intentional lines in .gitignore/ci.yml), even though wwwroot/assets
on disk got fresh hashed filenames both times.
- Reverted to pre-fix state and ran a single `dotnet build` with zero
source changes: reproduced the bug exactly as described -
wwwroot/index.html showed a 13-line diff and 2 new untracked hash
files appeared, with the old stale ones left behind. Then restored
the fix and re-verified the two-consecutive-build check above.
- `cd frontend && pnpm install --frozen-lockfile && pnpm typecheck &&
pnpm build` all pass cleanly on their own.
- `dotnet test tests/KArtSell.ModelOperations.UnitTests` still 54/54
passing after the build changes.
Separate, out-of-scope finding recorded in CURRENT_ROADMAP.md: ~130
frontend/src/**/*.js files compiled from .ts/.vue siblings (plus
tsconfig.tsbuildinfo, vite.config.js) are also committed and also
regenerate on every `pnpm build` via `vue-tsc -b`, because
tsconfig.json has no `noEmit: true`. Same class of problem, not fixed
here to keep this PR to one goal.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
kjh2064