fix: DEBT-022 - complete repo-wide jsonb/inet cast audit, fix OpenDartService

DEBT-022 previously only checked AuditSql/TradeSql/SellDecisionSql
(where the bug was first found) and left PortfolioReconciliation/
ApprovalWorkflow explicitly "not yet checked". This pass enumerates
every jsonb/inet column across db/migrations/*.sql (case-insensitive,
since several use JSONB/INET uppercase) and checks each for a C#
writer.

PortfolioReconciliation has no jsonb/inet columns at all.
ApprovalWorkflow's one jsonb column was already cast correctly.
Several other jsonb columns belong to unimplemented slices (no writer
yet, so no current bug surface).

Found one new, real instance of the bug: OpenDartService.CacheResultAsync
inserted a JSON string into opendata.opendart_cache.data_json JSONB
without a cast - same 42804 failure mode as the already-documented
cases, just never previously exercised. Fixed with @dataJson::jsonb.

dotnet build KArtSell.sln -c Release: clean. Not run against a live
database this session.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
2026-08-09 00:25:14 +09:00
parent ace9fe8a9c
commit cbcc4849ec
2 changed files with 2 additions and 2 deletions
+1 -1
View File
@@ -54,7 +54,7 @@
| DEBT-019 | Multiple duplicate cross-cutting abstractions (`IClock`, `IOutboxWriter`, `IKrxDataService`) | Medium (2) | Low (1) | Completed (partial) | Found and collapsed 3 separate cases where a slice reinvented an abstraction that already existed in `KArtSell.BuildingBlocks`: a second `IKrxDataService` (deleted, `ShadowRun.Services`), a second `IOutboxWriter`/`WriteAsync<T>` in `ReconcileTradeHandler.cs` (removed, switched to `BuildingBlocks.Reliability.IOutboxWriter`), and a second `IClock`/`SystemClock` in `ApprovalWorkflow/ApprovalPolicy.cs` (removed, switched to `BuildingBlocks.Time.IClock`). Root cause: successive sessions implementing a slice without searching `BuildingBlocks` first. Recommend a pre-implementation checklist step ("does this abstraction already exist in BuildingBlocks?") for future slices. | @claude | Session 2026-08-07 (Phase 3 J/K/L hardening) |
| DEBT-020 | `model_operations.models` and `compliance` schema never created by any migration | High (3) | Low (1) | Completed | `0036`/`0038` reference `model_operations.models(id)` via FK and `OpenDartDailyBatchJob.cs` queries it directly, but no migration ever ran `CREATE TABLE model_operations.models`; `0037` wrote to `compliance.*` tables without `CREATE SCHEMA compliance`. Any fresh database — including the actual deploy target (178.104.200.7), confirmed via a live failed SCP/DbMigrator deploy on 2026-08-07 — failed at migration `0036`/`0037`. Fixed via new `0035_model_operations_models.sql` (minimal: id/ticker/published_at/correlation_id/revision only — full Model Card schema is separate future work) and `CREATE SCHEMA IF NOT EXISTS compliance;` added to `0037`. Full chain 0000→0040 now verified fresh-install + idempotent re-run clean. | @claude | Session 2026-08-07 (deploy failure triage) |
| DEBT-021 | Dapper never configured for snake_case↔PascalCase column mapping | High (3) | Low (1) | Completed | `Dapper.DefaultTypeMap.MatchNamesWithUnderscores` was never set anywhere in the codebase, so every `QueryAsync<T>`/`QuerySingleOrDefaultAsync<T>` result-mapping onto a snake_case DB column (e.g. `event_type``EventType`) silently returned null/default for that property instead of throwing — masking the bug in every Sql class across every module. Confirmed via `ApprovalWorkflowTests.InsertAndRetrieveProposal_RoundTrips` and `AuditTrailTests.InsertAuditEvent_CreatesImmutableRecord` both getting real rows back with null fields. Fixed centrally via a `[ModuleInitializer]` in `KArtSell.BuildingBlocks/Data/DapperBootstrap.cs` (runs once per process regardless of entry point — Host/DbMigrator/tests). | @claude | Session 2026-08-07 (deploy failure triage) |
| DEBT-022 | jsonb/inet columns written as plain text without an explicit cast | Medium (2) | Low (1) | Completed (partial) | Dapper does not know to cast a `string` parameter to `jsonb`/`inet` for Npgsql; `AuditSql.InsertAuditEventAsync` (`details`, `ip_address`), `AuditSql.RedactAuditEventDetailsAsync` (duplicate `SET details =` assignment, separately fixed), `TradeSql.InsertTradeAsync`/`UpdateTradeStatusAsync` (`kis_response`), and `SellDecisionSql.InsertDecisionAsync` (`oos_performance`) all failed with `42804: column "x" is of type jsonb but expression is of type text` the first time they were run against a real schema. Fixed with explicit `::jsonb`/`::inet` casts at each call site (mechanical, no behavior change). `AuditSql`'s jsonb read-back (`Dictionary<string,object>` from a jsonb column) also needed a raw-DTO + `JsonSerializer.Deserialize` mapping since Dapper has no built-in jsonb→Dictionary conversion either. **Not yet checked**: `PortfolioReconciliation`/`ApprovalWorkflow` Sql classes for the same pattern beyond what surfaced in this session's test runs — a full audit of jsonb/inet columns across all Sql classes is still open. | @claude | Session 2026-08-07 (deploy failure triage) |
| DEBT-022 | jsonb/inet columns written as plain text without an explicit cast | Medium (2) | Low (1) | Completed | Dapper does not know to cast a `string` parameter to `jsonb`/`inet` for Npgsql; `AuditSql.InsertAuditEventAsync` (`details`, `ip_address`), `AuditSql.RedactAuditEventDetailsAsync` (duplicate `SET details =` assignment, separately fixed), `TradeSql.InsertTradeAsync`/`UpdateTradeStatusAsync` (`kis_response`), and `SellDecisionSql.InsertDecisionAsync` (`oos_performance`) all failed with `42804: column "x" is of type jsonb but expression is of type text` the first time they were run against a real schema. Fixed with explicit `::jsonb`/`::inet` casts at each call site (mechanical, no behavior change). `AuditSql`'s jsonb read-back (`Dictionary<string,object>` from a jsonb column) also needed a raw-DTO + `JsonSerializer.Deserialize` mapping since Dapper has no built-in jsonb→Dictionary conversion either. **2026-08-09: full audit completed** (repo-wide, not just Portfolio/Approval). Enumerated every `jsonb`/`inet` column across `db/migrations/*.sql` (case-insensitive — several use `JSONB`/`INET` uppercase, which an earlier lowercase-only grep would have missed), then checked each one for a C# writer. Findings: `PortfolioReconciliation`'s tables (`portfolio_management.holdings`/`reconciliation_logs`) have no `jsonb`/`inet` columns at all — nothing to fix. `ApprovalWorkflow`'s one `jsonb` column (`approval_events.details`) was already cast correctly in `InsertEventAsync`. Several other `jsonb` columns (`evidence_snapshot.payload`, execution-assurance/model-feedback tables under `evaluation`/`governance`) have no C# writer yet at all — those slices (VS-05/09/19 etc.) are unimplemented, so there's no bug surface yet; flag for re-check whenever they get built. **One new, real instance of this exact bug found and fixed**: `OpenDartService.CacheResultAsync` (`src/KArtSell.Host/Observability/OpenDartService.cs`) inserted a serialized JSON string into `opendata.opendart_cache.data_json JSONB` without a cast — same `42804` failure mode as the others, just never previously exercised/caught. Fixed with `@dataJson::jsonb`. `dotnet build -c Release` clean; not run against a live database this session (see the rest of this session's entries for why). | @claude | Session 2026-08-07 (deploy failure triage, discovery), Session 2026-08-09 (full audit + OpenDartService fix) |
| DEBT-023 | `ApprovalSql.InsertProposalAsync` fails on `DateOnly` parameter | Medium (2) | Low (1) | Completed | Stale entry, corrected 2026-08-08: this described `ApprovalSql.cs` under `src/KArtSell.Modules.ModelOperations/ApprovalWorkflow/` — that per-call-site fix (`::date` cast + `"yyyy-MM-dd"` string parameter, not a centralized type handler) landed in commit `2ccf74c` but this row was never updated to reflect it. That whole file was then deleted as dead code while resolving DEBT-017 (2026-08-08); its surviving sibling, `Features/ApprovalWorkflow/Sql.cs`, was found to have the *same* unfixed bug independently and received the identical fix in that session — see DEBT-017. No centralized `DateOnly` type handler was added; this remains a per-call-site fix pattern, so any *other* `DateOnly`-typed Dapper INSERT elsewhere in the codebase should still be checked individually rather than assumed safe. | @claude | commit 2ccf74c; DEBT-017 (this session) |
| DEBT-024 | New integration tests don't insert FK parent rows / one pure-logic test flakes under full-suite run | Low (1) | Low (1) | Backlog | `TradeExecutionTests` constructs `Trade` with a random `sellDecisionId` that was never inserted into `sell_decisions`, so every insert now correctly fails its FK constraint (`trades_sell_decision_id_fkey`) once the schema was actually complete (see DEBT-020) — test-only gap, not a production code defect; needs the tests updated to insert a parent `models`+`sell_decisions` row first. Separately, `SellPriorityRankerTests.CalculateScore_HardImpairment_ReturnsLowestScore` (pure logic, no DB) passed in isolation but returned 1000 instead of the expected 950 (age-boost not applied) when run as part of the full suite — not yet root-caused; may be test-order/parallelization state leakage rather than a `SellPriorityRanker` bug. Also, `DbUpMigrationTests.*` (pre-existing, unrelated to this session) fail locally with `42501: must be owner of database kartsell_migration_test` — a local Postgres role permission gap, not a code issue. | @claude | Session 2026-08-07 (deploy failure triage) |
| DEBT-025 | `Features/ApprovalWorkflow` has no `GET /approvals/{id}` endpoint | Medium (2) | Low (1) | Completed (DB verification pending) | Added `GetApprovalByIdEndpoint` (`GET /approvals/{id}`) + `ApprovalDetailResponse` (includes `Evidence`), and `ApprovalWorkflowSql.GetEvidenceForProposalAsync`. Evidence attached during approval (PBO/DSR/OOS artifact links) is now readable via HTTP. Two new tests added (`GetEvidenceForProposalAsync_ReturnsEvidenceAttachedDuringApproval` + the endpoint itself). `dotnet build -c Release` clean (0/0). **Not verified against a live database** — same 127.0.0.1:5432 connection-refused blocker as DEBT-017/026; do not mark fully verified until a real Postgres run passes. | @claude | DEBT-017 (2026-08-08), `src/KArtSell.Modules.ModelOperations/Features/ApprovalWorkflow/README.md` |
@@ -153,7 +153,7 @@ public class OpenDartService
{
const string sql = """
INSERT INTO opendata.opendart_cache (ticker, quarter, data_json, expires_at, published_at)
VALUES (@ticker, @quarter, @dataJson, @expiresAt, @publishedAt)
VALUES (@ticker, @quarter, @dataJson::jsonb, @expiresAt, @publishedAt)
ON CONFLICT (ticker, quarter) DO UPDATE SET
data_json = EXCLUDED.data_json,
expires_at = EXCLUDED.expires_at,