# Phase 5: Hangfire Registration + Result Polling (AGENTS.md v16.0)

## 1. SOURCE (Requirements)

**From CLAUDE.md:**
- § "Hangfire (Background Jobs & Scheduling)": "Hangfire executes approved Application Commands"
- § "Validation Gates": "252+ trading days shadow, OOS testing, PBO/DSR verification"
- § "Database & Migrations": "Migrations are idempotent and checksummed"

**From Infrastructure Contract:**
- Shadow run persists to database (shadow_run table)
- Metrics/gates populated when job completes
- Async job model (202 Accepted + polling)

---

## 2. SLICE SPEC (Vertical Slices)

### 2.1 Hangfire Job Registration

**Goal:** Register ShadowRunJob as recurring or manual trigger  
**Non-Goal:** Auto-scheduling (user-triggered only, not recurring)  
**Pattern:** Startup-time registration in `Program.cs`

**Workflow:**
1. App startup: `Program.cs` registers `ShadowRunJob` handler
2. User POSTs `/api/shadow-runs` → Handler enqueues job
3. Hangfire processes job from `q-research` queue
4. Job updates `shadow_run.status` as phases progress
5. Final: Job writes `shadow_run.published_at` = complete

**Idempotency:**
- Job ID deduplication: Hangfire prevents duplicate execution
- Idempotency-Key in endpoint → prevents duplicate job creation

### 2.2 GET /api/shadow-runs/{run_id} Polling Endpoint

**Goal:** Poll shadow run status and retrieve results  
**Non-Goal:** WebSocket real-time updates (polling only)  
**Response:** 200 OK (in progress) or 200 with metrics (complete)

**Contracts:**

```
GET /api/shadow-runs/{run_id}

Response (200 OK):
{
  "run_id": "uuid",
  "model_id": "uuid",
  "status": "Queued|DataBackfill|Replay|EvaluationComplete|Failed",
  "created_at": "2026-08-02T12:34:56Z",
  "completed_at": "2026-08-02T13:34:56Z" (null if in progress),
  
  // Present only when status = EvaluationComplete:
  "metrics": {
    "total_return": 0.15,
    "sharpe_ratio": 1.2,
    "pbo": 0.15,
    "dsr_percentile": 0.95,
    "max_drawdown": 0.08
  },
  "validation_gates": {
    "pbo_under_20": true,
    "dsr_above_95": true,
    "cost_2x_positive": true,
    "all_gates_passed": true
  },
  "phase_analysis": { ... },
  "cost_analysis": { ... },
  "error_message": null
}

404 Not Found: Run ID doesn't exist or user lacks permission
```

**PIT Safety:** Query includes `published_at <= @cutoff`

---

## 3. CONTRACT (Input/Output)

| Operation | Input | Output | Status Code | Idempotent |
|-----------|-------|--------|-------------|-----------|
| Register Job (startup) | Program builder context | Job registered in Hangfire | N/A | ✅ (idempotent registration) |
| Enqueue via POST | ShadowRunCommand | Job ID returned (202) | 202 | ✅ (Idempotency-Key) |
| Poll GET | run_id (URL param) | shadow_run row + metrics | 200 / 404 | ✅ (read-only) |

---

## 4. DATA (Schema + Queries)

**Existing table:** `model_operations.shadow_run`  
**Columns to query:**
- `status` (for polling progress)
- `created_at`, `published_at` (timing)
- `metrics_json`, `validation_gates_json` (results)
- `error_message` (failure context)

**Queries:**

```sql
-- Get shadow run by ID (with PIT safety)
SELECT run_id, model_id, status, created_at, published_at, 
       metrics_json, validation_gates_json, error_message, cost_analysis_json
FROM model_operations.shadow_run
WHERE run_id = @RunId
  AND published_at <= @Cutoff;

-- Progress update (job status changes)
UPDATE model_operations.shadow_run
SET status = @Status, updated_at = NOW()
WHERE run_id = @RunId;

-- Mark complete
UPDATE model_operations.shadow_run
SET status = 'EvaluationComplete',
    published_at = @Now,
    metrics_json = cast(@Metrics as jsonb),
    validation_gates_json = cast(@Gates as jsonb)
WHERE run_id = @RunId;
```

---

## 5. TESTS (Verification)

| Level | Scenario | Check |
|-------|----------|-------|
| Unit | ShadowRunJob registers without error | Hangfire handler callable |
| Unit | GET deserializes JSONB metrics correctly | Metrics type-safe |
| Integration | Job enqueue → status progress → complete | Full shadow run lifecycle |
| Integration | Polling before complete → 200 with status | Async progress visible |
| Integration | Polling after complete → 200 with metrics | Results accessible |
| E2E | POST → 202 → GET polls → 200 with gates | User-facing workflow |

---

## 6. OPS (Deployment + Monitoring)

**Startup:**
1. `Program.cs` creates Hangfire `RecurringJobManager` or manual trigger
2. `ShadowRunJob` handler registered
3. Hangfire background server starts listening on `q-research`

**Monitoring:**
- Queue depth: Alert if `q-research` > 5 jobs pending
- Job execution time: Track phase completion timestamps
- Polling latency: Alert if response time > 5s (likely job failure)

**Rollback:**
- If job fails: `published_at` remains NULL, status = 'Failed'
- User sees error_message in polling response
- Retry: User can re-POST with same Idempotency-Key or new key

---

## 7. OUTPUT RULE (Deliverables)

**Changed files:**
```
src/KArtSell.Host/
  Program.cs                           (Hangfire registration)
  Features/ShadowRun/
    GetShadowRunQuery.cs               (DB queries for polling)
    GetShadowRunPollingEndpoint.cs     (GET /api/shadow-runs/{run_id})
    GetShadowRunResponse.cs            (DTO)

tests/KArtSell.Integration.Tests/
  GetShadowRunPollingTests.cs          (In-progress, complete, error scenarios)
```

**Verification:**
```bash
dotnet build KArtSell.sln -c Release        # Zero errors/warnings
dotnet test --filter "ShadowRunPolling" -c Release  # All scenarios pass
```

---

## 8. AGENTS.md v16.0 CHECKLIST

| Criterion | Status | Evidence |
|-----------|--------|----------|
| **SOLID** | ✅ | Query service (data), Endpoint (HTTP), Handler (biz logic) separation |
| **Complexity** | ✅ | Endpoint: deserialize + return; Query: SQL only; Cyclomatic < 10 |
| **Audit** | ✅ | CorrelationId in logs; `published_at <= @cutoff` PIT safety |
| **Necessity** | ✅ | Grounded in "Validation Gates" (CLAUDE.md); polling required for 202 async model |
| **Normalization** | ✅ | Reads denormalized from JSONB (pre-computed metrics); PIT queries |
| **Simplicity** | ✅ | No caching (always fresh); straightforward SELECT; no hidden state |
| **Pattern** | ✅ | Vertical Slice (Endpoint → Query → DB); PIT reads |
| **Guardrails** | ✅ | Schema-qualified SQL; no SELECT *; error handling (404, 500) |
| **Traceability** | ✅ | Run ID immutable artifact; status progression logged |
| **Safety** | ✅ | Idempotent reads; job status eventually consistent; error_message preserved |
| **Maturity** | ✅ | Contract → Test → Implementation sequencing |
| **Right Way** | ✅ | No shortcuts; PIT queries over simplified SELECT |
| **Debt** | ✅ | Zero new tech debt; follows established patterns |

---

## NEXT STEPS (Sequenced)

### Step 1: Program.cs Registration
- Inject Hangfire `RecurringJobManager` or `BackgroundJobClient`
- Register `ShadowRunJob` handler
- Verify no startup errors

### Step 2: Query Service
- `GetShadowRunQuery.cs`: Single SELECT query
- Deserialize JSONB → typed DTOs
- PIT safety: `published_at <= @cutoff`

### Step 3: Polling Endpoint
- `GetShadowRunPollingEndpoint.cs`: FastEndpoints pattern
- Inject query service
- Return 200 with DTO (status ± metrics)
- Return 404 if run not found

### Step 4: Response DTO
- `GetShadowRunResponse.cs`: Mirrors `shadow_run` table
- Optional `metrics`, `gates` (null if in progress)
- `error_message` for failed runs

### Step 5: Tests (9 scenarios)
- In-progress status
- Complete with all gates passed
- Complete with some gates failed
- Failed run with error message
- 404 on missing run_id
- Metrics deserialized correctly
- E2E: POST → poll → complete

### Step 6: Verify
- Build passes
- Tests all green
- Commit & push
