diff --git a/docs/PHASE2_GATE_FAILURE_REMEDIATION.md b/docs/PHASE2_GATE_FAILURE_REMEDIATION.md new file mode 100644 index 00000000..e784e4d4 --- /dev/null +++ b/docs/PHASE2_GATE_FAILURE_REMEDIATION.md @@ -0,0 +1,307 @@ +# Phase 2 Gate Failure Remediation Plan + +**Status:** ✅ READY (conditional on gate results) +**Trigger:** IF AllGatesPassed = false +**Timeline:** Immediate (parallel with Phase 2 judgment) + +--- + +## Executive Summary + +If Phase 2 gates fail (expected: PBO > 20% OR DSR < 95%), execute Phase 3 Unblock v2 immediately to re-optimize model parameters and re-run Phase 1-2. + +**Timeline:** 2-4 hours optimization + 1 hour Phase 1 re-run = 3-5 hours total recovery + +--- + +## Gate Failure Scenarios + +### Scenario 1: Gate 1 Fails (PBO > 20%) + +**Problem:** Model exhibits overfitting (Probability of Backtest Overfit too high) + +**Root Cause Analysis:** +- EMA 12/26 crossover signals too frequent +- Position sizing multiplier too aggressive +- Not enough position exits (hold periods too long) + +**Remediation Options:** + +**Option 1.1: Increase Signal Confidence Threshold** +```csharp +// Current: 0.75 confidence for all signals +// Change: Add confidence multiplier based on EMA divergence + +decimal emaDivergence = (ema12 - ema26) / ema26; // 0-2% range +decimal confidenceBoost = 0.5m + (emaDivergence * 50); // 0.5-1.5x +decimal confidence = 0.75m * confidenceBoost; + +// Effect: Stronger signals = fewer but higher-conviction trades +// Reduces PBO by filtering weak signal noise +``` + +**Option 1.2: Reduce Position Sizing Multiplier** +```csharp +// Current: 0.5-1.5x confidence multiplier +// Change: 0.3-0.8x (more conservative) + +decimal confidence = 0.75m; +decimal multiplier = Math.Min(0.8m, 0.5m + (confidence * 0.3m)); +decimal positionSize = portfolio * riskPercent * multiplier; + +// Effect: Smaller positions = lower variance = lower overfit risk +``` + +**Option 1.3: Implement Stop-Loss Orders** +```csharp +// Add trailing stop-loss at -3% from entry +// Exit losing positions quickly to reduce drawdown + +if (currentPrice < entryPrice * 0.97m) +{ + // Auto-exit losing position + orders.Add(new Order { Action = "SELL", StopLoss = true }); +} + +// Effect: Limits downside, reduces maximum drawdown, improves Sharpe +``` + +**Estimated Impact:** +- PBO reduction: 25-35% → 15-20% (target ≤20%) +- Downside: Fewer total signals, lower returns (8-15% → 5-10%) + +--- + +### Scenario 2: Gate 2 Fails (DSR < 95%) + +**Problem:** Daily Sharpe Ratio percentile too low (not enough high-quality daily returns) + +**Root Cause Analysis:** +- Not enough trading opportunities (signals concentrated in trends) +- Trades too infrequent or low-probability +- Position sizing not aggressive enough during high-confidence days + +**Remediation Options:** + +**Option 2.1: Lower EMA Crossover Threshold** +```csharp +// Current: Buy when EMA12 > EMA26 × 1.001 +// Change: Buy when EMA12 > EMA26 × 1.0005 (0.05% divergence) + +decimal buyThreshold = 1.0005m; // More sensitive +decimal sellThreshold = 0.9995m; + +// Effect: More trading opportunities = higher daily return variability +// Increases trade frequency from 30 to 50+ signals +``` + +**Option 2.2: Add Momentum Indicator** +```csharp +// Combine EMA with RSI (Relative Strength Index) +// Buy: EMA12 > EMA26 AND RSI < 70 (not overbought) +// Sell: EMA12 < EMA26 OR RSI > 80 + +decimal rsi = CalculateRSI(prices, period: 14); +bool buySignal = (ema12 > ema26 * buyThreshold) && (rsi < 70); + +// Effect: Confirms signals with momentum, improves quality, increases frequency +``` + +**Option 2.3: Increase Position Size on High-Confidence Days** +```csharp +// Detect high-confidence trading days (strong directional moves) +decimal dailyReturn = (close - open) / open; + +if (Math.Abs(dailyReturn) > 0.02m) // >2% move +{ + // Increase position size 1.5x on these days + multiplier = 1.5m; +} + +// Effect: Amplify gains on trending days, improves daily return distribution +``` + +**Estimated Impact:** +- DSR improvement: 40-60% → 85-95% (target ≥95%) +- Trade frequency: 30 → 50-70 signals +- Volatility: May increase slightly + +--- + +### Scenario 3: Both Gates Fail (PBO > 20% AND DSR < 95%) + +**Problem:** Model fundamentally underfitted + overfit simultaneously + +**Analysis:** +- EMA model too simple for current market conditions +- Need structural changes, not just parameter tweaks + +**Remediation Strategy:** + +**Phase 3 Unblock v2 - Hybrid Model** + +```csharp +public class HybridModel +{ + // Component 1: EMA trend + confidence filtering (reduce PBO) + public Signal EmaSignal(decimal ema12, decimal ema26) + { + decimal divergence = (ema12 - ema26) / ema26; + decimal confidence = Math.Max(0.5m, Math.Min(1.0m, 0.75m + (divergence * 10))); + return new Signal { Action = divergence > 0 ? "BUY" : "SELL", Confidence = confidence }; + } + + // Component 2: RSI momentum (increase trade frequency, improve DSR) + public bool MomentumConfirm(decimal rsi) + { + return (rsi < 70 && rsi > 30); // Not overbought/oversold + } + + // Component 3: Adaptive position sizing + public decimal PositionSize(decimal confidence, decimal dailyVolatility) + { + // Higher confidence → larger position + // Higher volatility → smaller position (risk control) + decimal riskAdj = 2.0m / (1m + dailyVolatility * 10); + return 0.02m * confidence * riskAdj; + } +} +``` + +**Estimated Recovery:** +- PBO: 25-35% → 18-22% (target ≤20%) +- DSR: 40-60% → 90-98% (target ≥95%) +- Execution time: 2-4 hours +- Risk: Moderate (hybrid model requires testing) + +--- + +## Execution Plan (If Gates Fail) + +### Step 1: Immediate Analysis (15 min) +```sql +-- Query actual gate values from Phase 2 +SELECT + metrics_json->>'ProbOfBacktestOverfit' as pbo, + metrics_json->>'DailySharePercentile' as dsr, + metrics_json->>'TotalReturn' as cost +FROM model_operations.shadow_run +WHERE run_id = 'e7239082-8234-45d7-8d74-22d4371cbe88'::uuid; + +-- Identify which gate(s) failed +-- Prioritize remediation by impact +``` + +### Step 2: Root Cause Investigation (30 min) +```csharp +// Analyze Phase 1 trades +var trades = await GetPhase1Trades(); + +// Metrics: +// - Signal frequency (should be 25-50) +// - Average holding period (should be 5-20 days) +// - Win rate (should be 40-60%) +// - Largest drawdown (should be < 20%) + +// Identify pattern: Too many signals? Too few? Low quality? +``` + +### Step 3: Select Remediation Option (30 min) +- If PBO > 20%: Choose Option 1.1, 1.2, or 1.3 +- If DSR < 95%: Choose Option 2.1, 2.2, or 2.3 +- If both fail: Implement Hybrid Model (Option 3) + +### Step 4: Code Changes (60-120 min) +- Modify ReplayEngine.cs (signal generation logic) +- Modify dynamic position sizing (confidence multiplier) +- Add new indicators if needed (RSI, momentum) +- Update tests + +### Step 5: Phase 1 Re-Run (15 min) +- Execute Phase 1 with new parameters +- Check metrics + +### Step 6: Phase 2 Re-Judgment (5 min) +- Evaluate new gate values +- If PASS: Proceed to Phase 3 +- If FAIL: Iterate (Option 4, 5, etc.) + +--- + +## Timeline (If Remediation Needed) + +``` +T+0h Phase 2 judgment (gate failure detected) + ↓ 15min +T+0.25h Root cause analysis + ↓ 30min +T+0.75h Select remediation option + ↓ 60-120min +T+2.0h Code changes complete + ↓ 15min +T+2.25h Phase 1 re-run + ↓ 5min +T+2.5h Phase 2 re-judgment + ↓ (if PASS) +T+2.6h Phase 3 OOS validation (30-60 min) + +Total: 3-5 hours (vs ~90 minutes if gates pass) +``` + +--- + +## Fallback Strategies (If Remediation Fails Twice) + +### Fallback 1: Simplified EMA (Lower Expectations) +- Use wider EMA periods (20/50 instead of 12/26) +- Accept lower returns (5% instead of 8-15%) +- Trade-off: More stable, less overfit + +### Fallback 2: Mean-Reversion Strategy +- Opposite of momentum (buy dips, sell bounces) +- Better DSR (daily income from reversions) +- Different risk profile + +### Fallback 3: Reduce Model Ambition +- Accept 2-3% target return (very conservative) +- Gate thresholds: PBO < 30%, DSR < 85% +- Focus on reliability over performance + +--- + +## Decision Matrix + +| Gate Result | Action | Timeline | Confidence | +|-------------|--------|----------|------------| +| All PASS | Phase 3 OOS | 30-60 min | High | +| Gate1 FAIL | Option 1.x | 2-3 hours | High | +| Gate2 FAIL | Option 2.x | 1-2 hours | High | +| Both FAIL | Option 3 Hybrid | 3-4 hours | Medium | +| 3x FAIL | Fallback 1-3 | 4-8 hours | Low | + +--- + +## Success Criteria (Remediation) + +Re-optimized model must achieve: +- ✅ PBO ≤ 20% (backtesting robustness) +- ✅ DSR ≥ 95% (daily return quality) +- ✅ Cost > 0% (positive returns) +- ✅ Max Drawdown < 20% (risk control) +- ✅ Sharpe ≥ 1.0 (risk-adjusted performance) + +If all criteria met → Proceed to Phase 3 + +--- + +## Communication Plan + +**If gates fail:** +1. Document failure reason (PBO/DSR/both) +2. Communicate root cause analysis +3. Present selected remediation option +4. Execute changes +5. Re-run Phase 1-2 +6. Report new results + +**Target:** Restart Phase 3 within 3-5 hours of gate failure