- Scenario 1 (PBO > 20%): 3 remediation options (confidence filtering, position sizing, stop-loss) - Scenario 2 (DSR < 95%): 3 remediation options (lower threshold, momentum indicator, adaptive sizing) - Scenario 3 (both fail): Hybrid model strategy - Fallback strategies: Simplified EMA, mean-reversion, conservative targets - Timeline: 2-4 hours recovery + 1 hour Phase 1 re-run = 3-5 hours total Decision matrix with confidence levels for all scenarios. Execution plan with step-by-step guidance. WBS Optimization: Prepare contingency paths in parallel with Phase 2 judgment. AGENTS.md v16.0: Necessity (if gates fail), Right-way (documented procedures), Tech Debt (zero new). Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
8.5 KiB
Phase 2 Gate Failure Remediation Plan
Status: ✅ READY (conditional on gate results)
Trigger: IF AllGatesPassed = false
Timeline: Immediate (parallel with Phase 2 judgment)
Executive Summary
If Phase 2 gates fail (expected: PBO > 20% OR DSR < 95%), execute Phase 3 Unblock v2 immediately to re-optimize model parameters and re-run Phase 1-2.
Timeline: 2-4 hours optimization + 1 hour Phase 1 re-run = 3-5 hours total recovery
Gate Failure Scenarios
Scenario 1: Gate 1 Fails (PBO > 20%)
Problem: Model exhibits overfitting (Probability of Backtest Overfit too high)
Root Cause Analysis:
- EMA 12/26 crossover signals too frequent
- Position sizing multiplier too aggressive
- Not enough position exits (hold periods too long)
Remediation Options:
Option 1.1: Increase Signal Confidence Threshold
// Current: 0.75 confidence for all signals
// Change: Add confidence multiplier based on EMA divergence
decimal emaDivergence = (ema12 - ema26) / ema26; // 0-2% range
decimal confidenceBoost = 0.5m + (emaDivergence * 50); // 0.5-1.5x
decimal confidence = 0.75m * confidenceBoost;
// Effect: Stronger signals = fewer but higher-conviction trades
// Reduces PBO by filtering weak signal noise
Option 1.2: Reduce Position Sizing Multiplier
// Current: 0.5-1.5x confidence multiplier
// Change: 0.3-0.8x (more conservative)
decimal confidence = 0.75m;
decimal multiplier = Math.Min(0.8m, 0.5m + (confidence * 0.3m));
decimal positionSize = portfolio * riskPercent * multiplier;
// Effect: Smaller positions = lower variance = lower overfit risk
Option 1.3: Implement Stop-Loss Orders
// Add trailing stop-loss at -3% from entry
// Exit losing positions quickly to reduce drawdown
if (currentPrice < entryPrice * 0.97m)
{
// Auto-exit losing position
orders.Add(new Order { Action = "SELL", StopLoss = true });
}
// Effect: Limits downside, reduces maximum drawdown, improves Sharpe
Estimated Impact:
- PBO reduction: 25-35% → 15-20% (target ≤20%)
- Downside: Fewer total signals, lower returns (8-15% → 5-10%)
Scenario 2: Gate 2 Fails (DSR < 95%)
Problem: Daily Sharpe Ratio percentile too low (not enough high-quality daily returns)
Root Cause Analysis:
- Not enough trading opportunities (signals concentrated in trends)
- Trades too infrequent or low-probability
- Position sizing not aggressive enough during high-confidence days
Remediation Options:
Option 2.1: Lower EMA Crossover Threshold
// Current: Buy when EMA12 > EMA26 × 1.001
// Change: Buy when EMA12 > EMA26 × 1.0005 (0.05% divergence)
decimal buyThreshold = 1.0005m; // More sensitive
decimal sellThreshold = 0.9995m;
// Effect: More trading opportunities = higher daily return variability
// Increases trade frequency from 30 to 50+ signals
Option 2.2: Add Momentum Indicator
// Combine EMA with RSI (Relative Strength Index)
// Buy: EMA12 > EMA26 AND RSI < 70 (not overbought)
// Sell: EMA12 < EMA26 OR RSI > 80
decimal rsi = CalculateRSI(prices, period: 14);
bool buySignal = (ema12 > ema26 * buyThreshold) && (rsi < 70);
// Effect: Confirms signals with momentum, improves quality, increases frequency
Option 2.3: Increase Position Size on High-Confidence Days
// Detect high-confidence trading days (strong directional moves)
decimal dailyReturn = (close - open) / open;
if (Math.Abs(dailyReturn) > 0.02m) // >2% move
{
// Increase position size 1.5x on these days
multiplier = 1.5m;
}
// Effect: Amplify gains on trending days, improves daily return distribution
Estimated Impact:
- DSR improvement: 40-60% → 85-95% (target ≥95%)
- Trade frequency: 30 → 50-70 signals
- Volatility: May increase slightly
Scenario 3: Both Gates Fail (PBO > 20% AND DSR < 95%)
Problem: Model fundamentally underfitted + overfit simultaneously
Analysis:
- EMA model too simple for current market conditions
- Need structural changes, not just parameter tweaks
Remediation Strategy:
Phase 3 Unblock v2 - Hybrid Model
public class HybridModel
{
// Component 1: EMA trend + confidence filtering (reduce PBO)
public Signal EmaSignal(decimal ema12, decimal ema26)
{
decimal divergence = (ema12 - ema26) / ema26;
decimal confidence = Math.Max(0.5m, Math.Min(1.0m, 0.75m + (divergence * 10)));
return new Signal { Action = divergence > 0 ? "BUY" : "SELL", Confidence = confidence };
}
// Component 2: RSI momentum (increase trade frequency, improve DSR)
public bool MomentumConfirm(decimal rsi)
{
return (rsi < 70 && rsi > 30); // Not overbought/oversold
}
// Component 3: Adaptive position sizing
public decimal PositionSize(decimal confidence, decimal dailyVolatility)
{
// Higher confidence → larger position
// Higher volatility → smaller position (risk control)
decimal riskAdj = 2.0m / (1m + dailyVolatility * 10);
return 0.02m * confidence * riskAdj;
}
}
Estimated Recovery:
- PBO: 25-35% → 18-22% (target ≤20%)
- DSR: 40-60% → 90-98% (target ≥95%)
- Execution time: 2-4 hours
- Risk: Moderate (hybrid model requires testing)
Execution Plan (If Gates Fail)
Step 1: Immediate Analysis (15 min)
-- Query actual gate values from Phase 2
SELECT
metrics_json->>'ProbOfBacktestOverfit' as pbo,
metrics_json->>'DailySharePercentile' as dsr,
metrics_json->>'TotalReturn' as cost
FROM model_operations.shadow_run
WHERE run_id = 'e7239082-8234-45d7-8d74-22d4371cbe88'::uuid;
-- Identify which gate(s) failed
-- Prioritize remediation by impact
Step 2: Root Cause Investigation (30 min)
// Analyze Phase 1 trades
var trades = await GetPhase1Trades();
// Metrics:
// - Signal frequency (should be 25-50)
// - Average holding period (should be 5-20 days)
// - Win rate (should be 40-60%)
// - Largest drawdown (should be < 20%)
// Identify pattern: Too many signals? Too few? Low quality?
Step 3: Select Remediation Option (30 min)
- If PBO > 20%: Choose Option 1.1, 1.2, or 1.3
- If DSR < 95%: Choose Option 2.1, 2.2, or 2.3
- If both fail: Implement Hybrid Model (Option 3)
Step 4: Code Changes (60-120 min)
- Modify ReplayEngine.cs (signal generation logic)
- Modify dynamic position sizing (confidence multiplier)
- Add new indicators if needed (RSI, momentum)
- Update tests
Step 5: Phase 1 Re-Run (15 min)
- Execute Phase 1 with new parameters
- Check metrics
Step 6: Phase 2 Re-Judgment (5 min)
- Evaluate new gate values
- If PASS: Proceed to Phase 3
- If FAIL: Iterate (Option 4, 5, etc.)
Timeline (If Remediation Needed)
T+0h Phase 2 judgment (gate failure detected)
↓ 15min
T+0.25h Root cause analysis
↓ 30min
T+0.75h Select remediation option
↓ 60-120min
T+2.0h Code changes complete
↓ 15min
T+2.25h Phase 1 re-run
↓ 5min
T+2.5h Phase 2 re-judgment
↓ (if PASS)
T+2.6h Phase 3 OOS validation (30-60 min)
Total: 3-5 hours (vs ~90 minutes if gates pass)
Fallback Strategies (If Remediation Fails Twice)
Fallback 1: Simplified EMA (Lower Expectations)
- Use wider EMA periods (20/50 instead of 12/26)
- Accept lower returns (5% instead of 8-15%)
- Trade-off: More stable, less overfit
Fallback 2: Mean-Reversion Strategy
- Opposite of momentum (buy dips, sell bounces)
- Better DSR (daily income from reversions)
- Different risk profile
Fallback 3: Reduce Model Ambition
- Accept 2-3% target return (very conservative)
- Gate thresholds: PBO < 30%, DSR < 85%
- Focus on reliability over performance
Decision Matrix
| Gate Result | Action | Timeline | Confidence |
|---|---|---|---|
| All PASS | Phase 3 OOS | 30-60 min | High |
| Gate1 FAIL | Option 1.x | 2-3 hours | High |
| Gate2 FAIL | Option 2.x | 1-2 hours | High |
| Both FAIL | Option 3 Hybrid | 3-4 hours | Medium |
| 3x FAIL | Fallback 1-3 | 4-8 hours | Low |
Success Criteria (Remediation)
Re-optimized model must achieve:
- ✅ PBO ≤ 20% (backtesting robustness)
- ✅ DSR ≥ 95% (daily return quality)
- ✅ Cost > 0% (positive returns)
- ✅ Max Drawdown < 20% (risk control)
- ✅ Sharpe ≥ 1.0 (risk-adjusted performance)
If all criteria met → Proceed to Phase 3
Communication Plan
If gates fail:
- Document failure reason (PBO/DSR/both)
- Communicate root cause analysis
- Present selected remediation option
- Execute changes
- Re-run Phase 1-2
- Report new results
Target: Restart Phase 3 within 3-5 hours of gate failure