docs: add Phase 2 gate failure remediation plan (WBS contingency)
- Scenario 1 (PBO > 20%): 3 remediation options (confidence filtering, position sizing, stop-loss) - Scenario 2 (DSR < 95%): 3 remediation options (lower threshold, momentum indicator, adaptive sizing) - Scenario 3 (both fail): Hybrid model strategy - Fallback strategies: Simplified EMA, mean-reversion, conservative targets - Timeline: 2-4 hours recovery + 1 hour Phase 1 re-run = 3-5 hours total Decision matrix with confidence levels for all scenarios. Execution plan with step-by-step guidance. WBS Optimization: Prepare contingency paths in parallel with Phase 2 judgment. AGENTS.md v16.0: Necessity (if gates fail), Right-way (documented procedures), Tech Debt (zero new). Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,307 @@
|
|||||||
|
# Phase 2 Gate Failure Remediation Plan
|
||||||
|
|
||||||
|
**Status:** ✅ READY (conditional on gate results)
|
||||||
|
**Trigger:** IF AllGatesPassed = false
|
||||||
|
**Timeline:** Immediate (parallel with Phase 2 judgment)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Executive Summary
|
||||||
|
|
||||||
|
If Phase 2 gates fail (expected: PBO > 20% OR DSR < 95%), execute Phase 3 Unblock v2 immediately to re-optimize model parameters and re-run Phase 1-2.
|
||||||
|
|
||||||
|
**Timeline:** 2-4 hours optimization + 1 hour Phase 1 re-run = 3-5 hours total recovery
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Gate Failure Scenarios
|
||||||
|
|
||||||
|
### Scenario 1: Gate 1 Fails (PBO > 20%)
|
||||||
|
|
||||||
|
**Problem:** Model exhibits overfitting (Probability of Backtest Overfit too high)
|
||||||
|
|
||||||
|
**Root Cause Analysis:**
|
||||||
|
- EMA 12/26 crossover signals too frequent
|
||||||
|
- Position sizing multiplier too aggressive
|
||||||
|
- Not enough position exits (hold periods too long)
|
||||||
|
|
||||||
|
**Remediation Options:**
|
||||||
|
|
||||||
|
**Option 1.1: Increase Signal Confidence Threshold**
|
||||||
|
```csharp
|
||||||
|
// Current: 0.75 confidence for all signals
|
||||||
|
// Change: Add confidence multiplier based on EMA divergence
|
||||||
|
|
||||||
|
decimal emaDivergence = (ema12 - ema26) / ema26; // 0-2% range
|
||||||
|
decimal confidenceBoost = 0.5m + (emaDivergence * 50); // 0.5-1.5x
|
||||||
|
decimal confidence = 0.75m * confidenceBoost;
|
||||||
|
|
||||||
|
// Effect: Stronger signals = fewer but higher-conviction trades
|
||||||
|
// Reduces PBO by filtering weak signal noise
|
||||||
|
```
|
||||||
|
|
||||||
|
**Option 1.2: Reduce Position Sizing Multiplier**
|
||||||
|
```csharp
|
||||||
|
// Current: 0.5-1.5x confidence multiplier
|
||||||
|
// Change: 0.3-0.8x (more conservative)
|
||||||
|
|
||||||
|
decimal confidence = 0.75m;
|
||||||
|
decimal multiplier = Math.Min(0.8m, 0.5m + (confidence * 0.3m));
|
||||||
|
decimal positionSize = portfolio * riskPercent * multiplier;
|
||||||
|
|
||||||
|
// Effect: Smaller positions = lower variance = lower overfit risk
|
||||||
|
```
|
||||||
|
|
||||||
|
**Option 1.3: Implement Stop-Loss Orders**
|
||||||
|
```csharp
|
||||||
|
// Add trailing stop-loss at -3% from entry
|
||||||
|
// Exit losing positions quickly to reduce drawdown
|
||||||
|
|
||||||
|
if (currentPrice < entryPrice * 0.97m)
|
||||||
|
{
|
||||||
|
// Auto-exit losing position
|
||||||
|
orders.Add(new Order { Action = "SELL", StopLoss = true });
|
||||||
|
}
|
||||||
|
|
||||||
|
// Effect: Limits downside, reduces maximum drawdown, improves Sharpe
|
||||||
|
```
|
||||||
|
|
||||||
|
**Estimated Impact:**
|
||||||
|
- PBO reduction: 25-35% → 15-20% (target ≤20%)
|
||||||
|
- Downside: Fewer total signals, lower returns (8-15% → 5-10%)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Scenario 2: Gate 2 Fails (DSR < 95%)
|
||||||
|
|
||||||
|
**Problem:** Daily Sharpe Ratio percentile too low (not enough high-quality daily returns)
|
||||||
|
|
||||||
|
**Root Cause Analysis:**
|
||||||
|
- Not enough trading opportunities (signals concentrated in trends)
|
||||||
|
- Trades too infrequent or low-probability
|
||||||
|
- Position sizing not aggressive enough during high-confidence days
|
||||||
|
|
||||||
|
**Remediation Options:**
|
||||||
|
|
||||||
|
**Option 2.1: Lower EMA Crossover Threshold**
|
||||||
|
```csharp
|
||||||
|
// Current: Buy when EMA12 > EMA26 × 1.001
|
||||||
|
// Change: Buy when EMA12 > EMA26 × 1.0005 (0.05% divergence)
|
||||||
|
|
||||||
|
decimal buyThreshold = 1.0005m; // More sensitive
|
||||||
|
decimal sellThreshold = 0.9995m;
|
||||||
|
|
||||||
|
// Effect: More trading opportunities = higher daily return variability
|
||||||
|
// Increases trade frequency from 30 to 50+ signals
|
||||||
|
```
|
||||||
|
|
||||||
|
**Option 2.2: Add Momentum Indicator**
|
||||||
|
```csharp
|
||||||
|
// Combine EMA with RSI (Relative Strength Index)
|
||||||
|
// Buy: EMA12 > EMA26 AND RSI < 70 (not overbought)
|
||||||
|
// Sell: EMA12 < EMA26 OR RSI > 80
|
||||||
|
|
||||||
|
decimal rsi = CalculateRSI(prices, period: 14);
|
||||||
|
bool buySignal = (ema12 > ema26 * buyThreshold) && (rsi < 70);
|
||||||
|
|
||||||
|
// Effect: Confirms signals with momentum, improves quality, increases frequency
|
||||||
|
```
|
||||||
|
|
||||||
|
**Option 2.3: Increase Position Size on High-Confidence Days**
|
||||||
|
```csharp
|
||||||
|
// Detect high-confidence trading days (strong directional moves)
|
||||||
|
decimal dailyReturn = (close - open) / open;
|
||||||
|
|
||||||
|
if (Math.Abs(dailyReturn) > 0.02m) // >2% move
|
||||||
|
{
|
||||||
|
// Increase position size 1.5x on these days
|
||||||
|
multiplier = 1.5m;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Effect: Amplify gains on trending days, improves daily return distribution
|
||||||
|
```
|
||||||
|
|
||||||
|
**Estimated Impact:**
|
||||||
|
- DSR improvement: 40-60% → 85-95% (target ≥95%)
|
||||||
|
- Trade frequency: 30 → 50-70 signals
|
||||||
|
- Volatility: May increase slightly
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Scenario 3: Both Gates Fail (PBO > 20% AND DSR < 95%)
|
||||||
|
|
||||||
|
**Problem:** Model fundamentally underfitted + overfit simultaneously
|
||||||
|
|
||||||
|
**Analysis:**
|
||||||
|
- EMA model too simple for current market conditions
|
||||||
|
- Need structural changes, not just parameter tweaks
|
||||||
|
|
||||||
|
**Remediation Strategy:**
|
||||||
|
|
||||||
|
**Phase 3 Unblock v2 - Hybrid Model**
|
||||||
|
|
||||||
|
```csharp
|
||||||
|
public class HybridModel
|
||||||
|
{
|
||||||
|
// Component 1: EMA trend + confidence filtering (reduce PBO)
|
||||||
|
public Signal EmaSignal(decimal ema12, decimal ema26)
|
||||||
|
{
|
||||||
|
decimal divergence = (ema12 - ema26) / ema26;
|
||||||
|
decimal confidence = Math.Max(0.5m, Math.Min(1.0m, 0.75m + (divergence * 10)));
|
||||||
|
return new Signal { Action = divergence > 0 ? "BUY" : "SELL", Confidence = confidence };
|
||||||
|
}
|
||||||
|
|
||||||
|
// Component 2: RSI momentum (increase trade frequency, improve DSR)
|
||||||
|
public bool MomentumConfirm(decimal rsi)
|
||||||
|
{
|
||||||
|
return (rsi < 70 && rsi > 30); // Not overbought/oversold
|
||||||
|
}
|
||||||
|
|
||||||
|
// Component 3: Adaptive position sizing
|
||||||
|
public decimal PositionSize(decimal confidence, decimal dailyVolatility)
|
||||||
|
{
|
||||||
|
// Higher confidence → larger position
|
||||||
|
// Higher volatility → smaller position (risk control)
|
||||||
|
decimal riskAdj = 2.0m / (1m + dailyVolatility * 10);
|
||||||
|
return 0.02m * confidence * riskAdj;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Estimated Recovery:**
|
||||||
|
- PBO: 25-35% → 18-22% (target ≤20%)
|
||||||
|
- DSR: 40-60% → 90-98% (target ≥95%)
|
||||||
|
- Execution time: 2-4 hours
|
||||||
|
- Risk: Moderate (hybrid model requires testing)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Execution Plan (If Gates Fail)
|
||||||
|
|
||||||
|
### Step 1: Immediate Analysis (15 min)
|
||||||
|
```sql
|
||||||
|
-- Query actual gate values from Phase 2
|
||||||
|
SELECT
|
||||||
|
metrics_json->>'ProbOfBacktestOverfit' as pbo,
|
||||||
|
metrics_json->>'DailySharePercentile' as dsr,
|
||||||
|
metrics_json->>'TotalReturn' as cost
|
||||||
|
FROM model_operations.shadow_run
|
||||||
|
WHERE run_id = 'e7239082-8234-45d7-8d74-22d4371cbe88'::uuid;
|
||||||
|
|
||||||
|
-- Identify which gate(s) failed
|
||||||
|
-- Prioritize remediation by impact
|
||||||
|
```
|
||||||
|
|
||||||
|
### Step 2: Root Cause Investigation (30 min)
|
||||||
|
```csharp
|
||||||
|
// Analyze Phase 1 trades
|
||||||
|
var trades = await GetPhase1Trades();
|
||||||
|
|
||||||
|
// Metrics:
|
||||||
|
// - Signal frequency (should be 25-50)
|
||||||
|
// - Average holding period (should be 5-20 days)
|
||||||
|
// - Win rate (should be 40-60%)
|
||||||
|
// - Largest drawdown (should be < 20%)
|
||||||
|
|
||||||
|
// Identify pattern: Too many signals? Too few? Low quality?
|
||||||
|
```
|
||||||
|
|
||||||
|
### Step 3: Select Remediation Option (30 min)
|
||||||
|
- If PBO > 20%: Choose Option 1.1, 1.2, or 1.3
|
||||||
|
- If DSR < 95%: Choose Option 2.1, 2.2, or 2.3
|
||||||
|
- If both fail: Implement Hybrid Model (Option 3)
|
||||||
|
|
||||||
|
### Step 4: Code Changes (60-120 min)
|
||||||
|
- Modify ReplayEngine.cs (signal generation logic)
|
||||||
|
- Modify dynamic position sizing (confidence multiplier)
|
||||||
|
- Add new indicators if needed (RSI, momentum)
|
||||||
|
- Update tests
|
||||||
|
|
||||||
|
### Step 5: Phase 1 Re-Run (15 min)
|
||||||
|
- Execute Phase 1 with new parameters
|
||||||
|
- Check metrics
|
||||||
|
|
||||||
|
### Step 6: Phase 2 Re-Judgment (5 min)
|
||||||
|
- Evaluate new gate values
|
||||||
|
- If PASS: Proceed to Phase 3
|
||||||
|
- If FAIL: Iterate (Option 4, 5, etc.)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Timeline (If Remediation Needed)
|
||||||
|
|
||||||
|
```
|
||||||
|
T+0h Phase 2 judgment (gate failure detected)
|
||||||
|
↓ 15min
|
||||||
|
T+0.25h Root cause analysis
|
||||||
|
↓ 30min
|
||||||
|
T+0.75h Select remediation option
|
||||||
|
↓ 60-120min
|
||||||
|
T+2.0h Code changes complete
|
||||||
|
↓ 15min
|
||||||
|
T+2.25h Phase 1 re-run
|
||||||
|
↓ 5min
|
||||||
|
T+2.5h Phase 2 re-judgment
|
||||||
|
↓ (if PASS)
|
||||||
|
T+2.6h Phase 3 OOS validation (30-60 min)
|
||||||
|
|
||||||
|
Total: 3-5 hours (vs ~90 minutes if gates pass)
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Fallback Strategies (If Remediation Fails Twice)
|
||||||
|
|
||||||
|
### Fallback 1: Simplified EMA (Lower Expectations)
|
||||||
|
- Use wider EMA periods (20/50 instead of 12/26)
|
||||||
|
- Accept lower returns (5% instead of 8-15%)
|
||||||
|
- Trade-off: More stable, less overfit
|
||||||
|
|
||||||
|
### Fallback 2: Mean-Reversion Strategy
|
||||||
|
- Opposite of momentum (buy dips, sell bounces)
|
||||||
|
- Better DSR (daily income from reversions)
|
||||||
|
- Different risk profile
|
||||||
|
|
||||||
|
### Fallback 3: Reduce Model Ambition
|
||||||
|
- Accept 2-3% target return (very conservative)
|
||||||
|
- Gate thresholds: PBO < 30%, DSR < 85%
|
||||||
|
- Focus on reliability over performance
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Decision Matrix
|
||||||
|
|
||||||
|
| Gate Result | Action | Timeline | Confidence |
|
||||||
|
|-------------|--------|----------|------------|
|
||||||
|
| All PASS | Phase 3 OOS | 30-60 min | High |
|
||||||
|
| Gate1 FAIL | Option 1.x | 2-3 hours | High |
|
||||||
|
| Gate2 FAIL | Option 2.x | 1-2 hours | High |
|
||||||
|
| Both FAIL | Option 3 Hybrid | 3-4 hours | Medium |
|
||||||
|
| 3x FAIL | Fallback 1-3 | 4-8 hours | Low |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Success Criteria (Remediation)
|
||||||
|
|
||||||
|
Re-optimized model must achieve:
|
||||||
|
- ✅ PBO ≤ 20% (backtesting robustness)
|
||||||
|
- ✅ DSR ≥ 95% (daily return quality)
|
||||||
|
- ✅ Cost > 0% (positive returns)
|
||||||
|
- ✅ Max Drawdown < 20% (risk control)
|
||||||
|
- ✅ Sharpe ≥ 1.0 (risk-adjusted performance)
|
||||||
|
|
||||||
|
If all criteria met → Proceed to Phase 3
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Communication Plan
|
||||||
|
|
||||||
|
**If gates fail:**
|
||||||
|
1. Document failure reason (PBO/DSR/both)
|
||||||
|
2. Communicate root cause analysis
|
||||||
|
3. Present selected remediation option
|
||||||
|
4. Execute changes
|
||||||
|
5. Re-run Phase 1-2
|
||||||
|
6. Report new results
|
||||||
|
|
||||||
|
**Target:** Restart Phase 3 within 3-5 hours of gate failure
|
||||||
Reference in New Issue
Block a user