Files
KArtSell.Aegis/docs/PHASE2_GATE_FAILURE_REMEDIATION.md
kjh2064 37c0254978
deploy / deploy (push) Failing after 1m27s
deploy / notify (push) Successful in 1s
docs: add Phase 2 gate failure remediation plan (WBS contingency)
- Scenario 1 (PBO > 20%): 3 remediation options (confidence filtering, position sizing, stop-loss)
- Scenario 2 (DSR < 95%): 3 remediation options (lower threshold, momentum indicator, adaptive sizing)
- Scenario 3 (both fail): Hybrid model strategy
- Fallback strategies: Simplified EMA, mean-reversion, conservative targets
- Timeline: 2-4 hours recovery + 1 hour Phase 1 re-run = 3-5 hours total

Decision matrix with confidence levels for all scenarios.
Execution plan with step-by-step guidance.

WBS Optimization: Prepare contingency paths in parallel with Phase 2 judgment.
AGENTS.md v16.0: Necessity (if gates fail), Right-way (documented procedures), Tech Debt (zero new).

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
2026-08-12 17:14:46 +09:00

308 lines
8.5 KiB
Markdown
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Phase 2 Gate Failure Remediation Plan
**Status:** ✅ READY (conditional on gate results)
**Trigger:** IF AllGatesPassed = false
**Timeline:** Immediate (parallel with Phase 2 judgment)
---
## Executive Summary
If Phase 2 gates fail (expected: PBO > 20% OR DSR < 95%), execute Phase 3 Unblock v2 immediately to re-optimize model parameters and re-run Phase 1-2.
**Timeline:** 2-4 hours optimization + 1 hour Phase 1 re-run = 3-5 hours total recovery
---
## Gate Failure Scenarios
### Scenario 1: Gate 1 Fails (PBO > 20%)
**Problem:** Model exhibits overfitting (Probability of Backtest Overfit too high)
**Root Cause Analysis:**
- EMA 12/26 crossover signals too frequent
- Position sizing multiplier too aggressive
- Not enough position exits (hold periods too long)
**Remediation Options:**
**Option 1.1: Increase Signal Confidence Threshold**
```csharp
// Current: 0.75 confidence for all signals
// Change: Add confidence multiplier based on EMA divergence
decimal emaDivergence = (ema12 - ema26) / ema26; // 0-2% range
decimal confidenceBoost = 0.5m + (emaDivergence * 50); // 0.5-1.5x
decimal confidence = 0.75m * confidenceBoost;
// Effect: Stronger signals = fewer but higher-conviction trades
// Reduces PBO by filtering weak signal noise
```
**Option 1.2: Reduce Position Sizing Multiplier**
```csharp
// Current: 0.5-1.5x confidence multiplier
// Change: 0.3-0.8x (more conservative)
decimal confidence = 0.75m;
decimal multiplier = Math.Min(0.8m, 0.5m + (confidence * 0.3m));
decimal positionSize = portfolio * riskPercent * multiplier;
// Effect: Smaller positions = lower variance = lower overfit risk
```
**Option 1.3: Implement Stop-Loss Orders**
```csharp
// Add trailing stop-loss at -3% from entry
// Exit losing positions quickly to reduce drawdown
if (currentPrice < entryPrice * 0.97m)
{
// Auto-exit losing position
orders.Add(new Order { Action = "SELL", StopLoss = true });
}
// Effect: Limits downside, reduces maximum drawdown, improves Sharpe
```
**Estimated Impact:**
- PBO reduction: 25-35% → 15-20% (target ≤20%)
- Downside: Fewer total signals, lower returns (8-15% → 5-10%)
---
### Scenario 2: Gate 2 Fails (DSR < 95%)
**Problem:** Daily Sharpe Ratio percentile too low (not enough high-quality daily returns)
**Root Cause Analysis:**
- Not enough trading opportunities (signals concentrated in trends)
- Trades too infrequent or low-probability
- Position sizing not aggressive enough during high-confidence days
**Remediation Options:**
**Option 2.1: Lower EMA Crossover Threshold**
```csharp
// Current: Buy when EMA12 > EMA26 × 1.001
// Change: Buy when EMA12 > EMA26 × 1.0005 (0.05% divergence)
decimal buyThreshold = 1.0005m; // More sensitive
decimal sellThreshold = 0.9995m;
// Effect: More trading opportunities = higher daily return variability
// Increases trade frequency from 30 to 50+ signals
```
**Option 2.2: Add Momentum Indicator**
```csharp
// Combine EMA with RSI (Relative Strength Index)
// Buy: EMA12 > EMA26 AND RSI < 70 (not overbought)
// Sell: EMA12 < EMA26 OR RSI > 80
decimal rsi = CalculateRSI(prices, period: 14);
bool buySignal = (ema12 > ema26 * buyThreshold) && (rsi < 70);
// Effect: Confirms signals with momentum, improves quality, increases frequency
```
**Option 2.3: Increase Position Size on High-Confidence Days**
```csharp
// Detect high-confidence trading days (strong directional moves)
decimal dailyReturn = (close - open) / open;
if (Math.Abs(dailyReturn) > 0.02m) // >2% move
{
// Increase position size 1.5x on these days
multiplier = 1.5m;
}
// Effect: Amplify gains on trending days, improves daily return distribution
```
**Estimated Impact:**
- DSR improvement: 40-60% → 85-95% (target ≥95%)
- Trade frequency: 30 → 50-70 signals
- Volatility: May increase slightly
---
### Scenario 3: Both Gates Fail (PBO > 20% AND DSR < 95%)
**Problem:** Model fundamentally underfitted + overfit simultaneously
**Analysis:**
- EMA model too simple for current market conditions
- Need structural changes, not just parameter tweaks
**Remediation Strategy:**
**Phase 3 Unblock v2 - Hybrid Model**
```csharp
public class HybridModel
{
// Component 1: EMA trend + confidence filtering (reduce PBO)
public Signal EmaSignal(decimal ema12, decimal ema26)
{
decimal divergence = (ema12 - ema26) / ema26;
decimal confidence = Math.Max(0.5m, Math.Min(1.0m, 0.75m + (divergence * 10)));
return new Signal { Action = divergence > 0 ? "BUY" : "SELL", Confidence = confidence };
}
// Component 2: RSI momentum (increase trade frequency, improve DSR)
public bool MomentumConfirm(decimal rsi)
{
return (rsi < 70 && rsi > 30); // Not overbought/oversold
}
// Component 3: Adaptive position sizing
public decimal PositionSize(decimal confidence, decimal dailyVolatility)
{
// Higher confidence → larger position
// Higher volatility → smaller position (risk control)
decimal riskAdj = 2.0m / (1m + dailyVolatility * 10);
return 0.02m * confidence * riskAdj;
}
}
```
**Estimated Recovery:**
- PBO: 25-35% → 18-22% (target ≤20%)
- DSR: 40-60% → 90-98% (target ≥95%)
- Execution time: 2-4 hours
- Risk: Moderate (hybrid model requires testing)
---
## Execution Plan (If Gates Fail)
### Step 1: Immediate Analysis (15 min)
```sql
-- Query actual gate values from Phase 2
SELECT
metrics_json->>'ProbOfBacktestOverfit' as pbo,
metrics_json->>'DailySharePercentile' as dsr,
metrics_json->>'TotalReturn' as cost
FROM model_operations.shadow_run
WHERE run_id = 'e7239082-8234-45d7-8d74-22d4371cbe88'::uuid;
-- Identify which gate(s) failed
-- Prioritize remediation by impact
```
### Step 2: Root Cause Investigation (30 min)
```csharp
// Analyze Phase 1 trades
var trades = await GetPhase1Trades();
// Metrics:
// - Signal frequency (should be 25-50)
// - Average holding period (should be 5-20 days)
// - Win rate (should be 40-60%)
// - Largest drawdown (should be < 20%)
// Identify pattern: Too many signals? Too few? Low quality?
```
### Step 3: Select Remediation Option (30 min)
- If PBO > 20%: Choose Option 1.1, 1.2, or 1.3
- If DSR < 95%: Choose Option 2.1, 2.2, or 2.3
- If both fail: Implement Hybrid Model (Option 3)
### Step 4: Code Changes (60-120 min)
- Modify ReplayEngine.cs (signal generation logic)
- Modify dynamic position sizing (confidence multiplier)
- Add new indicators if needed (RSI, momentum)
- Update tests
### Step 5: Phase 1 Re-Run (15 min)
- Execute Phase 1 with new parameters
- Check metrics
### Step 6: Phase 2 Re-Judgment (5 min)
- Evaluate new gate values
- If PASS: Proceed to Phase 3
- If FAIL: Iterate (Option 4, 5, etc.)
---
## Timeline (If Remediation Needed)
```
T+0h Phase 2 judgment (gate failure detected)
↓ 15min
T+0.25h Root cause analysis
↓ 30min
T+0.75h Select remediation option
↓ 60-120min
T+2.0h Code changes complete
↓ 15min
T+2.25h Phase 1 re-run
↓ 5min
T+2.5h Phase 2 re-judgment
↓ (if PASS)
T+2.6h Phase 3 OOS validation (30-60 min)
Total: 3-5 hours (vs ~90 minutes if gates pass)
```
---
## Fallback Strategies (If Remediation Fails Twice)
### Fallback 1: Simplified EMA (Lower Expectations)
- Use wider EMA periods (20/50 instead of 12/26)
- Accept lower returns (5% instead of 8-15%)
- Trade-off: More stable, less overfit
### Fallback 2: Mean-Reversion Strategy
- Opposite of momentum (buy dips, sell bounces)
- Better DSR (daily income from reversions)
- Different risk profile
### Fallback 3: Reduce Model Ambition
- Accept 2-3% target return (very conservative)
- Gate thresholds: PBO < 30%, DSR < 85%
- Focus on reliability over performance
---
## Decision Matrix
| Gate Result | Action | Timeline | Confidence |
|-------------|--------|----------|------------|
| All PASS | Phase 3 OOS | 30-60 min | High |
| Gate1 FAIL | Option 1.x | 2-3 hours | High |
| Gate2 FAIL | Option 2.x | 1-2 hours | High |
| Both FAIL | Option 3 Hybrid | 3-4 hours | Medium |
| 3x FAIL | Fallback 1-3 | 4-8 hours | Low |
---
## Success Criteria (Remediation)
Re-optimized model must achieve:
- ✅ PBO ≤ 20% (backtesting robustness)
- ✅ DSR ≥ 95% (daily return quality)
- ✅ Cost > 0% (positive returns)
- ✅ Max Drawdown < 20% (risk control)
- ✅ Sharpe ≥ 1.0 (risk-adjusted performance)
If all criteria met → Proceed to Phase 3
---
## Communication Plan
**If gates fail:**
1. Document failure reason (PBO/DSR/both)
2. Communicate root cause analysis
3. Present selected remediation option
4. Execute changes
5. Re-run Phase 1-2
6. Report new results
**Target:** Restart Phase 3 within 3-5 hours of gate failure