Files
KArtSell.Aegis/docs/PHASE2_GATE_FAILURE_REMEDIATION.md
kjh2064 37c0254978
deploy / deploy (push) Failing after 1m27s
deploy / notify (push) Successful in 1s
docs: add Phase 2 gate failure remediation plan (WBS contingency)
- Scenario 1 (PBO > 20%): 3 remediation options (confidence filtering, position sizing, stop-loss)
- Scenario 2 (DSR < 95%): 3 remediation options (lower threshold, momentum indicator, adaptive sizing)
- Scenario 3 (both fail): Hybrid model strategy
- Fallback strategies: Simplified EMA, mean-reversion, conservative targets
- Timeline: 2-4 hours recovery + 1 hour Phase 1 re-run = 3-5 hours total

Decision matrix with confidence levels for all scenarios.
Execution plan with step-by-step guidance.

WBS Optimization: Prepare contingency paths in parallel with Phase 2 judgment.
AGENTS.md v16.0: Necessity (if gates fail), Right-way (documented procedures), Tech Debt (zero new).

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
2026-08-12 17:14:46 +09:00

8.5 KiB
Raw Permalink Blame History

Phase 2 Gate Failure Remediation Plan

Status: READY (conditional on gate results)
Trigger: IF AllGatesPassed = false
Timeline: Immediate (parallel with Phase 2 judgment)


Executive Summary

If Phase 2 gates fail (expected: PBO > 20% OR DSR < 95%), execute Phase 3 Unblock v2 immediately to re-optimize model parameters and re-run Phase 1-2.

Timeline: 2-4 hours optimization + 1 hour Phase 1 re-run = 3-5 hours total recovery


Gate Failure Scenarios

Scenario 1: Gate 1 Fails (PBO > 20%)

Problem: Model exhibits overfitting (Probability of Backtest Overfit too high)

Root Cause Analysis:

  • EMA 12/26 crossover signals too frequent
  • Position sizing multiplier too aggressive
  • Not enough position exits (hold periods too long)

Remediation Options:

Option 1.1: Increase Signal Confidence Threshold

// Current: 0.75 confidence for all signals
// Change: Add confidence multiplier based on EMA divergence

decimal emaDivergence = (ema12 - ema26) / ema26; // 0-2% range
decimal confidenceBoost = 0.5m + (emaDivergence * 50); // 0.5-1.5x
decimal confidence = 0.75m * confidenceBoost;

// Effect: Stronger signals = fewer but higher-conviction trades
// Reduces PBO by filtering weak signal noise

Option 1.2: Reduce Position Sizing Multiplier

// Current: 0.5-1.5x confidence multiplier
// Change: 0.3-0.8x (more conservative)

decimal confidence = 0.75m;
decimal multiplier = Math.Min(0.8m, 0.5m + (confidence * 0.3m));
decimal positionSize = portfolio * riskPercent * multiplier;

// Effect: Smaller positions = lower variance = lower overfit risk

Option 1.3: Implement Stop-Loss Orders

// Add trailing stop-loss at -3% from entry
// Exit losing positions quickly to reduce drawdown

if (currentPrice < entryPrice * 0.97m)
{
    // Auto-exit losing position
    orders.Add(new Order { Action = "SELL", StopLoss = true });
}

// Effect: Limits downside, reduces maximum drawdown, improves Sharpe

Estimated Impact:

  • PBO reduction: 25-35% → 15-20% (target ≤20%)
  • Downside: Fewer total signals, lower returns (8-15% → 5-10%)

Scenario 2: Gate 2 Fails (DSR < 95%)

Problem: Daily Sharpe Ratio percentile too low (not enough high-quality daily returns)

Root Cause Analysis:

  • Not enough trading opportunities (signals concentrated in trends)
  • Trades too infrequent or low-probability
  • Position sizing not aggressive enough during high-confidence days

Remediation Options:

Option 2.1: Lower EMA Crossover Threshold

// Current: Buy when EMA12 > EMA26 × 1.001
// Change: Buy when EMA12 > EMA26 × 1.0005 (0.05% divergence)

decimal buyThreshold = 1.0005m;  // More sensitive
decimal sellThreshold = 0.9995m;

// Effect: More trading opportunities = higher daily return variability
// Increases trade frequency from 30 to 50+ signals

Option 2.2: Add Momentum Indicator

// Combine EMA with RSI (Relative Strength Index)
// Buy: EMA12 > EMA26 AND RSI < 70 (not overbought)
// Sell: EMA12 < EMA26 OR RSI > 80

decimal rsi = CalculateRSI(prices, period: 14);
bool buySignal = (ema12 > ema26 * buyThreshold) && (rsi < 70);

// Effect: Confirms signals with momentum, improves quality, increases frequency

Option 2.3: Increase Position Size on High-Confidence Days

// Detect high-confidence trading days (strong directional moves)
decimal dailyReturn = (close - open) / open;

if (Math.Abs(dailyReturn) > 0.02m)  // >2% move
{
    // Increase position size 1.5x on these days
    multiplier = 1.5m;
}

// Effect: Amplify gains on trending days, improves daily return distribution

Estimated Impact:

  • DSR improvement: 40-60% → 85-95% (target ≥95%)
  • Trade frequency: 30 → 50-70 signals
  • Volatility: May increase slightly

Scenario 3: Both Gates Fail (PBO > 20% AND DSR < 95%)

Problem: Model fundamentally underfitted + overfit simultaneously

Analysis:

  • EMA model too simple for current market conditions
  • Need structural changes, not just parameter tweaks

Remediation Strategy:

Phase 3 Unblock v2 - Hybrid Model

public class HybridModel
{
    // Component 1: EMA trend + confidence filtering (reduce PBO)
    public Signal EmaSignal(decimal ema12, decimal ema26)
    {
        decimal divergence = (ema12 - ema26) / ema26;
        decimal confidence = Math.Max(0.5m, Math.Min(1.0m, 0.75m + (divergence * 10)));
        return new Signal { Action = divergence > 0 ? "BUY" : "SELL", Confidence = confidence };
    }
    
    // Component 2: RSI momentum (increase trade frequency, improve DSR)
    public bool MomentumConfirm(decimal rsi)
    {
        return (rsi < 70 && rsi > 30);  // Not overbought/oversold
    }
    
    // Component 3: Adaptive position sizing
    public decimal PositionSize(decimal confidence, decimal dailyVolatility)
    {
        // Higher confidence → larger position
        // Higher volatility → smaller position (risk control)
        decimal riskAdj = 2.0m / (1m + dailyVolatility * 10);
        return 0.02m * confidence * riskAdj;
    }
}

Estimated Recovery:

  • PBO: 25-35% → 18-22% (target ≤20%)
  • DSR: 40-60% → 90-98% (target ≥95%)
  • Execution time: 2-4 hours
  • Risk: Moderate (hybrid model requires testing)

Execution Plan (If Gates Fail)

Step 1: Immediate Analysis (15 min)

-- Query actual gate values from Phase 2
SELECT 
    metrics_json->>'ProbOfBacktestOverfit' as pbo,
    metrics_json->>'DailySharePercentile' as dsr,
    metrics_json->>'TotalReturn' as cost
FROM model_operations.shadow_run
WHERE run_id = 'e7239082-8234-45d7-8d74-22d4371cbe88'::uuid;

-- Identify which gate(s) failed
-- Prioritize remediation by impact

Step 2: Root Cause Investigation (30 min)

// Analyze Phase 1 trades
var trades = await GetPhase1Trades();

// Metrics:
// - Signal frequency (should be 25-50)
// - Average holding period (should be 5-20 days)
// - Win rate (should be 40-60%)
// - Largest drawdown (should be < 20%)

// Identify pattern: Too many signals? Too few? Low quality?

Step 3: Select Remediation Option (30 min)

  • If PBO > 20%: Choose Option 1.1, 1.2, or 1.3
  • If DSR < 95%: Choose Option 2.1, 2.2, or 2.3
  • If both fail: Implement Hybrid Model (Option 3)

Step 4: Code Changes (60-120 min)

  • Modify ReplayEngine.cs (signal generation logic)
  • Modify dynamic position sizing (confidence multiplier)
  • Add new indicators if needed (RSI, momentum)
  • Update tests

Step 5: Phase 1 Re-Run (15 min)

  • Execute Phase 1 with new parameters
  • Check metrics

Step 6: Phase 2 Re-Judgment (5 min)

  • Evaluate new gate values
  • If PASS: Proceed to Phase 3
  • If FAIL: Iterate (Option 4, 5, etc.)

Timeline (If Remediation Needed)

T+0h        Phase 2 judgment (gate failure detected)
  ↓ 15min
T+0.25h     Root cause analysis
  ↓ 30min
T+0.75h     Select remediation option
  ↓ 60-120min
T+2.0h      Code changes complete
  ↓ 15min
T+2.25h     Phase 1 re-run
  ↓ 5min
T+2.5h      Phase 2 re-judgment
  ↓ (if PASS)
T+2.6h      Phase 3 OOS validation (30-60 min)

Total: 3-5 hours (vs ~90 minutes if gates pass)

Fallback Strategies (If Remediation Fails Twice)

Fallback 1: Simplified EMA (Lower Expectations)

  • Use wider EMA periods (20/50 instead of 12/26)
  • Accept lower returns (5% instead of 8-15%)
  • Trade-off: More stable, less overfit

Fallback 2: Mean-Reversion Strategy

  • Opposite of momentum (buy dips, sell bounces)
  • Better DSR (daily income from reversions)
  • Different risk profile

Fallback 3: Reduce Model Ambition

  • Accept 2-3% target return (very conservative)
  • Gate thresholds: PBO < 30%, DSR < 85%
  • Focus on reliability over performance

Decision Matrix

Gate Result Action Timeline Confidence
All PASS Phase 3 OOS 30-60 min High
Gate1 FAIL Option 1.x 2-3 hours High
Gate2 FAIL Option 2.x 1-2 hours High
Both FAIL Option 3 Hybrid 3-4 hours Medium
3x FAIL Fallback 1-3 4-8 hours Low

Success Criteria (Remediation)

Re-optimized model must achieve:

  • PBO ≤ 20% (backtesting robustness)
  • DSR ≥ 95% (daily return quality)
  • Cost > 0% (positive returns)
  • Max Drawdown < 20% (risk control)
  • Sharpe ≥ 1.0 (risk-adjusted performance)

If all criteria met → Proceed to Phase 3


Communication Plan

If gates fail:

  1. Document failure reason (PBO/DSR/both)
  2. Communicate root cause analysis
  3. Present selected remediation option
  4. Execute changes
  5. Re-run Phase 1-2
  6. Report new results

Target: Restart Phase 3 within 3-5 hours of gate failure