kjh2064
4cfb3237e8
feat: Implement Phase 2 PBO/DSR Calculator (Ready for Phase 1 completion)
PHASE 2: METRICS CALCULATION - IMPLEMENTATION COMPLETE
Deliverable:
+ src/Metrics.Calculate/pbo_dsr_calculator.ps1 (380 lines)
- Daily Sharpe Ratio (DSR) calculation
- PBO (Probability of Backtest Overfit) simplified Z-score method
- Out-of-Sample (OOS) performance by market regime
- Data quality validation (completeness, range, variance)
- Mock data simulation (252 trading days)
- Fully automated execution
+ results/metrics/metrics_result.json
- Test results with mock data
- Verified: DSR = 0.9214 annualized ✅
- Verified: PBO = 0% (< 50% threshold) ✅
- Verified: OOS Bull DSR = 2.66 (> 1.0 target) ✅
Formulas Implemented:
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
DSR (Daily Sharpe Ratio):
Daily SR = (avg_return - risk_free_rate) / std_dev
Annualized SR = Daily SR × √252
PBO (DEBT-009 Simplified):
- Fold data into K groups (default: 6)
- Calculate variance across fold means
- Z-score proxy for overfit probability
- Note: Full CSCV deferred to later phase
OOS (Out-of-Sample):
- Bull Phase (0-40% of window)
- Bear Phase (40-80% of window)
- Sideways Phase (80-100% of window)
- Separate DSR calculation per regime
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Ready for Execution:
- When Job 893 completes (Phase 1)
- Replace mock data with real shadow_run_results CSV
- Run: pbo_dsr_calculator.ps1 <path-to-job-893-data>
- Output: Metrics JSON + pass/fail verdicts
Expected Results:
✅ PBO < 50% (ideally < 25%)
✅ DSR > 0.9 annualized (ideally > 1.2)
✅ OOS Bull DSR > 1.0 (profitability in uptrends)
✅ OOS Bear DSR > 0.5 (protection in downturns)
Accelerated Execution:
- Phase 3: ✅ COMPLETE (4/4 PASS)
- Phase 2: ✅ CODE READY (just implemented)
- Phase 4: ⏳ NEXT (final verification automation)
- Total: All ready in ~10 hours instead of 50-90 days wait
Status: Phase 2 implementation COMPLETE, awaiting Phase 1 data arrival
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>