데이터 게이트 검증기와 DAG 연결

This commit is contained in:
2026-06-18 01:57:19 +09:00
parent d7f9d3a944
commit 318eb87a26
8 changed files with 422 additions and 8 deletions
+2 -2
View File
@@ -36,7 +36,7 @@
| **D2 공식 결정론** | 149개 공식 ID 전부 lifecycle 등록 | 269개 등록 (100%) ✅ | | **D2 공식 결정론** | 149개 공식 ID 전부 lifecycle 등록 | 269개 등록 (100%) ✅ |
| **D3 리스크 제어** | Core/Satellite/Cash 버킷 밴드 위반 0건 | RISK_ON 밴드 내 유지 중 | | **D3 리스크 제어** | Core/Satellite/Cash 버킷 밴드 위반 0건 | RISK_ON 밴드 내 유지 중 |
| **D4 알파 피드백** | 예측→실현 수익 루프 30건 이상 누적 | 0건 (DATA_GATED ~2026-07-15) | | **D4 알파 피드백** | 예측→실현 수익 루프 30건 이상 누적 | 0건 (DATA_GATED ~2026-07-15) |
| **D5 실행 자동화** | run_all 1회 실행으로 전체 파이프라인 완결 | 90단계 DAG 구축 완료 ✅ | | **D5 실행 자동화** | run_all 1회 실행으로 전체 파이프라인 완결 | 96단계 DAG 구축 완료 ✅ |
--- ---
@@ -586,7 +586,7 @@ CI 게이트:
honest_proof_score: 50.95 → 목표: ≥70 (T+20 30건 → 70.95 자동 달성 예상) honest_proof_score: 50.95 → 목표: ≥70 (T+20 30건 → 70.95 자동 달성 예상)
자동화: 자동화:
run_all 성공률: 90단계 DAG PASS → 목표: ≥95% ✅ (step_count=90, wave_0~9) run_all 성공률: 96단계 DAG PASS → 목표: ≥95% ✅ (step_count=96, wave_0~9)
CI/CD 커버리지: 100% → 목표: 100% ✅ (Synology act_runner 온라인, 4게이트 PASS) CI/CD 커버리지: 100% → 목표: 100% ✅ (Synology act_runner 온라인, 4게이트 PASS)
수동 개입 횟수: 매일 → 목표: ≤1회/주 (setupDailyRunAllTrigger 설정 후) 수동 개입 횟수: 매일 → 목표: ≤1회/주 (setupDailyRunAllTrigger 설정 후)
``` ```
+6
View File
@@ -16,8 +16,14 @@
"ops:package": "python tools/refresh_trading_calendar.py && python tools/prepare_upload_zip.py --validation-mode release --profile", "ops:package": "python tools/refresh_trading_calendar.py && python tools/prepare_upload_zip.py --validation-mode release --profile",
"prepare-upload-zip": "python tools/refresh_trading_calendar.py && python tools/prepare_upload_zip.py", "prepare-upload-zip": "python tools/refresh_trading_calendar.py && python tools/prepare_upload_zip.py",
"ops:audit": "python tools/harness_coverage_auditor.py", "ops:audit": "python tools/harness_coverage_auditor.py",
"build-prediction-accuracy-harness": "python tools/build_prediction_accuracy_harness_v2.py",
"build-alpha-feedback-loop": "python tools/build_alpha_feedback_loop_v2.py",
"build-operational-alpha-calibration": "python tools/build_operational_alpha_calibration_v2.py",
"build-realized-performance": "python tools/build_realized_performance_v1.py", "build-realized-performance": "python tools/build_realized_performance_v1.py",
"validate-completion-harness": "python tools/validate_completion_harness_instructions_v1.py", "validate-completion-harness": "python tools/validate_completion_harness_instructions_v1.py",
"validate-prediction-accuracy-harness": "python tools/validate_prediction_accuracy_harness_v2.py",
"validate-alpha-feedback-loop": "python tools/validate_alpha_feedback_loop_v2.py",
"validate-operational-alpha-calibration": "python tools/validate_operational_alpha_calibration_v2.py",
"validate-realized-performance": "python tools/validate_realized_performance_v1.py", "validate-realized-performance": "python tools/validate_realized_performance_v1.py",
"validate-gas-recovery": "python tools/validate_gas_orchestration_recovery_v1.py", "validate-gas-recovery": "python tools/validate_gas_orchestration_recovery_v1.py",
"ops:clean": "python tools/clean_temp_artifacts_v1.py", "ops:clean": "python tools/clean_temp_artifacts_v1.py",
+4 -4
View File
@@ -1,9 +1,9 @@
{ {
"formula_id": "AUDIT_REPOSITORY_ENTROPY_V2", "formula_id": "AUDIT_REPOSITORY_ENTROPY_V2",
"gate": "PASS", "gate": "PASS",
"total_file_count": 1884, "total_file_count": 1890,
"package_script_count": 23, "package_script_count": 29,
"temp_json_count": 188, "temp_json_count": 191,
"budget": { "budget": {
"schema_version": "repository_entropy_budget.v1", "schema_version": "repository_entropy_budget.v1",
"max_total_files": 2200, "max_total_files": 2200,
@@ -15,5 +15,5 @@
"keep package scripts within release envelope" "keep package scripts within release envelope"
] ]
}, },
"source_zip_sha256": "469e09441818688b861efa6d6ee1bd6123806150c2940608ba964b44bcf50eb0" "source_zip_sha256": "1b582cc260c09b41dfd2c361bdd40705ac003b88ad8a09b42bbba615189cbf2b"
} }
+77 -2
View File
@@ -1,5 +1,5 @@
schema_version: release_dag.v3 schema_version: release_dag.v3
step_count: 90 step_count: 96
goal: Linearize package.json scripts into a validated DAG execution graph. goal: Linearize package.json scripts into a validated DAG execution graph.
execution_order: execution_order:
# 토폴로지 정렬 기준 병렬 실행 wave (의존성 없는 노드들을 동시에 실행 가능) # 토폴로지 정렬 기준 병렬 실행 wave (의존성 없는 노드들을 동시에 실행 가능)
@@ -85,13 +85,19 @@ execution_order:
wave_6: wave_6:
- build_algorithm_guidance_proof - build_algorithm_guidance_proof
- build_artifact_chain_hash - build_artifact_chain_hash
- build_alpha_feedback_loop
- build_honest_proof_gap_analyzer - build_honest_proof_gap_analyzer
- build_operational_alpha_calibration
- build_prediction_accuracy_harness
- validate_json_generator_outputs - validate_json_generator_outputs
- validate_alpha_feedback_loop
- validate_llm_copy_only - validate_llm_copy_only
- validate_llm_determinism - validate_llm_determinism
- validate_llm_regression - validate_llm_regression
- validate_low_capability - validate_low_capability
- validate_provenance - validate_provenance
- validate_prediction_accuracy_harness
- validate_operational_alpha_calibration
- validate_render_diff - validate_render_diff
- validate_report_numeric_consistency - validate_report_numeric_consistency
- validate_report_section_completeness - validate_report_section_completeness
@@ -163,6 +169,42 @@ dag:
artifact_policy: "keep" artifact_policy: "keep"
note: "WBS-4.1 realized performance replay summary — non-blocking diagnostic" note: "WBS-4.1 realized performance replay summary — non-blocking diagnostic"
build_prediction_accuracy_harness:
id: build_prediction_accuracy_harness
command: ["python", "tools/build_prediction_accuracy_harness_v2.py"]
inputs: ["tools/build_prediction_accuracy_harness_v2.py", "Temp/proposal_evaluation_history.json"]
outputs: ["Temp/prediction_accuracy_harness_v2.json"]
depends_on: ["update_proposal_evaluation_history"]
timeout_sec: 30
cache_key: "build_prediction_accuracy_harness_v2"
strict: false
artifact_policy: "keep"
note: "WBS-4.2 prediction accuracy harness — non-blocking diagnostic"
build_alpha_feedback_loop:
id: build_alpha_feedback_loop
command: ["python", "tools/build_alpha_feedback_loop_v2.py"]
inputs: ["tools/build_alpha_feedback_loop_v2.py", "Temp/proposal_evaluation_history.json"]
outputs: ["Temp/alpha_feedback_loop_v2.json"]
depends_on: ["update_proposal_evaluation_history"]
timeout_sec: 30
cache_key: "build_alpha_feedback_loop_v2"
strict: false
artifact_policy: "keep"
note: "WBS-4.3 alpha feedback loop — non-blocking diagnostic"
build_operational_alpha_calibration:
id: build_operational_alpha_calibration
command: ["python", "tools/build_operational_alpha_calibration_v2.py"]
inputs: ["tools/build_operational_alpha_calibration_v2.py", "Temp/outcome_quality_score_v1.json", "Temp/prediction_accuracy_harness_v2.json", "Temp/trade_quality_from_t5_v1.json", "Temp/smart_cash_recovery_v5.json"]
outputs: ["Temp/operational_alpha_calibration_v2.json"]
depends_on: ["build_prediction_accuracy_harness", "build_alpha_feedback_loop", "build_realized_performance"]
timeout_sec: 30
cache_key: "build_operational_alpha_calibration_v2"
strict: false
artifact_policy: "keep"
note: "WBS-4.3 operational alpha calibration — non-blocking diagnostic"
build_factor_shadow_eligibility: build_factor_shadow_eligibility:
id: build_factor_shadow_eligibility id: build_factor_shadow_eligibility
command: ["python", "tools/build_factor_shadow_eligibility_v1.py"] command: ["python", "tools/build_factor_shadow_eligibility_v1.py"]
@@ -581,6 +623,39 @@ dag:
strict: true strict: true
artifact_policy: "keep" artifact_policy: "keep"
validate_prediction_accuracy_harness:
id: validate_prediction_accuracy_harness
command: ["python", "tools/validate_prediction_accuracy_harness_v2.py"]
inputs: ["tools/validate_prediction_accuracy_harness_v2.py", "Temp/prediction_accuracy_harness_v2.json"]
outputs: ["Temp/validate_prediction_accuracy_harness_v2.json"]
depends_on: ["build_prediction_accuracy_harness"]
timeout_sec: 30
cache_key: "validate_prediction_accuracy_harness_v2"
strict: true
artifact_policy: "keep"
validate_alpha_feedback_loop:
id: validate_alpha_feedback_loop
command: ["python", "tools/validate_alpha_feedback_loop_v2.py"]
inputs: ["tools/validate_alpha_feedback_loop_v2.py", "Temp/alpha_feedback_loop_v2.json"]
outputs: ["Temp/validate_alpha_feedback_loop_v2.json"]
depends_on: ["build_alpha_feedback_loop"]
timeout_sec: 30
cache_key: "validate_alpha_feedback_loop_v2"
strict: true
artifact_policy: "keep"
validate_operational_alpha_calibration:
id: validate_operational_alpha_calibration
command: ["python", "tools/validate_operational_alpha_calibration_v2.py"]
inputs: ["tools/validate_operational_alpha_calibration_v2.py", "Temp/operational_alpha_calibration_v2.json"]
outputs: ["Temp/validate_operational_alpha_calibration_v2.json"]
depends_on: ["build_operational_alpha_calibration"]
timeout_sec: 30
cache_key: "validate_operational_alpha_calibration_v2"
strict: true
artifact_policy: "keep"
validate_realized_performance: validate_realized_performance:
id: validate_realized_performance id: validate_realized_performance
command: ["python", "tools/validate_realized_performance_v1.py"] command: ["python", "tools/validate_realized_performance_v1.py"]
@@ -1214,7 +1289,7 @@ dag:
command: ["python", "tools/prepare_upload_zip.py", "--skip-validate", "--skip-convert", "--validation-mode", "package-only"] command: ["python", "tools/prepare_upload_zip.py", "--skip-validate", "--skip-convert", "--validation-mode", "package-only"]
inputs: ["tools/prepare_upload_zip.py"] inputs: ["tools/prepare_upload_zip.py"]
outputs: [] outputs: []
depends_on: ["audit_entropy", "validate_specs", "validate_active_manifest", "validate_report_sync", "validate_report_numeric_consistency", "validate_field_dict", "validate_provenance", "validate_low_capability", "validate_golden_coverage", "validate_calibration", "validate_schema_model", "validate_gas_adapter", "validate_agents_shrink", "validate_no_replay_live_mix", "validate_realized_performance", "validate_runtime_source_whitelist", "validate_cash_ledger", "validate_factor_lifecycle", "validate_factor_lifecycle_completeness", "validate_metric_alias_collision", "validate_architecture_boundaries", "validate_module_io_coverage", "validate_artifact_chain_hash", "validate_artifact_sync", "validate_renderer_no_calc", "validate_packaged_refs", "validate_property_invariants", "validate_anti_late_entry", "validate_rule_lifecycle", "validate_change_requests", "validate_completion_harness_instructions", "validate_engine_health_card", "validate_llm_regression", "validate_llm_copy_only", "build_final_decision", "build_final_context", "build_provenance_ledger", "build_live_replay_separation", "build_late_chase_attribution", "build_profit_giveback_ratchet", "build_shadow_ledger", "build_operating_cadence_signal", "build_engine_health_card", "build_module_io_coverage", "build_artifact_chain_hash", "build_report", "build_bundle", "build_schema_models", "build_architecture_boundaries", "validate_decision_trace", "validate_factor_conflicts", "validate_no_lookahead", "validate_execution_sim", "validate_render_diff", "build_shadow_promotion", "validate_llm_determinism", "build_time_stop_forecast", "validate_live_activation", "build_rebalance_sheet"] depends_on: ["audit_entropy", "validate_specs", "validate_active_manifest", "validate_report_sync", "validate_report_numeric_consistency", "validate_field_dict", "validate_provenance", "validate_low_capability", "validate_golden_coverage", "validate_calibration", "validate_schema_model", "validate_gas_adapter", "validate_agents_shrink", "validate_no_replay_live_mix", "validate_prediction_accuracy_harness", "validate_alpha_feedback_loop", "validate_operational_alpha_calibration", "validate_realized_performance", "validate_runtime_source_whitelist", "validate_cash_ledger", "validate_factor_lifecycle", "validate_factor_lifecycle_completeness", "validate_metric_alias_collision", "validate_architecture_boundaries", "validate_module_io_coverage", "validate_artifact_chain_hash", "validate_artifact_sync", "validate_renderer_no_calc", "validate_packaged_refs", "validate_property_invariants", "validate_anti_late_entry", "validate_rule_lifecycle", "validate_change_requests", "validate_completion_harness_instructions", "validate_engine_health_card", "validate_llm_regression", "validate_llm_copy_only", "build_final_decision", "build_final_context", "build_provenance_ledger", "build_live_replay_separation", "build_late_chase_attribution", "build_profit_giveback_ratchet", "build_shadow_ledger", "build_operating_cadence_signal", "build_engine_health_card", "build_module_io_coverage", "build_artifact_chain_hash", "build_report", "build_bundle", "build_schema_models", "build_architecture_boundaries", "validate_decision_trace", "validate_factor_conflicts", "validate_no_lookahead", "validate_execution_sim", "validate_render_diff", "build_shadow_promotion", "validate_llm_determinism", "build_time_stop_forecast", "validate_live_activation", "build_rebalance_sheet", "build_prediction_accuracy_harness", "build_alpha_feedback_loop", "build_operational_alpha_calibration"]
timeout_sec: 60 timeout_sec: 60
cache_key: "prepare_zip_v1" cache_key: "prepare_zip_v1"
strict: true strict: true
+5
View File
@@ -89,6 +89,11 @@ TEMP_KEEP_FILES = {
"canonical_artifact_resolver_v1.json", "canonical_artifact_resolver_v1.json",
"final_execution_decision_v2.json", "final_execution_decision_v2.json",
"prediction_accuracy_harness_v2.json", "prediction_accuracy_harness_v2.json",
"validate_prediction_accuracy_harness_v2.json",
"alpha_feedback_loop_v2.json",
"validate_alpha_feedback_loop_v2.json",
"operational_alpha_calibration_v2.json",
"validate_operational_alpha_calibration_v2.json",
"realized_performance_v1.json", "realized_performance_v1.json",
"validate_realized_performance_v1.json", "validate_realized_performance_v1.json",
"single_truth_ledger_v2.json", "single_truth_ledger_v2.json",
+92
View File
@@ -0,0 +1,92 @@
"""validate_alpha_feedback_loop_v2.py — ALPHA_FEEDBACK_LOOP_VALIDATE_V2
Temp/alpha_feedback_loop_v2.json의 구조와 데이터 게이트 상태를 검증한다.
"""
from __future__ import annotations
import argparse
import json
from pathlib import Path
from typing import Any
ROOT = Path(__file__).resolve().parents[1]
DEFAULT_INPUT = ROOT / "Temp" / "alpha_feedback_loop_v2.json"
DEFAULT_OUT = ROOT / "Temp" / "validate_alpha_feedback_loop_v2.json"
FORMULA_ID = "ALPHA_FEEDBACK_LOOP_VALIDATE_V2"
def _load(path: Path) -> Any:
if not path.exists():
return {}
try:
return json.loads(path.read_text(encoding="utf-8"))
except Exception:
return {}
def _is_dict(value: Any) -> bool:
return isinstance(value, dict)
def main() -> int:
ap = argparse.ArgumentParser()
ap.add_argument("--input", default=str(DEFAULT_INPUT))
ap.add_argument("--out", default=str(DEFAULT_OUT))
args = ap.parse_args()
input_path = Path(args.input)
input_path = input_path if input_path.is_absolute() else ROOT / input_path
out_path = Path(args.out)
out_path = out_path if out_path.is_absolute() else ROOT / out_path
payload = _load(input_path)
errors: list[str] = []
if not _is_dict(payload):
errors.append("payload must be object")
else:
if payload.get("formula_id") != "ALPHA_FEEDBACK_LOOP_V2":
errors.append("formula_id mismatch")
status = str(payload.get("status") or "")
if status not in {"DATA_INSUFFICIENT", "ANALYZED"}:
errors.append(f"status={status}")
if "cases_analyzed" not in payload or not isinstance(payload.get("cases_analyzed"), int):
errors.append("cases_analyzed must be int")
if "recommended_adjustments" not in payload or not isinstance(payload.get("recommended_adjustments"), list):
errors.append("recommended_adjustments must be list")
cases = payload.get("cases_analyzed")
recs = payload.get("recommended_adjustments")
if isinstance(cases, int):
if cases < 10 and status != "DATA_INSUFFICIENT":
errors.append("cases_analyzed < 10 requires DATA_INSUFFICIENT")
if cases >= 10 and status != "ANALYZED":
errors.append("cases_analyzed >= 10 requires ANALYZED")
if isinstance(recs, list) and status == "DATA_INSUFFICIENT" and recs:
errors.append("DATA_INSUFFICIENT must not carry recommendations")
if status == "ANALYZED":
for key in [
"active_signal_rate_pct", "active_signal_n",
"passive_signal_rate_pct", "passive_signal_n",
"combined_rate_pct", "sell_signal_rate_pct", "sell_signal_n",
"pa1_current_ratio", "pa1_thesis_sum", "pa1_antithesis_sum",
"component_analysis", "note",
]:
if key not in payload:
errors.append(f"missing field: {key}")
result = {
"formula_id": FORMULA_ID,
"gate": "PASS" if not errors else "FAIL",
"checked_file": str(Path(args.input).as_posix()),
"errors": errors,
}
out_path.write_text(json.dumps(result, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
print(json.dumps(result, ensure_ascii=False, indent=2))
return 0 if not errors else 1
if __name__ == "__main__":
raise SystemExit(main())
@@ -0,0 +1,105 @@
"""validate_operational_alpha_calibration_v2.py — OPERATIONAL_ALPHA_CALIBRATION_VALIDATE_V2
Temp/operational_alpha_calibration_v2.json의 최소 계약을 검증한다.
"""
from __future__ import annotations
import argparse
import json
from pathlib import Path
from typing import Any
ROOT = Path(__file__).resolve().parents[1]
DEFAULT_INPUT = ROOT / "Temp" / "operational_alpha_calibration_v2.json"
DEFAULT_OUT = ROOT / "Temp" / "validate_operational_alpha_calibration_v2.json"
FORMULA_ID = "OPERATIONAL_ALPHA_CALIBRATION_VALIDATE_V2"
def _load(path: Path) -> Any:
if not path.exists():
return {}
try:
return json.loads(path.read_text(encoding="utf-8"))
except Exception:
return {}
def _is_dict(value: Any) -> bool:
return isinstance(value, dict)
def main() -> int:
ap = argparse.ArgumentParser()
ap.add_argument("--input", default=str(DEFAULT_INPUT))
ap.add_argument("--out", default=str(DEFAULT_OUT))
args = ap.parse_args()
input_path = Path(args.input)
input_path = input_path if input_path.is_absolute() else ROOT / input_path
out_path = Path(args.out)
out_path = out_path if out_path.is_absolute() else ROOT / out_path
payload = _load(input_path)
errors: list[str] = []
if not _is_dict(payload):
errors.append("payload must be object")
else:
if payload.get("formula_id") != "OPERATIONAL_ALPHA_CALIBRATION_V2":
errors.append("formula_id mismatch")
if payload.get("gate") not in {"PERFORMANCE_READY", "NOT_READY"}:
errors.append(f"gate={payload.get('gate')}")
if "performance_ready" not in payload or not isinstance(payload.get("performance_ready"), bool):
errors.append("performance_ready must be bool")
if "confidence_score" not in payload or not isinstance(payload.get("confidence_score"), (int, float)):
errors.append("confidence_score must be numeric")
for key in ["metrics", "targets", "readiness_reasons"]:
if key not in payload:
errors.append(f"missing field: {key}")
metrics = payload.get("metrics")
if isinstance(metrics, dict):
for key in [
"outcome_quality_score",
"t20_operational_sample",
"t20_operational_pass_rate",
"t5_operational_sample",
"t5_operational_pass_rate",
"trade_quality_t5_score",
"value_damage_pct_avg",
]:
if key not in metrics:
errors.append(f"missing field: metrics.{key}")
targets = payload.get("targets")
if isinstance(targets, dict):
for key in [
"outcome_quality_score_min",
"t20_operational_sample_min",
"t20_operational_pass_rate_min",
"t5_operational_sample_min",
"t5_operational_pass_rate_min",
"trade_quality_t5_score_min",
"value_damage_pct_avg_max",
]:
if key not in targets:
errors.append(f"missing field: targets.{key}")
reasons = payload.get("readiness_reasons")
if isinstance(reasons, list):
if payload.get("gate") == "PERFORMANCE_READY" and reasons:
errors.append("PERFORMANCE_READY must not have readiness reasons")
if payload.get("gate") == "NOT_READY" and not reasons:
errors.append("NOT_READY must have readiness reasons")
result = {
"formula_id": FORMULA_ID,
"gate": "PASS" if not errors else "FAIL",
"checked_file": str(Path(args.input).as_posix()),
"errors": errors,
}
out_path.write_text(json.dumps(result, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
print(json.dumps(result, ensure_ascii=False, indent=2))
return 0 if not errors else 1
if __name__ == "__main__":
raise SystemExit(main())
@@ -0,0 +1,131 @@
"""validate_prediction_accuracy_harness_v2.py — PREDICTION_ACCURACY_HARNESS_VALIDATE_V2
Temp/prediction_accuracy_harness_v2.json의 기본 구조와 허용된 데이터 게이트 상태를 검증한다.
현재는 운영 T+5/T+20 표본이 부족할 수 있으므로 INSUFFICIENT_SAMPLES는 허용한다.
"""
from __future__ import annotations
import argparse
import json
from pathlib import Path
from typing import Any
ROOT = Path(__file__).resolve().parents[1]
DEFAULT_INPUT = ROOT / "Temp" / "prediction_accuracy_harness_v2.json"
DEFAULT_OUT = ROOT / "Temp" / "validate_prediction_accuracy_harness_v2.json"
FORMULA_ID = "PREDICTION_ACCURACY_HARNESS_VALIDATE_V2"
ALLOWED_CALIBRATION = {
"CALIBRATED",
"MONITOR",
"PAE_CALIBRATION_REQUIRED",
"BUY_PROPOSAL_FROZEN_RECOMMEND",
"INSUFFICIENT_SAMPLES",
}
def _load(path: Path) -> Any:
if not path.exists():
return {}
try:
return json.loads(path.read_text(encoding="utf-8"))
except Exception:
return {}
def _is_dict(value: Any) -> bool:
return isinstance(value, dict)
def _ensure_fields(payload: dict[str, Any], path: str, fields: list[str], errors: list[str]) -> None:
block = payload
if path:
for part in path.split("."):
block = block.get(part) if isinstance(block, dict) else None
if not isinstance(block, dict):
errors.append(f"{path or 'root'} must be object")
return
for field in fields:
if field not in block:
errors.append(f"missing field: {path + '.' if path else ''}{field}")
def main() -> int:
ap = argparse.ArgumentParser()
ap.add_argument("--input", default=str(DEFAULT_INPUT))
ap.add_argument("--out", default=str(DEFAULT_OUT))
args = ap.parse_args()
input_path = Path(args.input)
input_path = input_path if input_path.is_absolute() else ROOT / input_path
out_path = Path(args.out)
out_path = out_path if out_path.is_absolute() else ROOT / out_path
payload = _load(input_path)
errors: list[str] = []
if not _is_dict(payload):
errors.append("payload must be object")
else:
if payload.get("formula_id") != "PREDICTION_ACCURACY_HARNESS_V2":
errors.append("formula_id mismatch")
calibration_state = str(payload.get("calibration_state") or "")
if calibration_state not in ALLOWED_CALIBRATION:
errors.append(f"calibration_state={calibration_state}")
for key in [
"as_of_date",
"data_origin_audit",
"windows",
"evaluation_methodology",
]:
if key not in payload:
errors.append(f"missing field: {key}")
audit = payload.get("data_origin_audit")
if isinstance(audit, dict):
for key in [
"operational_sample_count",
"replay_sample_count",
"untagged_row_count",
"unrealized_outcome_row_count",
"replay_in_live_stats",
"operational_only_accuracy",
]:
if key not in audit:
errors.append(f"missing field: data_origin_audit.{key}")
for key in [
"t1_op_rate", "t1_sample", "t5_op_rate", "t5_sample",
"t20_op_rate", "t20_sample", "t20_replay_rate", "t20_replay_sample",
"t20_replay_avg_return_pct", "t20_replay_stdev_return_pct",
"window_90d_rate",
]:
if key not in payload:
errors.append(f"missing field: {key}")
windows = payload.get("windows")
if isinstance(windows, dict):
_ensure_fields(windows, "t1", ["all", "30d", "7d"], errors)
_ensure_fields(windows, "t5", ["all", "active_passive", "30d", "90d"], errors)
_ensure_fields(windows, "t20", ["operational", "operational_30d", "replay", "replay_return_dist"], errors)
else:
errors.append("windows must be object")
t5_sample = payload.get("t5_sample")
if isinstance(t5_sample, int) and t5_sample < 30 and calibration_state != "INSUFFICIENT_SAMPLES":
errors.append("t5_sample < 30 requires INSUFFICIENT_SAMPLES")
result = {
"formula_id": FORMULA_ID,
"gate": "PASS" if not errors else "FAIL",
"checked_file": str(Path(args.input).as_posix()),
"errors": errors,
}
out_path.write_text(json.dumps(result, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
print(json.dumps(result, ensure_ascii=False, indent=2))
return 0 if not errors else 1
if __name__ == "__main__":
raise SystemExit(main())