diff --git a/AGENTS.md b/AGENTS.md index 8d714bc..8d4ade1 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -130,7 +130,7 @@ - **Python 인터프리터**: Windows 로컬 환경에서는 반드시 `python`을 사용한다 (`python3` 금지). - `python` → Python 3.13.5 (`Python313/`) — yaml/openpyxl/yfinance 등 프로젝트 패키지 설치됨 - `python3` → Python 3.12 (Windows Store) — 프로젝트 패키지 미설치 → `ModuleNotFoundError` 유발 - - Synology CI는 `/usr/bin/python3`를 사용하므로 `.gitea/workflows/ci.yml`은 `python3` 유지 + - 클라우드 서버(hz-prod-01)는 `/usr/bin/python3`를 사용하므로 `.gitea/workflows/ci.yml`은 `python3` 유지 ## 6. 검증 규칙 - `python tools/validate_specs.py` diff --git a/docs/SYNOLOGY_ACT_RUNNER_REFACTOR_PR_BODY.md b/docs/SYNOLOGY_ACT_RUNNER_REFACTOR_PR_BODY.md deleted file mode 100644 index fdae266..0000000 --- a/docs/SYNOLOGY_ACT_RUNNER_REFACTOR_PR_BODY.md +++ /dev/null @@ -1,26 +0,0 @@ -# Synology Act Runner Split PR Body - -## Title - -`chore: split Synology act_runner start and re-registration scripts` - -## Body - -- Added `tools/re_register_act_runner_synology.sh` for explicit host-mode re-registration. -- Added `tools/start_act_runner_synology.sh` for boot-time daemon start only. -- Kept `tools/setup_act_runner.sh` as the bootstrap path, but made the re-registration flow explicit and repeatable. -- Switched the runner registration labels to `self-hosted:host,snapshot-admin-host:host` so the job runs in host mode instead of Docker job containers and can be targeted by a dedicated deployment label. -- Updated `docs/SYNOLOGY_SNAPSHOT_ADMIN_POC.md` and `docs/ROADMAP_WBS.md` so the operator flow and WBS notes match the new runner split. -- The `snapshot_admin.yml` workflow is split into push smoke validation and manual full validation, which reduces routine CI cost while preserving the full web smoke path on demand. -- The deploy workflow now waits for `127.0.0.1:8787/api/state` readiness before asserting success, so startup latency does not fail the run spuriously. -- The `ci.yml` workflow now keeps `push` traffic on the core gate only, with UI/storage validation retained for non-push events. - -## Verification - -- `python tools/validate_snapshot_admin_workflow_v1.py` -- `python -c "import yaml, pathlib; yaml.safe_load(pathlib.Path('.gitea/workflows/snapshot_admin.yml').read_text(encoding='utf-8'))"` -- `git diff -- .gitea/workflows/snapshot_admin.yml tools/setup_act_runner.sh docs/SYNOLOGY_SNAPSHOT_ADMIN_POC.md docs/ROADMAP_WBS.md` -- Deploy job evidence: - - `healthcheck` retried after startup and passed - - `snapshot-admin-web-v6` returned from the verification step - - `Job succeeded` diff --git a/docs/SYNOLOGY_KIS_COLLECTION_SETUP.md b/docs/SYNOLOGY_KIS_COLLECTION_SETUP.md deleted file mode 100644 index d1ed76a..0000000 --- a/docs/SYNOLOGY_KIS_COLLECTION_SETUP.md +++ /dev/null @@ -1,95 +0,0 @@ -# Synology KIS Data Collection Setup - -This note answers how to run: - -```powershell -$env:KIS_APP_Key="..." -$env:KIS_APP_Secret="..." -python tools/run_kis_data_collection_v1.py --input-json GatherTradingData.json --sqlite-db src/quant_engine/kis_data_collection.db --output-json Temp/kis_data_collection_v1.json --kis-account real -``` - -on Synology DSM. - -## Rule - -Synology is Linux-based, so use `export` or a sourced env file. Do not use Windows `$env:` syntax. - -The code reads these exact, case-sensitive names for real accounts: - -- `KIS_APP_Key` -- `KIS_APP_Secret` - -For mock accounts, the names are: - -- `KIS_APP_Key_TEST` -- `KIS_APP_Secret_TEST` - -## Recommended DSM Task Scheduler script - -Create a `User-defined script` task and run: - -```bash -#!/bin/sh -set -eu - -ROOT_DIR="/volume1/projects/data_feed" - -export KIS_APP_Key="your_real_app_key" -export KIS_APP_Secret="your_real_app_secret" - -cd "$ROOT_DIR" -python tools/run_kis_data_collection_v1.py \ - --input-json GatherTradingData.json \ - --sqlite-db src/quant_engine/kis_data_collection.db \ - --output-json Temp/kis_data_collection_v1.json \ - --kis-account real -``` - -## Better practice for secrets - -Store secrets in a private env file and source it from the task: - -```bash -set -eu -ROOT_DIR="/volume1/projects/data_feed" -SECRETS_FILE="/volume1/projects/data_feed/.secrets/kis_real.env" - -. "$SECRETS_FILE" -cd "$ROOT_DIR" -python tools/run_kis_data_collection_v1.py \ - --input-json GatherTradingData.json \ - --sqlite-db src/quant_engine/kis_data_collection.db \ - --output-json Temp/kis_data_collection_v1.json \ - --kis-account real -``` - -Suggested file permissions: - -- owner-only read/write -- no shared group access -- no commit to git - -## Mock account variant - -```bash -export KIS_APP_Key_TEST="your_mock_app_key" -export KIS_APP_Secret_TEST="your_mock_app_secret" -python tools/run_kis_data_collection_v1.py \ - --input-json GatherTradingData.json \ - --sqlite-db src/quant_engine/kis_data_collection.db \ - --output-json Temp/kis_data_collection_v1.json \ - --kis-account mock \ - --no-live-kis -``` - -## What the collector writes - -- SQLite: `src/quant_engine/kis_data_collection.db` -- JSON summary: `Temp/kis_data_collection_v1.json` - -The latest collected summary in this workspace shows: - -- `row_count = 25` -- `kis_open_api = 21` -- `gathertradingdata_json = 25` - diff --git a/docs/SYNOLOGY_SNAPSHOT_ADMIN_COMMIT_MESSAGE_TEMPLATE.md b/docs/SYNOLOGY_SNAPSHOT_ADMIN_COMMIT_MESSAGE_TEMPLATE.md deleted file mode 100644 index de255a8..0000000 --- a/docs/SYNOLOGY_SNAPSHOT_ADMIN_COMMIT_MESSAGE_TEMPLATE.md +++ /dev/null @@ -1,37 +0,0 @@ -# Synology Snapshot Admin Commit Message Template - -Use this after a real Synology verification or a final documentation-only update. - -## Recommended format - -```text -WBS-7.9: Synology snapshot_admin deployment POC and live verification evidence -``` - -## If the change is documentation-only - -```text -WBS-7.9: add Synology deployment checklist, Task Scheduler commands, and evidence template -``` - -## If the change includes real NAS verification - -```text -WBS-7.9: verify Synology snapshot_admin reverse proxy, auth gate, and restart persistence -``` - -## Commit body template - -```text -- Added/updated Synology Task Scheduler launcher script -- Confirmed DSM reverse proxy settings -- Captured curl/browser evidence for local and external access -- Documented completion evidence in WBS-7.9 checklist -``` - -## Suggested workflow - -1. Run the validation commands. -2. Fill `docs/SYNOLOGY_SNAPSHOT_ADMIN_EVIDENCE_TEMPLATE.md`. -3. Commit with one of the messages above. -4. Push only after the evidence file is complete. diff --git a/docs/SYNOLOGY_SNAPSHOT_ADMIN_DEPLOYMENT_CHECKLIST.md b/docs/SYNOLOGY_SNAPSHOT_ADMIN_DEPLOYMENT_CHECKLIST.md deleted file mode 100644 index d264e69..0000000 --- a/docs/SYNOLOGY_SNAPSHOT_ADMIN_DEPLOYMENT_CHECKLIST.md +++ /dev/null @@ -1,153 +0,0 @@ -# Synology Snapshot Admin Deployment Checklist - -This checklist is the POC-ready version with concrete values. - -## 1. Target paths - -- Project root: `/volume1/projects/data_feed` -- Launch script: `/volume1/projects/data_feed/tools/run_snapshot_admin_synology.sh` -- Local DB: `/volume1/projects/data_feed/src/quant_engine/snapshot_admin.db` -- Local seed JSON: `/volume1/projects/data_feed/GatherTradingData.json` -- PID file: `/volume1/projects/data_feed/Temp/snapshot_admin.pid` -- Log file: `/volume1/projects/data_feed/Temp/snapshot_admin.log` - -See also: [`docs/SYNOLOGY_SNAPSHOT_ADMIN_DEPLOYMENT_CHECKLIST_FILLED.md`](C:/Temp/data_feed/docs/SYNOLOGY_SNAPSHOT_ADMIN_DEPLOYMENT_CHECKLIST_FILLED.md) -and [`docs/SYNOLOGY_SNAPSHOT_ADMIN_FIREWALL_PROXY_TABLE.md`](C:/Temp/data_feed/docs/SYNOLOGY_SNAPSHOT_ADMIN_FIREWALL_PROXY_TABLE.md) - -## 2. Service account - -- Preferred: dedicated DSM local user `snapshot-admin` -- Fallback for first POC: `root` -- Required permission: read/write access to `/volume1/projects/data_feed` - -## 3. Environment variables - -Set these before the Task Scheduler task runs. - -- `SNAPSHOT_ADMIN_AUTH_USER=snapshot-admin` -- `SNAPSHOT_ADMIN_AUTH_PASSWORD=` -- `SNAPSHOT_ADMIN_HOST=127.0.0.1` -- `SNAPSHOT_ADMIN_PORT=8787` -- `SNAPSHOT_ADMIN_ALLOW_REMOTE=0` -- `SNAPSHOT_ADMIN_PID_FILE=/volume1/projects/data_feed/Temp/snapshot_admin.pid` -- `SNAPSHOT_ADMIN_LOG_FILE=/volume1/projects/data_feed/Temp/snapshot_admin.log` -- `SNAPSHOT_ADMIN_STATE_URL=http://127.0.0.1:8787/api/state` -- `SNAPSHOT_ADMIN_PUBLIC_STATE_URL=https://admin.example.com/api/state` - -## 4. Task Scheduler tasks - -### Boot task - -- Name: `snapshot-admin-start` -- Trigger: `Boot-up` -- User: `snapshot-admin` or `root` -- Command: - -```bash -bash /volume1/projects/data_feed/tools/run_snapshot_admin_synology.sh start -``` - -### Healthcheck task - -- Name: `snapshot-admin-healthcheck` -- Trigger: `Scheduled Task` -- Interval: every 5 minutes -- User: same as boot task -- Command: - -```bash -bash /volume1/projects/data_feed/tools/run_snapshot_admin_synology.sh healthcheck -``` - -### Restart task - -- Name: `snapshot-admin-restart` -- Trigger: manual only -- User: same as boot task -- Command: - -```bash -bash /volume1/projects/data_feed/tools/run_snapshot_admin_synology.sh restart -``` - -## 4b. Gitea Actions runner label - -Use a unique host label so the deployment job is not mixed with generic self-hosted work. - -- Runner label: `snapshot-admin-host` -- Registration example: - -```bash -REG_TOKEN="" \ -GITEA_URL="http://192.168.123.100:8418" \ -RUNNER_LABEL="snapshot-admin-host" \ -bash tools/re_register_act_runner_synology.sh -``` - -- Workflow selector: - -```yaml -runs-on: [self-hosted, snapshot-admin-host] -``` - -## 4c. Queue handling - -- If the deploy workflow stays queued, it usually means the host runner is busy. -- Check the job currently holding the runner before re-dispatching. -- Do not keep dispatching deploy runs back-to-back. The workflow already uses `concurrency` to cancel in-progress duplicates. - -## 5. Reverse proxy - -- DSM path: `Control Panel > Login Portal > Advanced > Reverse Proxy` -- Rule name: `snapshot-admin` -- Source: - - Protocol: `HTTPS` - - Hostname: `admin.example.com` - - Port: `443` - - Path: `/` -- Destination: - - Protocol: `HTTP` - - Hostname: `127.0.0.1` - - Port: `8787` -- TLS certificate: certificate matching `admin.example.com` - -## 6. Firewall - -- Allow inbound `443/TCP` -- Block inbound `8787/TCP` from WAN -- If needed, allowlist office/VPN CIDRs only - -## 7. Verification order - -1. Start the service. -2. Confirm `bash /volume1/projects/data_feed/tools/run_snapshot_admin_synology.sh healthcheck` prints `healthcheck ok`. -3. Confirm local `curl -i http://127.0.0.1:8787/api/state`. - - Expect `200 OK`. - - Expect JSON with `version.app = snapshot-admin-web-v7`. -4. Confirm external `curl -i https://admin.example.com/api/state` returns `401`. - - Expect `WWW-Authenticate: Basic`. -5. Confirm authenticated `curl -u 'snapshot-admin:' https://admin.example.com/api/state` returns `200`. - - Expect the same `version.app` value as the local endpoint. -6. Confirm `curl -i https://admin.example.com/tables` after Basic Auth. - - Expect `200 OK` and the Tabler grid page. -7. Open browser `https://admin.example.com/`. - - Expect Basic Auth prompt, then UI render. -8. Open browser `https://admin.example.com/tables`. - - Expect Basic Auth prompt, then grid render. -9. Restart the task or NAS. -10. Repeat steps 2-8 and confirm the response pattern is unchanged. - -## 7b. Evidence rule - -- Do not mark `WBS-7.9` complete until the external `401`/`200` curl pair, both browser screenshots, and the reverse proxy rule screenshot are archived together. -- Loopback-only smoke tests are useful, but they do not replace the NAS-side live verification. - -## 7c. One-page field run sheet - -For a compact field execution order, use [`docs/SYNOLOGY_SNAPSHOT_ADMIN_FINAL_EXECUTION_ONE_PAGER.md`](C:/Temp/data_feed/docs/SYNOLOGY_SNAPSHOT_ADMIN_FINAL_EXECUTION_ONE_PAGER.md). - -## 8. Completion wording - -Use the following text only after evidence is collected: - -> WBS-7.9 실배포 검증 완료: Synology NAS에서 `tools/run_snapshot_admin_synology.sh` 기반 서비스가 `127.0.0.1:8787`에 정상 기동되고, DSM Reverse Proxy `HTTPS:443 -> HTTP 127.0.0.1:8787` 경유 외부 접속이 Basic Auth와 함께 `200 OK`로 확인되었으며, 미인증 요청은 `401 Unauthorized`로 차단되었다. `/` 및 `/tables` 렌더링과 재시작 후 지속성도 확인되었고, 증빙은 `docs/SYNOLOGY_SNAPSHOT_ADMIN_EVIDENCE_TEMPLATE.md` 양식으로 보관되었다. diff --git a/docs/SYNOLOGY_SNAPSHOT_ADMIN_DEPLOYMENT_CHECKLIST_FILLED.md b/docs/SYNOLOGY_SNAPSHOT_ADMIN_DEPLOYMENT_CHECKLIST_FILLED.md deleted file mode 100644 index 8e1aeb0..0000000 --- a/docs/SYNOLOGY_SNAPSHOT_ADMIN_DEPLOYMENT_CHECKLIST_FILLED.md +++ /dev/null @@ -1,114 +0,0 @@ -# Synology Snapshot Admin Deployment Checklist - Filled Example - -This is the deployment-ready example for the current repo state. -Replace only the hostname, certificate name, and strong password if your NAS uses different values. - -## 1. Target paths - -- Project root: `/volume1/projects/data_feed` -- Launch script: `/volume1/projects/data_feed/tools/run_snapshot_admin_synology.sh` -- Local DB: `/volume1/projects/data_feed/src/quant_engine/snapshot_admin.db` -- Local seed JSON: `/volume1/projects/data_feed/GatherTradingData.json` -- PID file: `/volume1/projects/data_feed/Temp/snapshot_admin.pid` -- Log file: `/volume1/projects/data_feed/Temp/snapshot_admin.log` - -## 2. Service account - -- Preferred DSM user: `snapshot-admin` -- Fallback for first POC: `root` -- Folder access: read/write on `/volume1/projects/data_feed` - -## 3. Environment variables - -```bash -SNAPSHOT_ADMIN_AUTH_USER=snapshot-admin -SNAPSHOT_ADMIN_AUTH_PASSWORD= -SNAPSHOT_ADMIN_HOST=127.0.0.1 -SNAPSHOT_ADMIN_PORT=8787 -SNAPSHOT_ADMIN_ALLOW_REMOTE=0 -SNAPSHOT_ADMIN_PID_FILE=/volume1/projects/data_feed/Temp/snapshot_admin.pid -SNAPSHOT_ADMIN_LOG_FILE=/volume1/projects/data_feed/Temp/snapshot_admin.log -SNAPSHOT_ADMIN_STATE_URL=http://127.0.0.1:8787/api/state -SNAPSHOT_ADMIN_PUBLIC_STATE_URL=https://admin.example.com/api/state -``` - -## 4. Task Scheduler - -### Boot task - -- Name: `snapshot-admin-start` -- User: `snapshot-admin` -- Trigger: `Boot-up` -- Command: - -```bash -bash /volume1/projects/data_feed/tools/run_snapshot_admin_synology.sh start -``` - -### Healthcheck task - -- Name: `snapshot-admin-healthcheck` -- User: `snapshot-admin` -- Trigger: every 5 minutes -- Command: - -```bash -bash /volume1/projects/data_feed/tools/run_snapshot_admin_synology.sh healthcheck -``` - -### Manual restart task - -- Name: `snapshot-admin-restart` -- User: `snapshot-admin` -- Trigger: manual -- Command: - -```bash -bash /volume1/projects/data_feed/tools/run_snapshot_admin_synology.sh restart -``` - -## 5. Reverse proxy - -- DSM path: `Control Panel > Login Portal > Advanced > Reverse Proxy` -- Rule name: `snapshot-admin` -- Source protocol: `HTTPS` -- Source hostname: `admin.example.com` -- Source port: `443` -- Source path: `/` -- Destination protocol: `HTTP` -- Destination hostname: `127.0.0.1` -- Destination port: `8787` -- TLS certificate: `admin.example.com` certificate - -## 6. Firewall - -- Allow inbound `443/TCP` -- Block inbound `8787/TCP` from WAN -- Allowlist only trusted office/VPN ranges if needed - -## 7. Verification commands - -```bash -curl -i http://127.0.0.1:8787/api/state -curl -i https://admin.example.com/api/state -curl -u 'snapshot-admin:' https://admin.example.com/api/state -curl -I https://admin.example.com/ -curl -I https://admin.example.com/tables -``` - -## 7b. Final preflight - -Use [`docs/SYNOLOGY_SNAPSHOT_ADMIN_FINAL_PREFLIGHT_10.md`](C:/Temp/data_feed/docs/SYNOLOGY_SNAPSHOT_ADMIN_FINAL_PREFLIGHT_10.md) -immediately before you mark the deployment complete. - -## 8. Completion wording - -Use this exact wording when evidence is complete: - -> WBS-7.9 실배포 검증 완료: Synology NAS에서 `tools/run_snapshot_admin_synology.sh` 기반 서비스가 `127.0.0.1:8787`에 정상 기동되고, DSM Reverse Proxy `HTTPS:443 -> HTTP 127.0.0.1:8787` 경유 외부 접속이 Basic Auth와 함께 `200 OK`로 확인되었으며, 미인증 요청은 `401 Unauthorized`로 차단되었다. `/` 및 `/tables` 렌더링과 재시작 후 지속성도 확인되었고, 증빙은 `docs/SYNOLOGY_SNAPSHOT_ADMIN_EVIDENCE_TEMPLATE.md` 양식으로 보관되었다. - -## 9. What to replace - -- `admin.example.com` if your public hostname differs -- `` with your generated password -- TLS certificate name if the DSM certificate uses another label diff --git a/docs/SYNOLOGY_SNAPSHOT_ADMIN_EVIDENCE_TEMPLATE.md b/docs/SYNOLOGY_SNAPSHOT_ADMIN_EVIDENCE_TEMPLATE.md deleted file mode 100644 index ff50e04..0000000 --- a/docs/SYNOLOGY_SNAPSHOT_ADMIN_EVIDENCE_TEMPLATE.md +++ /dev/null @@ -1,62 +0,0 @@ -# Synology Snapshot Admin Evidence Template - -Use this template to close `WBS-7.9` after a real Synology deployment test. - -## Deployment metadata - -- NAS model: -- DSM version: -- Public hostname: -- Reverse proxy rule name: -- TLS certificate name: -- Service launcher: `tools/run_snapshot_admin_synology.sh` -- Python service bind mode: -- Auth mode: `Basic Auth` - -## Local checks - -- `curl -i http://127.0.0.1:8787/api/state` -- Result: -- `curl -i http://127.0.0.1:8787/tables` -- Result: - -## External checks - -- `curl -i https:///api/state` -- Result: -- `curl -u ':' https:///api/state` -- Result: -- `curl -i https:///tables` -- Result: - -## Browser checks - -- `https:///` -- Result: -- `https:///tables` -- Result: - -## Restart persistence - -- Restart method used: -- Restart time: -- `healthcheck` result after restart: -- Time elapsed after restart: - -## Evidence attachments - -- Screenshot: DSM reverse proxy rule -- Screenshot: browser `/` -- Screenshot: browser `/tables` -- Log snippet: `Temp/snapshot_admin.log` -- `curl` output archive: - -## Completion statement - -- `WBS-7.9` completion condition met: - - local endpoint `200` - - external unauthenticated `401` - - external authenticated `200` - - browser render verified - - restart persistence verified - - evidence archived diff --git a/docs/SYNOLOGY_SNAPSHOT_ADMIN_FINAL_EXECUTION_ONE_PAGER.md b/docs/SYNOLOGY_SNAPSHOT_ADMIN_FINAL_EXECUTION_ONE_PAGER.md deleted file mode 100644 index a8e1f9d..0000000 --- a/docs/SYNOLOGY_SNAPSHOT_ADMIN_FINAL_EXECUTION_ONE_PAGER.md +++ /dev/null @@ -1,89 +0,0 @@ -# Synology Snapshot Admin Final Execution One-Pager - -Use this sheet on the NAS during the live verification run. - -## Goal - -Confirm that `snapshot_admin_server_v1.py` runs on Synology with loopback binding, DSM reverse proxy exposure, and Basic Auth protection. - -## Required values - -- Project root: `/volume1/projects/data_feed` -- Launcher: `/volume1/projects/data_feed/tools/run_snapshot_admin_synology.sh` -- Local URL: `http://127.0.0.1:8787/api/state` -- Public URL: `https://admin.example.com/api/state` -- Public UI URL: `https://admin.example.com/` -- Public tables URL: `https://admin.example.com/tables` - -## Execution order - -1. Start the service. - - `bash /volume1/projects/data_feed/tools/run_snapshot_admin_synology.sh start` -2. Confirm the healthcheck. - - `bash /volume1/projects/data_feed/tools/run_snapshot_admin_synology.sh healthcheck` - - Expected: `healthcheck ok` -3. Confirm local loopback. - - `curl -i http://127.0.0.1:8787/api/state` - - Expected: `200 OK` - - Expected JSON field: `version.app = snapshot-admin-web-v7` -4. Confirm unauthenticated external access. - - `curl -i https://admin.example.com/api/state` - - Expected: `401 Unauthorized` - - Expected header: `WWW-Authenticate: Basic` -5. Confirm authenticated external access. - - `curl -u 'snapshot-admin:' https://admin.example.com/api/state` - - Expected: `200 OK` - - Expected same `version.app` as local loopback -6. Confirm tables page. - - `curl -i https://admin.example.com/tables` - - Expected: `200 OK` - - Expected: Tabler grid HTML -7. Confirm browser render. - - Open `https://admin.example.com/` - - Open `https://admin.example.com/tables` - - Expected: Basic Auth prompt, then render -8. Confirm persistence. - - Restart the task or NAS - - Re-run steps 2-7 - - Expected: identical response pattern after restart - -## Queue check - -If the deployment workflow stays queued for more than a few minutes: - -1. Confirm the runner is registered with the host label. - - `RUNNER_LABEL=snapshot-admin-host` - - Re-register with `bash tools/re_register_act_runner_synology.sh` after setting the registration token. -2. Confirm the runner daemon is running. - - `bash tools/start_act_runner_synology.sh` -3. Confirm the queue target is the host runner label. - - Deploy workflow uses `runs-on: [self-hosted, snapshot-admin-host]` -4. If another job is occupying the runner, wait for it to finish or cancel the stale workflow from Gitea. -5. Re-dispatch `snapshot_admin_deploy.yml` after the runner is idle. - -## Pass criteria - -- Loopback `200` confirmed. -- External unauthenticated `401` confirmed. -- External authenticated `200` confirmed. -- `/` and `/tables` browser render confirmed. -- Restart persistence confirmed. -- DSM reverse proxy and firewall screenshots archived. - -## Workflow success evidence - -If you need the deploy-job proof from the NAS runner before the full external closeout: - -- `healthcheck` retried after startup and passed on the NAS runner. -- `snapshot-admin-web-v6` was returned by the deploy verification step. -- The workflow finished with `Job succeeded`. - -This proves the deploy job can launch, wait for readiness, and validate locally on Synology. -It does not replace the external reverse-proxy/browser closeout evidence above. - -## Do not close WBS-7.9 unless - -- The `401`/`200` curl pair is saved. -- Both browser screenshots are saved. -- The DSM reverse proxy rule screenshot is saved. -- The completion wording in `docs/SYNOLOGY_SNAPSHOT_ADMIN_DEPLOYMENT_CHECKLIST.md` is used only after evidence is archived. diff --git a/docs/SYNOLOGY_SNAPSHOT_ADMIN_FINAL_PREFLIGHT_10.md b/docs/SYNOLOGY_SNAPSHOT_ADMIN_FINAL_PREFLIGHT_10.md deleted file mode 100644 index 0f09d4f..0000000 --- a/docs/SYNOLOGY_SNAPSHOT_ADMIN_FINAL_PREFLIGHT_10.md +++ /dev/null @@ -1,29 +0,0 @@ -# Synology Snapshot Admin Final Preflight 10 - -Use this immediately before declaring `WBS-7.9` complete. - -1. Confirm the Python service is running on `127.0.0.1:8787`. -2. Confirm `bash /volume1/projects/data_feed/tools/run_snapshot_admin_synology.sh healthcheck` returns `healthcheck ok`. -3. Confirm `curl -i http://127.0.0.1:8787/api/state` returns `200 OK`. -4. Confirm `curl -i https://admin.example.com/api/state` returns `401 Unauthorized` without credentials. -5. Confirm `curl -u 'snapshot-admin:' https://admin.example.com/api/state` returns `200 OK`. -6. Confirm `https://admin.example.com/` renders in a browser after Basic Auth. -7. Confirm `https://admin.example.com/tables` renders in a browser after Basic Auth. -8. Confirm the DSM reverse proxy rule still maps `HTTPS:443 -> HTTP 127.0.0.1:8787`. -9. Confirm the firewall still blocks `8787/TCP` from WAN. -10. Restart the service or NAS and repeat steps 2 through 7. - -## Evidence to archive - -- `curl` output for steps 3 through 5 -- Browser screenshots for steps 6 and 7 -- DSM reverse proxy screenshot for step 8 -- Firewall screenshot for step 9 -- Restart proof for step 10 - -## Pass condition - -Declare `WBS-7.9` complete only when all 10 steps pass and the evidence files are saved using: - -- [`docs/SYNOLOGY_SNAPSHOT_ADMIN_EVIDENCE_TEMPLATE.md`](C:/Temp/data_feed/docs/SYNOLOGY_SNAPSHOT_ADMIN_EVIDENCE_TEMPLATE.md) -- [`docs/SYNOLOGY_SNAPSHOT_ADMIN_DEPLOYMENT_CHECKLIST_FILLED.md`](C:/Temp/data_feed/docs/SYNOLOGY_SNAPSHOT_ADMIN_DEPLOYMENT_CHECKLIST_FILLED.md) diff --git a/docs/SYNOLOGY_SNAPSHOT_ADMIN_FIREWALL_PROXY_COPYPASTE.md b/docs/SYNOLOGY_SNAPSHOT_ADMIN_FIREWALL_PROXY_COPYPASTE.md deleted file mode 100644 index 2ea5386..0000000 --- a/docs/SYNOLOGY_SNAPSHOT_ADMIN_FIREWALL_PROXY_COPYPASTE.md +++ /dev/null @@ -1,31 +0,0 @@ -# Synology Snapshot Admin Firewall and Reverse Proxy Copy-Paste - -Use these values verbatim in DSM. - -## Reverse proxy - -- Rule name: `snapshot-admin` -- Source protocol: `HTTPS` -- Source hostname: `admin.example.com` -- Source port: `443` -- Source path: `/` -- Destination protocol: `HTTP` -- Destination hostname: `127.0.0.1` -- Destination port: `8787` - -## Firewall - -- Allow: `443/TCP` from WAN or trusted CIDR -- Deny: `8787/TCP` from WAN -- Optional allow: `443/TCP` from office/VPN CIDR only - -## Certificate binding - -- Hostname: `admin.example.com` -- Bind to: reverse proxy rule `snapshot-admin` - -## Notes - -- Do not expose `8787/TCP` directly. -- Keep Basic Auth enabled in the Python service. -- Use `127.0.0.1` for the destination host unless direct-bind testing is intentional. diff --git a/docs/SYNOLOGY_SNAPSHOT_ADMIN_FIREWALL_PROXY_TABLE.md b/docs/SYNOLOGY_SNAPSHOT_ADMIN_FIREWALL_PROXY_TABLE.md deleted file mode 100644 index f7d7363..0000000 --- a/docs/SYNOLOGY_SNAPSHOT_ADMIN_FIREWALL_PROXY_TABLE.md +++ /dev/null @@ -1,38 +0,0 @@ -# Synology Snapshot Admin Firewall and Reverse Proxy Table - -Use these values for the first POC. - -## Reverse proxy rule - -| Field | Value | -|---|---| -| Rule name | `snapshot-admin` | -| Source protocol | `HTTPS` | -| Source hostname | `admin.example.com` | -| Source port | `443` | -| Source path | `/` | -| Destination protocol | `HTTP` | -| Destination hostname | `127.0.0.1` | -| Destination port | `8787` | - -## Firewall rules - -| Rule | Action | Source | Destination | Port | -|---|---|---|---|---| -| Reverse proxy public entry | Allow | WAN or trusted public CIDR | NAS | `443/TCP` | -| Raw service port | Deny | WAN | NAS | `8787/TCP` | -| Optional office/VPN allowlist | Allow | Office/VPN CIDR only | NAS | `443/TCP` | - -## Certificate - -| Field | Value | -|---|---| -| Type | TLS certificate | -| Hostname | `admin.example.com` | -| Binding | Reverse proxy rule `snapshot-admin` | - -## Notes - -- Keep `8787/TCP` private. -- Keep Basic Auth enabled in the Python service. -- Use `127.0.0.1` for the backend destination unless you are explicitly testing direct bind mode. diff --git a/docs/SYNOLOGY_SNAPSHOT_ADMIN_POC.md b/docs/SYNOLOGY_SNAPSHOT_ADMIN_POC.md deleted file mode 100644 index a0bce98..0000000 --- a/docs/SYNOLOGY_SNAPSHOT_ADMIN_POC.md +++ /dev/null @@ -1,257 +0,0 @@ -# Synology Snapshot Admin POC - -This guide enables external access to the Python snapshot admin service on Synology without exposing the raw service port to the internet. - -## Recommended topology - -1. Keep the Python service bound to loopback only: - -```bash -python tools/run_snapshot_admin_server_v1.py \ - --host 127.0.0.1 \ - --port 8787 \ - --db src/quant_engine/snapshot_admin.db \ - --seed GatherTradingData.json -``` - -2. Put Synology DSM reverse proxy in front of it: - - Source: `https://:443` - - Destination: `http://127.0.0.1:8787` - - Keep the service port closed from direct WAN access. - -3. Add browser authentication with the built-in Basic Auth gate: - - Set `SNAPSHOT_ADMIN_AUTH_USER` - - Set `SNAPSHOT_ADMIN_AUTH_PASSWORD` - - Or pass `--auth-user` and `--auth-password` on the wrapper command - -4. Verify from the NAS: - -```bash -curl -i http://127.0.0.1:8787/api/state -curl -u "$SNAPSHOT_ADMIN_AUTH_USER:$SNAPSHOT_ADMIN_AUTH_PASSWORD" http://127.0.0.1:8787/api/state -``` - -5. Verify from outside the NAS: - - Open `https:///` - - The browser should prompt for Basic Auth - - `https:///tables` should render after login - -## DSM Checklist - -Use these exact values for the first POC. - -1. **DSM app path** - - `Control Panel` - - `Login Portal` - - `Advanced` - - `Reverse Proxy` - -2. **Create reverse proxy rule** - - Description: `snapshot-admin` - - Source protocol: `HTTPS` - - Source hostname: your public DNS name, for example `admin.example.com` - - Source port: `443` - - Source path: `/` - - Destination protocol: `HTTP` - - Destination hostname: `127.0.0.1` - - Destination port: `8787` - -3. **Certificate** - - Attach a valid TLS certificate for the public hostname - - Prefer a Synology-managed or imported certificate that matches `admin.example.com` - -4. **Firewall** - - Allow inbound `443/TCP` only for the reverse proxy endpoint - - Do not expose `8787/TCP` on WAN - - If the NAS must be reachable only from a VPN or office IP range, allowlist those ranges and block the rest - -5. **Service start policy** - - Start the Python service on boot or via DSM Task Scheduler - - Keep it bound to `127.0.0.1` unless you intentionally use direct bind mode - - If you use direct bind mode, keep `--allow-remote` and Basic Auth enabled together - - For Gitea Actions runner verification, register `act_runner` with a dedicated host label (`self-hosted:host,snapshot-admin-host:host`) if you want to avoid Docker job containers and the `Cleaning up container` log line - - Preferred launcher script: `tools/run_snapshot_admin_synology.sh` - - Gitea CI deploy path: trigger `.gitea/workflows/snapshot_admin_deploy.yml` `workflow_dispatch` and let the host runner call the launcher script - - Runner bootstrap: `tools/re_register_act_runner_synology.sh` - - Runner daemon start: `tools/start_act_runner_synology.sh` - -6. **Runner re-registration** - - Use this when you want to switch an existing runner from Docker mode to host mode: - -```bash -cd /volume1/projects/data_feed -REG_TOKEN="" \ -GITEA_URL="http://192.168.123.100:8418" \ -bash tools/re_register_act_runner_synology.sh -``` - - - Expected effect: - - removes the existing `.runner` registration file - - registers `self-hosted:host,snapshot-admin-host:host` - - writes an updated `config.yaml` - - If the old runner remains listed in Gitea, remove it from the repository runner page and re-run the command above - -7. **Runner start** - - After re-registration, start the daemon: - -```bash -bash tools/start_act_runner_synology.sh -``` - - - Expected effect: - - launches `act_runner daemon` using the existing config - - records `runner.pid` and `runner.log` under the runner directory - -## DSM Task Scheduler - -Create two scheduled tasks in `Control Panel > Task Scheduler`. - -1. **Boot task** - - Task name: `snapshot-admin-start` - - User: `root` or a dedicated service account with access to the project folder - - Event: `Boot-up` - - Command: - -```bash -bash /volume1/projects/data_feed/tools/run_snapshot_admin_synology.sh start -``` - -2. **Healthcheck task** - - Task name: `snapshot-admin-healthcheck` - - User: same as boot task - - Event: `Scheduled Task` - - Repeat: every 5 minutes - - Command: - -```bash -bash /volume1/projects/data_feed/tools/run_snapshot_admin_synology.sh healthcheck -``` - -3. **Manual restart task** - - Task name: `snapshot-admin-restart` - - User: same as boot task - - Event: `Scheduled Task` - - Repeat: manual only, or keep disabled until needed - - Command: - -```bash -bash /volume1/projects/data_feed/tools/run_snapshot_admin_synology.sh restart -``` - -## Direct bind mode - -Direct binding to `0.0.0.0` is allowed only when both auth values are configured: - -```bash -python tools/run_snapshot_admin_server_v1.py \ - --host 0.0.0.0 \ - --port 8787 \ - --allow-remote \ - --auth-user "$SNAPSHOT_ADMIN_AUTH_USER" \ - --auth-password "$SNAPSHOT_ADMIN_AUTH_PASSWORD" -``` - -Use this only if you have a separate firewall or VPN rule in place. The default POC path is still loopback + reverse proxy. - -## Validation - -Run the unit/web checks before and after deployment: - -```bash -python -m pytest tests/unit/test_snapshot_admin_web_v1.py -q -python tools/validate_snapshot_admin_web_v1.py -``` - -The auth gate is part of the service now, so public exposure without credentials is rejected by the server itself. - -## Curl checklist - -Use this as the POC run sheet. - -1. Local service check: - -```bash -curl -i http://127.0.0.1:8787/api/state -``` - -Expected: -- `200 OK` -- JSON payload contains `version.app` - -2. Reverse proxy auth challenge: - -```bash -curl -i https:///api/state -``` - -Expected: -- `401 Unauthorized` -- `WWW-Authenticate: Basic` - -3. Reverse proxy authenticated access: - -```bash -curl -u ':' https:///api/state -``` - -Expected: -- `200 OK` -- JSON payload contains the same `version.app` - -4. UI rendering: - -```bash -curl -I https:/// -curl -I https:///tables -``` - -Expected: -- `200 OK` after auth -- HTML response, not a redirect to the raw port - -5. Restart persistence: - -```bash -bash tools/run_snapshot_admin_synology.sh restart -bash tools/run_snapshot_admin_synology.sh healthcheck -``` - -Expected: -- `healthcheck ok` -- The proxy URL continues to answer after the service restarts - -## Live verification - -Use this sequence on the actual Synology box after the reverse proxy rule is in place: - -1. Start the service and confirm the local health endpoint: - -```bash -curl -i http://127.0.0.1:8787/api/state -``` - -2. Confirm the auth gate: - -```bash -curl -i https:///api/state -``` - -Expected result: -- `401 Unauthorized` when no credentials are provided -- `200 OK` when valid Basic Auth credentials are supplied - -3. Confirm the browser surface: - - Open `https:///` - - Sign in with the Basic Auth credentials - - Open `https:///tables` - - Confirm rows render from the three SQLite sources - -4. Confirm the deployment survives a process restart: - - Restart the Python service or the task that launches it - - Re-run `curl -i http://127.0.0.1:8787/api/state` - - Re-open the browser URL and confirm login still works - -5. Archive evidence: - - Save the `curl` outputs - - Save a screenshot of `/` and `/tables` - - Record the DSM reverse proxy rule values and certificate name diff --git a/tools/re_register_act_runner_synology.sh b/tools/re_register_act_runner_synology.sh deleted file mode 100644 index 86a24f9..0000000 --- a/tools/re_register_act_runner_synology.sh +++ /dev/null @@ -1,58 +0,0 @@ -#!/bin/bash -set -eu - -GITEA_URL="${GITEA_URL:-http://192.168.123.100:8418}" -REG_TOKEN="${REG_TOKEN:-}" -RUNNER_NAME="${RUNNER_NAME:-synology-runner}" -RUNNER_LABEL="${RUNNER_LABEL:-snapshot-admin-host}" -RUNNER_DIR="${RUNNER_DIR:-/volume1/gitea/act_runner}" -ACT_RUNNER_VERSION="${ACT_RUNNER_VERSION:-0.2.11}" - -if [ -z "$REG_TOKEN" ]; then - echo "ERROR: REG_TOKEN is required" - echo "usage: REG_TOKEN=... GITEA_URL=... bash tools/re_register_act_runner_synology.sh" - exit 1 -fi - -ARCH=$(uname -m) -case "$ARCH" in - x86_64) BINARY="act_runner-${ACT_RUNNER_VERSION}-linux-amd64" ;; - aarch64) BINARY="act_runner-${ACT_RUNNER_VERSION}-linux-arm64" ;; - armv7l) BINARY="act_runner-${ACT_RUNNER_VERSION}-linux-arm-7" ;; - *) echo "ERROR: 지원하지 않는 아키텍처: $ARCH"; exit 1 ;; -esac - -mkdir -p "$RUNNER_DIR/workspace" - -if [ ! -x "$RUNNER_DIR/act_runner" ]; then - DOWNLOAD_URL="https://gitea.com/gitea/act_runner/releases/download/v${ACT_RUNNER_VERSION}/${BINARY}" - echo "[install] $DOWNLOAD_URL" - curl -L --progress-bar "$DOWNLOAD_URL" -o "$RUNNER_DIR/act_runner" - chmod +x "$RUNNER_DIR/act_runner" -fi - -cat > "$RUNNER_DIR/config.yaml" </dev/null; then - echo "snapshot admin already running pid=$(cat "$PID_FILE")" - return 0 - fi - cmd=("$PYTHON_BIN" "${ROOT_DIR}/tools/run_snapshot_admin_server_v1.py") - if [ "$ALLOW_REMOTE" = "1" ]; then - cmd+=("--host" "0.0.0.0" "--allow-remote") - else - cmd+=("--host" "$HOST") - fi - cmd+=("--port" "$PORT" "--db" "$DB_PATH" "--seed" "$SEED_PATH") - if [ -n "$AUTH_USER" ]; then - cmd+=("--auth-user" "$AUTH_USER") - fi - if [ -n "$AUTH_PASSWORD" ]; then - cmd+=("--auth-password" "$AUTH_PASSWORD") - fi - nohup "${cmd[@]}" >> "$LOG_FILE" 2>&1 & - echo $! > "$PID_FILE" - echo "started snapshot admin pid=$!" -} - -stop_server() { - if [ ! -f "$PID_FILE" ] || ! kill -0 "$(cat "$PID_FILE")" 2>/dev/null; then - echo "snapshot admin not running" - rm -f "$PID_FILE" - return 0 - fi - kill "$(cat "$PID_FILE")" - sleep 2 - if kill -0 "$(cat "$PID_FILE")" 2>/dev/null; then - kill -9 "$(cat "$PID_FILE")" - fi - rm -f "$PID_FILE" - echo "stopped snapshot admin" -} - -healthcheck() { - if [ ! -f "$PID_FILE" ] || ! kill -0 "$(cat "$PID_FILE")" 2>/dev/null; then - echo "healthcheck failed: process not running" - return 1 - fi - if [ -n "$AUTH_USER" ] && [ -n "$AUTH_PASSWORD" ]; then - if curl -fsS -u "${AUTH_USER}:${AUTH_PASSWORD}" "$STATE_URL" >/dev/null 2>&1; then - echo "healthcheck ok: $STATE_URL" - return 0 - fi - else - if curl -fsS "$STATE_URL" >/dev/null 2>&1; then - echo "healthcheck ok: $STATE_URL" - return 0 - fi - fi - echo "healthcheck failed: $STATE_URL" - return 1 -} - -public_check() { - echo "curl -i ${PUBLIC_STATE_URL}" - echo "curl -u ':' ${PUBLIC_STATE_URL}" - echo "curl -i ${PUBLIC_STATE_URL%/api/state}/tables" -} - -case "${1:-start}" in - start) - start_server - ;; - stop) - stop_server - ;; - restart) - stop_server - start_server - ;; - status) - if [ -f "$PID_FILE" ] && kill -0 "$(cat "$PID_FILE")" 2>/dev/null; then - echo "running pid=$(cat "$PID_FILE")" - exit 0 - fi - echo "stopped" - exit 1 - ;; - healthcheck) - healthcheck - ;; - public-check) - public_check - ;; - *) - echo "usage: $0 {start|stop|restart|status|healthcheck|public-check}" - exit 2 - ;; -esac diff --git a/tools/start_act_runner_synology.sh b/tools/start_act_runner_synology.sh deleted file mode 100644 index c070c86..0000000 --- a/tools/start_act_runner_synology.sh +++ /dev/null @@ -1,26 +0,0 @@ -#!/bin/bash -set -eu - -RUNNER_DIR="${RUNNER_DIR:-/volume1/gitea/act_runner}" -PID_FILE="${PID_FILE:-$RUNNER_DIR/runner.pid}" -LOG_FILE="${LOG_FILE:-$RUNNER_DIR/runner.log}" - -if [ ! -x "$RUNNER_DIR/act_runner" ]; then - echo "ERROR: act_runner binary missing at $RUNNER_DIR/act_runner" - exit 1 -fi - -if [ ! -f "$RUNNER_DIR/config.yaml" ]; then - echo "ERROR: config missing at $RUNNER_DIR/config.yaml" - echo "Run tools/re_register_act_runner_synology.sh first." - exit 1 -fi - -if [ -f "$PID_FILE" ] && kill -0 "$(cat "$PID_FILE")" 2>/dev/null; then - echo "already running pid=$(cat "$PID_FILE")" - exit 0 -fi - -nohup "$RUNNER_DIR/act_runner" daemon --config "$RUNNER_DIR/config.yaml" >> "$LOG_FILE" 2>&1 & -echo $! > "$PID_FILE" -echo "started pid=$!"