kjh2064/QuantEngineByItz
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 9 Wiki Activity

fix: Run HTTP/CSS health checks over SSH against 127.0.0.1, not the public IP

Browse Source 
Root cause confirmed by direct test:
  curl --connect-timeout 5 http://178.104.200.7:5000/Account/Login -> 000

quantengine.service sets ASPNETCORE_URLS=http://127.0.0.1:5000 (loopback
only, by design -- Nginx is the only public entry point, proxying
quant.taxbaik.com to it). The Gitea Actions runner is not the
production host, so its direct curl to $DEPLOY_HOST:5000 was always
going to hit a closed port. Run #2005 is direct proof: "Deploy to
Production" succeeded, the site was reachable over HTTPS the whole
time, and journalctl was clean -- yet "Health Check & Verification"
burned through all 20 retries (60s) because it was polling the wrong
address entirely. This check has likely never once passed on this
service's actual network layout.

Fix: wrap the HTTP-200 / login-content / CSS retry loop in a single
SSH session that runs curl against 127.0.0.1:5000 on the production
server itself -- consistent with how the service-status and DB-error
checks already correctly run remotely. Removed the redundant
per-attempt SSH round trips for service status (now a plain local
command inside the same remote script) and dropped the separate
"Setup SSH (for service check)" step's curl usage entirely.
This commit is contained in:
김재현
2026-07-12 01:10:51 +09:00
parent d6b224dbb4
commit 3c3f2d56c8
1 changed files with 32 additions and 27 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.gitea/workflows/deploy-prod.yml
+32 -27
View File
@@ -254,48 +254,56 @@ jobs:
- name: Health Check
run: |
# IMPORTANT: quantengine.service binds ASPNETCORE_URLS to
# http://127.0.0.1:5000 (loopback only) -- Nginx is the only
# thing that reaches it from outside, via quant.taxbaik.com.
# The Gitea Actions runner is a separate host/container, so
# `curl http://$DEPLOY_HOST:5000/...` from here always hits a
# closed port and times out ("000") -- confirmed directly:
# curl --connect-timeout 5 http://178.104.200.7:5000/... -> 000
# Every previous run's Health Check silently burned through all
# 20 retries on this before failing, even on deployments that
# actually worked (see Run #2005: Deploy job succeeded, site was
# reachable over HTTPS and journalctl was clean the whole time).
# Fix: run the HTTP/CSS checks *on* the server against
# 127.0.0.1:5000, the same way the service-status and DB-error
# checks already correctly do via SSH.
ssh -i ~/.ssh/deploy_key \
-p ${{ env.DEPLOY_PORT }} \
-o StrictHostKeyChecking=accept-new \
${{ env.DEPLOY_USER }}@${{ env.DEPLOY_HOST }} bash -s << 'REMOTE'
set -e
ATTEMPTS=20
DEPLOY_HOST="${{ env.DEPLOY_HOST }}"
echo "【 Health Checks (max ${ATTEMPTS} attempts) 】"
for i in $(seq 1 $ATTEMPTS); do
# Check 1: HTTP 200
HTTP_CODE=$(curl -s -o /dev/null -w "%{http_code}" http://$DEPLOY_HOST:5000/Account/Login 2>/dev/null || echo "000")
HTTP_CODE=$(curl -s -o /dev/null -w "%{http_code}" http://127.0.0.1:5000/Account/Login 2>/dev/null || echo "000")
if [ "$HTTP_CODE" = "200" ]; then
echo "✓ [1/5] HTTP 200 OK (attempt $i)"
echo "✓ [1/6] HTTP 200 OK (attempt $i)"
# Check 2: Login Page
LOGIN_BODY=$(curl -s http://$DEPLOY_HOST:5000/Account/Login 2>/dev/null || echo "")
LOGIN_BODY=$(curl -s http://127.0.0.1:5000/Account/Login 2>/dev/null || echo "")
if echo "$LOGIN_BODY" | grep -q "login\|Login\|로그인"; then
echo "✓ [2/5] Login page content verified"
echo "✓ [2/6] Login page content verified"
else
echo "⚠ [2/5] Login page content verification skipped"
echo "⚠ [2/6] Login page content verification skipped"
fi
# Check 3: CSS loaded
CSS_CODE=$(curl -s -o /dev/null -w "%{http_code}" http://$DEPLOY_HOST:5000/css/admin.css 2>/dev/null || echo "000")
CSS_CODE=$(curl -s -o /dev/null -w "%{http_code}" http://127.0.0.1:5000/css/admin.css 2>/dev/null || echo "000")
if [ "$CSS_CODE" = "200" ]; then
echo "✓ [3/5] CSS file loaded"
echo "✓ [3/6] CSS file loaded"
else
echo "⚠ [3/5] CSS file check skipped (status: $CSS_CODE)"
echo "⚠ [3/6] CSS file check skipped (status: $CSS_CODE)"
fi
# Check 4: Service active
SERVICE_STATUS=$(ssh -i ~/.ssh/deploy_key \
-p 22 \
-o StrictHostKeyChecking=accept-new \
kjh2064@$DEPLOY_HOST \
"systemctl is-active quantengine" 2>/dev/null || echo "unknown")
SERVICE_STATUS=$(systemctl is-active quantengine 2>/dev/null || echo "unknown")
if [ "$SERVICE_STATUS" = "active" ]; then
echo "✓ [4/5] Service active (running)"
echo "✓ [4/6] Service active (running)"
else
echo "⚠ [4/5] Service status: $SERVICE_STATUS"
echo "⚠ [4/6] Service status: $SERVICE_STATUS"
fi
# Check 5: Release verified
echo "✓ [5/5] Deployment release: ${{ needs.deploy.outputs.release-tag }} (commit: ${{ needs.deploy.outputs.commit-hash }})"
echo "✓ [5/6] Deployment release: ${{ needs.deploy.outputs.release-tag }} (commit: ${{ needs.deploy.outputs.commit-hash }})"
# Check 6: DB connectivity (GET /Account/Login returns 200 even when
# the DB password is stale -- the page itself has no DB dependency.
@@ -304,11 +312,7 @@ jobs:
# 2026-07-12: this check would have caught it, the HTTP check alone
# did not.)
sleep 2
DB_ERRORS=$(ssh -i ~/.ssh/deploy_key \
-p 22 \
-o StrictHostKeyChecking=accept-new \
kjh2064@$DEPLOY_HOST \
"journalctl -u quantengine --since '1 minute ago' --no-pager 2>/dev/null | grep -c '28P01\|password authentication failed'" || echo "0")
DB_ERRORS=$(journalctl -u quantengine --since '1 minute ago' --no-pager 2>/dev/null | grep -c '28P01\|password authentication failed' || echo "0")
if [ "$DB_ERRORS" = "0" ]; then
echo "✓ [6/6] No DB authentication errors in recent logs"
else
@@ -332,6 +336,7 @@ jobs:
exit 1
fi
done
REMOTE
post-deploy-report:
name: Deployment Report