QuantEngineByItz/docs/archive/DATABASE_CONSOLIDATION_PLAN_2026_06_23.md

Database Consolidation Plan (2026-06-23)

Archive candidate: this document records consolidation history and must not be treated as an operational source of truth.

Current State: FRAGMENTED

• Canonical: src/quant_engine/ (2 files)
• Scattered: outputs/ (10) + Temp/ (3)
• Total: 15 database files

Issue

1. kis_data_collection.db in 3 locations:

   • src/quant_engine/ (CANONICAL)
   • legacy/archive locations
   • Temp/test_kis_data_collection.db

2. snapshot_admin.db in 4+ locations:

   • src/quant_engine/ (CANONICAL)
   • legacy/archive locations
   • Temp/snapshot_admin_*.db (multiple variants)
   • unrelated DBs in other subtrees

Solution

Step 1: Verify Canonical Copies (src/quant_engine/)

• kis_data_collection.db: 5 records [OK]
• snapshot_admin.db: 0 records (initialized) [OK]

Step 2: Archive Scattered Files (archive_db/)

Create archive directory with timestamp:

archive_db/
├── 2026-06-23_outputs_kis_data_collection/
├── 2026-06-23_outputs_snapshot_admin/
├── 2026-06-23_temp_test_files/
└── manifest.json (record what was archived)

Step 3: Clean Obsolete References

• Remove imports from legacy non-canonical DB paths
• Remove imports from archive/backup DB paths
• Update any code expecting these paths

Step 4: Update Documentation

• Update all references to use: src/quant_engine/
• Update deployment docs (Synology)
• Update CI/CD workflows

Benefits

• Single source of truth
• Easier backup/recovery
• Clear separation: live vs. archived
• Faster data access
• Simplified deployment

Files to Delete (After Archiving)

• obsolete duplicate DBs outside canonical src/quant_engine/
• transient Temp/ validation DBs after use