gabriel / musehub public
docs BREAKING docs/database-architecture-canonical #1 / 1
AI Agent gabriel · 7 hours ago · Sep 8, 2026 · Diff

docs: canonical database architecture doc, major discovery on staging (#154)

While verifying Section 5 (Production Architecture), a routine staging deploy failed at the migration step. Investigation found staging has been running on a real AWS RDS instance (musehub-staging-db, created 2026-04-05, the same day as the original IAM users) this entire time - not self-hosted Postgres as every doc in this project claimed. The Docker Postgres container everyone assumed was staging's database has been stopped for 2 months with real accumulated data, diverged from RDS.

Investigated the divergence directly rather than assuming: RDS has 28 repos/8 identities/11555 commits, the stopped container has 35 repos/2 identities/2116 commits. 33 of the container's 35 repos are disposable benchmark artifacts from a single 2026-05-13 test run; the only 3 real repos already exist in RDS with their own newer history. Nothing unique is trapped in the stopped container.

Root cause: deploy/secrets.sh only ever knew about DB_PASSWORD, never DATABASE_URL - when it regenerated staging's .env for #156, it silently dropped the RDS connection string. Only didn't break immediately because the already-running container doesn't re-read .env after startup.

Created docs/database-architecture.md as the canonical, secret-free source of truth for DB architecture across all three environments, with a 4-phase plan to standardize on RDS for both staging and production (decided with Gabriel) rather than the self-hosted pattern production currently uses.

Also includes an earlier, unrelated fix: deploy.sh used 'docker rm -f' to stop the old blue/green slot, which sends SIGKILL immediately and never gave the app's lifespan shutdown handler (closes the DB pool, stops the Playwright browser) a chance to run. Changed to 'docker stop --time 15' first so in-flight requests and graceful shutdown actually happen.

sha256:9e73bdc9c63fc86b74f90db1ae47676596d7fcd6e006ace35a1f228dfabfc77d sha
sha256:23efc08a3fcec5132abb7a4827626dd99f59bf69f64eeeaefad3c1d72a08fe36 parent
+11 symbols
2 changed · 1061 in snapshot files
sha256:d27143791a1a9f1a46d4cc25a7f0d97dadb14749ba6972a42f4f515afe9a45c7 snapshot
+11
symbols added
2
files changed
1061
files in snapshot
0
dead code introduced
Semantic Changes 11 symbols
+ .vscode/
+ alembic/
+ alembic/versions/
+ deploy/
+ deploy/load-tests/
+ docs/
+ docs/architecture/
+ docs/guides/
+ docs/issues/
+ docs/issues/assets/
+ docs/legal/
+ docs/mists/
+ docs/production-readiness/
+ docs/reference/
+ docs/releases/
+ docs/roadmaps/
+ musehub/
+ musehub/api/
+ musehub/api/routes/
+ musehub/api/routes/api/
+ musehub/api/routes/musehub/
+ musehub/auth/
+ musehub/core/
+ musehub/crypto/
+ musehub/db/
+ musehub/debug/
+ musehub/graph/
+ musehub/maintenance/
+ musehub/mcp/
+ musehub/mcp/tools/
+ musehub/mcp/write_tools/
+ musehub/middleware/
+ musehub/models/
+ musehub/muse_cli/
+ musehub/proposals/
+ musehub/protocol/
+ musehub/security/
+ musehub/services/
+ musehub/storage/
+ musehub/templates/
+ musehub/templates/mcp/
+ musehub/templates/musehub/
+ musehub/templates/musehub/fragments/
+ musehub/templates/musehub/macros/
+ musehub/templates/musehub/opengraph/
+ musehub/templates/musehub/pages/
+ musehub/templates/musehub/partials/
+ musehub/templates/musehub/static/
+ musehub/templates/musehub/static/vendor/
+ musehub/types/
+ scripts/
+ src/
+ src/scss/
+ src/scss/components/
+ src/scss/pages/
+ src/scss/patterns/
+ src/scss/theme/
+ src/ts/
+ src/ts/pages/
+ tests/
+ tests/unit/
+ tools/
+ workflows/
~ docs/database-architecture.md .md 11 symbols added
+ MuseHub Database Architecture — Canonical Source of Truth section MuseHub Database Architecture — Canonical Source of Truth L1–143
+ Current state (as of 2026-09-08) — verified live, not assumed section Current state (as of 2026-09-08) — verified live, not assumed L10–24
+ table section table L12–17
+ How this was discovered, and what it means (2026-09-08 investigation) section How this was discovered, and what it means (2026-09-08 investigation) L24–64
+ Migration plan — phased, no step skipped section Migration plan — phased, no step skipped L82–143
+ Phase 0 — Immediate housekeeping (safe, do first, low risk) section Phase 0 — Immediate housekeeping (safe, do first, low risk) L84–94
+ Phase 1 — Formally correct staging's secrets pipeline (small, low risk) section Phase 1 — Formally correct staging's secrets pipeline (small, low risk) L94–109
+ Phase 2 — Migrate production from self-hosted Postgres to RDS section Phase 2 — Migrate production from self-hosted Postgres to RDS L109–122
+ Phase 3 — Retire the manual backup pipeline section Phase 3 — Retire the manual backup pipeline L122–132
+ Phase 4 — Documentation and ticket reconciliation section Phase 4 — Documentation and ticket reconciliation L132–143
+ Target architecture (decided 2026-09-08) section Target architecture (decided 2026-09-08) L64–82
~ deploy/deploy.sh .sh
Files Changed
+1 ~1
1061 in snapshot
+ docs/database-architecture.md .md
~ deploy/deploy.sh .sh
← Older Oldest on docs/database-architecture-canonical
All commits
Newer → Latest on docs/database-architecture-canonical

0 comments

No comments yet. Be the first to start the discussion.

To add a comment, use the Muse CLI: muse hub commit comment sha256:9e73bdc9c63fc86b74f90db1ae47676596d7fcd6e006ace35a1f228dfabfc77d --body "your comment"