Production readiness — Section 17: Testing Before Launch

0 Anchors

— Blast radius

— Churn 30d

0 Proposals

Verified the automated test suite exists and collects cleanly: 7,028 tests, zero collection errors. Actually running the full suite for pass/fail status is still open. Several items (backup/restore testing, canonical-redirect testing) cannot pass until their prerequisite sections (8, 6) are built.

Also confirmed failing today: "verify neither maintainer is a single point of operational failure" — currently fails due to the shared deploy credential and Aaron's missing AWS identity (Section 1).

◎ Activity1

●

gabriel opened this issue 68 days ago

○

gabriel 37 days ago

Ran the actual test suite for the first time (previously only confirmed it collects): 7,028 tests, 7,003 passed / 15 failed / 24 skipped. Root-caused and fixed 9 of 15 failures — a real bug in a test helper (_attach_repo_to_domain set the wrong DB column, domain_id instead of marketplace_domain_id — application code was correct, only the test fixture was wrong) plus a clean regression (Dockerfile's PLAYWRIGHT_BROWSERS_PATH never added to the approved safe-env-var test list). Remaining 5 failures are a test-environment gap (missing blob storage config), not app bugs.

Also confirmed live, externally:

Canonical redirects (www→apex, http→https) — both 301, verified via curl
TLS from an external network — valid cert, correct CN, via openssl
Maintainer-SPOF test now passes — was failing per the original doc; resolved via #150 (Aaron has independent SSO identity, deploys staging/production under his own identity)
Rollback already tested for real (#159)

Remaining real gaps (integration/load/soak testing, tabletop exercise, break-glass test, the OpenGraph test fix) split to #179.

Closing.

✓

closed this issue 37 days ago

Assignee

gabriel human

Release

no commits linked to this issue

create

muse hub issue create \
  --title "..." \
  --body "..." \
  --label bug \
  --anchor path/to/file.py::Symbol \
  --commit-anchor <sha> \
  --repo gabriel/musehub

read

muse hub issue get 165 --json
muse hub issue list --state open --json

update

muse hub issue edit 165 \
  --anchor path/to/file.py::Symbol \
  --repo gabriel/musehub

comment

muse hub issue comment 165 \
  --body "Fixed in <sha>" \
  --repo gabriel/musehub

reopen

muse hub issue reopen 165 \
  --repo gabriel/musehub

create

create_issue({
  repo_id: "sha256:a265796360c3b1b8700b5682ced5f6b044a2c0d3a2c58918892a5aa494db6c75",
  title: "...",
  body: "...",
  labels: ["bug"],
  symbol_anchors: [
    "path/to/file.py::Symbol"
  ],
  commit_anchors: ["<sha>"]
})

read

get_issue({
  repo_id: "sha256:a265796360c3b1b8700b5682ced5f6b044a2c0d3a2c58918892a5aa494db6c75",
  issue_number: 165
})

list_issues({
  repo_id: "sha256:a265796360c3b1b8700b5682ced5f6b044a2c0d3a2c58918892a5aa494db6c75",
  state: "open"
})

update

edit_issue({
  repo_id: "sha256:a265796360c3b1b8700b5682ced5f6b044a2c0d3a2c58918892a5aa494db6c75",
  issue_number: 165,
  symbol_anchors: ["path/to/file.py::Symbol"]
})

comment

create_issue_comment({
  repo_id: "sha256:a265796360c3b1b8700b5682ced5f6b044a2c0d3a2c58918892a5aa494db6c75",
  issue_number: 165,
  body: "..."
})

reopen

reopen_issue({
  repo_id: "sha256:a265796360c3b1b8700b5682ced5f6b044a2c0d3a2c58918892a5aa494db6c75",
  issue_number: 165
})

create

curl -X POST \
  http://localhost:10003/api/repos/sha256:a265796360c3b1b8700b5682ced5f6b044a2c0d3a2c58918892a5aa494db6c75/issues \
  -H "Content-Type: application/json" \
  -H "Authorization: MSign handle=\"...\" ts=... sig=\"...\"" \
  -d '{
    "title": "...",
    "body": "...",
    "labels": ["bug"],
    "symbol_anchors": ["path/to/file.py::Symbol"]
  }'

read

# get one issue
curl http://localhost:10003/api/repos/sha256:a265796360c3b1b8700b5682ced5f6b044a2c0d3a2c58918892a5aa494db6c75/issues/165

# list open issues
curl "http://localhost:10003/api/repos/sha256:a265796360c3b1b8700b5682ced5f6b044a2c0d3a2c58918892a5aa494db6c75/issues?state=open"

update

curl -X PATCH \
  http://localhost:10003/api/repos/sha256:a265796360c3b1b8700b5682ced5f6b044a2c0d3a2c58918892a5aa494db6c75/issues/165 \
  -H "Content-Type: application/json" \
  -H "Authorization: MSign handle=\"...\" ts=... sig=\"...\"" \
  -d '{"title": "...", "body": "..."}'

comment

curl -X POST \
  http://localhost:10003/api/repos/sha256:a265796360c3b1b8700b5682ced5f6b044a2c0d3a2c58918892a5aa494db6c75/issues/165/comments \
  -H "Content-Type: application/json" \
  -H "Authorization: MSign handle=\"...\" ts=... sig=\"...\"" \
  -d '{"body": "Fixed in <sha>"}'

reopen

curl -X POST \
  http://localhost:10003/api/repos/sha256:a265796360c3b1b8700b5682ced5f6b044a2c0d3a2c58918892a5aa494db6c75/issues/165/reopen \
  -H "Authorization: MSign handle=\"...\" ts=... sig=\"...\""