gabriel / musehub public
09-ci-pipeline.md markdown
75 lines 4.9 KB
Raw
sha256:75d2964a7ec35f461373e3c83b208df829526661ea6a17060f95df5d50d21fb0 docs+infra: re-verify security monitoring (#161), enable Se… Sonnet 5 patch 3 days ago

Section 9 — CI Pipeline

Companion to musehub-production-readiness-checklist.md. "GitHub Actions" / "pull requests" / "CODEOWNERS" translate to MuseHub-native equivalents per the Decision Log — but this section's investigation found those equivalents mostly don't exist as product features yet, not just "unconfigured." That's a more significant gap than a typical checklist item.

What actually exists today

  • A real, well-built internal background job queue (musehub/services/musehub_jobs.py): claim_next_job uses SELECT ... FOR UPDATE SKIP LOCKED for safe concurrent claiming across workers, reclaim_stale_jobs resets jobs stuck in running after a crash, and there's a dependency barrier so fetch.mpack.prebuild won't claim ahead of a same-repo mpack.index. This is genuinely good engineering — it directly answers Section 5's earlier "are background jobs idempotent/safely retryable" question (yes, confirmed).
  • But the only job types that exist are internal: mpack.index and fetch.mpack.prebuild — post-push indexing/prebuild work, not user-defined CI (tests, lint, type-check).
  • A musehub-runner container is provisioned (docker:24-dind, privileged, with RUNNER_TOKEN/MUSEHUB_URL env vars) — infrastructure for running isolated CI jobs exists at the container level, but I could not find the client code that would poll MuseHub for jobs and execute them (searched for RUNNER_TOKEN/"runner" references in musehub/ — none found outside the compose file and docs/infrastructure.md). This looks like scaffolding for a CI runner that isn't wired to a real job-execution client yet.

What doesn't exist yet (product gaps, not just configuration gaps)

  • No branch protection. No code enforcing protected branches, required reviews, or blocking direct pushes to main/dev.
  • No required status checks. Nothing in musehub_proposals.py's merge_proposal gates on CI status — a proposal can merge regardless of whether any checks passed.
  • No CODEOWNERS-equivalent. No per-path required-reviewer concept found anywhere.
  • No user-defined pipeline execution (formatting, linting, type-checking, unit/integration tests, migration checks, dependency/secret/container/IaC scanning) triggered by push or proposal.

This means Section 9 isn't "reconfigure GitHub Actions as MuseHub CI" — it's "these MuseHub platform features don't exist yet." That's a materially bigger scope than the checklist implies, and worth Gabriel knowing explicitly: closing this section means building product features in musehub/, not just writing deployment config.

Checklist assessment

Given the above, nearly every item in this section is genuinely not started, and it's more honest to say so plainly than to mark partial credit:

  • [ ] Require proposals for protected branches — not started (no branch protection exists)
  • [ ] Protect main/dev — not started
  • [ ] Require CI checks before merging — not started (no CI-status gating in merge_proposal)
  • [ ] Require review from another maintainer for sensitive changes — not started
  • [ ] Define required-reviewer paths (CODEOWNERS-equivalent) — not started
  • [ ] Run formatting/linting/type-checking/unit/integration/migration-check/dependency-scan/ secret-scan/container-scan/IaC-scan — none of these run automatically anywhere today; they exist as manual developer commands (pytest, etc.) but nothing triggers them on push
  • [ ] Immutable versioned build artifacts with commit SHA — partially: Docker images are tagged and pushed to ECR (musehub/musehub:<tag>), which is a form of immutable artifact, but there's no automated build-on-push — images are built manually via push.sh on a laptop
  • [ ] Produce artifact once, promote through environments — not done, IMAGE_TAG redeploy support exists in push.sh (so promoting a specific tag is possible), but the default flow rebuilds for each environment rather than promoting the staging-tested image
  • [ ] SBOM generation — not started
  • [ ] Dependency update automation — not started
  • [ ] Urgent-patch workflow — not documented
  • [ ] CI/deployment log retention — not applicable, no CI exists to log
  • [ ] Prevent force-push/branch deletion on protected branches — not started

Recommendation for the later build phase

Given how much is missing, prioritize in this order:

  1. Required-status-check gating on merge_proposal — the highest-leverage single feature, since everything else in Sections 9–11 assumes this exists.
  2. Branch protection (block direct push to main/dev, require proposals).
  3. Wire the actual musehub-runner job-execution client — decide whether it runs a fixed pipeline (lint/test/build) or user-defined config (more general, more work).
  4. CODEOWNERS-equivalent, once 1–3 exist and there's a real merge gate to attach required reviewers to.
File History 1 commit
sha256:75d2964a7ec35f461373e3c83b208df829526661ea6a17060f95df5d50d21fb0 docs+infra: re-verify security monitoring (#161), enable Se… Sonnet 5 patch 3 days ago