publish_muse_release.sh failed against production: missing AWS profile + IAM-denied S3 GetObject

Summary

Discovered live while publishing muse 0.2.1rc4 to production. deploy/publish_muse_release.sh failed on two separate, sequential bugs when run against production (it had presumably never been run against production before, or account guardrails changed since).

Bug 1 — no AWS profile for production's SSM calls

Every aws ssm send-command/aws ssm get-command-invocation call had no --profile, so it silently ran under the ambient default credentials (musehub-infra, the Nonproduction account) regardless of which environment was requested. Production's instance lives in a separate AWS account with no static IAM user by design. Symptom:

An error occurred (InvalidInstanceId) when calling the SendCommand operation: Instances not in a valid state for account

— which reads like a broken instance, not a missing --profile.

Fix: resolve PROFILE_ARGS per environment (empty for staging, --profile musehub-production for production) and apply it to every SSM call. S3 calls deliberately keep using the ambient default credentials in both cases — the release bucket is a single shared, public-read bucket, not per-environment.

Bug 2 — production instance role denied s3:GetObject on the (public) release bucket

Once bug 1 was fixed and the SSM call reached the right account, the remote command's own aws s3 cp (authenticated via the production instance's IAM role) got:

fatal error: An error occurred (403) when calling the HeadObject operation: Forbidden

Diagnosed directly on the instance: an anonymous curl and aws s3 cp --no-sign-request both got HTTP 200, while the signed request using the instance's own role got 403. The bucket's policy grants s3:GetObject to Principal: "*" and public access block is fully disabled — genuinely public. An explicit IAM deny (an account-level guardrail on production) overrides that public Allow for authenticated principals; anonymous requests never touch IAM at all.

Fix: added --no-sign-request to the remote instance-side download, matching how install.sh itself fetches releases over plain HTTPS with no credentials — there was never a reason to authenticate for this GET since the bucket is deliberately public-read.

Verification

Reran the actual publish against production after each fix — 1st attempt failed with the account error, 2nd with the 403, 3rd (both fixes applied) succeeded end to end including all 20 smoke_muse.sh checks against https://musehub.ai. muse 0.2.1rc4 is now live on both staging and production.

Tests

tests/test_deployment.py::TestPublishMuseReleaseProfile — 5 tests, executing the real case-statement/PROFILE_ARGS logic in a subprocess and statically guarding every SSM call site and the --no-sign-request flag. Full tests/test_deployment.py: 42 passed.

Source

Found by Claude Sonnet 5 while publishing muse 0.2.1rc4 (the midi-domain opt-in gate) to production, 2026-09-11.