MuseHub Production-Readiness Checklist

Master checklist for promoting MuseHub from staging to production, automating deployments, sharing operational ownership between Gabriel and Aaron, and applying security, reliability, backup, documentation, and infrastructure best practices.

0. Decisions and Inventory

• [ ] Document the current staging architecture.
  • [ ] AWS account and region
  • [ ] Compute platform: ECS, EC2, App Runner, Elastic Beanstalk, Lambda, EKS, etc.
  • [ ] Load balancer, API Gateway, CloudFront, or other ingress
  • [ ] Database engine and topology
  • [ ] Redis, queues, object storage, search, background workers
  • [ ] VPC, subnets, route tables, NAT gateways, and security groups
  • [ ] Secrets and configuration sources
  • [ ] Logging and monitoring
  • [ ] Current deployment process
• [ ] Inventory every manually created AWS resource.
• [ ] Inventory every external dependency.
  • [ ] GitHub
  • [ ] Namecheap
  • [ ] Email provider
  • [ ] OAuth providers
  • [ ] Payment or billing providers
  • [ ] Error tracking
  • [ ] Third-party APIs
• [ ] Identify all stateful and mission-critical data.
• [ ] Define expected initial traffic and reasonable growth assumptions.
• [ ] Define availability expectations.
• [ ] Define acceptable downtime during deployment.
• [ ] Define recovery objectives.
  • [ ] RPO: maximum acceptable data loss
  • [ ] RTO: maximum acceptable restoration time
• [ ] Decide the canonical public URLs.
  • [ ] musehub.com
  • [ ] www.musehub.com
  • [ ] API hostname, such as api.musehub.com
  • [ ] Staging hostname, such as staging.musehub.com
• [ ] Choose either the root domain or www as canonical.
• [ ] Redirect the noncanonical hostname permanently to the canonical one.
• [ ] Decide whether DNS remains hosted at Namecheap or moves to Route 53.
• [ ] Decide whether production and non-production will use separate AWS accounts.

---

1. Ownership and Shared Access

No shared human accounts, shared passwords, or shared AWS access keys.

• [ ] Confirm the domain is held by the company rather than an individual wherever practical.
• [ ] Confirm AWS account ownership and billing contacts.
• [ ] Confirm GitHub organization ownership.
• [ ] Add both Gabriel and Aaron as appropriate organizational owners or administrators.
• [ ] Ensure neither person is the sole recovery path for:
  • [ ] Namecheap
  • [ ] AWS
  • [ ] GitHub
  • [ ] DNS
  • [ ] Database
  • [ ] Production secrets
  • [ ] Billing
  • [ ] Backups
• [ ] Use company-controlled email addresses for infrastructure accounts.
• [ ] Configure at least two trusted recovery paths.
• [ ] Store emergency recovery documentation in a location accessible to both founders.
• [ ] Create a documented joiner–mover–leaver access procedure.
• [ ] Create a quarterly access-review process.
• [ ] Record ownership for every operational area:
  • [ ] AWS administration
  • [ ] DNS and certificates
  • [ ] Deployments
  • [ ] Database administration
  • [ ] Incident response
  • [ ] Billing
  • [ ] Backups
• [ ] Create an emergency break-glass procedure.
• [ ] Test that Aaron can independently:
  • [ ] Sign in to AWS
  • [ ] Inspect production
  • [ ] Deploy staging
  • [ ] Approve or deploy production
  • [ ] Inspect logs and alarms
  • [ ] Restore from backup
  • [ ] Update DNS when authorized
• [ ] Test that Gabriel can perform the same tasks without relying on undocumented knowledge.

---

2. AWS Account Structure

Recommended Target

• [ ] Create or verify an AWS Organization.
• [ ] Use distinct AWS accounts for:
  • [ ] Management
  • [ ] Production
  • [ ] Non-production
• [ ] Consider separate security and log-archive accounts as the operation grows.
• [ ] Place production and non-production into separate organizational units.
• [ ] Apply Service Control Policies only after testing them carefully.
• [ ] Centralize billing.
• [ ] Enable account-level budget alerts.
• [ ] Set alternate security, operations, and billing contacts on every account.
• [ ] Restrict enabled AWS regions to the regions MuseHub actually uses, where feasible.
• [ ] Document any temporary decision to keep staging and production in one AWS account as accepted technical debt.

---

3. Identity, Authentication, and Permissions

Root and Recovery Access

• [ ] Protect every AWS root account with phishing-resistant MFA where possible.
• [ ] Do not create root access keys.
• [ ] Do not use root for routine administration.
• [ ] Secure the root email account with MFA.
• [ ] Store root recovery information securely.
• [ ] Document the legitimate uses of root.
• [ ] Configure multi-person approval for emergency root use if practical.

Human Access

• [ ] Configure AWS IAM Identity Center.
• [ ] Create separate identities for Gabriel and Aaron.
• [ ] Require MFA for both.
• [ ] Prefer hardware security keys or passkeys.
• [ ] Define permission sets, for example:
  • [ ] Administrator
  • [ ] ProductionOperator
  • [ ] Developer
  • [ ] ReadOnly
  • [ ] Billing
• [ ] Give routine access through the least-privileged applicable role.
• [ ] Require explicit elevation for administrative work.
• [ ] Avoid permanent AdministratorAccess for everyday use where practical.
• [ ] Use IAM Access Analyzer to inspect and refine permissions.
• [ ] Review unused roles, users, policies, and credentials.
• [ ] Document who can approve and perform production changes.

Workload Access

• [ ] Give every application or service its own IAM role.
• [ ] Do not place AWS access keys in source code.
• [ ] Do not place long-lived AWS access keys in GitHub secrets.
• [ ] Restrict every deployment role to:
  • [ ] The correct repository
  • [ ] The correct branch, tag, or GitHub environment
  • [ ] The correct AWS account
  • [ ] The required AWS resources only
• [ ] Separate staging and production deployment roles.
• [ ] Validate trust policies and permission boundaries.
• [ ] Prefer temporary credentials everywhere.

---

4. Infrastructure as Code

• [ ] Select one Infrastructure-as-Code system:
  • [ ] Terraform/OpenTofu
  • [ ] AWS CDK
  • [ ] CloudFormation
• [ ] Capture the existing staging infrastructure in code.
• [ ] Stop making undocumented console-only changes.
• [ ] Build reusable modules rather than duplicating entire configurations.
• [ ] Provide separate environment configuration for:
  • [ ] Staging
  • [ ] Production
• [ ] Keep production and staging state isolated.
• [ ] Encrypt remote IaC state.
• [ ] Enable versioning and locking for remote state.
• [ ] Restrict access to IaC state because it may contain sensitive metadata.
• [ ] Pin provider, module, and action versions.
• [ ] Add formatting and validation checks.
• [ ] Add static security scanning.
• [ ] Require a reviewed plan before production changes.
• [ ] Detect configuration drift.
• [ ] Document how to import existing resources into IaC.
• [ ] Add environment-standard tags:
  • [ ] Application
  • [ ] Environment
  • [ ] Owner
  • [ ] Managed-by
  • [ ] Cost-center
  • [ ] Data-classification
• [ ] Make a fresh environment reproducible from source control and documented prerequisites.
• [ ] Add a teardown guard so production cannot be accidentally destroyed.
• [ ] Add deletion protection to critical resources.
• [ ] Add policy checks that prevent production resources from being publicly exposed unintentionally.
• [ ] Ensure IaC changes are reviewed by someone other than the author when practical.

---

5. Production Architecture

• [ ] Reassess whether staging’s sizing and topology are production-appropriate.
• [ ] Do not copy staging data into production unless explicitly sanitized and required.
• [ ] Use separate production:
  • [ ] Database
  • [ ] Credentials
  • [ ] Secrets
  • [ ] Object-storage buckets
  • [ ] Queues
  • [ ] Cache
  • [ ] Logs
  • [ ] Encryption keys where appropriate
• [ ] Deploy production across at least two Availability Zones where supported and justified.
• [ ] Keep databases and internal services in private subnets.
• [ ] Expose only the public ingress layer.
• [ ] Eliminate direct public database access.
• [ ] Use security groups by service relationship rather than broad CIDR ranges.
• [ ] Prohibit unrestricted administrative ports such as public SSH.
• [ ] Use AWS Systems Manager Session Manager rather than public SSH where applicable.
• [ ] Configure health checks.
• [ ] Configure graceful application shutdown.
• [ ] Configure connection draining.
• [ ] Set sensible application and load-balancer timeouts.
• [ ] Configure autoscaling or document why fixed capacity is appropriate.
• [ ] Set resource requests, limits, or instance sizes deliberately.
• [ ] Verify production can survive a routine instance or task replacement.
• [ ] Confirm background jobs are idempotent or safely retryable.
• [ ] Confirm database migrations are compatible with rolling deployments.
• [ ] Add rate limiting and abuse controls.
• [ ] Consider AWS WAF for public endpoints.
• [ ] Configure DDoS protections appropriate to the architecture.
• [ ] Establish AWS service quota monitoring for relevant resources.
• [ ] Document every intentional single point of failure.
• [ ] Create a plan to remove unacceptable single points of failure.

---

6. DNS, Domain, and TLS

Namecheap Account

• [ ] Enable MFA on Namecheap for every authorized administrator.
• [ ] Use unique individual access rather than shared credentials where the account model permits.
• [ ] Enable domain lock.
• [ ] Verify registrant and recovery contact details.
• [ ] Enable automatic renewal.
• [ ] Verify the payment method will not unexpectedly expire.
• [ ] Document who may change nameservers or DNS.
• [ ] Export or document all existing DNS records before making changes.
• [ ] Preserve mail-related MX, SPF, DKIM, and DMARC records.
• [ ] Confirm both Gabriel and Aaron have an approved recovery path.

DNS Design

• [ ] Decide between:
  • [ ] Namecheap-hosted DNS
  • [ ] Route 53-hosted DNS with Namecheap remaining registrar
• [ ] Prefer Route 53 when AWS-native alias records and IaC-managed DNS are valuable.
• [ ] Create a Route 53 hosted zone if selected.
• [ ] Reproduce every required existing record before switching nameservers.
• [ ] Lower DNS TTL ahead of cutover.
• [ ] Configure the root/apex record.
• [ ] Configure www.
• [ ] Configure api if needed.
• [ ] Keep staging on a clearly separate hostname.
• [ ] Do not expose internal AWS service hostnames unnecessarily.
• [ ] Verify IPv4 and IPv6 behavior.
• [ ] Configure a canonical-host redirect.
• [ ] Test DNS from multiple resolvers after cutover.
• [ ] Raise TTL again after the deployment is stable.
• [ ] Manage DNS records through IaC where practical.

TLS

• [ ] Request an AWS Certificate Manager certificate.
• [ ] Include every required hostname.
• [ ] Prefer DNS validation.
• [ ] Retain ACM validation records for automatic renewal.
• [ ] Redirect all HTTP traffic to HTTPS.
• [ ] Use modern TLS policies.
• [ ] Configure HSTS only after HTTPS and subdomain behavior are verified.
• [ ] Confirm certificate-expiration monitoring.
• [ ] Test the full certificate chain.
• [ ] Confirm cookies use:
  • [ ] Secure
  • [ ] HttpOnly
  • [ ] Appropriate SameSite settings

---

7. Secrets and Application Configuration

• [ ] Inventory all secrets.
• [ ] Remove secrets from source control and repository history.
• [ ] Store secrets in AWS Secrets Manager or SSM Parameter Store as appropriate.
• [ ] Encrypt secrets with KMS.
• [ ] Use separate staging and production secrets.
• [ ] Give workloads access only to the secrets they require.
• [ ] Do not expose production secrets to pull-request workflows.
• [ ] Do not print secrets in logs.
• [ ] Configure secret rotation where supported.
• [ ] Rotate credentials during the production launch.
• [ ] Document emergency rotation procedures.
• [ ] Separate secrets from nonsecret configuration.
• [ ] Validate required configuration at application startup.
• [ ] Fail safely when critical configuration is missing.
• [ ] Remove default credentials and sample keys.
• [ ] Audit OAuth callback URLs and API allowlists for production.
• [ ] Audit every external webhook secret.
• [ ] Document ownership and rotation frequency for every production secret.

---

8. Database Security, Migrations, and Backups

Database Security

• [ ] Enable encryption at rest.
• [ ] Enforce encryption in transit.
• [ ] Use a production-specific KMS key where appropriate.
• [ ] Store credentials in Secrets Manager.
• [ ] Avoid administrator credentials for normal application traffic.
• [ ] Create separate roles for:
  • [ ] Application runtime
  • [ ] Migrations
  • [ ] Read-only operations
  • [ ] Administration
• [ ] Keep the database off the public internet.
• [ ] Restrict security-group access to the application layer.
• [ ] Enable deletion protection.
• [ ] Enable database-level audit logging where appropriate.
• [ ] Monitor storage, CPU, memory, connections, locks, and slow queries.

Backups and Recovery

• [ ] Enable automated backups.
• [ ] Set an explicit retention period.
• [ ] Enable point-in-time recovery where supported.
• [ ] Configure scheduled longer-term snapshots.
• [ ] Configure cross-account backup copies for critical production data.
• [ ] Consider cross-region copies based on the agreed disaster model.
• [ ] Protect backup vaults from routine production credentials.
• [ ] Encrypt backup recovery points.
• [ ] Enable backup-vault lock or equivalent immutability where justified.
• [ ] Monitor backup-job failures.
• [ ] Define retention and deletion policies.
• [ ] Document the restoration procedure.
• [ ] Perform an actual restore test before launch.
• [ ] Schedule recurring restore tests.
• [ ] Measure restoration time against the RTO.
• [ ] Confirm restored data satisfies the RPO.
• [ ] Record the latest successful restore test.
• [ ] Verify backups are restorable by both Gabriel and Aaron.
• [ ] Back up any other mission-critical state, including object storage and configuration data.

Schema Migrations

• [ ] Run migrations automatically but as an explicit deployment step.
• [ ] Ensure only one migration runner executes at a time.
• [ ] Back up before high-risk migrations.
• [ ] Prefer expand-and-contract migrations.
• [ ] Avoid combining destructive schema changes with dependent application changes in one irreversible step.
• [ ] Test migrations against a production-like dataset.
• [ ] Define rollback or forward-repair procedures.
• [ ] Record migration state and duration.
• [ ] Alert on migration failure.
• [ ] Ensure older and newer application versions can coexist during rolling or blue/green deployments where required.

---

9. CI Pipeline

• [ ] Require pull requests for protected branches.
• [ ] Protect main.
• [ ] Require CI checks before merging.
• [ ] Require review from another maintainer for sensitive changes.
• [ ] Define CODEOWNERS for:
  • [ ] Infrastructure
  • [ ] Deployment workflows
  • [ ] Authentication and security
  • [ ] Database migrations
• [ ] Run:
  • [ ] Formatting
  • [ ] Linting
  • [ ] Type checking
  • [ ] Unit tests
  • [ ] Integration tests
  • [ ] Migration checks
  • [ ] Dependency vulnerability scanning
  • [ ] Secret scanning
  • [ ] Container or artifact scanning
  • [ ] IaC validation and security scanning
• [ ] Generate immutable, versioned build artifacts.
• [ ] Record the Git commit SHA in every artifact.
• [ ] Produce the artifact once and promote the same artifact through environments.
• [ ] Generate an SBOM where practical.
• [ ] Pin third-party GitHub Actions to immutable commit SHAs for sensitive workflows.
• [ ] Minimize GitHub Actions token permissions.
• [ ] Avoid executing untrusted pull-request code with production-capable credentials.
• [ ] Configure dependency update automation.
• [ ] Define an urgent-patch workflow.
• [ ] Retain CI and deployment logs for an explicit period.
• [ ] Prevent force-pushes and branch deletion on protected branches.

---

10. Automated Staging Deployment

• [ ] Create a GitHub staging environment.
• [ ] Deploy automatically after successful merges to main, or adopt a deliberate release branch if preferred.
• [ ] Authenticate to AWS through GitHub OIDC.
• [ ] Do not store long-lived AWS keys in GitHub.
• [ ] Restrict the staging role to the staging AWS account and resources.
• [ ] Run infrastructure planning.
• [ ] Apply approved or policy-compliant infrastructure changes.
• [ ] Deploy the immutable application artifact.
• [ ] Run database migrations.
• [ ] Run smoke tests.
• [ ] Run health checks.
• [ ] Publish the deployment URL and commit SHA.
• [ ] Notify both maintainers of success or failure.
• [ ] Preserve deployment logs.
• [ ] Automatically roll back application deployment when health checks fail, where safe.
• [ ] Keep database rollback separate and explicit.
• [ ] Prevent concurrent staging deployments from colliding.
• [ ] Make the deployment status visible in GitHub.
• [ ] Record artifact digest, migration version, deployer, and timestamp.

---

11. Automated Production Deployment

• [ ] Create a GitHub production environment.
• [ ] Use a distinct production AWS role.
• [ ] Restrict that role to the production account and resources.
• [ ] Trigger production deployment through an explicit release action:
  • [ ] Version tag
  • [ ] GitHub release
  • [ ] Manual promotion of a verified artifact
• [ ] Do not rebuild the application for production.
• [ ] Promote the exact artifact tested in staging.
• [ ] Require successful staging deployment and tests first.
• [ ] Require production approval.
• [ ] Prevent self-approval when practical.
• [ ] Permit both Gabriel and Aaron to approve.
• [ ] Require approval from someone other than the person initiating the deployment when practical.
• [ ] Display the complete deployment plan before approval.
• [ ] Back up stateful systems before risky changes.
• [ ] Run migrations as a controlled step.
• [ ] Deploy using:
  • [ ] Rolling deployment
  • [ ] Blue/green deployment
  • [ ] Canary deployment
• [ ] Run post-deployment smoke tests.
• [ ] Monitor error rate, latency, and health during rollout.
• [ ] Stop or roll back when thresholds are exceeded.
• [ ] Record who approved and initiated each deployment.
• [ ] Record commit SHA, artifact identifier, and migration version.
• [ ] Generate release notes.
• [ ] Notify both operators after completion.
• [ ] Prevent concurrent production deployments.
• [ ] Add a documented manual deployment path for emergencies.
• [ ] Add a documented rollback command or workflow.
• [ ] Test rollback before launch.

---

12. Observability

• [ ] Centralize application logs.
• [ ] Use structured logs.
• [ ] Include:
  • [ ] Timestamp
  • [ ] Environment
  • [ ] Service
  • [ ] Request or correlation ID
  • [ ] Release version
  • [ ] Severity
• [ ] Prevent credentials, tokens, and sensitive personal data from entering logs.
• [ ] Set explicit log-retention periods.
• [ ] Encrypt logs.
• [ ] Collect infrastructure and application metrics.
• [ ] Build dashboards for:
  • [ ] Request rate
  • [ ] Error rate
  • [ ] Latency
  • [ ] Saturation
  • [ ] Task or instance health
  • [ ] Database capacity
  • [ ] Database connections
  • [ ] Queue depth
  • [ ] Background-job failures
• [ ] Add external uptime checks.
• [ ] Configure alerts for:
  • [ ] Site unavailable
  • [ ] Elevated 5xx responses
  • [ ] Elevated latency
  • [ ] Unhealthy deployment
  • [ ] Database storage pressure
  • [ ] CPU or memory exhaustion
  • [ ] Backup failure
  • [ ] Certificate problems
  • [ ] Queue backlog
  • [ ] Security findings
  • [ ] Unexpected cost growth
• [ ] Route alerts to both Gabriel and Aaron.
• [ ] Define warning versus page-worthy alert severity.
• [ ] Test alert delivery.
• [ ] Ensure every urgent alert links to a runbook.
• [ ] Configure error tracking with release correlation.
• [ ] Define service-level indicators and initial service objectives.
• [ ] Add deployment markers to dashboards.
• [ ] Verify logs and metrics remain available during an application outage.

---

13. Security Monitoring and Auditability

• [ ] Enable CloudTrail.
• [ ] Use a multi-region trail for management activity.
• [ ] Include read and write management events.
• [ ] Encrypt CloudTrail logs.
• [ ] Enable log-file validation.
• [ ] Store logs in a nonpublic, access-restricted bucket.
• [ ] Send relevant CloudTrail events to CloudWatch Logs.
• [ ] Enable AWS Config where cost and scope permit.
• [ ] Enable GuardDuty.
• [ ] Enable Security Hub and AWS Foundational Security Best Practices controls.
• [ ] Enable IAM Access Analyzer.
• [ ] Review public and cross-account resource access.
• [ ] Enable S3 Block Public Access at the account level unless explicitly incompatible.
• [ ] Enable ECR or image vulnerability scanning if containers are used.
• [ ] Monitor changes to IAM, KMS, CloudTrail, DNS, and security groups.
• [ ] Alert on root-account activity.
• [ ] Alert on failed console logins and suspicious authentication.
• [ ] Define a vulnerability-remediation SLA.
• [ ] Run a production security review before launch.
• [ ] Record and assign every security finding.
• [ ] Review high-severity findings before every production launch.

---

14. Application Security

• [ ] Document the authentication and authorization model.
• [ ] Test authorization independently of UI visibility.
• [ ] Enforce server-side access control.
• [ ] Use secure password hashing if MuseHub stores passwords.
• [ ] Require email verification where relevant.
• [ ] Implement secure account recovery.
• [ ] Rate-limit login and recovery endpoints.
• [ ] Protect against CSRF where applicable.
• [ ] Validate and sanitize untrusted input.
• [ ] Use parameterized database queries.
• [ ] Restrict CORS to deliberate origins.
• [ ] Define a Content Security Policy.
• [ ] Add standard security headers.
• [ ] Validate uploaded file type, size, and storage behavior.
• [ ] Protect against path traversal and unsafe archive extraction.
• [ ] Scan uploaded content if the risk model warrants it.
• [ ] Set request-body and upload limits.
• [ ] Review SSRF exposure from URL-fetching features.
• [ ] Ensure production debug mode is disabled.
• [ ] Ensure stack traces are not returned to users.
• [ ] Review WebSocket or SSE authentication if used.
• [ ] Document data classification and retention.
• [ ] Add an account and data deletion path where applicable.
• [ ] Run an OWASP-oriented review before launch.
• [ ] Review session expiration and token revocation behavior.
• [ ] Verify privileged actions require fresh authorization where appropriate.

---

15. Cost Controls

• [ ] Apply consistent cost-allocation tags.
• [ ] Create monthly AWS budgets.
• [ ] Alert Gabriel and Aaron at multiple thresholds.
• [ ] Enable Cost Anomaly Detection.
• [ ] Review NAT Gateway, data-transfer, and logging costs.
• [ ] Set log-retention limits.
• [ ] Set backup-retention limits consistent with recovery requirements.
• [ ] Right-size production after load testing.
• [ ] Review idle resources.
• [ ] Document expected baseline monthly cost.
• [ ] Document the cost implications of autoscaling.
• [ ] Configure service quotas and limits deliberately.
• [ ] Review production cost weekly during the first month.
• [ ] Add owner and purpose tags to every billable resource.
• [ ] Alert on untagged or unexpectedly expensive resources where practical.

---

16. Documentation and Runbooks

• [ ] Create a system architecture diagram.
• [ ] Create a data-flow diagram.
• [ ] Document every AWS account and region.
• [ ] Document DNS and certificate ownership.
• [ ] Document environment differences.
• [ ] Document the IaC repository structure.
• [ ] Document local development setup.
• [ ] Document staging deployment.
• [ ] Document production deployment.
• [ ] Document rollback.
• [ ] Document database migration procedures.
• [ ] Document database restoration.
• [ ] Document secret rotation.
• [ ] Document certificate and DNS troubleshooting.
• [ ] Document user-access provisioning and removal.
• [ ] Document break-glass access.
• [ ] Document incident response.
• [ ] Document backup retention and restoration tests.
• [ ] Document common alarms and remediation.
• [ ] Create a service inventory with owners.
• [ ] Create a dependency inventory.
• [ ] Create an operational decision log.
• [ ] Record accepted risks and deferred work.
• [ ] Keep documentation versioned beside the relevant code where practical.
• [ ] Confirm Aaron can follow the documentation without Gabriel’s help.
• [ ] Confirm Gabriel can follow it without relying on personal memory.
• [ ] Add a production launch checklist and rollback decision tree.
• [ ] Add contact information and escalation paths.
• [ ] Review documentation after every incident or major deployment.

---

17. Testing Before Launch

• [ ] Run the complete automated test suite.
• [ ] Run production-like integration tests.
• [ ] Test authentication and authorization.
• [ ] Test a clean infrastructure deployment in a disposable environment.
• [ ] Test migration from the current schema.
• [ ] Test application rollback.
• [ ] Test failed migration handling.
• [ ] Test a database restore.
• [ ] Test backup integrity.
• [ ] Test task or instance failure.
• [ ] Test deployment during active traffic.
• [ ] Test autoscaling if enabled.
• [ ] Run basic load and soak tests.
• [ ] Verify timeout behavior.
• [ ] Verify retry behavior.
• [ ] Verify idempotency for jobs and webhooks.
• [ ] Test DNS and TLS from external networks.
• [ ] Test both root and www.
• [ ] Verify canonical redirects.
• [ ] Test monitoring and alert delivery.
• [ ] Run a tabletop incident exercise with Gabriel and Aaron.
• [ ] Test the break-glass process.
• [ ] Verify neither maintainer is a single point of operational failure.
• [ ] Test production access from a clean workstation.
• [ ] Test what happens when a secret, database, queue, or external API is unavailable.

---

18. Cutover Plan

• [ ] Freeze unrelated infrastructure changes during cutover.
• [ ] Confirm the production environment is healthy.
• [ ] Confirm production secrets are installed.
• [ ] Confirm migrations have been rehearsed.
• [ ] Confirm current backups succeeded.
• [ ] Confirm a restoration has been tested.
• [ ] Confirm dashboards and alerts are active.
• [ ] Confirm Aaron and Gabriel are both available for the cutover.
• [ ] Lower DNS TTL in advance.
• [ ] Deploy the approved production artifact.
• [ ] Run migrations.
• [ ] Run internal smoke tests.
• [ ] Update DNS.
• [ ] Verify:
  • [ ] DNS resolution
  • [ ] TLS
  • [ ] Root domain
  • [ ] www
  • [ ] API
  • [ ] Authentication
  • [ ] Critical user paths
  • [ ] Background workers
  • [ ] Outbound email
  • [ ] Object uploads and downloads
• [ ] Monitor logs, error rate, and latency closely.
• [ ] Keep the previous environment available for rollback during the agreed window.
• [ ] Execute rollback if predetermined thresholds are crossed.
• [ ] Raise DNS TTL after stability is confirmed.
• [ ] Record the launch artifact and configuration versions.
• [ ] Announce launch status to stakeholders.
• [ ] Record all manual actions taken during cutover.
• [ ] Confirm production backups run successfully after cutover.

---

19. Post-Launch

• [ ] Review production metrics after the first hour.
• [ ] Review again after 24 hours.
• [ ] Review again after one week.
• [ ] Inspect AWS costs after 24 hours and one week.
• [ ] Review security findings.
• [ ] Confirm backups continue to succeed.
• [ ] Perform another restore test after production has real data.
• [ ] Remove temporary cutover access and resources.
• [ ] Remove obsolete staging-derived credentials.
• [ ] Resolve or formally record every launch exception.
• [ ] Conduct a blameless launch retrospective.
• [ ] Update documentation with lessons learned.
• [ ] Schedule:
  • [ ] Monthly dependency updates
  • [ ] Monthly access review during the early production period
  • [ ] Quarterly disaster-recovery exercise
  • [ ] Quarterly permissions review
  • [ ] Quarterly cost review
  • [ ] Annual domain and recovery-contact review

---

Recommended Implementation Order

P0 — Before Creating Production

• [ ] Inventory staging.
• [ ] Establish shared organizational ownership and MFA.
• [ ] Decide AWS account boundaries.
• [ ] Select and implement Infrastructure as Code.
• [ ] Define RPO, RTO, and domain topology.
• [ ] Decide the canonical hostname and DNS provider.

P1 — Build Production

• [ ] Provision the production account and environment through IaC.
• [ ] Configure identity, networking, encryption, and secrets.
• [ ] Configure the database, automated backups, and restore testing.
• [ ] Configure GitHub OIDC and separate deployment roles.
• [ ] Build staging and production pipelines.
• [ ] Add logs, dashboards, alarms, and audit trails.
• [ ] Complete operational documentation.

P2 — Launch

• [ ] Configure certificates and DNS.
• [ ] Rehearse migrations, deployment, and rollback.
• [ ] Run security and load tests.
• [ ] Complete the cutover.
• [ ] Validate production and document the outcome.

---

Suggested Ownership Split

Workstream	Primary	Reviewer / Backup	Status
Staging architecture inventory			Not started
AWS Organizations and accounts			Not started
IAM Identity Center and permissions			Not started
Infrastructure as Code			Not started
Networking and production architecture			Not started
Namecheap, DNS, and TLS			Not started
Database and backups			Not started
GitHub CI/CD			Not started
Logging, metrics, and alerts			Not started
Security review			Not started
Documentation and runbooks			Not started
Cutover coordination			Not started

---

Decision Log

Date	Decision	Rationale	Owner

---

Accepted Risks and Deferred Work

Item	Risk	Mitigation	Target Date	Owner