gabriel / musehub public
feat patch phase4 task/phase1-object-store-invariant #3 / 6
AI Agent gabriel · 10 days ago · May 29, 2026 · Diff

feat(phase4): wire fetch serves commits and snapshots from S3 (issue #63)

Adds _snap_row_to_wire_s3() and _commit_to_wire_s3() in musehub_wire_shared: - When storage_uri is set, reads canonical bytes from S3 and parses them - Falls back to DB reconstruction (_snap_row_to_wire / _to_wire_commit) when storage_uri is null (pre-backfill rows)

Updates all 3 _snap_row_to_wire call sites and 2 _to_wire_commit call sites in wire_fetch_mpack to use the async S3-first variants.

Hoists backend = get_backend() to top of wire_fetch_mpack so it is available at all call sites.

Key invariant: snapshot with zeroed manifest_blob still serves correct manifest to pulling clients because S3 is the source of truth.

21 tests GREEN. 133 existing tests unaffected.

sha256:9d0ffea20e344782dc6a969d4a240b3d7c96392b5dc03bbd9421890cb78c6f19 sha
sha256:450998d182617fa93b737cbbdb3fe956c61566051739acec8c63ec5e7b4587f8 parent
+36 ~12 symbols
sha256:7455c93a6e49cf3701cd46aa5d884b68726e5ad589a6b924c4bf212afa0f8c62 snapshot
+36
symbols added
~12
symbols modified
0
dead code introduced
Semantic Changes 48 symbols
~ tests/test_object_store_invariant_phase4.py .py 32 symbols added
+ TestWireFetchCommitFromS3 class class TestWireFetchCommitFromS3 L236–294
+ test_commit_s3_bytes_decode_correctly method async_method test_commit_s3_bytes_decode_correctly L251–275
+ test_commit_storage_uri_is_set method async_method test_commit_storage_uri_is_set L237–249
+ test_wire_fetch_returns_commit method async_method test_wire_fetch_returns_commit L277–294
+ TestWireFetchFallback class class TestWireFetchFallback L301–349
+ test_snapshot_fallback_when_no_storage_uri method async_method test_snapshot_fallback_when_no_storage_uri L302–349
+ TestWireFetchSnapshotFromS3 class class TestWireFetchSnapshotFromS3 L126–229
+ test_snapshot_manifest_served_from_s3 method async_method test_snapshot_manifest_served_from_s3 L127–158
+ test_snapshot_storage_uri_is_set method async_method test_snapshot_storage_uri_is_set L217–229
+ test_snapshot_wire_manifest_correct_even_when_manifest_blob_zeroed method async_method test_snapshot_wire_manifest_correct_even_when_manifest_blob_zeroed L160–215
+ _backend function function _backend L43–45
+ _make_commit_with_snapshot function async_function _make_commit_with_snapshot L61–119
+ _make_repo function async_function _make_repo L48–58
+ _uid function function _uid L39–40
+ AsyncSession import import AsyncSession L23–23
+ MusehubBranch import import MusehubBranch L27–27
+ MusehubCommit import import MusehubCommit L27–27
+ MusehubCommitGraph import import MusehubCommitGraph L27–27
+ MusehubCommitRef import import MusehubCommitRef L27–27
+ MusehubMPackIndex import import MusehubMPackIndex L27–27
+ MusehubRepo import import MusehubRepo L27–27
+ MusehubSnapshot import import MusehubSnapshot L27–27
+ MusehubSnapshotRef import import MusehubSnapshotRef L27–27
+ annotations import import annotations L17–17
+ compute_branch_id import import compute_branch_id L26–26
+ compute_identity_id import import compute_identity_id L26–26
+ compute_repo_id import import compute_repo_id L26–26
+ datetime import import datetime L20–20
+ fake_id import import fake_id L25–25
+ json import import json L19–19
+ pytest import import pytest L22–22
+ secrets import import secrets L21–21
~ musehub/services/musehub_wire_fetch.py .py 2 symbols added, 12 symbols modified
+ _commit_to_wire_s3 import import _commit_to_wire_s3 L29–29
+ _snap_row_to_wire_s3 import import _snap_row_to_wire_s3 L29–29
~ FetchMPackResult
~ FetchNotIndexedError
~ FetchPresignResult
~ MPackValidationError
~ _reconstruct_manifest
~ _snap_row_to_wire
~ _to_wire_commit
~ _utc_now
~ logger
~ wire_fetch
~ wire_fetch_mpack
~ wire_fetch_presign
~ musehub/services/musehub_wire_shared.py .py 2 symbols added
+ _commit_to_wire_s3 function async_function _commit_to_wire_s3 L361–400
+ _snap_row_to_wire_s3 function async_function _snap_row_to_wire_s3 L334–358
Older feat(phase3): write snapshot objects to S3 at all 3 write sites … sha256:450998d182617fa93b737cbbdb3fe956c61566051739acec8c63ec5e7b4587f8 All commits Newer feat(phase5): backfill script + tests for object store invariant… sha256:6191353352f5187f4dc208220e243f3cf703ac322cad8018775716e9d21bce86

0 comments

No comments yet. Be the first to start the discussion.

To add a comment, use the Muse CLI: muse hub commit comment sha256:9d0ffea20e344782dc6a969d4a240b3d7c96392b5dc03bbd9421890cb78c6f19 --body "your comment"