gabriel / musehub public
Open #93
filed by gabriel human · 4 days ago

musehub drops snapshot directories (and empties some manifests) on push → clones fail with snapshot hash mismatch

0 Anchors
Blast radius
Churn 30d
0 Proposals

Summary

musehub stores some pushed snapshots with their directories dropped (and, via a separate legacy delta-base bug, some with emptied manifests). Because a snapshot's id is hash_snapshot(manifest, directories), a stored snapshot whose directories (or manifest) were lost no longer reproduces its snapshot_id when a client re-hashes it on clone. The client rejects the snapshot, drops its commit, and every descendant fails parent not in mpack — yielding an empty working tree. This is why muse clone https://staging.musehub.ai/gabriel/muse fails while the local source repo and a localhost clone are clean.

This is not a muse code migrate bug and not corruption in the muse object store. The source repo is correct; the corruption is musehub-side, in pushed/stored data.

Evidence

For gabriel/muse on staging, the affected snapshots:

snapshot_id (prefix) musehub entry_count musehub n_dirs correct (local raw object)
eea07a1f… 1062 0 1062 manifest + 31 directories
708d5734… 0 0 full manifest + dirs
3d5ae8b5… 0 0 full manifest + dirs
edd649a9… 0 0 full manifest + dirs

Proof it's a directories (not manifest) problem for eea07a1f:

RAW local object : entries=1062  dirs=31  hash_snapshot(manifest, dirs) = sha256:eea07a1f…  ✓ (== id)
served by staging: entries=1062  dirs=0   hash_snapshot(manifest, [])   = sha256:3305e95e…  ✗
manifests are byte-identical (0 differing entries)

So the manifest is intact; only the directories field was lost in storage, which alone breaks the snapshot id.

Two distinct defects

  1. Directories dropped on push/store. eea07a1f has a full 1062-entry manifest but n_dirs=0 on staging (should be 31). The push path or the snapshot serialization is not persisting the directories array (note: directories are NOT fully derivable from manifest paths — directories_from_manifest yields a different set, so they must be stored, not re-derived).
  2. Manifests emptied (legacy delta-base bug). 708d5734/3d5ae8b5/edd649a9 are stored with entry_count=0. This is the wire-push base-resolution path resolving an external delta-only parent to {} (tracked separately; the serving-side reconstruction in _snap_row_to_wire_s3 mitigates these on read, but does not restore lost directories).

Impact

  • muse clone of any repo containing such a snapshot fails: snapshot rejected → commit dropped → parent not in mpack cascade → empty working tree.
  • Affects the official gabriel/muse repo on staging (HEAD unclonable).

Repair (operator)

The POST /{owner}/{slug}/repair-snapshot endpoint (wire_repair_snapshot) validates hash_snapshot(manifest, directories) == snapshot_id and stores the corrected record. The source repo holds the correct (manifest, directories). Repair driven by a signed request (muse sign request … --body-file payload.json) with the raw manifest+directories read from the source repo's object (NOT muse read-snapshot, which currently drops directories — see below).

Fixes

  • Push/store path must persist the snapshot directories array end-to-end.
  • Verify a snapshot's stored content reproduces its snapshot_id at unpack time (reject / log a hard integrity error rather than silently storing a snapshot that can't be re-hashed).
  • Close the empty-manifest delta-base bug (external delta-only parent resolving to {}).
  • (muse, separate) muse read-snapshot drops the directories field — fix so it returns the full stored record.
  • Repair the 4 corrupt gabriel/muse staging snapshots from the source repo's correct data.

Provenance

Surfaced during the muse clone hash-mismatch investigation (2026-06). Originally mis-diagnosed as a muse code migrate snapshot-rekey gap; a raw-object scan of the source repo showed all snapshots are correctly keyed (manifest + directories), relocating the bug to musehub's push/store/serve handling of directories.

Activity
gabriel opened this issue 4 days ago
No activity yet. Use the CLI to comment.