musehub drops snapshot directories (and empties some manifests) on push → clones fail with snapshot hash mismatch
Summary
musehub stores some pushed snapshots with their directories dropped (and, via a
separate legacy delta-base bug, some with emptied manifests). Because a snapshot's id is
hash_snapshot(manifest, directories), a stored snapshot whose directories (or manifest) were
lost no longer reproduces its snapshot_id when a client re-hashes it on clone. The client
rejects the snapshot, drops its commit, and every descendant fails parent not in mpack —
yielding an empty working tree. This is why muse clone https://staging.musehub.ai/gabriel/muse
fails while the local source repo and a localhost clone are clean.
This is not a muse code migrate bug and not corruption in the muse object store. The
source repo is correct; the corruption is musehub-side, in pushed/stored data.
Evidence
For gabriel/muse on staging, the affected snapshots:
| snapshot_id (prefix) | musehub entry_count |
musehub n_dirs |
correct (local raw object) |
|---|---|---|---|
eea07a1f… |
1062 | 0 | 1062 manifest + 31 directories |
708d5734… |
0 | 0 | full manifest + dirs |
3d5ae8b5… |
0 | 0 | full manifest + dirs |
edd649a9… |
0 | 0 | full manifest + dirs |
Proof it's a directories (not manifest) problem for eea07a1f:
RAW local object : entries=1062 dirs=31 hash_snapshot(manifest, dirs) = sha256:eea07a1f… ✓ (== id)
served by staging: entries=1062 dirs=0 hash_snapshot(manifest, []) = sha256:3305e95e… ✗
manifests are byte-identical (0 differing entries)
So the manifest is intact; only the directories field was lost in storage, which alone breaks
the snapshot id.
Two distinct defects
- Directories dropped on push/store.
eea07a1fhas a full 1062-entry manifest butn_dirs=0on staging (should be 31). The push path or the snapshot serialization is not persisting thedirectoriesarray (note: directories are NOT fully derivable from manifest paths —directories_from_manifestyields a different set, so they must be stored, not re-derived). - Manifests emptied (legacy delta-base bug).
708d5734/3d5ae8b5/edd649a9are stored withentry_count=0. This is the wire-push base-resolution path resolving an external delta-only parent to{}(tracked separately; the serving-side reconstruction in_snap_row_to_wire_s3mitigates these on read, but does not restore lost directories).
Impact
muse cloneof any repo containing such a snapshot fails: snapshot rejected → commit dropped →parent not in mpackcascade → empty working tree.- Affects the official
gabriel/muserepo on staging (HEAD unclonable).
Repair (operator)
The POST /{owner}/{slug}/repair-snapshot endpoint (wire_repair_snapshot) validates
hash_snapshot(manifest, directories) == snapshot_id and stores the corrected record. The source
repo holds the correct (manifest, directories). Repair driven by a signed request
(muse sign request … --body-file payload.json) with the raw manifest+directories read from the
source repo's object (NOT muse read-snapshot, which currently drops directories — see below).
Fixes
- Push/store path must persist the snapshot
directoriesarray end-to-end. - Verify a snapshot's stored content reproduces its
snapshot_idat unpack time (reject / log a hard integrity error rather than silently storing a snapshot that can't be re-hashed). - Close the empty-manifest delta-base bug (external delta-only parent resolving to
{}). - (muse, separate)
muse read-snapshotdrops thedirectoriesfield — fix so it returns the full stored record. - Repair the 4 corrupt
gabriel/musestaging snapshots from the source repo's correct data.
Provenance
Surfaced during the muse clone hash-mismatch investigation (2026-06). Originally mis-diagnosed
as a muse code migrate snapshot-rekey gap; a raw-object scan of the source repo showed all
snapshots are correctly keyed (manifest + directories), relocating the bug to musehub's
push/store/serve handling of directories.