Open #58

filed by gabriel human · 18 days ago

Muse Wire Protocol — MVP hardening (clone/push/fetch/pull master tracking issue)

0 Anchors

— Blast radius

— Churn 30d

0 Proposals

Muse Wire Protocol — MVP hardening (master tracking issue)

Status update (2026-07-07)

All 8 correctness sub-tickets (MWP-1 through MWP-8) are complete — see the sub-tickets section below and the acceptance criteria, all checked off. The wire protocol itself is hardened as designed.

aaronrene onboarding, concretely underway:

Registered MuseHub account confirmed (aaronrene, 3 existing repos).
Now a write collaborator on both muse and musehub — no fork needed, can push branches and open merge proposals directly.
Onboarding reference doc published as a mist: https://staging.musehub.ai/gabriel/mists/4dB2qCpPV5vG — the standard branch→commit→push→MP→review cycle, spelled out with exact commands.

A separate, more serious finding surfaced while verifying the above — not a wire-protocol correctness bug like MWP-1..8, but an authorization gap: push_unpack_mpack (the actual push handler) has no repo-specific write-authorization check at all today — any authenticated MuseHub identity can push to any repo, public or private, regardless of collaborator status. Traced end to end, confirmed in code (not assumed). Tracked separately as musehub#131: https://staging.musehub.ai/gabriel/musehub/issues/131 — scoped as its own fix-plus-sweep, since this class of gap may exist in other routes too. This finding is why "aaronrene can push with confidence" is true operationally today (his collaborator grant is real and works for MP creation) but the deeper authorization story for push itself still needs closing — #131 tracks that, not this ticket.

Background

The Muse Wire Protocol (MWP) — clone, push, fetch, pull — is the single hardest, most time-consuming subsystem in the workspace. It is functionally equivalent to git smart-HTTP, rebuilt from scratch on an mpack packfile format with content-addressed objects, presigned R2/MinIO uploads, and an async server-side prebuild/index pipeline.

It mostly works. The blocker to declaring it MVP-ready — so aaronrene can clone/push/pull/fetch with confidence and start picking up tickets — is a small number of correctness gaps in the prebuild/index/negotiation pipeline plus a largely-disabled test suite. This issue is the result of a full read-through of both sides of the protocol. It records the architecture, the confirmed root causes (grounded in file:line), and the proposed sub-tickets.

This is a hardening pass, not a rewrite. The architecture is sound. The fixes are targeted.

Proximate triggers

Clone-after-push staleness — push a repo, clone it (works), push another commit or two, clone again → the new commits/blobs are missing from the clone, yet clone exits 0.
#55 — push have-negotiation ignored sibling refs, causing redundant multi-hundred-MB uploads and broken pipes when a branch lags a sibling already on the remote.

Architecture recap (for onboarding)

All four commands move data as a single binary mpack (b"MUSE" magic, section table, 32-byte SHA-256 footer). Objects are content-addressed: sha256:<hex> is the integrity check.

Command	Client entry	Server endpoint(s)	Notes
`push`	`muse/cli/commands/push.py` → `transport.push_mpack_*`	`POST /push/mpack-presign` → PUT to R2 → `POST /push/unpack-mpack`	client builds delta mpack, uploads direct to R2, server indexes
`clone`	`muse/cli/commands/clone.py`	`POST /fetch/mpack` with `have=[]`	no dedicated clone endpoint — clone is a full fetch served from the prebuilt-mpack cache
`fetch`	`muse/cli/commands/fetch.py` → `transport.fetch_mpack`	`POST /fetch/mpack`	have/want delta
`pull`	`muse/cli/commands/pull.py` (reuses `fetch_mpack`)	`POST /fetch/mpack`	fetch + merge/ff

Server push pipeline (musehub_wire_push.py::wire_push_unpack_mpack): reads mpack from R2 → inserts objects/snapshots/commits → computes MusehubCommitGraph generations (step 9, synchronous) → advances branch pointer (step 10, atomic CAS) → enqueues async jobs (step 11): mpack.index, fetch.mpack.prebuild, intel.*, gc.

Server fetch pipeline (musehub_wire_fetch.py::wire_fetch_mpack): on have=[] (clone), looks up musehub_fetch_mpack_cache — HIT requires every want tip cached AND all pointing to one mpack_id; MISS raises MPackNotReadyError (HTTP 503 + Retry-After). Delta builds walk the commit graph via _walk_commit_delta.

Prebuild (process_fetch_mpack_prebuild_job): after each push, builds one combined mpack covering all branch tips and writes a cache row per tip → next clone is a sub-second cache HIT.

Confirmed root causes

RC-1 — Generation fallback to 0 silently corrupts the fetch walk (CRITICAL — the clone staleness bug)

musehub_wire_push.py:926:

_gen = (max(_parent_gens) + 1) if _parent_gens else 0

When a pushed commit's parent generation cannot be resolved — the parent is not in this mpack (it is on the remote, excluded by the have set) and its MusehubCommitGraph row is not returned by the lookup at musehub_wire_push.py:870-874 — the new commit is written with generation 0 instead of parent_gen + 1.

That single wrong generation poisons the fetch-side walk. _walk_commit_delta (musehub_wire_fetch.py:102-111) is a generation-bounded range scan:

.where(MusehubCommitGraph.generation > min_have_gen)
.where(MusehubCommitGraph.generation <= max_want_gen)

With the new tip at gen 0, max_want_gen = 0, so the scan excludes every real ancestor (gens 1..N). The BFS truncates, and the tip-snapshot resolver (musehub_wire_fetch.py:484-490, ORDER BY generation DESC LIMIT 1) selects an older commit's snapshot. The assembled mpack therefore carries the old tip's blobs but is cached against the new tip id. Clone returns 200 with a working tree missing the latest commits — exactly the reported symptom.

The fetch code already half-knows this — see the [BLOB-DEBUG] block at musehub_wire_fetch.py:495-509: "A mismatch means CommitGraph is stale for the newest push — the old snapshot's manifest_blob would be missing new blobs."

RC-2 — `_walk_commit_delta` has no fallback when the graph is inconsistent

musehub_wire_fetch.py:66 — if True: # fast path always active — makes the real parent-pointer DAG walk (the "legacy fallback" at lines 159-184) dead code. The generation range-scan is the only path. There is no safety net when generations are wrong or missing (RC-1), so a single bad row produces a silently truncated mpack instead of a correct (if slower) walk.

RC-3 — Job idempotency drops fresh payloads on back-to-back pushes

musehub_jobs.py:72-84 (enqueue_job) and :158-164 (enqueue_push_intel) dedup on (repo_id, job_type, status="pending"). If a fetch.mpack.prebuild (or mpack.index) job from push A is still pending when push B lands, push B's job is skipped. The pending job still carries push A's payload / tip set, so push B's new tip never gets a prebuild or index until yet another push displaces it. Rapid pushes ⇒ permanently stale clone cache.

RC-4 — Prebuild can run before index (foundation-job ordering race)

musehub_jobs.py:170-189: fetch.mpack.prebuild and mpack.index are both _FOUNDATION_TYPES and share the same created_at. claim_next_job (musehub_jobs.py:206-231) orders by created_at with no tiebreaker, so the prebuild may run first. wire_fetch_mpack then hits the index-coverage check (musehub_wire_fetch.py:580-593), raises FetchNotIndexedError, the prebuild swallows it (musehub_wire_fetch.py:1041), and no cache row is written. The clone then 503s until a retry/another push.

RC-5 — Client does not retry on 503 (clone-immediately-after-push fails)

The fetch route correctly returns 503 + Retry-After while the prebuild/index is in flight (wire.py:694-705). But the client (transport.py::fetch_mpack) raises TransportError(503), and clone.py:454-464 catches it, rmtrees the target, and exits INTERNAL_ERROR. There is no honor of Retry-After, no poll loop. A clone issued seconds after a push fails hard even though the server self-heals within a second or two.

RC-6 — The wire test suite is largely disabled ✅ Fixed by MWP-6

~19 wire/mpack test files are skipped, 11 with reason="muse wire protocol in flux" — including test_issue_61_clone_after_push.py, the exact clone-after-push contract. The protocol's own regression net is off, which is why these regressions reach manual testing. MVP cannot be declared with the suite dark.

Fixed: MWP-6 revived all flux-skipped wire files across 6 phases. 55 tests under pytest -m wire now run green (47 in-process + 8 slow subprocess E2E). The MWP6_00 gate (test_mwp6_wire_suite_enabled.py) enforces no-flux-skip forever.

RC-8 — Repair endpoints and force-push never invalidate `musehub_fetch_mpack_cache`

wire_repair_object, wire_repair_snapshot, and wire_repair_commit (musehub_wire_push.py) each end with await session.commit() → return {"repaired": True} and never touch musehub_fetch_mpack_cache. Likewise, wire_push_unpack_mpack with force=True advances the branch pointer and enqueues intel jobs but never deletes old-tip cache rows.

The cache is keyed by (repo_id, tip_commit_id) with a 7-day TTL. After a repair, the prebuilt mpack cached against the tip still carries the pre-repair bytes — the repair fixed the DB row but the fetch path presigns whatever mpack_id the cache row addresses and exits 0. A clone issued within 7 days of the repair receives corrupted content.

This was the mechanism behind the gabriel/muse rc10 incident: two main-line merge commits had their signer_public_key field stamped by the rc10 migration. repair-commit fixed the musehub_commits rows — but staging clones stayed broken because the prebuilt mpack kept being served. The manual workaround was:

UPDATE musehub_fetch_mpack_cache SET expires_at = NOW() - INTERVAL '1 second' WHERE repo_id = ...

A correct fix requires active invalidation in the same DB transaction as the repair, so a cache row can never outlive its corresponding object/snapshot/commit content.

RC-7 — #55 (sibling-ref push negotiation): main path fixed, needs verification + dry-run alignment

The live-push path already uses all sibling refs as the walk boundary (push.py:819-822, branch_have = remote_branch_heads.values()), and blob dedup subtracts the sibling tip's full manifest (mpack.py:330-336, 362). Client tests exist (tests/test_push_branch_have.py, tests/test_push_have_filter.py). What remains: (a) confirm #55 is actually resolved end-to-end against staging and close it, and (b) the dry-run path still scopes have to a single branch (push.py:668-674) — this is #32 and should be aligned so --dry-run counts match the real push.

Goal — definition of MVP done

aaronrene (or any dev) can, against staging, reliably:

muse clone <url> — get a complete, correct working tree every time, including immediately after a push (with bounded client retry on 503).
muse push — upload only the true delta; never re-upload objects already on the remote via any ref; survive back-to-back pushes.
muse fetch / muse pull — receive exactly the new commits/blobs and converge.
Do all of the above repeatedly with no stale clones, no phantom large uploads, no manual cache-busting.
The wire regression suite is re-enabled and green as the proof.

Proposed sub-tickets

Create as needed; each is independently shippable. Repo target in brackets.

MWP-1 [musehub] — Make commit-graph generations authoritative (fixes RC-1). ✅ COMPLETE → filed as musehub#106 (closed): https://staging.musehub.ai/gabriel/musehub/issues/106 Never write generation 0 for a commit that has parents. If a parent generation is unresolved at push time, resolve it (recursive lookup / on-demand backfill) or fail the push loudly. Add an invariant check + regression test for the push-child-of-remote-commit case. Shipped in 6 phases (13 tests, tests/test_mwp1_generation_authority.py): _resolve_generation_with_backfill (on-demand bottom-up backfill from musehub_commits), repair_corrupt_commit_generations (idempotent repair of existing artefacts), check_commit_graph_invariant (no gen-0-with-parents), the [MWP1_13] fetch-side repair guard, and the MWP1_14 end-to-end push→clone→push→clone regression gate. Merged to dev + main (sha256:da49a05fd62cda…).
MWP-2 [musehub] — Add a correctness fallback to _walk_commit_delta (fixes RC-2). ✅ CLOSED → musehub#107 closed: https://staging.musehub.ai/gabriel/musehub/issues/107 When the generation range-scan truncates (a wrong/missing generation makes the BFS dead-end), fall back to the authoritative MusehubCommit.parent_ids DAG walk so the returned commit set is never incomplete. Removed the if True: dead-code guard (line 66); promoted the legacy walk to a tested helper (_walk_commit_delta_dag) with Kahn's topo-sort (deque.popleft(), O(n)) and max_nodes=100_000 bound. 23 tests GREEN (10 MWP2 + 13 MWP1). All 6 acceptance criteria met. MWP2_12 (acceptance gate) proves fallback alone ships a complete clone even with repair patched out.
MWP-3 [musehub] — Fix job enqueue for back-to-back pushes (fixes RC-3). ✅ COMPLETE → musehub#108 (closed): https://staging.musehub.ai/gabriel/musehub/issues/108 fetch.mpack.prebuild / mpack.index must always reflect the latest tip set. Two-face fix shipped: (1) process_fetch_mpack_prebuild_job re-reads live MusehubBranch tips at runtime — coarse dedup becomes self-coalescing; (2) enqueue_job deduped per mpack_key via JSONB predicate — distinct mpacks are non-substitutable work and can never be dropped by a coarse pending-job check. RC-4 ordering is explicitly out of scope (→ MWP-4). 36 tests GREEN across 5 suites (14 MWP-3 + adjacent regressions fixed). MWP3_10 (two-push full drain) + MWP3_11 (N=10 stress partial drain) prove E2E correctness end-to-end.
MWP-4 [musehub] — Order prebuild strictly after index (fixes RC-4). ✅ COMPLETE → musehub#109 (closed): https://staging.musehub.ai/gabriel/musehub/issues/109 Three-layer fix shipped in 5 phases (12 tests, tests/test_mwp4_prebuild_ordering.py): (1) created_at stagger — prebuild gets t+1s, index stays at t+0s (defense-in-depth); (2) correlated NOT EXISTS barrier in claim_next_job — prebuild un-claimable while any mpack.index for the same repo_id is pending/running (load-bearing); (3) narrow except FetchNotIndexedError: raise in process_fetch_mpack_prebuild_job — index gaps surface as retryable fail_job failures instead of silent empty-cache. MWP4_11 (E2E claim loop) + MWP4_12 (N=10 stress) prove zero FetchNotIndexedError and a correct cache at the latest tip after any burst of back-to-back pushes. Commit range: sha256:2c523da453513 → sha256:46854101acdde on task/mwp-4-prebuild-after-index.
MWP-5 [muse] — Clone/fetch honor Retry-After with a bounded poll (fixes RC-5). ✅ COMPLETE → muse#59 (closed): https://staging.musehub.ai/gabriel/muse/issues/59 Extracted HttpTransport._fetch_mpack_once; added bounded retry loop in fetch_mpack tracking total_waited against a configurable budget (default 120s). `--retry-timeout` / `--no-retry` flags on `clone`, `fetch`, `pull`. Terminal 503 message improved; JSON envelope carries `retryable: true`. 19 tests GREEN (MWP5_00–MWP5_18 including real-urllib E2E on a live localhost server). Zero regressions across adjacent suites.
MWP-6 [musehub + muse] — Re-enable the wire regression suite (fixes RC-6). Un-skip test_issue_61_clone_after_push.py and the ..._flux mpack/wire tests; fix or rewrite each against the current protocol; gate MVP on green.
MWP-7 [muse] — Verify and close #55; align --dry-run negotiation (RC-7 / #32). ✅ COMPLETE → muse#62 (closed): https://staging.musehub.ai/gabriel/muse/issues/62 Five-phase E2E + TDD campaign on task/mwp-7-sibling-negotiation. Phase 1/2 (GT/LF): 9 unit tests confirm the live-push sibling-ref negotiation path is correct (push.py:819-822, branch_have = remote_branch_heads.values()). Phase 3 (VS): real staging push with a 17-commit dev + 1-commit main topology — commits_sent=1, objects_sent=1; clone verified integrity all_ok:true. #55 closed. Phase 4 (DR): fixed dry-run have-set: replaced get_remote_head(push_branch) with _all_known_have_anchors(root) (all sibling tracking refs) — matches live-path logic. Phase 5 (RG): removed c != local_head filter so up-to-date dry-runs report 0 correctly. 21 tests GREEN (pytest -m wire); 85 adjacent push-corpus tests GREEN. #32 closed. Commit: sha256:6ec37c93009f7c30451ea8db62a0a1b764bb79fa93231b5506bb6855251b3e5d.
MWP-8 [musehub] — Cache invalidation for repair and force-push (fixes RC-8). ✅ COMPLETE → musehub#111 (closed): https://staging.musehub.ai/gabriel/musehub/issues/111 Five-phase TDD campaign on task/mwp-8-cache-invalidation. Phase 0 (AU): proved the stale-HIT gap — AU_01..AU_05 go RED before any fix. Phase 1 (IV): added invalidate_fetch_mpack_cache(session, repo_id, tips=None) to musehub_gc.py; deletes rows AND best-effort-deletes their R2 objects regardless of TTL; tips=None → repo-wide (repair path), tips=[...] → tip-scoped (force-push). Does NOT commit — caller owns the transaction (D2). 6 unit tests GREEN (IV_01..IV_06). Phase 2 (RP): wired invalidation into wire_repair_object, wire_repair_snapshot, wire_repair_commit; each calls invalidate_fetch_mpack_cache then _enqueue_repair_prebuild in the same transaction before session.commit(). 4 contract tests GREEN (RP_04..RP_07 incl. atomicity rollback guard). Phase 3 (FP): wired tip-scoped invalidation into wire_push_unpack_mpack force-push path; only old+new tips busted — sibling cache rows survive (D1/D3 design decisions). 4 tests GREEN (FP_02..FP_05 incl. blast-radius and prebuild-still-enqueued proofs). Phase 4 (EX): added _enqueue_repair_prebuild helper; verified cache-bust logged via caplog; EX_02 is the E2E narrative (@pytest.mark.wire): repair → MISS → prebuild → HIT serves fresh content. 3 tests GREEN (EX_01..EX_03). Phase 5 (RG): 54 adjacent corpus tests GREEN; 16 MWP-8 tests collected + GREEN; master tracker (muse#58) updated; musehub#111 closed with audit verdict. 16 tests total in tests/test_mwp8_cache_invalidation.py.

Acceptance criteria

Push C1→C2→C3, clone, push C4, clone again → second clone contains C4 and its blobs (RC-1, RC-2). Codified as a non-skipped regression test. ✅ Fixed by MWP-1 + MWP-2
Two pushes within one prebuild interval both end with a correct, fresh clone cache (RC-3 ✅ fixed by MWP-3, RC-4 ✅ fixed by MWP-4).
muse clone issued immediately after muse push succeeds via bounded retry, never hard-fails on the first 503 (RC-5). ✅ Fixed by MWP-5 (muse#59)
Pushing main when it lags an already-pushed dev sends only the delta (one merge commit, ~zero new blobs); #55 closed with evidence (RC-7). ✅ Fixed by MWP-7
--dry-run commit/object counts equal the real push counts (#32). ✅ Fixed by MWP-7
✅ The previously-skipped wire/mpack test suite is re-enabled and green (RC-6). Fixed by MWP-6.
A repaired object/snapshot/commit is never masked by a stale prebuilt mpack — cache invalidated in the same transaction as the repair; clone issued immediately after repair returns the corrected content (RC-8). ✅ Fixed by MWP-8 (musehub#111)

Out of scope

Any redesign of the mpack binary format or the presigned-upload model — both are sound.
Per-object presign / fetch/objects fallback paths beyond what clone/fetch need for MVP.
Shallow-clone (--depth) hardening beyond current behavior.
Code-intel / structural-intel jobs (independent of correctness of the wire data path).

◎ Activity1

●

gabriel opened this issue 18 days ago

○

gabriel 18 days ago

MWP-1 complete. Commit-graph generations are now authoritative.

musehub#106 shipped in 6 load-bearing phases (13 tests, all GREEN, tests/test_mwp1_generation_authority.py) and is merged to dev + main (sha256:da49a05fd62cda…):

_resolve_generation_with_backfill — on-demand bottom-up generation backfill from musehub_commits (the authoritative DAG) whenever a push hits an unresolved parent.
repair_corrupt_commit_generations — idempotent repair of pre-existing gen-0-with-parents artefacts.
check_commit_graph_invariant — data-integrity gate: no musehub_commit_graph row may have generation 0 with non-empty parents.
[MWP1_13] fetch-side guard — on tip-snapshot mismatch, repair generations in-place before assembling the clone mpack.
MWP1_14 — non-skipped end-to-end push→clone→push→clone regression gate.

MWP-2 filed → musehub#107 (https://staging.musehub.ai/gabriel/musehub/issues/107), assigned to gabriel. RC-2 is the read-side complement to RC-1: _walk_commit_delta still has no safety net when a generation is wrong or a graph row is missing — the generation-bounded range scan truncates and the clone silently drops commits. MWP-2 removes the if True: dead-code guard and promotes the authoritative MusehubCommit.parent_ids DAG walk to a tested fallback behind a free consistency check, so the current walk result is always complete even before MWP-1's repair runs. Plan is multi-phase TDD with symbol anchors, ready for a sonnet 4.6 agent to pick up.

Assignee

gabriel human

Release

no commits linked to this issue

create

muse hub issue create \
  --title "..." \
  --body "..." \
  --label bug \
  --anchor path/to/file.py::Symbol \
  --commit-anchor <sha> \
  --repo gabriel/muse

read

muse hub issue get 58 --json
muse hub issue list --state open --json

update

muse hub issue edit 58 \
  --anchor path/to/file.py::Symbol \
  --repo gabriel/muse

comment

muse hub issue comment 58 \
  --body "Fixed in <sha>" \
  --repo gabriel/muse

muse hub issue close 58 \
  --repo gabriel/muse

create

create_issue({
  repo_id: "sha256:200e8689fe34a831289bc1eca17633b2069d595379b7c2f57a158e35d8291bec",
  title: "...",
  body: "...",
  labels: ["bug"],
  symbol_anchors: [
    "path/to/file.py::Symbol"
  ],
  commit_anchors: ["<sha>"]
})

read

get_issue({
  repo_id: "sha256:200e8689fe34a831289bc1eca17633b2069d595379b7c2f57a158e35d8291bec",
  issue_number: 58
})

list_issues({
  repo_id: "sha256:200e8689fe34a831289bc1eca17633b2069d595379b7c2f57a158e35d8291bec",
  state: "open"
})

update

edit_issue({
  repo_id: "sha256:200e8689fe34a831289bc1eca17633b2069d595379b7c2f57a158e35d8291bec",
  issue_number: 58,
  symbol_anchors: ["path/to/file.py::Symbol"]
})

comment

create_issue_comment({
  repo_id: "sha256:200e8689fe34a831289bc1eca17633b2069d595379b7c2f57a158e35d8291bec",
  issue_number: 58,
  body: "..."
})

close_issue({
  repo_id: "sha256:200e8689fe34a831289bc1eca17633b2069d595379b7c2f57a158e35d8291bec",
  issue_number: 58
})

create

curl -X POST \
  http://localhost:10003/api/repos/sha256:200e8689fe34a831289bc1eca17633b2069d595379b7c2f57a158e35d8291bec/issues \
  -H "Content-Type: application/json" \
  -H "Authorization: MSign handle=\"...\" ts=... sig=\"...\"" \
  -d '{
    "title": "...",
    "body": "...",
    "labels": ["bug"],
    "symbol_anchors": ["path/to/file.py::Symbol"]
  }'

read

# get one issue
curl http://localhost:10003/api/repos/sha256:200e8689fe34a831289bc1eca17633b2069d595379b7c2f57a158e35d8291bec/issues/58

# list open issues
curl "http://localhost:10003/api/repos/sha256:200e8689fe34a831289bc1eca17633b2069d595379b7c2f57a158e35d8291bec/issues?state=open"

update

curl -X PATCH \
  http://localhost:10003/api/repos/sha256:200e8689fe34a831289bc1eca17633b2069d595379b7c2f57a158e35d8291bec/issues/58 \
  -H "Content-Type: application/json" \
  -H "Authorization: MSign handle=\"...\" ts=... sig=\"...\"" \
  -d '{"title": "...", "body": "..."}'

comment

curl -X POST \
  http://localhost:10003/api/repos/sha256:200e8689fe34a831289bc1eca17633b2069d595379b7c2f57a158e35d8291bec/issues/58/comments \
  -H "Content-Type: application/json" \
  -H "Authorization: MSign handle=\"...\" ts=... sig=\"...\"" \
  -d '{"body": "Fixed in <sha>"}'

curl -X POST \
  http://localhost:10003/api/repos/sha256:200e8689fe34a831289bc1eca17633b2069d595379b7c2f57a158e35d8291bec/issues/58/close \
  -H "Authorization: MSign handle=\"...\" ts=... sig=\"...\""

Muse Wire Protocol — MVP hardening (clone/push/fetch/pull master tracking issue)

Muse Wire Protocol — MVP hardening (master tracking issue)

Status update (2026-07-07)

Background

Proximate triggers

Architecture recap (for onboarding)

Confirmed root causes

RC-1 — Generation fallback to 0 silently corrupts the fetch walk (CRITICAL — the clone staleness bug)

RC-2 — _walk_commit_delta has no fallback when the graph is inconsistent

RC-3 — Job idempotency drops fresh payloads on back-to-back pushes

RC-4 — Prebuild can run before index (foundation-job ordering race)

RC-5 — Client does not retry on 503 (clone-immediately-after-push fails)

RC-6 — The wire test suite is largely disabled ✅ Fixed by MWP-6

RC-8 — Repair endpoints and force-push never invalidate musehub_fetch_mpack_cache

RC-7 — #55 (sibling-ref push negotiation): main path fixed, needs verification + dry-run alignment

Goal — definition of MVP done

Proposed sub-tickets

Acceptance criteria

Out of scope

RC-2 — `_walk_commit_delta` has no fallback when the graph is inconsistent

RC-8 — Repair endpoints and force-push never invalidate `musehub_fetch_mpack_cache`