MWP-7 [muse] — Verify & close #55 (sibling-ref push negotiation); align --dry-run to the live push path (#32)

Master tracker: muse#58 — Muse Wire Protocol MVP hardening. Repo target: muse (the push client). No musehub server changes. Closes on completion: muse#55, muse#32. Predecessors (all done): MWP-1..MWP-6. The wire regression suite is green (pytest -m wire) and the MWP6_00 anti-skip gate is live.

This ticket is fully spec'd for another agent to execute. Every deliverable carries a test ID and a concrete code anchor (file.py::Symbol or file.py:line). Read the Code Intelligence Appendix at the bottom first — it lists every symbol you will touch and the exact muse code command to read it.

IMPORTANT — this project uses Muse (not git) for version control.

• Never run git, gh, or any git subcommand. Repo paths: ~/ecosystem/muse, ~/ecosystem/musehub, ~/ecosystem/agentception.
• Use muse code grep / symbols / cat / impact / deps before reading files directly. Every command accepts --json / -j.
• Use muse -C ~/ecosystem/muse <command> when CWD differs from the target repo.
• Commit with full provenance: muse commit -m "..." --agent-id claude-code --model-id claude-sonnet-4-6 --sign.
• Branch first; never commit to dev/main. Never merge to dev/main without gabriel's explicit permission.

Background

The two open bugs this ticket owns

#55 — push have-negotiation ignores sibling refs. Observed: after pushing dev to staging (17 commits, 46 blobs), pushing main (which is dev + one merge commit) attempted to upload 1,375 commits / 379 MB and died with [Errno 32] Broken pipe. The merge-base of local main and local dev is exactly dev's tip — every object is already on the remote via dev. Only the single merge commit needed to go over the wire. The have set was scoped to the target branch's remote tip, so objects reachable via sibling refs (dev) were treated as missing.

#32 — push --dry-run overcounts commits_sent. Observed: push local main --force-with-lease --dry-run reported commits_sent: 1271; the live push sent commits_sent: 0. The dry-run walks want − {remote_branch_tip} only — it does not simulate the full have negotiation the live push performs over all sibling refs. An agent or human seeing 1271 before a push will reasonably stop and investigate — eroding trust in dry-run output.

What the live push path looks like today (the RC-7 claim)

The master tracker's RC-7 asserts the live push path was already repaired for #55, and that MWP-7 is therefore "verify + close + align dry-run." The code does look intentional:

• Commit-walk boundary — push.py:819-822 (inside push.py::run) builds branch_have from all remote_branch_heads.values():

  branch_have: list[str] = [
      h for h in remote_branch_heads.values()
      if _is_valid_commit_id(h)
  ]
  

  passed into _push_mpack(..., branch_have=branch_have) → walk_commits(root, [local_head], have=branch_have).

• Blob dedup — mpack.py::walk_commits (mpack.py:330-336) unions every have-commit's full snapshot manifest into have_blobs, then blobs_to_send = manifest_blobs − have_blobs (mpack.py:362). Because Muse snapshots are flat full-repo manifests, a sibling tip's manifest contains every blob reachable at that tip — so subtracting it removes everything already on the remote via that sibling.

• have for objects — push.py:749-752 is likewise the union of all remote heads.

So someone wrote code intending to close #55. But #55 is still open. We do not actually know whether (a) the fix is complete and was simply never verified+closed, or (b) the fix is partial (e.g. a residual over-count in a force-with-lease or diverged-history edge). This ticket does not assume RC-7 is correct — it establishes ground truth empirically first, then branches.

What is NOT yet aligned — the dry-run path

The dry-run block in push.py::run (push.py:663-692) is inconsistent with the live path and with itself:

if dry_run:
    have: list[str] = [                                   # ← objects: ALL tracking refs
        c for c in _all_known_have_anchors(root)
        if c != local_head and commit_exists(root, c)
    ]
    dry_branch_have: list[str] = []                        # ← commits: ONLY target branch tip
    _cached = get_remote_head(remote, push_branch, root)
    if _cached and commit_exists(root, _cached):
        dry_branch_have = [_cached]
    dry_walk = walk_commits(root, [local_head], have=dry_branch_have)
    dry_commits = len(dry_walk["commits"])                 # ← over-counts (the #32 bug)
    dry_objects = len(collect_blob_ids(root, [local_head], have=have))

• dry_commits walks back to only get_remote_head(remote, push_branch) — the target branch's cached tip. Sibling refs are ignored → the #32 over-count.
• dry_objects already uses all tracking refs (_all_known_have_anchors). So commits and objects in the same dry-run use different have sets. The object count is already sibling-aware; the commit count is not.

The fix is to make dry_commits use the same all-sibling-refs anchor set as dry_objects, so dry-run mirrors the live path's negotiation logic.

Architecture recap — the push negotiation, end to end

muse push (push.py::run, entry registered by push.py::register) negotiates in two phases. Read these symbols before touching anything (see Appendix):

push.py::_push_mpack (push.py:192):

commit_walk = walk_commits(root, [local_head], have=branch_have)   # commit boundary
... blobs_to_send = commit_walk["all_blob_ids"]                    # already net of have_blobs
returns (PushResult, commits_sent, objects_sent)

mpack.py::walk_commits (mpack.py:303) is the single source of truth for what enters a push mpack:

• iter_ancestors(repo_root, commit_ids, prune=lambda cid: cid in have_set) — BFS stops at any have-commit.
• have_blobs = union of every have-commit's full flat manifest (mpack.py:330-336).
• blobs_to_send = manifest_blobs − have_blobs (mpack.py:362).

Key invariant for this ticket: walk_commits takes a single flat have list and uses it for both the commit-prune boundary and the blob subtraction. There is no separate "commit-have" vs "blob-have". This is why passing all sibling heads as have fixes both dimensions of #55 at once.

Goal — definition of MVP done for MWP-7

1. #55 is empirically resolved and closed with evidence. Pushing main when it lags an already-pushed sibling dev on the same remote sends only the delta — the merge commit and zero (or near-zero) net-new blobs — never the full history.
2. --dry-run mirrors the live negotiation. For any branch, push --dry-run's commits_sent / objects_sent equal what the live push actually sends, using the same all-sibling-refs have logic. #32 is closed.
3. Both are codified as non-skipped regression tests wired into the suite (and, where the wire marker applies, into pytest -m wire).
4. No regression in the existing push test corpus (≈20 test_push_* / test_*push* files).

Phases — TDD-first, load-bearing, branch-on-result

Execution rule: each phase must be fully green before the next begins. Write the test first (red), then implement to green. Test IDs appear in both this plan and the test file. Phase 1 is the gate that decides Phase 2's shape — do not pre-judge it.

Phase 1 — Reproduce & establish ground truth (the gate)

Build the #55 scenario as a deterministic, in-process reproduction and find out, with evidence, whether the current live path already sends only the delta. This phase writes no production code — it only adds tests and records the verdict.

Scenario under test (the #55 shape):

remote has:  dev → tip D   (a linear chain root..D, full manifests on remote)
local has:   dev → D   AND   main → M, where M = merge commit with parent D (main = dev + 1)
push main →  remote_branch_heads = {"dev": D}     (main absent on remote, or at an old tip)
expected:    commits_sent == 1 (just M),  objects_sent == (only M's net-new blobs, ~0)

New test file: tests/test_mwp7_sibling_negotiation.py (marker: pytest.mark.wire).

Deliverables:

• [x] GT_01 — Helper builders in the test module: _chain(root, n) (linear commit chain returning ids), _merge_commit(root, parent, content) (a commit whose manifest = parent's manifest + 1 changed/added file). Reuse the patterns in tests/test_push_branch_have.py::_commit / _build_linear_chain and tests/test_push_have_filter.py::_make_commit rather than inventing new ones.
• [x] GT_02 — Commit-boundary truth. Assert len(walk_commits(root, [M], have=[D])["commits"]) == 1. Proves the commit walk stops at the sibling tip D. (Mirrors test_push_branch_have.py::test_BH2, but framed as the #55 merge-commit case.)
• [x] GT_03 — Blob-dedup truth. Assert walk_commits(root, [M], have=[D])["all_blob_ids"] contains only M's net-new blob(s) and none of D's manifest blobs. Directly exercises have_blobs subtraction (mpack.py:330-362) for the flat-manifest sibling case.
• [x] GT_04 — run()-level live path truth. Invoke push.py::run against a mocked transport whose _fetch_remote_info_safe returns branch_heads = {"dev": D} (main absent). Capture the branch_have and have actually passed into _push_mpack (spy pattern from test_push_branch_have.py::test_BH5). Assert both contain D, and assert the reported commits_sent == 1. This is the in-process proxy for the staging E2E.
• [x] GT_05 — Diverged / force edge. Repeat GT_04 with main present on the remote at an older, diverged tip M_old (not an ancestor of M). Record actual commits_sent. This is the most likely place a residual #55 bug hides (the up-to-date short-circuit at push.py:789 and the have/remote_head split at push.py:749-757).
• [x] GT_06 — Verdict record. Add a module docstring section GROUND TRUTH VERDICT: that states, based on GT_02..GT_05 outcomes, whether the live path is already correct (all green) or partially broken (which assertion is red and why). This verdict selects Phase 2's mode.

Phase 1 exit gate: GT_01..GT_06 committed; verdict recorded in the issue (post a comment, do not edit this body) and in the test module docstring.

Phase 2 — Complete the live-path fix (conditional on Phase 1's verdict)

This phase has two modes. Phase 1 decides which one you execute. Document the chosen mode in the commit message and an issue comment.

Mode A — verdict "already correct" (expected, per RC-7)

No production change to the live path. Convert the Phase-1 proxies into the permanent contract:

• [x] LF_01 — Promote GT_04 to an explicit regression assertion named test_LF_push_main_after_dev_sends_only_merge_commit, with a docstring citing #55 and the observed 1375 → 1 reduction. Keep it in tests/test_mwp7_sibling_negotiation.py.
• [x] LF_02 — Add test_LF_no_net_new_blobs_when_sibling_has_all_objects asserting objects_sent == 0 for the pure-merge case (merge commit references only blobs already in D's manifest). If a merge legitimately adds a blob, parametrize for == K.
• [x] LF_03 — Run the adjacent corpus green: tests/test_push_branch_have.py, tests/test_push_have_filter.py, tests/test_push_step1_dag_walk.py, tests/test_cmd_push_hardening.py (one file at a time, python3 -m pytest <file> -q --tb=short). Zero regressions.

Mode B — verdict "partially broken" (only if a GT_* is red)

Implement the minimal fix at the exact failing anchor. Most-likely sites, in priority order:

• [ ] LF_10 — If branch_have is wrong: fix push.py:819-822 so the commit-walk boundary is the validated union of all remote_branch_heads.values() (not just the target tip).
• [ ] LF_11 — If have (blob dedup) is wrong: fix push.py:749-752 so the blob-have anchor is the same validated union. (Both have and branch_have must reference the same set — see the walk_commits single-flat-have invariant above.)
• [ ] LF_12 — If the up-to-date / diverged short-circuit mis-fires (push.py:782-808): correct the remote_head selection so a sibling-covered push is not mis-reported.
• [ ] LF_13 — Make every red GT_* green. Then run the Mode-A LF_01..LF_03 deliverables (the regression assertions apply regardless of mode).
• [ ] LF_14 — muse code impact "muse/cli/commands/push.py::run" --json and muse code impact "muse/core/mpack.py::walk_commits" --json to confirm the blast radius; run any additional impacted tests surfaced.

Phase 2 exit gate: all GT_* green; LF_* (chosen mode) green; adjacent push corpus green.

Phase 3 — Verify against staging & close #55 with evidence

The in-process tests prove the logic; #55 is a staging bug, so it closes on staging evidence. This phase runs a real push to https://staging.musehub.ai.

Use a throwaway repo so we never disturb a real one. muse hub repo create --no-init so the remote starts empty and the first push is a clean ff-from-null (the same --no-init trick MWP-6 used for its E2E fixtures).

Procedure (record exact commits_sent / objects_sent from each push):

• [x] VS_01 — Create gabriel/mwp7-sibling-verify on staging via muse -C ~/ecosystem/muse hub repo create --name mwp7-sibling-verify --no-init --hub https://staging.musehub.ai --json. Add the remote; build a local repo with dev = linear chain (e.g. 17 commits) and main = dev + one merge commit (mirror the #55 report's shape).
• [x] VS_02 — muse push staging dev → record counts (expect full chain on first push).
• [x] VS_03 — muse push staging main → assert from the [PUSH step 1] new_commits / [PUSH step 3] stderr telemetry and the JSON envelope that commits_sent == 1 and objects_sent is ~0 (only the merge commit). This is the literal #55 acceptance: 1375 → 1. Capture the full stderr [PUSH step *] trace as evidence.
• [x] VS_04 — Negative control: add a genuinely new commit with a new blob on main, push, confirm counts reflect exactly that one commit + one blob (proves we didn't over-correct into under-sending).
• [x] VS_05 — Clone the staging repo into a fresh dir and muse verify --json → the working tree is complete and correct (no missing blobs from the delta-only push). Ties #55 back to the MWP-1/2 clone-correctness guarantees.
• [x] VS_06 — Tear down: muse hub repo delete the throwaway repo.
• [x] VS_07 — Close #55: post a comment on #55 with the before/after counts and the [PUSH step *] evidence, then muse hub issue update 55 --status closed --hub https://staging.musehub.ai. (Comment separately; never edit the original description.)

Phase 3 exit gate: staging evidence captured; #55 closed with a comment citing the numbers.

Phase 4 — Align --dry-run to the verified live path (#32)

Now that the live path is the confirmed reference, make dry-run mirror it. Write the failing test first.

New tests: add to tests/test_mwp7_sibling_negotiation.py (or a sibling tests/test_mwp7_dry_run_alignment.py — pick one and note it).

• [x] DR_01 — Red repro of #32. Build the same dev/main repo with tracking refs written for the remote (.muse/remotes/<remote>/dev = D). Invoke push <remote> main --dry-run --json. Assert commits_sent == 1 (matching the live path). This must be RED against current code (today it walks back to only main's cached tip and over-counts). Mirror the #32 report: assert the current wrong value in a comment, the correct value in the assertion.
• [x] DR_02 — Red repro, object dimension. Assert dry-run objects_sent equals the live net-new blob count for the same scenario. (Objects already use _all_known_have_anchors, so this may pass today — if so, document it as already-aligned and keep the assertion as a guard.)
• [x] DR_03 — Fix. In push.py:663-692, replace the target-branch-only dry_branch_have with the same all-tracking-refs anchor set already computed for have (_all_known_have_anchors(root)), so dry_commits and dry_objects negotiate against the identical set: python if dry_run: have = [c for c in _all_known_have_anchors(root) if c != local_head and commit_exists(root, c)] dry_walk = walk_commits(root, [local_head], have=have) # ← same set as objects dry_commits = len(dry_walk["commits"]) dry_objects = len(collect_blob_ids(root, [local_head], have=have)) Delete the now-dead dry_branch_have / _cached / get_remote_head block.
• [x] DR_04 — DR_01 / DR_02 now green.
• [x] DR_05 — Preserve the offline contract. tests/test_cmd_push_hardening.py:: ...test_dry_run_makes_no_http_calls (≈L544) must stay green — the recommended fix keeps dry-run offline (uses local tracking refs, no GET /refs). Run it explicitly.
• [x] DR_06 — Update the dry-run docstring/comment in push.py::run (the "no network calls" comment at push.py:662) to state precisely: dry-run negotiates against the local tracking-ref mirror of remote heads — it equals the live push when tracking refs are fresh (i.e. after a muse fetch); stale tracking refs may still drift. (See Design Decision D1.)

Phase 4 exit gate: DR_01..DR_06 green; test_dry_run_makes_no_http_calls green.

Phase 5 — Close #32, wire regression into the suite, finalize

• [ ] RG_01 — End-to-end #32 scenario (the literal report). A test that fast-forwards local main to match a remote dev already in the tracking refs and asserts push --dry-run main reports commits_sent: 0 — the exact 1271 → 0 case from #32. Map to #32's VL_01.
• [ ] RG_02 — Non-zero control. Genuinely new commits → dry-run reports the correct non-zero count (maps to #32 VL_02).
• [ ] RG_03 — Suite wiring. Confirm tests/test_mwp7_sibling_negotiation.py (and any dry-run sibling file) carry pytest.mark.wire and appear under pytest -m wire. Confirm the MWP6_00 gate (tests/test_mwp6_wire_suite_enabled.py) still passes (no flux-skip introduced).
• [ ] RG_04 — muse code test --json on the changed files; full Phase-2 adjacent corpus green one more time.
• [ ] RG_05 — Close #32: comment on #32 with the dry-run-vs-live parity evidence, then muse hub issue update 32 --status closed --hub https://staging.musehub.ai.
• [ ] RG_06 — Update master tracker muse#58: tick the two open acceptance criteria — "Pushing main when it lags an already-pushed dev sends only the delta; #55 closed with evidence (RC-7)" and "--dry-run commit/object counts equal the real push counts (#32)" — and mark MWP-7 ✅ COMPLETE in the sub-tickets list. (Read the local source file first, edit locally, push --body-file — never cowboy-push.)
• [ ] RG_07 — Merge prep: ensure all work is on the feature branch task/mwp-7-sibling-negotiation; stop there and hand back to gabriel for the dev/main merge (do not self-merge).

Phase 5 exit gate: #55 and #32 closed; muse#58 updated; all MWP-7 tests green and wired.

Branching & commits

muse -C ~/ecosystem/muse checkout dev
muse -C ~/ecosystem/muse checkout -b task/mwp-7-sibling-negotiation \
  --intent "verify+close #55 sibling-ref push negotiation; align --dry-run to live (#32)" \
  --resumable
# ... per-phase commits ...
muse -C ~/ecosystem/muse commit -m "test(mwp7/phase1): ground-truth #55 sibling negotiation (GT_01..GT_06)" \
  --agent-id claude-code --model-id claude-sonnet-4-6 --sign

One commit per phase minimum; test-first within each. Do not merge to dev/main — that is gabriel's call.

Acceptance criteria (the whole-ticket gate)

• [ ] GT_01..GT_06 codify the #55 scenario and record an explicit ground-truth verdict.
• [ ] Live path proven (Mode A) or fixed (Mode B): pushing main that lags sibling dev sends only the merge commit and ~zero net-new blobs (LF_*).
• [ ] Staging evidence: VS_03 shows commits_sent == 1 (the literal 1375 → 1); VS_05 clone verifies complete. #55 closed with a comment citing the numbers.
• [ ] --dry-run commits_sent / objects_sent equal the live push for the sibling-lag scenario (DR_*, RG_01..RG_02), with the offline contract preserved (DR_05). #32 closed with parity evidence.
• [ ] All MWP-7 tests carry pytest.mark.wire, run green under pytest -m wire, and the MWP6_00 anti-skip gate stays green (RG_03).
• [ ] Zero regressions across the existing test_push_* / test_*push* corpus.
• [ ] muse#58's two open RC-7/#32 acceptance criteria ticked; MWP-7 marked ✅ COMPLETE.

Design decisions

D1 — Dry-run stays offline. DECIDED (gabriel, 2026-06-29). #32's stated fix (FX_01) says dry-run should fetch all remote heads (GET /refs, like live step 0). We deliberately do not do that. Dry-run is a local prediction: it negotiates against the local tracking-ref mirror of remote state (_all_known_have_anchors) — the same model the live push's --force-with-lease safety check already trusts, and the same mental model as git push --dry-run (which also never fetches). Making dry-run silently hit the network would break the documented "no network calls" contract, the existing contract test test_cmd_push_hardening.py::...test_dry_run_makes_no_http_calls, and the user expectation that dry-run is cheap and local.

The fix therefore aligns the negotiation logic (use all sibling refs, not just the target branch) while keeping dry-run offline. Consequence — stated plainly in the UX via DR_06: dry-run counts equal the live push when tracking refs are fresh (i.e. after a muse fetch); stale tracking refs may drift, exactly as the live --force-with-lease lease can be stale. This is the correct, principled behaviour — not a compromise. The network-fetch variant is explicitly rejected and out of scope.

D2 — One flat have set, not two. walk_commits uses a single have list for both the commit-prune boundary and the blob subtraction. Do not introduce a separate "blob-have" — the flat-manifest model makes the single set correct for both. The live path's two variables (have, branch_have) currently hold the same logical set; keep them consistent (or unify) rather than letting them diverge.

Out of scope

• Any change to the mpack binary format, the presigned-upload model, or the server (musehub) negotiation. MWP-7 is client-side only.
• The clone-staleness / generation pipeline (RC-1..RC-5) — done in MWP-1..MWP-5.
• Cache-invalidation audit (RC-8 / MWP-8) — separate ticket.
• Shallow-clone (--depth) negotiation.
• Re-architecting --dry-run into a live-parity network call — explicitly rejected (see D1).

Code Intelligence Appendix — symbol anchors

Read these before editing. Every anchor is a muse code target.

Primary edit sites

Reference / dependency reads

Blast-radius checks (run before/after editing)

muse code impact "muse/cli/commands/push.py::run" --json
muse code impact "muse/core/mpack.py::walk_commits" --json
muse code deps   "muse/cli/commands/push.py" --json

Existing tests to read (don't reinvent fixtures)

Test run discipline

muse -C ~/ecosystem/muse code test --json                 # only tests for changed files
python3 -m pytest tests/test_mwp7_sibling_negotiation.py -q --tb=short
python3 -m pytest -m wire -q --tb=short                    # the wire suite gate
# NEVER: python3 -m pytest tests/   (whole-suite run is forbidden)