gabriel / muse public
Closed #55 Bug
filed by gabriel human · 18 days ago

push have-negotiation ignores sibling refs — redundant large uploads when branch lags a sibling

0 Anchors
— Blast radius
— Churn 30d
0 Proposals

muse push — have-negotiation ignores sibling refs on remote

Background

When pushing a branch to a remote, the push negotiation computes blobs_to_send by comparing the local tip against the remote's tip for that specific branch only. It does not account for objects already present on the remote via other refs.

Observed behaviour

After pushing dev to staging successfully (17 commits, 46 blobs), pushing main to the same staging remote attempted to upload 1,375 commits and 379 MB of data — failing with [Errno 32] Broken pipe.

Local main is staging/dev + one merge commit. The merge-base of local main and local dev is exactly dev's tip (sha256:f9828e…). All objects are already present on staging from the dev push. Only the merge commit itself needs to go over the wire.

Root cause

The "have" set used during push negotiation is scoped to the target branch's remote tip. Objects reachable from other remote refs (e.g. staging/dev) are not included in the have set, so the delta computation over-counts blobs_to_send by treating already-uploaded objects as missing.

Expected behaviour

blobs_to_send should be computed against the union of all objects reachable from all refs present on the remote — not just the branch being pushed. When main is a direct descendant of dev and dev is already on the remote, the push for main should send only the new commits and their net-new objects (in this case, one merge commit and zero new blobs).

Impact

  • Pushes to branches that lag behind a sibling ref on the same remote become disproportionately large.
  • Large packs time out or hit broken-pipe errors even when no new data is needed.
  • Workaround: push the most-advanced branch first (e.g. dev), then accept that main may still require a large redundant upload until this is fixed.
◎ Activity1
●
gabriel opened this issue 18 days ago
○
gabriel 17 days ago

MWP-7 Phase 3 — Staging E2E evidence: #55 is closed

Verdict: RC-7 is correct. The live path already handles sibling-ref negotiation.

Test setup (VS_01)

Throwaway repo gabriel/mwp7-sibling-verify on staging. Local repo:

  • dev — 17-commit linear chain (each commit adds one file, one blob)
  • main — dev + 1 merge commit (merge-file.txt)

This mirrors the #55 report exactly (the 1375-commit / 379 MB push scenario).

VS_02 — Push dev (full chain)

commits_sent: 17,  objects_sent: 17

Full chain sent, as expected (remote starts empty).

VS_03 — Push main after dev is already on remote (the #55 scenario)

[PUSH step 0] GET /refs → branch_heads = {'dev': 'sha256:72083e80...'} (main absent)
[PUSH step 0]   have = ['sha256:72083e80...']
[PUSH step 1]   new_commits = 1
[PUSH step 2] have_blobs=17  manifest_blobs=18  blobs_to_send=1
[PUSH step 3]   { commits: 1, snapshots: 1, blobs: 1 }
[PUSH step 7] → { blobs_written=1, commits_written=1 }

JSON result:

{ "status": "pushed", "commits_sent": 1, "objects_sent": 1 }

1375 → 1. The live path reads all remote branch heads (including dev) into the have set at push.py:819-822, so walk_commits prunes at dev's tip and sends only the merge commit. The broken-pipe is gone.

VS_04 — Negative control

Added a genuinely new commit (nc-file.txt) to main and pushed:

{ "status": "pushed", "commits_sent": 1, "objects_sent": 1 }

Exactly 1 new commit + 1 new blob — no over-sending, no under-sending.

VS_05 — Clone + integrity check

{ "all_ok": true, "commits_checked": 19, "objects_checked": 19, "failures": [] }

Delta-only push produced a complete, verifiable repo. All 19 commits (17 dev + merge + neg-control) and 19 objects intact — no missing blobs.

#55 is empirically closed. Regression tests LF_01 / LF_02 in tests/test_mwp7_sibling_negotiation.py lock this behaviour permanently.

✓
closed this issue 17 days ago