# MWP QA — Verifying the Muse Wire Protocol (clone / push / fetch / pull) ## Background The Muse Wire Protocol (MWP) is how `muse clone`, `muse push`, `muse fetch`, and `muse pull` move commits, snapshots, and blobs between your machine and a MuseHub server. It's functionally equivalent to git's smart-HTTP protocol, rebuilt on a binary packfile format (`mpack`) with content-addressed objects and presigned uploads direct to object storage (R2 on staging/prod, MinIO on localhost). This mist is a copy-paste-able verification pass for anyone picking up work against `https://staging.musehub.ai` for the first time. Run it top to bottom once when you first get access — if every step below matches its expected result, the wire protocol is working correctly for you and you're clear to start real work. If anything deviates from the expected result, stop and report it (don't work around it silently) — see **Troubleshooting** at the bottom. This guide was produced from [muse#63](https://staging.musehub.ai/gabriel/muse/issues/63), a full black-box verification pass of the wire protocol MVP against real staging, run after closing [muse#58](https://staging.musehub.ai/gabriel/muse/issues/58)'s 8 sub-tickets (MWP-1..MWP-8). Every recipe below was actually executed against staging or localhost and its exact result recorded (with `` substituted back in for whichever handle actually ran it) — this is not speculative documentation. Wherever you see ``, substitute your own handle or a unique scratch-repo suffix — every recipe below uses it consistently for both the repo-name suffix and the owner segment of the URL, since they're the same value once you're authenticated as yourself. --- ## Prerequisites **1. Install the CLI:** ```bash curl -fsSL https://staging.musehub.ai/install.sh | sh muse --version ``` **2. Register your identity** (Ed25519 keypair, no password/JWT): ```bash muse auth keygen --hub https://staging.musehub.ai muse auth register --hub https://staging.musehub.ai --handle muse auth whoami ``` `keygen` stores a BIP-39 mnemonic in your OS keychain and never prints it to the terminal. Back it up immediately: ```bash # macOS security find-generic-password -s muse -a mnemonic -w | pbcopy # paste into your password manager ``` Losing the mnemonic means permanent loss of your derived keys. --- ## Architecture at a glance | Command | What it does | Server endpoint | |---|---|---| | `muse push` | Uploads only the commits/blobs the remote doesn't already have | `POST /push/mpack-presign` → PUT to object storage → `POST /push/unpack-mpack` | | `muse clone` | Downloads a complete repo — there's no separate clone endpoint, it's a fetch with `have=[]` | `POST /fetch/mpack` | | `muse fetch` | Downloads new commits/blobs into your local repo, without touching your working tree or branch pointer | `POST /fetch/mpack` | | `muse pull` | `fetch` + fast-forward merge (or a reported conflict if history diverged) | `POST /fetch/mpack` (then local merge) | Objects are content-addressed (`sha256:` is the integrity check), so `push`/`fetch` only ever transfer the true delta — you'll see this verified concretely in Step 2 below. **A convenience worth knowing up front:** `muse init` auto-seeds three remotes — `local`, `staging`, `production` — pointing at `//`. So as long as the hub repo you create has the **same name as your local directory**, you never need `muse remote add` at all — `muse push staging main` just works out of the box. All recipes below rely on this; if you name your hub repo differently from your directory, you'll need `muse remote set-url staging ` first. `--json` output below is piped through `jq` for readability — that's a convention, not a requirement; drop `| jq` any time you want raw output. --- ## Quick Start — 5 minute smoke test Confirms all 4 verbs work end to end against staging. ```bash # Create and push a brand-new repo — directory name and hub repo name must # match so the auto-seeded 'staging' remote resolves correctly cd /tmp && rm -rf mwp-smoke- && mkdir mwp-smoke- && cd mwp-smoke- muse init --json | jq ``` ```json { "muse_version": "0.2.0rc15", "schema": 1, "exit_code": 0, "duration_ms": 1.945, "warnings": [], "status": "ok", "repo_id": "sha256:ff51dfb3547e43fdfde6d080e59e7b25047c1a72e3bdebf8c140d8c6f1c7c6b8", "branch": "main", "domain": "code", "path": "/private/tmp/mwp-smoke-/.muse", "reinitialized": false, "bare": false, "remotes": { "local": "https://localhost:1337//mwp-smoke-", "staging": "https://staging.musehub.ai//mwp-smoke-", "production": "https://musehub.ai//mwp-smoke-" } } ``` ```bash echo "hello" > README.md && muse code add README.md muse commit -m "init" --agent-id --model-id manual muse hub repo create --name mwp-smoke- --no-init --hub https://staging.musehub.ai --json | jq ``` ```json { "muse_version": "0.2.0rc15", "schema": 1, "exit_code": 0, "duration_ms": 451.494, "warnings": [], "repo_id": "sha256:932c532eb980f812828f2822f512b21d8821403b28f992206497e413e0d59dc4", "name": "mwp-smoke-", "owner": "", "slug": "mwp-smoke-", "visibility": "public", "default_branch": "main", "clone_url": "https://staging.musehub.ai//mwp-smoke-", "url": "https://staging.musehub.ai//mwp-smoke-" } ``` ```bash muse push staging main --json | jq ``` ```json { "muse_version": "0.2.0rc15", "schema": 1, "exit_code": 0, "duration_ms": 2142.301, "warnings": [], "status": "pushed", "remote": "staging", "branch": "main", "head": "sha256:d05f7e8bdfeee15f79635cd962fee48de3ab7967c5c0bc706385a567f725318e", "commits_sent": 1, "objects_sent": 1, "force": false, "dry_run": false } ``` **Expect:** `"status": "pushed"`, `exit_code: 0`. (No `muse remote add` step — `staging` was already configured by `muse init`.) ```bash # Clone it back down cd /tmp && rm -rf mwp-smoke-clone muse clone https://staging.musehub.ai//mwp-smoke- mwp-smoke-clone --json | jq ``` ```json { "muse_version": "0.2.0rc15", "schema": 1, "exit_code": 0, "duration_ms": 1418.982, "warnings": [], "status": "cloned", "url": "https://staging.musehub.ai//mwp-smoke-", "directory": "/private/tmp/mwp-smoke-clone", "branch": "main", "commits_received": 1, "blobs_written": 1, "skipped_blobs": 0, "head": "sha256:d05f7e8bdfeee15f79635cd962fee48de3ab7967c5c0bc706385a567f725318e", "dry_run": false, "shallow_commits": [] } ``` ```bash cat mwp-smoke-clone/README.md ``` **Expect:** `"status": "cloned"`, `exit_code: 0`, `README.md` contains `hello`. ```bash # Push a second commit, then fetch + pull it into the clone cd /tmp/mwp-smoke- echo "v2" >> README.md && muse code add README.md muse commit -m "v2" --agent-id --model-id manual muse push staging main --json | jq cd /tmp/mwp-smoke-clone muse fetch origin --branch main --json | jq # downloads the new commit, does NOT update your working tree cat README.md # still shows only "hello" — fetch != merge, this is correct muse pull origin main --json | jq # fetch + fast-forward merge cat README.md # now shows "hello" and "v2" ``` **Expect:** after `fetch`, the working tree is unchanged (proves `fetch` and `pull` are genuinely different operations, not aliases). After `pull`, `README.md` shows both lines. If all of the above matches, clean up and move to the detailed recipes below only if you want deeper confidence, or skip straight to real work: ```bash cd /tmp/mwp-smoke- && muse hub repo delete --hub https://staging.musehub.ai --yes rm -rf /tmp/mwp-smoke- /tmp/mwp-smoke-clone ``` **Careful:** `muse hub repo delete` with no target resolves the repo from your *current directory's* remote config — always run it from inside the repo you mean to delete, or pass the name explicitly (`muse hub repo delete / --yes`) if you're elsewhere. --- ## Detailed verb-by-verb recipes Each recipe below was run against real staging as part of muse#63's verification pass, with the exact result recorded. Use `` as a placeholder for your handle / a unique scratch-repo suffix throughout, so your scratch repos don't collide with anyone else's. ### `muse push` — uploads only the true delta ```bash cd /tmp && rm -rf mwp-qa-push- && mkdir mwp-qa-push- && cd mwp-qa-push- muse init --json | jq echo "base" > file.txt && muse code add file.txt muse commit -m "base" --agent-id --model-id manual muse hub repo create --name mwp-qa-push- --no-init --hub https://staging.musehub.ai --json | jq muse push staging main --json | jq '{objects_sent, commits_sent}' ``` **Expect:** first push sends `commits_sent: 1`, `objects_sent: 1` (the one new blob). ```bash # Push again with no changes — should send nothing muse push staging main --json | jq '{objects_sent, commits_sent}' ``` **Expect:** `commits_sent: 0` (nothing new; muse detects the remote is already up to date and no-ops cleanly, not an error). ```bash # Two branches sharing a base — the second branch's push should exclude # blobs already reachable via the first branch's tip muse checkout -b feature-a && echo "fa" > fa.txt && muse code add fa.txt muse commit -m "feature a" --agent-id --model-id manual muse push staging feature-a --json | jq muse checkout main && muse checkout -b feature-b && echo "fb" > fb.txt && muse code add fb.txt muse commit -m "feature b" --agent-id --model-id manual muse push staging feature-b --json | jq '.objects_sent' ``` **Expect:** `objects_sent: 1` — only `fb.txt`'s blob, not `file.txt`'s (which the remote already has via both `main` and `feature-a`). ### `muse clone` — always a complete, correct working tree ```bash cd /tmp && rm -rf mwp-qa-clone- time muse clone https://staging.musehub.ai//mwp-qa-push- mwp-qa-clone- --json | jq ``` **Expect:** `"status": "cloned"`, `exit_code: 0`. First clone of a repo right after it was created may take a few seconds longer (server builds a prebuilt packfile in the background) — if the server isn't ready yet, `muse clone` automatically retries with backoff; you'll see a `⏳ remote preparing fetch data...` message rather than an error, and it converges on its own. ```bash # Push then clone again immediately — this exact sequence was once a real bug # (musehub#113/MWP-1) where the clone could silently miss the newest commit cd /tmp/mwp-qa-push- muse checkout main # the push recipe above leaves you on feature-b — switch back echo "v2" >> file.txt && muse code add file.txt muse commit -m "v2" --agent-id --model-id manual muse push staging main --json | jq cd /tmp && rm -rf mwp-qa-clone-2- muse clone https://staging.musehub.ai//mwp-qa-push- mwp-qa-clone-2- --json | jq diff /tmp/mwp-qa-push-/file.txt /tmp/mwp-qa-clone-2-/file.txt ``` **Expect:** `diff` produces no output (files identical, `v2` line present in both). ### `muse fetch` — downloads without touching your working tree ```bash cd /tmp && rm -rf mwp-qa-fetch-x- mwp-qa-fetch-y- muse clone https://staging.musehub.ai//mwp-qa-push- mwp-qa-fetch-x- --json | jq muse clone https://staging.musehub.ai//mwp-qa-push- mwp-qa-fetch-y- --json | jq cd /tmp/mwp-qa-fetch-x- echo "from-x" >> file.txt && muse code add file.txt muse commit -m "from x" --agent-id --model-id manual muse push origin main --json | jq cd /tmp/mwp-qa-fetch-y- muse fetch origin --branch main --json | jq cat file.txt # should NOT show "from-x" yet ``` **Expect:** `fetch` exits 0 and downloads the new commit's data, but your local branch pointer and working tree are untouched — `file.txt` still shows the old content. This is the correct, intentional difference between `fetch` and `pull`. ### `muse pull` — fetch + merge, and what happens on a real conflict ```bash cd /tmp/mwp-qa-fetch-y- muse pull origin main --json | jq cat file.txt # NOW shows "from-x" ``` **Expect:** `"status": "fast_forward"`, working tree converges. **Conflict case** — if two people edit the same file on diverging branches, a `pull` will not silently pick a side or corrupt anything — it stops and surfaces the conflict: ```bash # From two independent clones, each commits a different change to the same line, # then the second one to pull sees: muse pull origin main --json | jq # {"status": "conflict", "exit_code": 1, "conflict_paths": ["file.txt"], ...} muse conflicts --json | jq # inspect both sides before resolving ``` If you ever see `"status": "conflict"`, do not run `muse checkout --ours` or `--theirs` blindly — read both sides (`muse conflicts --json`, then open the file and read the `<<<<<<< ours` / `======= theirs` markers) and resolve manually with `muse resolve `, then commit. --- ## Advanced: repeated push/clone loop (confidence check) If you want extra confidence before relying on this for real work, this loop mirrors what was run 10 times against staging with zero staleness and flat upload size on every iteration: ```bash cd /tmp && rm -rf mwp-qa-loop- && mkdir mwp-qa-loop- && cd mwp-qa-loop- muse init --json | jq muse hub repo create --name mwp-qa-loop- --no-init --hub https://staging.musehub.ai --json | jq for i in $(seq 1 5); do echo "iteration $i" >> log.txt muse code add log.txt muse commit -m "iter $i" --agent-id --model-id manual muse push staging main --json | jq "{i: $i, objects_sent}" rm -rf /tmp/mwp-qa-loop-clone-$i muse clone https://staging.musehub.ai//mwp-qa-loop- /tmp/mwp-qa-loop-clone-$i --json | jq tail -1 /tmp/mwp-qa-loop-clone-$i/log.txt done ``` **Expect:** `objects_sent` stays `1` every iteration (never grows), and each clone's last line always matches that iteration's commit — no staleness, no manual cache-busting needed at any point. **A ~30s wait on some iterations is normal, not a hang.** Against real staging, cloning immediately after a push can hit the server's async prebuild job before it's finished — you'll see `⏳ remote preparing fetch data (server busy, attempt 1); retrying in 30s …` on stderr, then the clone completes correctly once the retry lands. This is expected roughly half the time in a tight push→clone loop on staging (localhost doesn't show this, since there's no real prebuild latency to race against). It always converges well within the 120s retry budget — if you ever see `❌ Remote still preparing clone data after 120s`, that's the one case worth reporting, not the 30s wait itself. --- ## Troubleshooting | Symptom | Meaning | Fix | |---|---|---| | `⏳ remote preparing fetch data (server busy, attempt N); retrying in 30s …` | Server hasn't finished building the fetch cache for a very recent push yet | Normal — the client retries automatically for up to 120s. If it eventually fails with `❌ Remote still preparing clone data after 120s`, wait a bit and retry manually; if it happens repeatedly, report it. | | `❌ Repository not found on remote` (push) | The repo doesn't exist on the hub yet | `muse hub repo create --name --no-init --hub --json`, then retry the push. | | `Remote '' is not configured` | `muse clone` always names the remote `origin`; a manually-added remote keeps whatever name you gave it | `muse remote --json` to see configured names, or `muse remote set-url `. | | `push rejected: ... is not a fast-forward ... use --force to override` | Someone else pushed to the same branch first (or you rewrote history locally) | This is correct, safe behavior — pull/rebase first. Only use `--force` if you're deliberately rewriting shared history, and never on a shared branch without coordinating with whoever else is using it. | | `"status": "conflict"` on pull | Two branches diverged with real conflicting changes | See the conflict-case recipe above — read both sides, `muse resolve`, never blind `--ours`/`--theirs`. | | `muse fetch ` — `unrecognized arguments` | `fetch` takes `--branch`/`-b`, not a positional branch arg | `muse fetch --branch --json` | If something doesn't match its expected result above and none of these explain it, stop and report the exact command, its JSON output, and which step it was — don't work around it silently. --- ## Cleanup Scratch repos created while running this guide should be deleted when you're done so they don't accumulate on staging: ```bash cd /tmp/mwp-qa-push- && muse hub repo delete --hub https://staging.musehub.ai --yes cd /tmp/mwp-qa-loop- && muse hub repo delete --hub https://staging.musehub.ai --yes rm -rf /tmp/mwp-qa-*- /tmp/mwp-smoke- /tmp/mwp-smoke-clone ```