Episode 17 --- The Wire Protocol

Working YouTube title:
I Flipped One Bit On Purpose (What Actually Crosses The Network)

Thumbnail thought:
4746 bytes. one flipped bit. one rejection.

Target runtime: ~8:00

---

[0:00--0:20] COLD OPEN

[CAMERA --- Episode 16's closing line, on screen: "what actually crosses the wire when it does."]

GABRIEL:

Every push and fetch this season has been one line of output. Today I'm opening the envelope. Real bytes, real hex, and I'm going to flip one bit on purpose and watch it get caught.

[TITLE CARD --- fast]

THE WIRE PROTOCOL

[Music enters.]

---

[0:20--1:20] THE FORMAT, FROM THE SOURCE ITSELF

[TERMINAL]

$ muse code cat "muse/core/mpack.py" | sed -n '973,986p'

# Wire format:
#   [4B]  magic: b"MUSE"
#   [1B]  version: 1
#   [1B]  section_count: N
#   [N*17B] section table: each entry is (1B type, 8B offset, 8B length)
#   [...]   section data
#   [32B]   SHA-256 of every byte above (footer)

GABRIEL VO:

Five section types --- blobs, commits, snapshots, tags, metadata. Not five separate files. One binary blob, one HTTP body, one checksum. Let's build a real one and read it byte by byte.

---

[1:20--3:00] READING THE ENVELOPE

[TERMINAL]

$ python3 -c "... build_wire_mpack(mpack, meta={...}) ..."

total wire size: 4746 bytes
header (hex): 4d 55 53 45 01 05
magic=b'MUSE' version=1 section_count=5

GABRIEL:

4d 55 53 45 --- that's literally "MUSE" in ASCII. Version 1. Five sections.

  [0] type=1 (BLOBS)     offset=91   length=2545
  [1] type=2 (COMMITS)   offset=2636 length=1488
  [2] type=3 (SNAPSHOTS) offset=4124 length=449
  [3] type=4 (TAGS)      offset=4573 length=8
  [4] type=5 (META)      offset=4581 length=133

GABRIEL VO:

Every section's exact byte range, declared up front. No parsing the whole blob to find where commits start --- jump straight to offset 2636 and you're there.

$ # META section, decoded straight out of the bytes
{"repo_id":"demo","branch":"main","head_commit_id":"sha256:7f05297c..."}

GABRIEL:

That's not a pretty-printed API response. That's literally what was sitting at byte offset 4581, decoded live.

---

[3:00--4:00] THE PART WHERE I BREAK IT ON PURPOSE

[CAMERA]

$ python3 -c "
data = bytearray(open('wire.mpack','rb').read())
data[200] ^= 0xFF
parse_wire_mpack(bytes(data))
"

Correctly rejected: OSError: Wire MPack failed SHA-256 integrity check

GABRIEL VO:

One byte, deep inside the blobs section, flipped. Not truncated, not missing --- just wrong. The 32-byte SHA-256 footer catches it immediately, before a single section is even parsed. No silent corruption gets anywhere near the object store.

[beat]

That's the same footer check muse push runs on every single upload --- twice, actually. Let's watch it happen for real.

---

[4:00--5:40] A REAL PUSH, ALL SEVEN STEPS

[TERMINAL --- real verbose push output]

[PUSH step 0] GET .../refs → remote_head, have=[...]
[PUSH step 1] walk local DAG → want vs have → new_commits=1
[PUSH step 2] have_blobs=0  manifest_blobs=3  blobs_to_send=3
[PUSH step 3] pack into one mpack → size=4613 bytes
[PUSH step 4] mpack_key = sha256:2d1be6...
[PUSH step 5] POST /push/mpack-presign → upload_url (MinIO, expires in 1hr)
[PUSH step 6] pre-PUT integrity check: sha256(wire_bytes) == mpack_key ✓
              PUT → HTTP 200
              post-PUT integrity check: blob_id(mpack_bytes) == mpack_key ✓
[PUSH step 7] POST /push/unpack-mpack → blobs_written=3 commits_written=1

GABRIEL:

Seven steps, and two separate integrity checks around one upload --- before the PUT, and conceptually after, both comparing against the same content-addressed key. If a single byte flips in transit to object storage, the key itself won't match and the push fails loudly instead of quietly landing corrupted bytes.

---

[5:40--6:30] THE SECOND PUSH IS SMALLER, ON PURPOSE

[TERMINAL]

$ echo "second line" >> README.md && muse commit -m "..." --sign
$ muse push local main

[PUSH step 2] have_blobs=3  manifest_blobs=3  blobs_to_send=1
[PUSH step 3] pack into one mpack → size=2113 bytes

GABRIEL VO:

First push: three blobs, 4613 bytes. Second push, one line added to one file: one blob, 2113 bytes. have_blobs --- the remote already has two of these three objects, so only the genuinely new one travels. Content addressing isn't just how objects are named locally; it's the actual diff algorithm for what crosses the wire.

---

[6:30--7:30] FETCH IS A DIFFERENT SHAPE ENTIRELY

[TERMINAL]

$ muse clone https://localhost:1337/gabriel/wire-episode17

[transport] POST .../fetch/mpack  want=1 have=0
[transport] GET mpack ... size=0.01MB
[transport] sha256  match=True
[transport] unpackb  commits=2  snaps=2  blobs=3
[clone] apply_mpack DONE  blobs_written=3  commits_written=2

GABRIEL:

Push is a three-way negotiation --- presign, upload, unpack. Fetch is one POST to negotiate what's needed, one GET to retrieve it, one hash check, done. Asymmetric on purpose: pushing has to coordinate a write; fetching is just a read with a checksum.

[beat]

And notice: sha256 match=True on the way in too. The same footer check from three minutes ago, running automatically, every time, whether you're the one uploading or the one downloading.

---

[7:30--8:00] OUT

[TERMINAL --- fading to black]

GABRIEL VO:

That's every byte, in both directions, verified. Nothing this episode was broken --- which, six episodes running into real bugs, is its own kind of interesting. The wire protocol is exactly as solid as it needed to be.

[beat]

Next: what you can actually build with an API this well-specified, if you're not using the CLI at all.

[CUT TO BLACK]

musehub.ai

---

Production Notes

Episode 17 is a deliberate tonal break: after six straight episodes finding real defects, this one holds up completely under the same scrutiny. Say that explicitly near the end (7:30) rather than letting it pass silently --- the audience has been trained by this point to expect a bug, and the absence of one is itself informative about where this codebase's engineering effort actually went (core wire format, hashing, transport) versus where it's been thinner (CLI ergonomics, session lifecycle, docs).

The Bit-Flip Has To Be Visually Legible

Flipping data[200] is arbitrary and invisible unless the byte offset is shown landing inside the BLOBS section range from the table printed one beat earlier (offset 91, length 2545 --- 200 is inside that range). Make the connection explicit on screen: this isn't a random corrupted byte, it's specifically inside the payload the footer is supposed to protect.

Show The Retry, If It Happens

The clone in research hit a real "server busy, retrying in 30s" backoff before succeeding — a genuine async-job-queue behavior (the worker prebuilding the fetch mpack), not a bug. If it recurs at record time, it's worth keeping in rather than cutting for pacing: it's one more real piece of the actual system doing its job under normal async load, and cutting it would make the demo look more instantaneous than production actually is.

Everything Here Is Real

The wire mpack byte layout, the tamper test, both pushes, and the clone were all run against the actual current build and the real local MuseHub hub — not a mocked transport. Re-run make-wire-episode17-demo.sh at record time and re-verify the exact byte offsets/sizes still match; they will shift slightly if any upstream commit message or file content changes, which is expected — narrate the shape of the format, not these exact numbers, if they've drifted.

The Seed

The viewer arrives thinking:

Push and fetch are just "upload the diff, download the diff" --- not much to see at the wire level.

They should leave thinking:

There's a real binary format under here, with its own integrity guarantee, that behaves asymmetrically on purpose depending on which side of the wire you're on — and it held up to me trying to break it. What does a server built directly on top of that guarantee actually let me do?

That's the MuseHub API, next episode.