fix(push): chunk C frame into <=8MB batches to prevent CF stream truncation
Wall 5 root cause: the push_stream POST body contains a single 66 MB C (COMMIT_PACK) frame. Cloudflare truncates or resets the streaming body mid-frame. The server's msgpack Unpacker then misaligns, interpreting mid-frame binary bytes as a new top-level value. A random 0xDF byte (map32 format) followed by arbitrary bytes produces a garbage map length (e.g. 2,662,457,625) that exceeds max_map_len → ValueError.
Fix — two parts:
1. Client (transport.py): _frame_generator() now chunks commits+snapshots into batches of COMMIT_CHUNK_SIZE (50) commits per C frame. Each snapshot is sent exactly once (tracked via sent_snap_ids). Result: ~14 C frames of ~5-8 MB each instead of one 66 MB frame. Added MAX_C_FRAME_BYTES (8 MB) and COMMIT_CHUNK_SIZE (50) constants.
2. Server (musehub_wire.py): C frame handler changes from replace to extend for commits_wire and snapshots_wire. A single C frame still works. Multiple C frames accumulate their commits/snapshots correctly.
Tests (T19, T20 — 6 new tests, 24 total in test_presign_upload.py): - T19: truncation of large C frame produces max_map_len error (documents the failure mode); chunked frames survive individually; each chunk is below MAX_C_FRAME_BYTES - T20: server accumulates commits across multiple C frames; single C frame still works; snapshots dedup correctly
0 comments
muse hub commit comment sha256:f367ba20336a65046aa8107faa93af2ec3141bfc74dcf682b1ac927569073d9b --body "your comment"
No comments yet. Be the first to start the discussion.