fix: parallelize fetch/objects S3 reads to fix clone 504 timeout
Sequential backend.get() calls on S3/R2 issue one HTTP round-trip per object. A repo with 100+ objects can easily exceed Cloudflare's 100-second origin-response timeout, producing the 504 the user reported when cloning from staging.
- S3Backend.get_batch(): override the base-class sequential fallback with asyncio.gather() so all objects are fetched concurrently; total latency ≈ max(single-object latency) rather than N × single-object latency. - _pack_objects_response(): switch from a sequential for-loop over backend.get() to asyncio.gather() over all objects; semantics preserved (empty OID synthesised without I/O, cross-repo dedup via stored_repo_id). - Tests: 3 new tests in TestIntegrationS3BackendMocked — correctness (all found, missing omitted) and a timing test proving parallelism (N=6 @ 40 ms each must complete in <100 ms, not 240 ms).
0 comments
muse hub commit comment sha256:d0c3abc9b002b3d481cb859603182a321ca715d8a884e17a2960524d2f6b00fb --body "your comment"
No comments yet. Be the first to start the discussion.