gabriel / muse public
midi_merge.py python
587 lines 23.5 KB
Raw
sha256:06dba78c2a78e251b580422dd1fd547f3c8357ff18f7709a860873b2d24dbbbf chore: bump version to 0.2.0rc14 Sonnet 4.6 patch 1 day ago
1 """MIDI dimension-aware merge for the Muse MIDI plugin.
2
3 This module implements the multidimensional merge that makes Muse meaningfully
4 different from git. Git treats every file as an opaque byte sequence: any
5 two-branch change to the same file is a conflict. Muse understands that a
6 MIDI file has *independent orthogonal axes*, and two collaborators can touch
7 different axes of the same file without conflicting.
8
9 Dimensions
10 ----------
11
12 MIDI carries far more independent axes than a naive "notes vs. everything else"
13 split. The full dimension taxonomy maps every MIDI event type to exactly one
14 internal bucket:
15
16 +----------------------+--------------------------------------------------+
17 | Internal dimension | MIDI event types / CC numbers |
18 +======================+==================================================+
19 | ``notes`` | ``note_on`` / ``note_off`` |
20 +----------------------+--------------------------------------------------+
21 | ``pitch_bend`` | ``pitchwheel`` |
22 +----------------------+--------------------------------------------------+
23 | ``channel_pressure`` | ``aftertouch`` (mono channel pressure) |
24 +----------------------+--------------------------------------------------+
25 | ``poly_pressure`` | ``polytouch`` (per-note polyphonic aftertouch) |
26 +----------------------+--------------------------------------------------+
27 | ``cc_modulation`` | CC 1 — modulation wheel |
28 +----------------------+--------------------------------------------------+
29 | ``cc_volume`` | CC 7 — channel volume |
30 +----------------------+--------------------------------------------------+
31 | ``cc_pan`` | CC 10 — stereo pan |
32 +----------------------+--------------------------------------------------+
33 | ``cc_expression`` | CC 11 — expression controller |
34 +----------------------+--------------------------------------------------+
35 | ``cc_sustain`` | CC 64 — damper / sustain pedal |
36 +----------------------+--------------------------------------------------+
37 | ``cc_sostenuto`` | CC 66 — sostenuto pedal |
38 +----------------------+--------------------------------------------------+
39 | ``cc_soft_pedal`` | CC 67 — soft pedal (una corda) |
40 +----------------------+--------------------------------------------------+
41 | ``cc_portamento`` | CC 65 — portamento on/off |
42 +----------------------+--------------------------------------------------+
43 | ``cc_reverb`` | CC 91 — reverb send level |
44 +----------------------+--------------------------------------------------+
45 | ``cc_chorus`` | CC 93 — chorus send level |
46 +----------------------+--------------------------------------------------+
47 | ``cc_other`` | All other CC events (numbered controllers) |
48 +----------------------+--------------------------------------------------+
49 | ``program_change`` | ``program_change`` (patch / instrument select) |
50 +----------------------+--------------------------------------------------+
51 | ``tempo_map`` | ``set_tempo`` meta events |
52 +----------------------+--------------------------------------------------+
53 | ``time_signatures`` | ``time_signature`` meta events |
54 +----------------------+--------------------------------------------------+
55 | ``key_signatures`` | ``key_signature`` meta events |
56 +----------------------+--------------------------------------------------+
57 | ``markers`` | ``marker``, ``cue_marker``, ``text``, |
58 | | ``lyrics``, ``copyright`` meta events |
59 +----------------------+--------------------------------------------------+
60 | ``track_structure`` | ``track_name``, ``instrument_name``, ``sysex``, |
61 | | ``sequencer_specific`` and unknown meta events |
62 +----------------------+--------------------------------------------------+
63
64 Why fine-grained dimensions matter
65 -----------------------------------
66 With the old 4-bucket model, changing sustain pedal (CC64) and changing channel
67 volume (CC7) were the same dimension: they always conflicted. With 21 internal
68 dimensions they are independent — two agents can edit different aspects of the
69 same MIDI file without ever conflicting.
70
71 Independence rules
72 ------------------
73 - **Independent** (``independent_merge=True``): notes, pitch_bend, all CC
74 dimensions, channel_pressure, poly_pressure, program_change, key_signatures,
75 markers. Conflicts in these dimensions never block merging others.
76 - **Non-independent** (``independent_merge=False``): tempo_map, time_signatures,
77 track_structure. A conflict here blocks merging other dimensions until
78 resolved, because a tempo change shifts the musical meaning of every subsequent
79 tick position, and track structure changes affect routing.
80
81 Merge algorithm
82 ---------------
83 1. Parse ``base``, ``left``, and ``right`` MIDI bytes into event streams.
84 2. Convert to absolute-tick representation and bucket by dimension.
85 3. Hash each bucket; compare ``base ↔ left`` and ``base ↔ right`` to detect
86 per-dimension changes.
87 4. For each dimension apply the winning side determined by ``.museattributes``
88 strategy (or the standard one-sided-change rule when no conflict exists).
89 5. Reconstruct a valid MIDI file by merging winning dimension slices, sorting
90 by absolute tick, converting back to delta-time, and writing to bytes.
91
92 Public API
93 ----------
94 - :func:`extract_dimensions` — parse MIDI bytes → ``MidiDimensions``
95 - :func:`merge_midi_dimensions` — three-way dimension merge → bytes or ``None``
96 - :func:`dimension_conflict_detail` — per-dimension change report for logging
97 - :data:`INTERNAL_DIMS` — ordered list of all internal dimension names
98 - :data:`DIM_ALIAS` — user-facing ``.museattributes`` name → internal bucket
99 - :data:`NON_INDEPENDENT_DIMS` — dimensions that block others on conflict
100 """
101
102 import io
103 import json
104 import logging
105 from dataclasses import dataclass, field
106
107 import mido
108
109 logger = logging.getLogger(__name__)
110
111 from muse.core.types import blob_id
112 from muse.core.attributes import AttributeRule, resolve_strategy
113
114 # ---------------------------------------------------------------------------
115 # Dimension constants — the complete MIDI dimension taxonomy
116 # ---------------------------------------------------------------------------
117
118 #: Internal dimension names, ordered canonically.
119 #: Each MIDI event type maps to exactly one of these buckets.
120 INTERNAL_DIMS: list[str] = [
121 # --- Expressive note content ---
122 "notes", # note_on / note_off
123 "pitch_bend", # pitchwheel
124 "channel_pressure", # aftertouch (mono)
125 "poly_pressure", # polytouch (per-note)
126 # --- Named CC controllers (individually mergeable) ---
127 "cc_modulation", # CC 1
128 "cc_volume", # CC 7
129 "cc_pan", # CC 10
130 "cc_expression", # CC 11
131 "cc_sustain", # CC 64
132 "cc_portamento", # CC 65
133 "cc_sostenuto", # CC 66
134 "cc_soft_pedal", # CC 67
135 "cc_reverb", # CC 91
136 "cc_chorus", # CC 93
137 "cc_other", # all remaining CC numbers
138 # --- Patch / program selection ---
139 "program_change",
140 # --- Timeline / notation metadata (non-independent) ---
141 "tempo_map", # set_tempo — non-independent: affects all tick positions
142 "time_signatures", # time_signature — non-independent: affects bar structure
143 # --- Tonal context and notation ---
144 "key_signatures", # key_signature
145 "markers", # marker, cue_marker, text, lyrics, copyright
146 # --- Track structure (non-independent) ---
147 "track_structure", # track_name, instrument_name, sysex, unknown meta
148 ]
149
150 #: Dimensions whose conflicts block merging all other dimensions until resolved.
151 #: All other dimensions are merged in parallel regardless of conflicts here.
152 NON_INDEPENDENT_DIMS: frozenset[str] = frozenset({
153 "tempo_map",
154 "time_signatures",
155 "track_structure",
156 })
157
158 #: User-facing dimension names from .museattributes mapped to internal buckets.
159 #: Agents and humans use these names in merge strategy declarations.
160 DIM_ALIAS: _DimAliasMap = {
161 "pitch_bend": "pitch_bend",
162 "aftertouch": "channel_pressure",
163 "poly_aftertouch": "poly_pressure",
164 "modulation": "cc_modulation",
165 "volume": "cc_volume",
166 "pan": "cc_pan",
167 "expression": "cc_expression",
168 "sustain": "cc_sustain",
169 "portamento": "cc_portamento",
170 "sostenuto": "cc_sostenuto",
171 "soft_pedal": "cc_soft_pedal",
172 "reverb": "cc_reverb",
173 "chorus": "cc_chorus",
174 "automation": "cc_other",
175 "program": "program_change",
176 "tempo": "tempo_map",
177 "time_sig": "time_signatures",
178 "key_sig": "key_signatures",
179 "markers": "markers",
180 "track_structure": "track_structure",
181 }
182
183 #: All valid names (aliases + internal) → internal bucket.
184 _CANONICAL: _DimAliasMap = {**DIM_ALIAS, **{d: d for d in INTERNAL_DIMS}}
185
186 #: CC number → internal dimension name for named controllers.
187 _CC_DIM: dict[int, str] = {
188 1: "cc_modulation",
189 7: "cc_volume",
190 10: "cc_pan",
191 11: "cc_expression",
192 64: "cc_sustain",
193 65: "cc_portamento",
194 66: "cc_sostenuto",
195 67: "cc_soft_pedal",
196 91: "cc_reverb",
197 93: "cc_chorus",
198 }
199
200 # ---------------------------------------------------------------------------
201 # Data types
202 # ---------------------------------------------------------------------------
203
204 @dataclass
205 class DimensionSlice:
206 """Events belonging to one dimension of a MIDI file.
207
208 ``events`` is a list of ``(abs_tick, mido.Message)`` pairs sorted by
209 ascending absolute tick. ``content_hash`` is the SHA-256 digest of the
210 canonical JSON serialisation of the event list (used for change detection
211 without loading file bytes).
212 """
213
214 name: str
215 events: list[tuple[int, mido.Message]] = field(default_factory=list)
216 content_hash: str = ""
217
218 def __post_init__(self) -> None:
219 if not self.content_hash:
220 self.content_hash = _hash_events(self.events)
221
222 @dataclass
223 class MidiDimensions:
224 """All dimension slices extracted from one MIDI file.
225
226 ``slices`` maps internal dimension name → :class:`DimensionSlice`.
227 Every internal dimension in :data:`INTERNAL_DIMS` has an entry, even if
228 the corresponding event list is empty (hash of empty list is stable).
229 """
230
231 ticks_per_beat: int
232 file_type: int
233 slices: _DimSliceMap
234
235 def get(self, dim: str) -> DimensionSlice:
236 """Return the slice for a user-facing or internal dimension name."""
237 internal = _CANONICAL.get(dim, dim)
238 return self.slices[internal]
239
240 # ---------------------------------------------------------------------------
241 # Internal helpers
242 # ---------------------------------------------------------------------------
243
244 def _classify_event(msg: mido.Message) -> str | None:
245 """Map a mido Message to an internal dimension bucket.
246
247 Returns ``None`` for events that should be excluded from all buckets
248 (e.g. ``end_of_track`` is handled during reconstruction, not stored here).
249 Unknown messages that are meta events fall back to ``"track_structure"``.
250 True unknowns (no ``is_meta`` attribute) are discarded.
251 """
252 t = msg.type
253
254 # --- Note events ---
255 if t in ("note_on", "note_off"):
256 return "notes"
257
258 # --- Pitch / pressure ---
259 if t == "pitchwheel":
260 return "pitch_bend"
261 if t == "aftertouch":
262 return "channel_pressure"
263 if t == "polytouch":
264 return "poly_pressure"
265
266 # --- CC — split by controller number ---
267 if t == "control_change":
268 return _CC_DIM.get(msg.control, "cc_other")
269
270 # --- Program change ---
271 if t == "program_change":
272 return "program_change"
273
274 # --- Timeline metadata ---
275 if t == "set_tempo":
276 return "tempo_map"
277 if t == "time_signature":
278 return "time_signatures"
279 if t == "key_signature":
280 return "key_signatures"
281
282 # --- Section markers and text annotations ---
283 if t in ("marker", "cue_marker", "text", "lyrics", "copyright"):
284 return "markers"
285
286 # --- Track structure and routing ---
287 if t in ("track_name", "instrument_name", "sysex", "sequencer_specific"):
288 return "track_structure"
289
290 # --- End-of-track is reconstructed, not stored ---
291 if t == "end_of_track":
292 return None
293
294 # --- Unknown meta events → track structure (safe default) ---
295 if getattr(msg, "is_meta", False):
296 return "track_structure"
297
298 return None
299
300 type _MsgVal = int | str | list[int]
301 type _MsgDict = dict[str, _MsgVal] # serialised mido.Message
302 type _DimAliasMap = dict[str, str] # dimension alias → canonical name
303 type _DimSliceMap = dict[str, "DimensionSlice"] # dim name → slice
304 type _DimBuckets = dict[str, list[tuple[int, mido.Message]]] # dim → (tick, msg) list
305 type _DimReport = dict[str, str] # dim name → resolution summary
306
307 def _msg_to_dict(msg: mido.Message) -> _MsgDict:
308 """Serialise a mido Message to a JSON-compatible dict."""
309 from muse.core.validation import MAX_SYSEX_BYTES
310
311 d: _MsgDict = {"type": msg.type}
312 for attr in (
313 "channel", "note", "velocity", "control", "value",
314 "pitch", "program", "numerator", "denominator",
315 "clocks_per_click", "notated_32nd_notes_per_beat",
316 "tempo", "key", "scale", "text", "data",
317 ):
318 if hasattr(msg, attr):
319 raw = getattr(msg, attr)
320 if isinstance(raw, (bytes, bytearray)):
321 # Cap sysex / large byte payloads to prevent memory exhaustion
322 # when a crafted MIDI contains a giant sysex blob.
323 if len(raw) > MAX_SYSEX_BYTES:
324 logger.warning(
325 "⚠️ Sysex payload %d bytes exceeds cap (%d) — truncating",
326 len(raw), MAX_SYSEX_BYTES,
327 )
328 raw = raw[:MAX_SYSEX_BYTES]
329 d[attr] = list(raw)
330 elif isinstance(raw, str):
331 d[attr] = raw
332 elif isinstance(raw, int):
333 d[attr] = raw
334 return d
335
336 def _hash_events(events: list[tuple[int, mido.Message]]) -> str:
337 """``sha256:``-prefixed ID of the canonical JSON representation of an event list."""
338 payload = json.dumps(
339 [(tick, _msg_to_dict(msg)) for tick, msg in events],
340 sort_keys=True,
341 separators=(",", ":"),
342 ).encode()
343 return blob_id(payload)
344
345 def _to_absolute(track: mido.MidiTrack) -> list[tuple[int, mido.Message]]:
346 """Convert a delta-time track to a list of ``(abs_tick, msg)`` pairs."""
347 result: list[tuple[int, mido.Message]] = []
348 abs_tick = 0
349 for msg in track:
350 abs_tick += msg.time
351 result.append((abs_tick, msg))
352 return result
353
354 # ---------------------------------------------------------------------------
355 # Public: extract_dimensions
356 # ---------------------------------------------------------------------------
357
358 def extract_dimensions(midi_bytes: bytes) -> MidiDimensions:
359 """Parse *midi_bytes* and bucket events by dimension.
360
361 Every event type in the MIDI spec maps to exactly one of the
362 :data:`INTERNAL_DIMS` buckets. Empty buckets are present with an empty
363 event list so that callers can always index by dimension name.
364
365 Args:
366 midi_bytes: Raw bytes of a ``.mid`` file.
367
368 Returns:
369 A :class:`MidiDimensions` with one :class:`DimensionSlice` per
370 internal dimension. Events within each slice are sorted by ascending
371 absolute tick, then by event type for determinism when multiple events
372 share the same tick.
373
374 Raises:
375 ValueError: If *midi_bytes* cannot be parsed as a MIDI file.
376 """
377 try:
378 mid = mido.MidiFile(file=io.BytesIO(midi_bytes))
379 except Exception as exc:
380 raise ValueError(f"Failed to parse MIDI data: {exc}") from exc
381
382 buckets: _DimBuckets = {
383 dim: [] for dim in INTERNAL_DIMS
384 }
385
386 for track in mid.tracks:
387 for abs_tick, msg in _to_absolute(track):
388 bucket = _classify_event(msg)
389 if bucket is not None:
390 buckets[bucket].append((abs_tick, msg))
391
392 for dim in INTERNAL_DIMS:
393 buckets[dim].sort(key=lambda x: (x[0], x[1].type))
394
395 slices = {
396 dim: DimensionSlice(name=dim, events=events)
397 for dim, events in buckets.items()
398 }
399 return MidiDimensions(
400 ticks_per_beat=mid.ticks_per_beat,
401 file_type=mid.type,
402 slices=slices,
403 )
404
405 # ---------------------------------------------------------------------------
406 # Public: dimension_conflict_detail
407 # ---------------------------------------------------------------------------
408
409 def dimension_conflict_detail(
410 base: MidiDimensions,
411 left: MidiDimensions,
412 right: MidiDimensions,
413 ) -> _DimReport:
414 """Return a per-dimension change report for a conflicting file.
415
416 Returns a dict mapping internal dimension name to one of:
417
418 - ``"unchanged"`` — neither side changed this dimension.
419 - ``"left_only"`` — only the left (ours) side changed.
420 - ``"right_only"`` — only the right (theirs) side changed.
421 - ``"both"`` — both sides changed; a dimension-level conflict.
422
423 This is used by :func:`merge_midi_dimensions` and surfaced in
424 ``muse merge`` output for human-readable conflict diagnostics.
425 """
426 report: _DimReport = {}
427 for dim in INTERNAL_DIMS:
428 base_hash = base.slices[dim].content_hash
429 left_hash = left.slices[dim].content_hash
430 right_hash = right.slices[dim].content_hash
431 left_changed = base_hash != left_hash
432 right_changed = base_hash != right_hash
433 if left_changed and right_changed:
434 report[dim] = "both"
435 elif left_changed:
436 report[dim] = "left_only"
437 elif right_changed:
438 report[dim] = "right_only"
439 else:
440 report[dim] = "unchanged"
441 return report
442
443 # ---------------------------------------------------------------------------
444 # Reconstruction helpers
445 # ---------------------------------------------------------------------------
446
447 def _events_to_track(
448 events: list[tuple[int, mido.Message]],
449 ) -> mido.MidiTrack:
450 """Convert absolute-tick events to a mido MidiTrack with delta times."""
451 track = mido.MidiTrack()
452 prev_tick = 0
453 for abs_tick, msg in sorted(events, key=lambda x: (x[0], x[1].type)):
454 delta = abs_tick - prev_tick
455 new_msg = msg.copy(time=delta)
456 track.append(new_msg)
457 prev_tick = abs_tick
458 if not track or track[-1].type != "end_of_track":
459 track.append(mido.MetaMessage("end_of_track", time=0))
460 return track
461
462 def _reconstruct(
463 ticks_per_beat: int,
464 winning_slices: _DimBuckets,
465 ) -> bytes:
466 """Build a type-0 MIDI file from winning dimension event lists.
467
468 All dimension events are merged into a single track (type-0) for
469 maximum compatibility. The absolute-tick ordering is preserved and
470 duplicate end_of_track messages are removed.
471 """
472 all_events: list[tuple[int, mido.Message]] = []
473 for events in winning_slices.values():
474 all_events.extend(events)
475
476 all_events = [
477 (tick, msg) for tick, msg in all_events
478 if msg.type != "end_of_track"
479 ]
480 all_events.sort(key=lambda x: (x[0], x[1].type))
481
482 track = _events_to_track(all_events)
483 mid = mido.MidiFile(type=0, ticks_per_beat=ticks_per_beat)
484 mid.tracks.append(track)
485
486 buf = io.BytesIO()
487 mid.save(file=buf)
488 return buf.getvalue()
489
490 # ---------------------------------------------------------------------------
491 # Public: merge_midi_dimensions
492 # ---------------------------------------------------------------------------
493
494 def merge_midi_dimensions(
495 base_bytes: bytes,
496 left_bytes: bytes,
497 right_bytes: bytes,
498 attrs_rules: list[AttributeRule],
499 path: str,
500 ) -> tuple[bytes, dict[str, str]] | None:
501 """Attempt a dimension-level three-way merge of a MIDI file.
502
503 For each internal dimension (all 21 of them):
504
505 - If neither side changed → keep base.
506 - If only one side changed → take that side (clean auto-merge).
507 - If both sides changed → consult ``.museattributes`` strategy:
508
509 * ``ours`` / ``theirs`` → take the specified side; record in report.
510 * ``manual`` / ``auto`` / ``union`` → unresolvable; return ``None``.
511
512 Non-independent dimensions (``tempo_map``, ``time_signatures``,
513 ``track_structure``) that have bilateral conflicts cause an immediate
514 ``None`` return — their conflicts cannot be auto-resolved because they
515 affect the semantic meaning of all other dimensions.
516
517 Args:
518 base_bytes: MIDI bytes for the common ancestor.
519 left_bytes: MIDI bytes for the ours (left) branch.
520 right_bytes: MIDI bytes for the theirs (right) branch.
521 attrs_rules: Rule list from :func:`muse.core.attributes.load_attributes`.
522 path: Workspace-relative POSIX path (used for strategy lookup).
523
524 Returns:
525 A ``(merged_bytes, dimension_report)`` tuple when all dimension
526 conflicts can be resolved, or ``None`` when at least one dimension
527 conflict has no resolvable strategy.
528
529 *dimension_report* maps each internal dimension name to the side
530 chosen: ``"base"``, ``"left"``, ``"right"``, or the strategy string.
531 Only dimensions with non-empty event lists or conflicts are included.
532
533 Raises:
534 ValueError: If any of the byte strings cannot be parsed as MIDI.
535 """
536 base_dims = extract_dimensions(base_bytes)
537 left_dims = extract_dimensions(left_bytes)
538 right_dims = extract_dimensions(right_bytes)
539
540 detail = dimension_conflict_detail(base_dims, left_dims, right_dims)
541
542 winning_slices: _DimBuckets = {}
543 dimension_report: _DimReport = {}
544
545 for dim in INTERNAL_DIMS:
546 change = detail[dim]
547
548 if change == "unchanged":
549 winning_slices[dim] = base_dims.slices[dim].events
550 if base_dims.slices[dim].events:
551 dimension_report[dim] = "base"
552
553 elif change == "left_only":
554 winning_slices[dim] = left_dims.slices[dim].events
555 dimension_report[dim] = "left"
556
557 elif change == "right_only":
558 winning_slices[dim] = right_dims.slices[dim].events
559 dimension_report[dim] = "right"
560
561 else:
562 # Both sides changed — resolve via .museattributes strategy.
563 # Look up by user-facing aliases first, then internal name.
564 user_dim_names = [k for k, v in DIM_ALIAS.items() if v == dim]
565 user_dim_names.append(dim) # internal name is also a valid alias
566
567 strategy = "auto"
568 for user_dim in user_dim_names:
569 s = resolve_strategy(attrs_rules, path, user_dim)
570 if s != "auto":
571 strategy = s
572 break
573 if strategy == "auto":
574 strategy = resolve_strategy(attrs_rules, path, "*")
575
576 if strategy == "ours":
577 winning_slices[dim] = left_dims.slices[dim].events
578 dimension_report[dim] = f"ours ({dim})"
579 elif strategy == "theirs":
580 winning_slices[dim] = right_dims.slices[dim].events
581 dimension_report[dim] = f"theirs ({dim})"
582 else:
583 # Unresolvable conflict. Non-independent dims fail fast.
584 return None
585
586 merged_bytes = _reconstruct(base_dims.ticks_per_beat, winning_slices)
587 return merged_bytes, dimension_report
File History 1 commit
sha256:06dba78c2a78e251b580422dd1fd547f3c8357ff18f7709a860873b2d24dbbbf chore: bump version to 0.2.0rc14 Sonnet 4.6 patch 1 day ago