gabriel / muse public
domain-extensibility.md markdown
238 lines 14.1 KB
Raw
sha256:94f494a8f59e4b708ebb89e304209737ce34324a33aae83840a6f6b06d7b8d9d docs: add domain-extensibility.md — the two-axis breadth/de… Sonnet 5 1 day ago

The Muse Extensibility Surface: Breadth and Depth

This is a survey of how much design space a Muse domain plugin actually opens up — grounded in the real shipped plugins (muse/plugins/*) and the real merge engine (muse/core/merge_engine.py), not aspirational copy. For "how do I write one," see docs/guide/plugin-authoring-guide.md. This doc is the "why is the design space this large" companion.

The core idea: two independent axes

Most version-control systems give you one axis of flexibility at best. Git lets you choose a merge strategy (recursive, ours, octopus...) but the meaning of a line is fixed — always text. Muse separates two concerns that are usually welded together, and lets each vary independently:

  • Axis A — what you're versioning. The domain plugin decides what a "change" even means: a note, a chess move, a graph edge, a tensor cell, a function body. This is the six-method protocol plus five diffing algebras.
  • Axis B — how divergence gets reconciled. Independent of what the content is, there are six top-level strategies for resolving two divergent histories, a per-path override layer on top of that, and a learned-memory layer on top of that.

Every domain gets both axes for free, and they compose orthogonally — the combinatorics are the actual source of the "staggering" feeling, not any single piece in isolation.


Axis A: what you're versioning

The six-method contract

A domain plugin is a duck-typed Protocol — no base class, no registration decorator (muse/domain.py::MuseDomainPlugin):

Method Answers
snapshot(live_state) What does the current state actually contain?
diff(base, target) What's the minimal typed delta between two states?
merge(base, left, right) How do two divergent histories reconcile against their common ancestor?
drift(committed, live) Has the live state diverged from the last commit? (muse status)
apply(delta, live_state) How do you reconstruct a historical state? (muse checkout)
schema() What does this domain's data actually look like, structurally?

Implement these six and the domain gets branching, merging, log, diff, checkout, push/pull, and time-travel for free — the engine owns the DAG, you own the meaning of a change.

Five algebras, freely mixable per dimension

schema() declares a top_level shape and any number of dimensions, each independently typed as one of five kinds (muse/core/schema.py):

Kind Ops Real domains using it
sequence insert / delete / move, LCS or Myers diff midi (note streams), chess (moves — see below)
tree Zhang-Shasha edit distance, moves preserve subtree identity timeline (OTIO project→sequence→track hierarchy)
tensor sparse/block numerical diff, configurable ε threshold midi (CC automation curves — dtype="float32", diff_mode="sparse", epsilon=0.5)
set pure membership delta, unordered, content-addressed identity todo (tasks), identity (identities/relationships), mist (artifacts), social (posts/reactions/follow graph)
map recursive key-by-key delegation to any other kind, to arbitrary depth code (symbol tables), any domain with structured metadata

The important word is freely mixable. A map dimension's values can each be typed as sequence, set, tree, or another map — the engine picks the right diff algorithm per key, recursively. This is why the midi plugin can declare 21 independent dimensions in one domain (sustain pedal, channel volume, tempo, note streams, CC automation, each its own algebra) and two agents editing completely unrelated aspects of the same file simply never conflict — not because of a special case, but because independent dimensions are the default behavior of the schema system.

Two optional extension protocols

Protocol Extra method(s) Unlocks
AddressedMergePlugin merge_ops() Address-keyed Map merge — ops at different addresses commute automatically; only same-address, incompatible-intent ops surface a conflict. This is what lets code produce a conflict naming the exact function that both branches touched, not a line range.
HarmonyPlugin conflict_fingerprint() + similarity() Semantic conflict fingerprinting — Tier 3 of Harmony's resolution engine (see Axis B) replays a past resolution for a structurally similar conflict, not just a byte-identical one.

Neither is required. A domain plugin can be ~50 lines and still get full VCS semantics (the JsonDocPlugin reference example in the developer docs is under 80 lines, six methods, no extensions).

Depth exhibit: domain-specific invariants beyond diff/merge

The six methods aren't a ceiling — a domain can layer arbitrary structural invariants on top of the generic merge protocol. The identity domain (muse/plugins/identity/plugin.py) names three invariants for its graph, though only the first is actually enforced inside merge() today — worth being precise about, since the other two are a real illustration of scope a domain could claim without every domain needing to:

  • I1 — acyclicity (enforced at merge time). Any new relationship edge that would introduce a cycle into the identity graph becomes a merge conflict instead of being silently applied. The graph is provably never left in a cyclic state, even under concurrent edits from two branches that individually looked fine.
  • I2 — root distance. A derived property of the graph; the plugin's own docstring documents this as not enforced at merge time.
  • I3 — quorum soundness. Documented as a hub-level concern, not a merge-time one — soundness here depends on state Muse itself doesn't own, so it's deliberately left to MuseHub rather than forced into the plugin.

The real depth claim isn't "every domain enforces every invariant it names" — it's that a domain can express "what does it mean for this entire data structure to remain valid" as code that runs on every merge, selectively, exactly where that's the plugin's job and not the hub's. I1 proves the mechanism is real; I2/I3 show the boundary is a deliberate design choice, not a limitation of the protocol.


Axis B: how divergence gets reconciled

This axis is completely independent of what the content is. It applies uniformly whether you're merging MIDI, chess games, or JSON configs.

Six top-level merge strategies

Confirmed directly from muse/core/merge_engine.py's strategy table — each strategy is a (diff_unit, resolution) pair:

Strategy diff_unit resolution Behavior
recursive (default) three_way escalate Classic three-way merge against the common ancestor. Genuine conflicts are surfaced, not guessed at.
overlay snapshot prefer_theirs No common-ancestor diff at all — the incoming branch's final state is laid on top of yours. Wherever they differ, theirs wins, automatically, no conflict ever raised.
snapshot snapshot escalate Same no-ancestor comparison as overlay, but every difference between the two final states is surfaced for review instead of auto-resolved.
replay replay_ours escalate Takes everything changed since the common ancestor on your side and re-applies it on top of the incoming branch's state — a rebase-shaped reconciliation.
ours three_way prefer_ours A real three-way merge; conflicts are still detected, but always resolve deterministically to your side.
theirs three_way prefer_theirs Same, resolving to the incoming side.

The distinction between overlay/snapshot and ours/theirs is easy to underrate: the former two skip history entirely and compare final states; the latter two still do a real three-way diff against the ancestor and only use "prefer a side" to break ties on genuine conflicts. They answer different questions — "what's different between these two end states" vs. "what changed since we diverged, and who wins when that collides."

Per-path override layer — .museattributes

Independent of which top-level strategy is active, individual paths can declare their own resolution policy, gitattributes-style:

Strategy Behavior
domain Delegate to the active domain plugin (the default)
ours Always take our version; never raises a conflict
theirs Always take their version; never raises a conflict
union Additive merge where possible; conflict only if destructive
binary Opaque blob; conflict on any difference at all

A lockfile can be merge=ours while everything else in the same repo uses full domain-aware merging — no plugin code touched, one line in .museattributes.

The learned-memory layer — Harmony

Sitting on top of both of the above, Harmony is a four-tier resolution engine that gets smarter every time a conflict is resolved:

Tier 1 — Policy    declarative rule fires on a path pattern
Tier 2 — Exact     blob fingerprint matches a saved resolution
Tier 3 — Semantic  HarmonyPlugin.similarity() ≥ threshold → fuzzy replay
Tier 4 — Escalate  hub issue created; human or agent needed

Every manually-resolved conflict is recorded with confidence=1.0 (human-verified). The next time an equivalent conflict recurs — same blob (Tier 2), or a structurally equivalent one even with different bytes (Tier 3, if the domain implements HarmonyPlugin) — it replays automatically. The identity domain's conflict_fingerprint() hashes (from_handle, edge_type, to_handle) and deliberately ignores signatures and timestamps — two pushes adding the same logical relationship edge at different times fingerprint identically and replay the same resolution without a human ever seeing it twice.


How the axes compose

A single muse merge invocation is really: (domain's algebra choices per dimension) × (top-level strategy) × (any .museattributes per-path overrides) × (whatever Harmony already knows). None of these four are aware of each other's internals — the domain plugin never knows which top-level strategy invoked it; the top-level strategy never knows what algebra a dimension uses; Harmony never cares which domain or strategy produced the conflict it's fingerprinting. That decoupling is what makes the combinatorics real rather than a marketing multiplication — you can actually add a new domain, or a new top-level strategy, or a new attribute rule, without touching the other two axes at all.


The breadth, in evidence: every domain that actually ships today

muse/plugins/registry.py currently registers nine plugins (midi is conditional on an optional dependency). This list is worth stating plainly because it's dramatically wider than what's publicly documented (see the note at the end) — this is the actual, current breadth:

Domain Top-level shape What makes it a distinct exhibit
code map (5 dimensions: structure, symbols, imports, variables, metadata) Symbol-level addressed merge. A pure reformat produces zero delta — the diff is semantic, not textual.
identity set (identities, relationships) Merge-time acyclicity enforcement (I1) layered on top of ordinary set-merge, plus two more named invariants deliberately left to derivation/the hub — see the depth exhibit above.
midi set of files, 21 independent sub-dimensions The deepest dimension count of any shipped domain, and the only one that internally mixes all three of sequence (notes/meta events), tensor (CC automation curves), and tree (track hierarchy) in a single plugin — the clearest existing proof that the five algebras are meant to be composed, not chosen once per domain.
mist set, identity by_content The filename is the content hash. Every artifact type is first-class — MIDI, ABIs, JSON schemas, images, arbitrary binary — because identity never depends on interpreting the bytes.
chess sequence (SAN moves, LCS-diffable) Two branches diverging after a shared opening produce a "variation" conflict — literally reusing chess's own vocabulary for the situation, because the domain model matches how a chess player already thinks.
social set (posts, reactions, follow graph — independent) + non-independent profile dimension A worked example of mixing independence: three dimensions merge freely in parallel, one deliberately does not (concurrent profile edits conflict on purpose).
timeline tree (Zhang-Shasha), OTIO-compatible Proves the domain protocol reaches into an entirely different industry's existing interchange format (video editing) without inventing a new one.
todo set, content-addressed by task text Deliberately the simplest possible real domain — one file, no address-keyed extension, no CRDT. The floor of the design space, not the ceiling.
scaffold (template, not a real domain) The literal starting point — docs/guide/plugin-authoring-guide.md's "copy this and fill in the TODOs" example lives in the registry itself.

Nine domains, none of them text files with line-based diffing, all implementing the identical six-method contract, all getting branch/merge/ log/checkout/push/pull for free, all composable with any of the six top-level merge strategies and the .museattributes override layer.

A stale-docs note, found while researching this

musehub/templates/musehub/pages/docs_muse_domains.html (the public developer docs page this doc was written to accompany) currently states "Three domains ship with Muse" and names only code, identity, and mist. That was true at some earlier point but is six domains behind current reality — chess, midi, social, timeline, and todo are all real, registered, working plugins today. Worth a follow-up pass to bring that page in line with muse/plugins/registry.py (the actual source of truth), possibly alongside expanding it with the six top-level merge strategies and the .museattributes/Harmony layering described above, none of which the current page covers at all — it only documents Axis A (the domain protocol), not Axis B (merge strategy selection).

File History 1 commit
sha256:94f494a8f59e4b708ebb89e304209737ce34324a33aae83840a6f6b06d7b8d9d docs: add domain-extensibility.md — the two-axis breadth/de… Sonnet 5 1 day ago