Canonical version resolution — emergent SemVer as source of truth, PEP 440 as derived build artifact

Canonical Version Resolution — Emergent SemVer as Source of Truth, PEP 440 as a Derived Build Artifact

Background

Today two version strings exist for the same release, in two different formats, maintained in two different ways:

• pyproject.toml's version field (in both muse/pyproject.toml and musehub/pyproject.toml) is PEP 440, hand-typed, and must be bumped manually in both files, in both repos, on every release per the current runbook (musehub/.muse/agent.md, "Standard release flow," step 1: "Bump version in muse/pyproject.toml AND musehub/pyproject.toml, commit each on dev"). This is the literal mechanism by which the two values can drift out of sync — it is a two-file, two-repo, hand-synchronized edit with no verification that both edits actually happened or agree.
• muse release add <tag> / muse's own release-channel logic is SemVer 2.0, and is already partially emergent: muse/core/semver_classifier.py::classify_delta() computes a structural sem_ver_bump per commit from an AST diff of the changed public symbol graph, and muse release suggest (muse/cli/commands/release.py ::run_suggest) already aggregates every unreleased commit's bump and proposes the next tag, with a drivers list citing exactly which commits/symbols produced it.

muse --version (muse/_version.py::_read_version) currently prints the raw PEP 440 string read straight from pyproject.toml, with no translation to or acknowledgment of the SemVer form at all — so a nightly announced socially as v0.2.0-nightly.3 and a local muse --version showing 0.2.0.dev3 look like two different things to anyone who doesn't already know the translation table in musehub/docs/versioning.md.

Two additional facts change what the right fix is, not just whether one is needed:

1. Muse is not distributed via PyPI. publish_muse_release.sh builds an sdist with python3 -m build --sdist (backend: hatchling), uploads the tarball to S3, and serves it from staging.musehub.ai/releases/ for curl | sh install. PEP 440 is required here only because hatchling (a Python build backend) validates the version field against it — it is a self-imposed, internal build-tooling constraint, not a requirement from any external distribution channel pip/twine would otherwise impose.
2. A Rust port is a stated future direction, and the entire reason muse/tools/typing_audit.py exists — a zero-tolerance type-safety ratchet banning Any, cast(), bare collections, untyped defs — is to keep the Python implementation idiomatically close enough to Rust that the port is straightforward. Cargo/crates.io require SemVer 2.0 natively; PEP 440 has zero relevance in that world. A version scheme that depends on PEP 440 today is exactly the kind of Python-only idiom this workspace has been actively engineering away from everywhere else.

The fix is not "pick one format" — hatchling will keep requiring a PEP-440-legal version field for as long as the build backend is Python-based, and that requirement doesn't disappear by preference. The fix is: make SemVer 2.0 the one canonical, human-facing, emergently-computed source of truth, and make PEP 440 a derived, invisible, build-time-only artifact that is never hand-typed and never shown to a user. Muse's own commit-graph classifier already computes most of what "emergent" means here — this plan turns that existing, per-commit machinery into a general-purpose "what is my version right now" resolver, usable with zero manual action for the common case, with an explicit, reviewable manual-override escape hatch for the rare case a human needs to pin an exact string.

Goal

• One canonical version-resolution function, callable from the CLI and from build scripts, that returns a SemVer 2.0 string with zero manual state required in the common case — the nightly/pre-release sequence number is derived from the commit graph (the count of commits since the last release tag), not tracked in a separate counter file anywhere.
• An explicit, checked-in, reviewable manual-override mechanism for the rare case a human needs to pin an exact version string rather than trust the emergent computation.
• muse --version and every other user-facing surface print the canonical SemVer string — never a bare PEP 440 string, never a translation a user has to do themselves.
• pyproject.toml's version field (in both muse and musehub) is generated from the resolver at build time, satisfying hatchling's validation, without ever being hand-typed by a human again.
• musehub/docs/versioning.md and the deploy runbook are rewritten to describe and enforce this model, removing the two-file manual-bump step that is today's actual drift-risk mechanism.

"Done" means: every deliverable in the Phases below is complete and tested, the acceptance criteria all pass, and the two files that currently require a hand-synchronized manual edit on every release no longer do.

Non-Goals / Out of Scope

• Performing the Rust port itself. This plan only ensures the version scheme doesn't need to change again when that happens — it does not touch the port.
• Redesigning semver_classifier.py's bump-inference heuristics (visibility tiers, stability tiers, the bump matrix, .muse/stability.toml). This plan consumes that existing machinery as-is; it does not change how a bump is decided, only how the current effective version is resolved and displayed from the bumps already being computed.
• PyPI publishing. Not in scope, not planned, and explicitly not the reason PEP 440 exists in this workspace (see Background).
• Deleting PEP 440 from existence. It must still be produced, because hatchling's build backend requires a PEP-440-legal version field to build an sdist at all. It is demoted to a derived, generated artifact — not removed.

Design

The core insight — the nightly sequence number is already free

musehub/docs/versioning.md already decided nightly numbering should be "sequential auto-increment, starting at dev1" rather than date- or timestamp-based, specifically because a date/timestamp can't answer "how many nightlies so far" the way a sequential count does. muse release suggest --json already computes and returns exactly that count today, as unreleased_count — the number of commits since the last release tag, walked via walk_commits_between. The emergent nightly sequence number this plan needs already exists as a field in existing, tested code. No new counter state, no new file, no new tracked value — the "sequence" is "how many commits away from the last release," which Muse already knows how to compute for any ref.

Manual override — an explicit, checked-in escape hatch

A .muse/version_override file (single line, SemVer 2.0 — e.g. 0.2.0-rc.1), checked first by the resolver. When present, it is returned verbatim with source: "manual_override" and full precedence — no blending with emergent computation, no partial override of just the pre-release segment. Its absence (the expected default, steady state) means fully emergent resolution. Placed under .muse/ rather than a bare repo-root VERSION file for the same reason .muse/stability.toml lives there: it's a Muse-specific mechanism, not a general-purpose project convention, and keeping it under .muse/ signals that distinction — see Open Questions if this placement should be reconsidered.

Resolution algorithm — resolve_version()

1. Check .muse/version_override — if present and non-empty, return {version: <contents>, source: "manual_override"} immediately. Skip every subsequent step.
2. Else, find the latest stable release tag (reuse list_releases, the same call run_suggest already makes).
3. Walk commits since that tag (reuse walk_commits_between) and aggregate sem_ver_bump across them (reuse the aggregation loop already in run_suggest, extracted into a standalone, independently-testable function — see Refactor requirement, below).
4. If unreleased_count == 0: HEAD is exactly at the release tag. Return that tag's version plainly, source: "release_tag".
5. Else: compute the next candidate version from the aggregated bump using the existing pre-1.0/standard SemVer arithmetic already implemented in run_suggest (extracted into compute_next_version(base_version, agg_bump)), then append -nightly.<unreleased_count>. Return with source: "emergent".
6. Derive the PEP-440-canonical string from the resolved SemVer via the exact mechanical table already published in versioning.md (-nightly.N → .devN, -alpha.N → aN, -beta.N → bN, -rc.N → rcN, no suffix → unchanged). This translation is pure string logic with no ambiguity — the table is already written down; this plan only needs to implement it as code and test it against every row.

Refactor requirement — extract, don't duplicate

run_suggest's aggregation walk and pre-1.0 arithmetic currently live inline inside the CLI handler in muse/cli/commands/release.py. This plan extracts the reusable pieces into muse/core/version_resolver.py as pure functions with no CLI/argparse dependency, then refactors run_suggest to call them — muse release suggest's existing observable output must not change (same JSON schema, same values, for every existing fixture/test). This is a refactor with a regression gate, not a rewrite.

New CLI surfaces

• muse version resolve --json — exposes the full resolved structure: version (canonical SemVer), pep440 (derived), source (manual_override| release_tag|emergent), base_tag, unreleased_count, inferred_bump.
• muse version resolve --format pep440 — prints just the derived PEP 440 string, for build scripts that need exactly that value (the hatchling version-source hook, publish_muse_release.sh).
• muse --version — changed from a static pyproject.toml read to calling the resolver and printing the canonical SemVer string.

Build-time PEP 440 generation

pyproject.toml's version field becomes hatchling-dynamic in both muse/pyproject.toml and musehub/pyproject.toml rather than a literal hand-typed string. The exact mechanism (a custom hatchling version-source plugin calling muse version resolve --format pep440 directly, vs. a pre-build step in publish_muse_release.sh that generates a small version file hatchling reads via its existing [tool.hatch.version] source = "regex"/"code" support) is a Phase 3 decision, made against hatchling's actual documented extension points — not assumed here (see Risks).

Docs and runbook

musehub/docs/versioning.md and .muse/agent.md's "Standard release flow" section are rewritten: the canonical source of truth is release tags plus the commit DAG, never a hand-maintained file; the two-file manual-bump step is removed entirely; cutting a real, named release becomes "cut a release tag" (already a normal muse release add operation) and nightlies require zero manual action at all.

Phases

Each phase is fully green (tests passing, deliverables checked off) before the next begins. Test IDs use the VER_NN prefix.

Phase 1 — Core resolver library

Extract and generalize the existing aggregation/arithmetic logic into muse/core/version_resolver.py; add the manual-override check; add the SemVer→PEP 440 derivation. Pure functions — no CLI/argparse, minimal I/O (reading the override file and the commit graph only).

• resolve_version(root, repo_id, branch) -> ResolvedVersion — VER_01
• compute_next_version(base_version, agg_bump) -> str, extracted from run_suggest, unit-tested against every arithmetic case run_suggest's existing tests already cover (pre-1.0 minor-bump-not-major, patch bump, standard 1.0+ major/minor/patch) — VER_02
• semver_to_pep440(semver_str) -> str implementing every row of the table in versioning.md (nightly, alpha, beta, rc, stable, including the v prefix strip) — VER_03
• .muse/version_override manual-override check, full precedence, tested for both presence and absence — VER_04
• run_suggest refactored to call the extracted functions with zero output change — captured as a golden-fixture regression test comparing pre- and post-refactor JSON output byte-for-byte on at least 3 existing scenarios — VER_05
• Fixture-repo integration test: a hand-constructed commit history with a known release tag and a known number of unreleased commits, asserting resolve_version's emergent nightly count matches unreleased_count exactly, and matches what muse release suggest --json independently reports for the same state — VER_06

Phase 2 — CLI surface

• muse version resolve --json — VER_10
• muse version resolve --format pep440 — VER_11
• muse --version updated to call the resolver and print the canonical SemVer string (not PEP 440) — VER_12
• Regression: muse release suggest --json output is byte-identical to the pre-Phase-1 baseline captured in VER_05 — VER_13

Phase 3 — Build-time PEP 440 generation (muse + musehub)

• Decide and document the exact hatchling dynamic-version mechanism (plugin vs. generated-file), justified against hatchling's actual documented extension points — VER_20
• Wire muse/pyproject.toml's version field to that mechanism, backed by muse version resolve --format pep440 — VER_21
• python3 -m build --sdist produces a tarball whose filename/version matches the resolver's output for a known fixture commit/tag state, with no hand-typed version = line remaining in pyproject.toml — VER_22
• Same wiring applied to musehub/pyproject.toml — VER_23
• publish_muse_release.sh's MUSE_VERSION env-var override reviewed and either removed or explicitly re-scoped as a third, clearly-logged precedence tier above .muse/version_override (see Open Questions) — VER_24

Phase 4 — Docs and runbook

• musehub/docs/versioning.md rewritten: canonical source is release tags + commit DAG, not a hand-maintained file; .muse/version_override documented as the escape hatch, including precedence order — VER_30
• musehub/.muse/agent.md's "Standard release flow" section rewritten to remove the two-file manual-bump step — VER_31
• musehub/docs/deploy.md runbook updated to match the new flow end-to-end — VER_32

Phase 5 — Cross-repo consistency guard

• A check (either a muse code check invariant or a small script run as part of the deploy pipeline) that flags a hand-edited literal version = string reappearing in either pyproject.toml — a regression guard against the old habit creeping back in after Phase 3 removes it — VER_40
• Explicit decision, documented and implemented, on whether muse and musehub must resolve to the same version string when deployed together, or whether musehub should instead declare a "compatible muse version" constraint — this is a real open design question (see below), not assumed by this plan — VER_41

Acceptance Criteria

• muse --version on a clean checkout with zero unreleased commits since the last release tag prints that tag's plain SemVer string, with no pre-release suffix.
• muse --version on a checkout with N unreleased commits since the last stable tag prints X.Y.Z-nightly.N, where N matches unreleased_count from muse release suggest --json exactly — verified by running both commands against the same fixture state and comparing.
• Placing a .muse/version_override file at repo root with an explicit string causes muse version resolve --json to return that string verbatim with source: "manual_override", ignoring commit history entirely, even when the commit graph would otherwise suggest a different version.
• python3 -m build --sdist succeeds for both muse and musehub and produces a tarball whose version matches muse version resolve --format pep440's output, with no hand-edited version = line present in either pyproject.toml.
• muse release suggest --json output is unchanged — byte-identical schema and values — versus the pre-refactor baseline, for every existing regression fixture.
• musehub/docs/versioning.md and the deploy runbook no longer instruct a human to hand-bump two files on every release.

Risks

• Refactor risk: extracting run_suggest's inline logic without changing its observable behavior is the single highest-risk step in Phase 1. Mitigated by capturing pre-refactor output as a golden fixture (VER_05) before touching the code, not after.
• hatchling mechanism risk: there are a few valid approaches to a dynamic version field (a custom version-source plugin vs. pre-build file generation) with different tradeoffs in build-time dependencies and complexity. Phase 3 must pick one and justify it against hatchling's actual documented extension points, not guess or default to whichever seems simplest without checking.
• Existing env-var override risk: publish_muse_release.sh already reads a MUSE_VERSION env var as an override today. Phase 3 must decide whether that survives alongside the new .muse/version_override file-based override, and if so, make the precedence between an ephemeral env var and a checked-in, reviewable file explicit and logged — silently letting both exist with undefined precedence would recreate exactly the kind of ambiguity this plan exists to remove.

Open Questions

• Should muse and musehub be required to resolve to matching version strings when deployed together (today's implicit assumption, baked into the "both repos share the same version string" line in the current runbook), or should musehub instead declare a SemVer-range compatibility constraint against the muse version it expects — more correct for two independently, emergently versioned components, but a bigger conceptual change than this plan assumes by default. Flagged for a decision during Phase 5, not resolved here.
• Exact hatchling dynamic-version mechanism (Phase 3) — needs to be decided against hatchling's real documented capabilities during implementation, not assumed in this plan.
• Does the MUSE_VERSION env-var override in publish_muse_release.sh remain as a third precedence tier above .muse/version_override, or does it get removed now that a checked-in, reviewable file-based override exists and is strictly better for auditability (an env var leaves no trace in history; a committed file does)?
• Should the manual-override file live at .muse/version_override (this plan's default, for consistency with .muse/stability.toml's placement) or as a bare repo-root VERSION file (a more common cross-ecosystem convention, explicitly what was proposed in discussion before this plan was written)? Flagged for confirmation before Phase 1 locks the exact path.

Implementation Order

Phase 1 → Phase 2 → Phase 3 → Phase 4 → Phase 5. Strictly sequential: nothing in Phase 2 can call a resolver that doesn't exist yet; Phase 3's build-time generation depends on Phase 2's --format pep440 CLI surface; Phase 4's docs describe behavior that must already exist to document accurately; Phase 5's consistency guard has nothing to guard until Phases 3–4 have actually removed the old hand-bump habit.

Plan-only issue. Implementation begins only once this plan is reviewed and the Open Questions above are resolved — particularly the override-file path and the MUSE_VERSION env-var precedence question, both of which affect Phase 1's and Phase 3's exact deliverables.