semver classifier flags breaking_changes on a reorder-only diff despite its own 'no semantic change' summary
Summary
While researching Build With Muse Episode 07 (Semantic Diff), found a real inconsistency in classify_delta() (muse/core/semver_classifier.py): a commit that purely reorders two top-level functions within a file — no symbol added, removed, or modified — gets classified with sem_ver_bump: "minor" and a non-empty breaking_changes list, despite the diff's own ops entry explicitly summarizing itself as no semantic change.
Reproduction
- Commit a file with two functions, e.g.:
def helper_a(x): return x def helper_b(x): return x * 2 - Commit again with just the two functions' order swapped (no other edits).
muse diff HEAD~1 HEADcorrectly reports zero symbol-level changes:M extra.py 1 modified file- But
muse read-commit HEAD --json | jq '.structured_delta'shows:{ "ops": [ { "op": "replace", "address": "extra.py", "old_summary": "extra.py (before)", "new_summary": "extra.py (reformatted — no semantic change)" } ], "summary": "1 modified file", "sem_ver_bump": "minor", "breaking_changes": ["extra.py"] }
The op's own new_summary says "no semantic change" in plain English, yet the same delta is classified as a breaking, minor-version-bump change one field over.
Hypothesis (not confirmed)
diff()'s fallback path for a file where nothing at the symbol level changed (a pure reorder) apparently produces a top-level replace op keyed by the whole file address ("extra.py") rather than a patch op with an empty child_ops list. classify_delta()'s _classify_replace() branch then treats a file-level replace op the same way it would treat replacing an actual symbol — defaulting to breaking/minor — with no way to distinguish "the file's bytes changed but I found zero semantic content changes" from an actual content replacement it couldn't otherwise interpret.
Ask
- Confirm whether this fallback path is intentional conservatism (file-level replace with no visibility into content = assume worst case) or a genuine gap.
- If intentional: the
new_summarytext is misleading and should not claim "no semantic change" while the classification says otherwise — pick one honest story. - If a gap: either emit a
patchop with emptychild_ops(matching the pattern used elsewhere, e.g. real rename/move ops) instead of a file-levelreplace, or teach_classify_replaceto recognize this specific no-op-content case and classify it asnone. - Add a regression test: reorder-only diff must have
sem_ver_bump == "none"andbreaking_changes == [].
Priority
Not urgent — deliberately not chasing this down mid-recording. Circle back once the Build With Muse Season 1 season is wrapped; this doesn't block any episode script since Episode 07 already avoided making any semver claim about this specific case.
Related
- Episode 07 script (
presentations/build-with-muse/episode-07/semantic-diff.md) — documents choosing not to use this detail on screen, and this ticket, in its own production notes.
Source
Found by Claude Sonnet 5 while researching Episode 07, 2026-09-12.
Fixed — confirmed your hypothesis exactly, and it was a gap, not intentional conservatism.
Confirmation (Ask #1)
build_diff_ops()(muse/plugins/code/symbol_diff.py) does exactly what you suspected: when every symbol'scontent_idin a modified file is unchanged (pure reorder here — same mechanism behind the comment-only case in #92), it emits a whole-fileReplaceOplabeled"<path> (reformatted — no semantic change)"instead of apatchop with emptychild_ops._classify_replace()(muse/core/semver_classifier.py) pattern-matchesnew_summaryfor"renamed to"/"signature"/"implementation"but had no case for this exact fallback text, so it fell through to the conservative "unrecognised summary" catch-all — classified breaking/minor, confidence 0.4 — directly contradicting the op's own "no semantic change" claim one field over.Gap, not intentional (Ask #3)
There's no principled reason to assume worst-case when the diff engine already determined, with certainty, that nothing semantic changed — the two fields were just disagreeing with each other. Went with your option 3's second suggestion: taught
_classify_replaceto recognize this specific case rather than changing the op shape, since it's a smaller, more surgical change than threading apatch/empty-child_opsshape through the whole fallback path. Added a branch checking for"no semantic change"innew_summary, classifying itinvisible(no API impact, no bump, confidence 1.0) — placed just before the catch-all, so the existing generic"reformatted"(without the"no semantic change"suffix) catch-all test is untouched and still passes.Regression tests (Ask #4)
tests/test_sem_ver.py::TestClassifyDeltaReplace::test_replace_no_semantic_change_is_invisible— synthetic-op unit test matching the file's existing style.tests/test_reorder_only_semver_classification.py— real-CLI end-to-end test reproducing your exact repro steps (commit two functions, commit again with only their order swapped, assertsem_ver_bump == "none"andbreaking_changes == []). This test reproduced your exact contradictory JSON (sem_ver_bump: "minor",breaking_changes: ["extra.py"]) when run against the pre-fix code.Confirmed genuine RED on both new tests (reverted the fix, both failed for the right reason — same symptom from your ticket body) before GREEN with the fix restored.
Verification
Targeted regression suite —
test_sem_ver.py(123 tests, including the untouched catch-all test), the two new tests,test_cmd_commit.py,test_cmd_diff.py(the only two direct callers ofclassify_deltapermuse code impact) — all pass (336 passed), zero regressions.Merged to
dev→main, pushed tolocal/staging/productionfor both branches.