gabriel / muse public
Closed #101 Bug
filed by gabriel human · 2 days ago

Some commits record a tracked file's path a second time in directories, causing false 'deleted' status

0 Anchors
Blast radius
Churn 30d
0 Proposals

Some commits record a tracked file's exact path a second time in directories, causing false "deleted" status

Background

Discovered incidentally while merging an unrelated doc-only commit into ~/ecosystem/presentations/build-with-muse's dev branch. muse status reported 19 files as deleted immediately after a clean branch switch, even though every one of those files exists on disk, unmodified, with correct content.

Repro / evidence

Repo: ~/ecosystem/presentations/build-with-muse, commit sha256:07c10b6184a651841b238710523048c817c5c062f2435c64628f6e407e9d55e3 ("Fix stale 'under seven minutes' claim in THE POINT section"), snapshot sha256:f7b950d6fecd6a30cf3b1063b172364d4e5776e567e5f7f5eaaae969477cc257.

muse read-snapshot sha256:f7b950d6fecd6a30cf3b1063b172364d4e5776e567e5f7f5eaaae969477cc257 --json

The manifest (files) correctly lists these paths with real content hashes:

"episode-03/everything-is-content-addressed.md" -> sha256:635d415d4b32e6c729b9cc0d4e2b1ea29c5295e44c3471f02622f5b94040d4e6

But the same snapshot's directories list also contains the exact same path:

"episode-03/everything-is-content-addressed.md"

19 paths across episode-03 through episode-21 are affected this way (full list in the JSON payload, all .md episode-script files). No real directory exists at any of these paths on disk — a regular file does.

Effect

muse status compares the committed directories list against the live working tree. Since no directory exists where one is (incorrectly) recorded, every one of these paths is reported as a deleted directory:

"deleted": [
    "episode-03/everything-is-content-addressed.md/",
    "episode-04/identity-without-passwords.md/",
    ... 17 more ...
]

This is misleading and alarming (it reads as data loss) but is not actual data loss — the files are intact, correctly tracked in manifest, and readable normally. It's a display/bookkeeping bug in how the affected commits' snapshots were constructed, not a working-tree or object-store integrity problem.

Root cause — not yet identified, hypothesis only

muse/core/snapshot.py::walk_workdir_with_dirs (the function that computes (manifest, directories) together from a live os.walk during commit) looks correct on inspection: it only ever appends real dirpath values from os.walk, never a file's own path. That rules out ordinary muse commit as the direct cause, unless a same-named directory genuinely existed on disk at commit time (unlikely for 19 different episode scripts).

More likely candidates, not yet investigated:

  • A merge/rebase/replay path (muse/core/rebase.py::replay_one or the merge engine) that unions a parent commit's directories with a new manifest without pruning entries that transitioned from directory-with-content to plain file (or vice versa) across the merge.
  • A commit created via a path that doesn't go through walk_workdir_with_dirs at all (e.g. commit-tree, a scripted/API commit, or an older code path predating musehub#60 Phase 1's SnapshotRecord.directories schema work).
  • These .md files may have started life as placeholder directories (e.g. someone ran mkdir episode-03/everything-is-content-addressed.md by mistake, committed it as an empty tracked directory, then later replaced it with a real file without the old directory entry ever being explicitly removed from a subsequent commit's carried-forward state).

Suggested investigation

  1. Walk this repo's commit history (muse log --json) and bisect which commit first introduced the duplicate entry for one of the affected paths (muse read <commit> --json --manifest at each step, checking both manifest and the snapshot's directories).
  2. Once the introducing commit is found, identify what command sequence produced it — that pins down whether this is a live commit bug, a merge/replay bug, or a stale-migration artifact.
  3. Add an invariant check (muse check or muse verify?) that flags any snapshot where a path appears in both manifest and directories simultaneously — that state should never be valid, and should be either rejected at commit time or at least surfaced as a warning rather than silently producing misleading status output later.

Acceptance criteria

  • Root cause identified and documented (which code path produces this).
  • A regression test reproducing the exact commit-time bug.
  • muse status no longer reports a false "deleted" for a path that is correctly tracked as a file.
  • Ideally: a muse verify/muse check invariant catches this class of corruption at write time, not just at read/status time.
Activity2
gabriel opened this issue 2 days ago
gabriel 12 hours ago

Tracked under parent roadmap ticket #104 (comprehensive fix for the empty-directory sentinel design flaw all three of these — plus one already-shipped fix — are symptoms of). Not closing this individually; #104 defines the multi-phase plan and the actual acceptance gate for considering this whole bug family resolved.

gabriel 11 hours ago

Fixed in sha256:47e5542341c5a936597b3974abf274d75185c50c3b89917c21d8face06fa6691 (shipped as muse 0.2.1rc10, part of the #104 roadmap fix).

Root cause identified via bisection (Phase 1 task): the exact introducing commit in ~/ecosystem/presentations/build-with-muse is sha256:903cc80e8b89919349486c10d2d255e8deea0098694ffd5d0f4fe0252b86617d ('Add Episode 03 script: Everything Is Content-Addressed') -- an ordinary muse commit transitioning a placeholder directory to a real file in the same commit that carries forward HEAD's committed empty dirs. Not a merge/rebase/cherry-pick path as hypothesized. The stale directory entry was carried forward, uncorrected, across every subsequent commit.

The already-shipped subtraction fix in plugin.py::snapshot() (0.2.1rc8) already prevents this for new commits going forward. This fix adds a locked-in regression test (TestManifestDirectoriesMutualExclusion in tests/test_empty_dir_sentinel_invariants.py) plus the kind discriminator that makes the underlying ambiguity structurally impossible.

Cleaned up the real-world instance: 16 of the 19 stale ghost-directory entries in ~/ecosystem/presentations/build-with-muse were removed in sha256:4ed52e48060146a7c75184bd215d4540af33468b128f40fec178eb8695cb8d3b (dev branch of that repo). The remaining 3 (episode-03/04/05) are entangled with real, unrelated pending script edits in that repo's working tree and were deliberately left for gabriel to resolve when those edits are committed -- they'll self-resolve automatically via the same subtraction fix, no extra action needed.

closed this issue 11 hours ago