gabriel / muse public
symlog-filename-too-long.md markdown
92 lines 4.0 KB
Raw
sha256:ac29b8ba9021514a03ab2782d92bf67671f0efa5b3b70d46f7598c5d4e923378 docs: record muse#74 Phase 4 live verification (SCR_13, SCR_14) Sonnet 5 13 days ago

symlog write fails silently when symbol path exceeds OS filename limit

Background

When committing a markdown file whose section headings are long, Muse emits a warning and skips the symbol journal entry:

⚠️ symlog write failed for commit sha256:bb322b… — symbol journal may be
incomplete: [Errno 63] File name too long:
'/Users/gabriel/ecosystem/muse/.muse/symlogs/docs/issues/mwp-5-clone-retry-503.md/
MWP-5%20%E2%80%94%20Clone%2Ffetch%2Fpull%20honor%20Retry-After%20with%20a%20bounded%20poll%20%28fixes%20RC-5%29.
Phases%20%E2%80%94%20load-bearing%2C%20TDD%2C%20never%20skip%20ahead.Phase%205%20%E2%80%94%20real-urllib%20E2E%20%2B%20docs%20%2B%20close%20%E2%9C%85'

The commit itself succeeds and is content-addressed correctly. Only the symlog entry for that symbol is missing — muse code symbol-log for the long-heading symbol will show no history.

Trigger: the symlog filename is derived by URL-encoding the symbol address (file.md::Section Heading). On macOS (APFS) and Linux (ext4/xfs) the per-component filename limit is 255 bytes. A long section heading with multi-byte Unicode (em-dashes , encoded as %E2%80%94, slashes, parentheses) blows past this quickly.

Root cause

The symlog writer constructs a filesystem path by joining the repo-relative file path with the URL-encoded symbol name and a .jsonl extension. No length check is performed before the open() call. [Errno 63] File name too long is caught and re-emitted as a warning, so the commit is not aborted — but the journal entry is permanently missing for that symbol.

Relevant code path (unverified exact line — run muse code grep "symlog" --json):

muse/core/symlog.py  (or equivalent)

Goal

  1. Detect when the constructed symlog filename would exceed the OS limit (255 bytes per component is the POSIX minimum; use that as the ceiling).
  2. Fall back to a deterministic truncation or hash-based filename that stays within the limit while remaining unique and human-readable where possible.
  3. The muse code symbol-log command must return correct history for symbols with long names — no silent gaps.
  4. The warning is replaced by either: (a) no output (silent success via the fallback), or (b) a debug-level log if the fallback fires, never a user-visible warning on a clean commit.

Proposed fix

Option A — truncate + hash suffix (recommended)

filename = encode(symbol_name)
if len(filename) > MAX_COMPONENT_BYTES:
    truncated = filename[:200]          # leave room for suffix
    suffix    = sha256(symbol_name)[:8] # collision-resistant short hash
    filename  = f"{truncated}_{suffix}"

The truncated prefix stays human-readable in ls output; the hash suffix guarantees uniqueness even across symbols that share a long common prefix.

Option B — always hash long names

If the symbol name exceeds a threshold, use sha256(full_name).hex() as the filename entirely and store the original name inside the .jsonl payload. Loses human-readability but is simpler to implement correctly.

Either option must be accompanied by a migration that renames any existing overlong paths (unlikely in practice — the error prevents them from being written in the first place, so no existing corrupt entries to fix).

Acceptance criteria

  • [ ] Committing a file with a section heading long enough to exceed 255 bytes (URL-encoded) produces no warning and a correct symlog entry.
  • [ ] muse code symbol-log "docs/issues/long-heading-file.md::Long Section…" returns the expected history.
  • [ ] Existing short-name symlogs are unaffected (no regression).
  • [ ] Unit test covering the boundary: symbol name at 254 bytes, 255 bytes, and 256 bytes (encoded) all produce valid filenames.

Out of scope

  • Changing the symlog storage format (.jsonl per symbol) — the format is fine.
  • Handling the rare case where the directory component (the file path portion) also exceeds 255 bytes — that would require a separate path-component truncation pass and is not triggered by today's codebase.
File History 2 commits
sha256:ac29b8ba9021514a03ab2782d92bf67671f0efa5b3b70d46f7598c5d4e923378 docs: record muse#74 Phase 4 live verification (SCR_13, SCR_14) Sonnet 5 13 days ago
sha256:2562dffa0a0822ac1bdea854f9b267843c6ce95b497a9dc5c55837c80a3ebd0a feat: domain_command_registry — Phase 1 of muse#74 (SCR_01-03) Sonnet 4.6 patch 13 days ago