gabriel / muse public
Closed #61 Bug
filed by gabriel human · 116 days ago

symlog write silently skipped when URL-encoded symbol name exceeds 255-byte filename limit

0 Anchors
— Blast radius
— Churn 30d
0 Proposals

symlog write fails silently when symbol path exceeds OS filename limit

Background

When committing a markdown file whose section headings are long, Muse emits a warning and skips the symbol journal entry:

⚠️ symlog write failed for commit sha256:bb322b… — symbol journal may be
incomplete: [Errno 63] File name too long:
'/Users/gabriel/ecosystem/muse/.muse/symlogs/docs/issues/mwp-5-clone-retry-503.md/
MWP-5%20%E2%80%94%20Clone%2Ffetch%2Fpull%20honor%20Retry-After%20with%20a%20bounded%20poll%20%28fixes%20RC-5%29.
Phases%20%E2%80%94%20load-bearing%2C%20TDD%2C%20never%20skip%20ahead.Phase%205%20%E2%80%94%20real-urllib%20E2E%20%2B%20docs%20%2B%20close%20%E2%9C%85'

The commit itself succeeds and is content-addressed correctly. Only the symlog entry for that symbol is missing — muse code symbol-log for the long-heading symbol will show no history.

Trigger: the symlog filename is derived by URL-encoding the symbol address (file.md::Section Heading). On macOS (APFS) and Linux (ext4/xfs) the per-component filename limit is 255 bytes. A long section heading with multi-byte Unicode (em-dashes —, encoded as %E2%80%94, slashes, parentheses) blows past this quickly.

Root cause

The symlog writer constructs a filesystem path by joining the repo-relative file path with the URL-encoded symbol name and a .jsonl extension. No length check is performed before the open() call. [Errno 63] File name too long is caught and re-emitted as a warning, so the commit is not aborted — but the journal entry is permanently missing for that symbol.

Relevant code path (unverified exact line — run muse code grep "symlog" --json):

muse/core/symlog.py  (or equivalent)

Goal

  1. Detect when the constructed symlog filename would exceed the OS limit (255 bytes per component is the POSIX minimum; use that as the ceiling).
  2. Fall back to a deterministic truncation or hash-based filename that stays within the limit while remaining unique and human-readable where possible.
  3. The muse code symbol-log command must return correct history for symbols with long names — no silent gaps.
  4. The warning is replaced by either: (a) no output (silent success via the fallback), or (b) a debug-level log if the fallback fires, never a user-visible warning on a clean commit.

Proposed fix

Option A — truncate + hash suffix (recommended)

filename = encode(symbol_name)
if len(filename) > MAX_COMPONENT_BYTES:
    truncated = filename[:200]          # leave room for suffix
    suffix    = sha256(symbol_name)[:8] # collision-resistant short hash
    filename  = f"{truncated}_{suffix}"

The truncated prefix stays human-readable in ls output; the hash suffix guarantees uniqueness even across symbols that share a long common prefix.

Option B — always hash long names

If the symbol name exceeds a threshold, use sha256(full_name).hex() as the filename entirely and store the original name inside the .jsonl payload. Loses human-readability but is simpler to implement correctly.

Either option must be accompanied by a migration that renames any existing overlong paths (unlikely in practice — the error prevents them from being written in the first place, so no existing corrupt entries to fix).

Acceptance criteria

  • Committing a file with a section heading long enough to exceed 255 bytes (URL-encoded) produces no warning and a correct symlog entry.
  • muse code symbol-log "docs/issues/long-heading-file.md::Long Section…" returns the expected history.
  • Existing short-name symlogs are unaffected (no regression).
  • Unit test covering the boundary: symbol name at 254 bytes, 255 bytes, and 256 bytes (encoded) all produce valid filenames.

Out of scope

  • Changing the symlog storage format (.jsonl per symbol) — the format is fine.
  • Handling the rare case where the directory component (the file path portion) also exceeds 255 bytes — that would require a separate path-component truncation pass and is not triggered by today's codebase.
◎ Activity2
●
gabriel opened this issue 116 days ago
○
gabriel 31 days ago

Recurred again just now (2026-09-22), filing the muse#104 empty-dir-sentinel roadmap ticket -- same signature:

⚠️ symlog write failed for commit sha256:09d4b6592647086ae8fe7356c0369b867aba50c718c3ae0b9622088c0490db3a — symbol journal may be incomplete: [Errno 63] File name too long: '/Users/gabriel/ecosystem/muse/.muse/symlogs/docs/issues/issue-104-empty-dir-sentinel-roadmap.md/Empty-directory%20sentinel...'

This keeps happening any time a markdown doc has a reasonably long section heading -- which is common for issue/roadmap docs specifically, i.e. exactly the kind of file this workspace writes constantly. Bumping priority mentally given the recurrence rate; the proposed fix in the local doc (docs/issues/symlog-filename-too-long.md, Option A -- truncate + hash suffix) still looks like the right shape.

○
gabriel 31 days ago

Fixed in sha256:d08abfdaa26df2e781fa4b7aae0ebe64cfdc41a3d8df3d38bcac09cd3c7aab85 (shipped as muse 0.2.1rc9).

Root cause: symlog_path() had no length check on the percent-encoded leaf filename, so any symbol name whose encoded form exceeded the OS's 255-byte NAME_MAX raised OSError inside append_symlog, which upstream callers in commit.py swallowed silently — the commit appeared to succeed but the journal entry was lost.

Fix: names whose percent-encoded form exceeds 200 UTF-8 bytes now fall back to a hashed leaf name (a !-prefixed SHA-256 digest — ! can never appear in normal percent-encoded output), with a small per-directory sidecar index (.leaf_index.json) mapping the hash back to the original symbol name for list_symlog_symbols/list_symlog_symbols_for_file to resolve on read.

Verified end-to-end against the real installed 0.2.1rc9 CLI: committed a file with a 251-character symbol name, confirmed no OSError, confirmed muse code symbols/muse code symbol-log both correctly see the entry, confirmed the on-disk hashed leaf + sidecar index round-trip.

TDD: 8 new tests in tests/test_core_symlog.py::TestLongSymbolNameFilenameLimit, confirmed RED (6/8 failing with the exact production OSError) before the fix, GREEN after. Full symlog + commit regression suite (306 tests across test_core_symlog.py, test_cmd_commit_symlog.py, test_cmd_symlog_lifecycle.py, test_cmd_symlog_read.py, test_cmd_symlog_refs.py, test_cmd_commit.py) passes with zero regressions.

✓
closed this issue 31 days ago