muse code test — implementation audit, cache behavior, and documentation

muse code test — Implementation Audit, Cache Behavior, and Documentation

Background

muse code test is one of the most powerful day-to-day commands in the workspace, but its runtime is erratic and its cache invalidation rules are opaque. Sometimes it returns in under a second; sometimes it blocks for tens of minutes with no explanation. This unpredictability makes it difficult to trust the tool in practice — agents (and humans) can't reason about when it is safe to call it, when they should wait, or when a slow run signals something broken rather than just a cold cache.

The implementation lives in three layers:

• muse/cli/commands/test_cmd.py — CLI entrypoint; orchestrates selection, execution, and output
• muse/core/test_selection.py — symbol-graph-driven test selector; uses CallGraphCache and SymbolCache
• muse/core/test_runner.py — pytest subprocess wrapper and result collector

The docstring in test_selection.py claims "a warm-cache run completes in <50 ms." The cold-cache rebuild path is not documented at all — neither what triggers it, how long it takes, nor how to tell the difference from the output.

Problem Statement

There are three distinct problems:

1. Cache invalidation is a black box. The cache lives in .muse/ and is keyed against committed snapshots. But it is not clear which operations bust it: new commits, muse code add, muse index rebuild, adding new import statements, or changing the set of test files. Agents trigger cold rebuilds without knowing it.

2. The slow path has no visible feedback. When the cache is cold, muse code test may spend minutes re-parsing every blob in the object store and rebuilding the call graph. There is no progress indicator, no log line saying "cache cold — rebuilding," and no estimated time. The command looks hung.

3. There is no documentation. The expected behavior, cache lifecycle, invalidation rules, and performance envelope are not documented anywhere users or agents can find them. The only documentation is the module docstring in test_selection.py, which describes the warm-cache case only.

Goal

After this issue is resolved:

• The exact cache invalidation rules are documented and verified by tests.
• muse code test emits a visible log line when it enters the cold-rebuild path, including what triggered the invalidation.
• A muse code test --cache-status flag (or similar) lets callers inspect whether the cache is warm before committing to a run.
• A user-facing reference doc (docs/code-test.md) covers behavior, performance expectations, invalidation triggers, and how to force a cache warm-up.

Phases

Phase 1 — Audit and document the cache invalidation rules

Read the full implementation of CallGraphCache and SymbolCache. Map the exact conditions under which each is invalidated. Verify against the test suite in tests/test_core_test_selection.py, tests/test_core_test_runner.py, and tests/test_test_selection_speedup.py.

Deliverables

• CT_01 — Written analysis (committed to docs/code-test-internals.md) covering: what the cache key is, what files it reads, what invalidates it, and the performance envelope for warm vs cold paths
• CT_02 — Existing test coverage gap map: which invalidation triggers have tests and which do not

Phase 2 — Fix the cold-path UX (progress feedback)

When muse code test enters the cold-rebuild path it must say so. A single logger.warning or stderr line is sufficient — users and agents need to know the command is working, not hung.

Deliverables

• CT_03 — Cold-rebuild path emits a human-readable log line: "⚠️ code-test: cache cold — rebuilding call graph (this may take a minute)" or equivalent
• CT_04 — --json output includes "cache_hit": true/false at the top level so callers can detect which path was taken
• CT_05 — Test: mock a cache miss, invoke muse code test --json, assert cache_hit == false and the warning line appears in output

Phase 3 — Fill invalidation test gaps

For every invalidation trigger identified in Phase 1 that lacks a test, add one.

Deliverables

• CT_06 — Test: new commit advances snapshot → cache is invalidated on next muse code test
• CT_07 — Test: unchanged snapshot → cache is reused (no re-parse); verified by counting object-store reads
• CT_08 — Test: new import added to a source file → call graph is rebuilt; newly reachable test is included in selection
• CT_09 — Test: test file added with no production changes → new test is included without full cache bust
• CT_10 — Test: muse code index rebuild followed by muse code test uses the fresh index, not a stale cache

Phase 4 — muse code test --cache-status flag

Add a lightweight flag that inspects the current cache state and exits without running any tests.

CLI

muse code test --cache-status [--json]

Output (--json)

{
  "exit_code": 0,
  "cache_hit": true,
  "cache_snapshot_id": "sha256:abc...",
  "head_snapshot_id":  "sha256:abc...",
  "stale": false,
  "cache_age_seconds": 42,
  "callgraph_symbols": 1847,
  "test_functions": 312
}

Deliverables

• CT_11 — muse code test --cache-status --json returns the schema above; exit 0 whether warm or cold
• CT_12 — stale: true is returned when cache snapshot ID ≠ HEAD snapshot ID
• CT_13 — stale: false is returned when they match
• CT_14 — Test: invoke --cache-status on a fresh repo (no cache file) → graceful JSON response with stale: true, no exception

Phase 5 — User-facing documentation

Write docs/code-test.md — the canonical reference for muse code test.

Deliverables

• CT_15 — docs/code-test.md committed; covers: how test selection works, cache lifecycle, invalidation triggers, warm vs cold performance expectations, --cache-status usage, and how to force a warm-up with muse code index rebuild
• CT_16 — Inline docstring in muse/core/test_selection.py::select_tests updated to cross-reference docs/code-test.md and document the cold path

Acceptance Criteria

• muse code test never appears to hang silently; cold rebuilds are always announced
• --json output always includes "cache_hit"
• Any cache miss traceable to a specific trigger has a test that proves the trigger works correctly
• docs/code-test.md exists and covers all behavior an agent or user needs to reason about the command correctly

Out of Scope

• Changing the cache storage format or invalidation strategy (audit only, no redesign)
• Distributed or shared caches
• Windows filesystem compatibility