gabriel / muse public
code-test.md markdown
144 lines 6.4 KB
Raw
sha256:ce542c4d0f6dacc9520c02d0e3378bf96bc377d118a7402f2bbc8af1749b08e0 feat(code-test): audit cache internals, add cache_hit/--cac… Sonnet 5 minor ⚠ breaking 9 hours ago

muse code test — symbol-graph–driven test selection

muse code test knows exactly which symbols changed and which tests call them (via the committed call graph), so it runs the minimum set of tests needed to validate a change — instead of guessing from file names or running the full suite every time.

How it works

  1. Diff — compares the HEAD snapshot's symbols against the working tree to find every modified, added, or deleted symbol.
  2. Graph — walks the call graph (BFS, bounded by --depth, default 3) from every test function to find which tests transitively call each changed symbol.
  3. Prioritise — orders tests by risk: failure streaks first, flaky tests second, unknown tests third.
  4. Execute — runs the selected tests as isolated subprocesses with configurable parallelism.
  5. Record — persists pass/fail results to .muse/cache/test_history.json for future prioritisation and flaky-test detection.

Usage

muse code test                                        # smart selection (default)
muse code test --all                                   # run everything, no selection
muse code test --symbol "billing.py::compute_total"    # force-select covering tests
muse code test tests/test_billing.py                    # run a specific file/node ID directly
muse code test --dry-run                                # show what would run, without running it
muse code test --ci                                     # run the full .muse/ci.toml gate suite
muse code test --history                                # show historical pass/fail summary
muse code test --cache-status                           # show cache warmth, run nothing

All subcommands accept --json for structured output.

Caching — why it's fast, and how to tell

Test selection depends on two on-disk caches:

  • .muse/cache/symbols.json — parsed symbol trees, keyed by file content hash.
  • .muse/cache/callgraph.json — per-file call subgraphs, keyed by file content hash.

Both are content-addressed: a cache entry is keyed by the hash of the file's bytes, so it's either present (warm — reused, no re-parse needed) or absent (cold — the file is freshly parsed and the result cached for next time). There is no "stale" cache entry in the traditional sense — a file either changed (new hash, cold until parsed once more) or it didn't (same hash, forever warm). See docs/code-test-internals.md for the full cache-key model and audit.

cache_hit in --json output

Every --json run (and --dry-run --json) includes a cache_hit field in its selection object:

{"selection": {"cache_hit": false, "...": "..."}}

cache_hit: false means at least one touched file was cold and had to be re-parsed this run — normal on the very first run after a commit touches new files, or after a long gap. On the cold path a warning is printed:

⚠️ code-test: cache cold — rebuilding symbol/call-graph data (N file(s) re-parsed)

This is purely informational — it never changes correctness, only speed. Test-selection results are identical whether the cache is warm or cold.

muse code test --cache-status

Inspect cache warmth for the current HEAD without running (or selecting) any tests — this never parses anything, so it's effectively instant:

muse code test --cache-status --json
{
  "mode": "cache-status",
  "cache_status": {
    "cache_hit": true,
    "stale": false,
    "head_snapshot_id": "sha256:abc123...",
    "semantic_files_total": 42,
    "files_warm": 42,
    "files_cold": 0,
    "symbol_cache_entries": 118,
    "callgraph_cache_entries": 97,
    "symbol_cache_age_seconds": 12.4,
    "callgraph_cache_age_seconds": 12.4
  }
}
Field Meaning
cache_hit true when every semantic file in HEAD's manifest is already warm.
stale true when at least one semantic file in HEAD's manifest has no cache entry yet (the inverse of cache_hit).
head_snapshot_id Snapshot ID at HEAD, or null on a repo with no commits.
semantic_files_total / files_warm / files_cold Per-file warmth breakdown across the HEAD manifest.
symbol_cache_entries / callgraph_cache_entries Total entries currently on disk in each cache (may exceed files_warm — old entries for files no longer in HEAD aren't actively pruned).
symbol_cache_age_seconds / callgraph_cache_age_seconds Seconds since each cache file was last written, or null if it doesn't exist yet.

On a fresh repository with no commits and no cache files, every count is 0/null and cache_hit is trivially true (nothing to be cold about) — --cache-status never raises, regardless of repo state.

Flags reference

Flag Effect
TARGET (positional) Pytest node IDs or file paths to run directly, bypassing graph selection.
--all, -a Run every discovered test, ignoring the working-tree diff.
--symbol ADDR, -s ADDR Force-select tests covering this symbol address (repeatable).
--depth N, -d N Call-graph BFS depth for selection (default 3).
--workers N, -w N Parallel worker count.
--timeout S Wall-clock budget per partition in seconds (0 = unlimited).
--dry-run Print selected targets without executing them.
--no-save Don't persist results to test history.
--history Print historical pass/fail summary and exit.
--flaky Print only flaky tests from history and exit.
--ci Run the full CI gate suite from .muse/ci.toml.
--cache-status Print cache warmth for HEAD and exit; never runs or selects tests.
--extra ARGS... Extra arguments forwarded verbatim to pytest.
--json, -j Emit structured JSON instead of human-readable text.

Why call-graph selection beats file-name heuristics

File-name heuristics (test_foo.py ↔ foo.py) break the moment a repository uses a non-obvious naming scheme, a shared test module, or a parameterized fixture that exercises symbols from many files. The call graph knows exactly what each test calls — it does not guess. muse code test falls back to the file-name heuristic only when a changed symbol has no call-graph coverage at all (reported via fallback_used in --json output).

See also

  • docs/code-test-internals.md — cache-key model, invalidation audit, and the gap-coverage map behind this command's test suite.
File History 1 commit
sha256:ce542c4d0f6dacc9520c02d0e3378bf96bc377d118a7402f2bbc8af1749b08e0 feat(code-test): audit cache internals, add cache_hit/--cac… Sonnet 5 minor ⚠ 9 hours ago