gabriel / musehub public
Closed #23 Enhancement
filed by gabriel human · 48 days ago

feat(intel): Breakage page — pure Python provider, route, template, SCSS, dashboard card

0 Anchors
Blast radius
Churn 30d
0 Proposals

Multi-Dimensional Code Intelligence Findings

Before designing this feature, muse code was run across six axes.

muse code breakage — current output shape

file_count:    767   (scanned against HEAD snapshot)
total_issues:  195
issue_types:
  stale_import  195   import references a name that no longer exists in HEAD

severity:
  warning  195

issue keys:  issue_type, file_path, description, severity

top affected files:
  musehub/services/musehub_wire.py                   9 issues
  tests/test_commit_signature_verification.py        7 issues
  musehub/crypto/keys.py                             3 issues
  musehub/services/musehub_intel_providers.py        3 issues

The only issue_type today is stale_import — a symbol is imported but no longer exists in the HEAD snapshot. The schema supports future types (e.g. missing_symbol, type_mismatch) via the issue_type column.

muse code hotspots — top churners (blast radius for this feature)

musehub/services/musehub_wire.py::wire_push_stream       40 changes
musehub/mcp/dispatcher.py::_call_tool                    22 changes
musehub/storage/backends.py::LocalBackend                16 changes
musehub/api/routes/wire.py::push_stream                  16 changes

muse code gravity — highest downstream blast radius

musehub/storage/backends.py::S3Backend._key             38.8 gravity_pct
musehub/storage/backends.py::S3Backend._get_client      38.7
musehub/storage/backends.py::LocalBackend._resolve_root 38.6
musehub/storage/backends.py::LocalBackend._path         38.5

Existing DB state

No musehub_intel_breakage_* table exists yet. A breakage_count column exists on musehub_intel_blast_risk (unrelated — don't reuse it). Next migration: 0017.


Pure-Python Detection Algorithm (no subprocess)

muse code breakage detects structural breakage in the working tree by comparing the HEAD snapshot's symbol table against import references. The pure-Python equivalent:

For each push (HEAD snapshot):

  1. Fetch HEAD snapshot manifest → map{ file_path: object_id }

  2. Build known_symbols set:
     For each file in manifest:
       parse_symbols(src, path) → collect all symbol names + qualified_names

  3. Build known_modules set from manifest file paths
     (e.g. "musehub/services/wire.py" → module "musehub.services.wire")

  4. For each Python file in manifest:
     parse_symbols(src, path) → find import records (kind="import")
     For each import:
       qualified_name = "import::<dotted.module>::<sym>"
       If sym not in known_symbols AND module not in known_modules:
         emit issue: { issue_type="stale_import", file_path, description, severity="warning" }

  5. Batch-upsert into musehub_intel_breakage_issues
     (idempotent: issue_id = blob_id(repo_id:ref:file_path:issue_type:description))

  6. Upsert summary into musehub_intel_breakage_meta
     (total_issues, warning_count, error_count, ref)

All data flows from get_backend(owner, slug).get(object_id) via parse_symbols() — no subprocess, no _run_muse, no on-disk repo.


Page Wireframe

╔══════════════════════════════════════════════════════════════════════════════╗
║  ← Intel Hub                                                                ║
║                                                                              ║
║    ⚡ BREAKAGE                                                               ║
║    Structural breakage detected at push time — stale imports, missing        ║
║    symbols, and cross-module reference failures in the HEAD snapshot.        ║
║                                                                              ║
╠══════════════════════════════════════════════════════════════════════════════╣
║                                                                              ║
║  ┌─────────────────┐  ┌─────────────────┐  ┌─────────────────┐             ║
║  │      195        │  │      195        │  │       0         │             ║
║  │   TOTAL ISSUES  │  │    WARNINGS     │  │     ERRORS      │             ║
║  └─────────────────┘  └─────────────────┘  └─────────────────┘             ║
║   (spectral txt)        (orange border)      (danger border)                ║
║                                                                              ║
║  FILTER  ┌──────────────────┐ ┌──────────┐ ┌──────────┐                   ║
║          │  stale_import    │ │ warnings │ │  errors  │                   ║
║          └──────────────────┘ └──────────┘ └──────────┘                   ║
║                                                                              ║
║  SHOW  [ 20 ]  [ 50 ]  [ 100 ]                                              ║
║                                                                              ║
║  ┌──────────────────────────────────────────────────────────────────────┐   ║
║  │  FILE                        TYPE             DESCRIPTION            │   ║
║  ├──────────────────────────────────────────────────────────────────────┤   ║
║  │  musehub/services/           stale_import     imports 'blob_id' but  │   ║
║  │    musehub_wire.py                            no symbol with that    │   ║
║  │                                               name exists in HEAD    │   ║
║  ├──────────────────────────────────────────────────────────────────────┤   ║
║  │  musehub/crypto/keys.py      stale_import     imports 'short_id'…    │   ║
║  ├──────────────────────────────────────────────────────────────────────┤   ║
║  │  tests/test_wire.py          stale_import     imports 'split_id'…    │   ║
║  └──────────────────────────────────────────────────────────────────────┘   ║
║                                                                              ║
╠══════════════════════════════════════════════════════════════════════════════╣
║  showing 20 of 195 issues                                                    ║
╚══════════════════════════════════════════════════════════════════════════════╝

Spectral Theme Tokens

Element Token Notes
Stat chip values var(--gradient-spectral) background-clip text
Total chip border var(--color-teal) 30% neutral count
Warning chip border var(--color-orange) 30% soft severity
Error chip border var(--color-danger) 45% hard severity
stale_import badge var(--color-orange) warning-level
error type badge var(--color-danger) error-level
File path text var(--font-mono), text-secondary 0.72rem
Description text var(--text-secondary) 0.72rem, wraps
Row hover var(--bg-surface) 0.12s ease
Column headers var(--text-muted) 0.65rem, uppercase
Clean state panel var(--color-success) 30% green border + bg tint

DB Schema (Migration 0017)

musehub_intel_breakage_issues

Column Type Notes
issue_id VARCHAR(128) PK — blob_id(repo_id:ref:file_path:type:desc)
repo_id VARCHAR(128) FK → musehub_repos CASCADE
issue_type VARCHAR(64) stale_import / future types
file_path VARCHAR(512)
description TEXT
severity VARCHAR(16) warning / error
ref VARCHAR(128) HEAD commit_id at time of scan

Indexes: ix_intel_breakage_issues_repo on (repo_id), ix_intel_breakage_issues_repo_type on (repo_id, issue_type)

musehub_intel_breakage_meta

Column Type Notes
repo_id VARCHAR(128) PK — FK → musehub_repos CASCADE
total_issues INTEGER
warning_count INTEGER
error_count INTEGER
file_count INTEGER total files scanned
ref VARCHAR(128)

Implementation Phases (load-bearing order)

Phase 1 — Migration 0017 + ORM models

Files: alembic/versions/0017_breakage_tables.py, musehub/db/musehub_models.py

  • Create musehub_intel_breakage_issues and musehub_intel_breakage_meta
  • Add both ORM models (MusehubIntelBreakageIssue, MusehubIntelBreakageMeta)
  • Full docstrings on both models

Phase 2 — BreakageProvider (pure Python, zero subprocess)

File: musehub/services/musehub_intel_providers.py

  • Walk HEAD snapshot manifest → parse_symbols() per file → build known_symbols + known_modules
  • Second pass: collect all import records → check against known sets → emit stale_import issues
  • Upsert issues + meta; issue_id = blob_id(repo_id:ref:file_path:issue_type:description)
  • Register as "intel.code.breakage" in _PROVIDER_REGISTRY and job_types_for_push
  • Return [("intel.code.breakage", {"total": N, "warnings": W, "errors": E})]
  • Full NumPy-style class docstring

Phase 3 — TDD: 34 Tests RED

File: tests/test_intel_breakage.py

All 34 tests written and confirmed RED before Phase 4.

  • T01–T05 DB model (columns, nullable, cascade, meta table, indexes)
  • T06–T12 Provider (no subprocess, stale_import detection, clean repo = 0 issues, idempotent, meta upserted, cross-file, empty manifest)
  • T13–T19 Route (200, empty state, 404, type filter, severity filter, top, stat chips)
  • T20–T24 E2E HTML (type badges, description text, clean panel, stat chips, dashboard link)
  • T25–T28 Data integrity (upsert idempotent, cross-repo isolation, meta reflects issue count, ref stored)
  • T29–T31 Performance (provider under 10s for 100-file scan, route under 500ms, index exists)
  • T32–T34 Security (XSS in description, SQL injection in type filter, invalid top → 200)

Phase 4 — Route Handler

File: musehub/api/routes/musehub/ui_intel.py

  • intel_breakage_page at GET /{owner}/{repo_slug}/intel/breakage
  • Query params: issue_type (all/stale_import, extensible), severity (all/warning/error), top (20/50/100 as str)
  • Aggregate from meta row for stat chips (never len(page))
  • Context: issues, total_issues, warning_count, error_count, file_count, selected_type, selected_severity, selected_top, valid_tops, valid_types

Phase 5 — Template

File: musehub/templates/musehub/pages/intel_breakage.html

  • 3 stat chips: Total / Warnings / Errors (danger border when errors > 0)
  • Clean-state panel (green, ✓ No structural breakage detected) when total_issues == 0
  • Type + severity filter pills + top selector
  • Issue rows: file path | type badge | description
  • All user content | e, all numbers | fmtnum
  • Empty state (no data yet) vs clean state (0 issues) are distinct

Phase 6 — SCSS (.bk-* namespace)

Files: src/scss/components/_breakage.scss, src/scss/pages/_breakage.scss

  • .bk-stat-card variants: --total (teal), --warn (orange), --error (danger)
  • .bk-stat-val with var(--gradient-spectral) background-clip text
  • .bk-type-badge variants: --stale-import (orange), --error (danger)
  • .bk-clean-panel: success green border + tint, flex row with icon
  • .bk-row grid: 2fr 9rem 3fr (file | type | description)
  • Responsive collapse of type badge at 700px

Phase 7 — Dashboard Card + Wire-Up

Files: musehub/api/routes/musehub/ui_intel.py, musehub/templates/musehub/pages/intel_dashboard.html

  • Dashboard queries: total + warning/error counts from meta row
  • Dashboard card: ⚡ BREAKAGE, total count, warning/error breakdown, clean indicator
  • Links to /intel/breakage
  • Deploy to staging (CSS rebuild + both containers)
  • Backfill job for gabriel/musehub and gabriel/muse

Docstring Standard

class BreakageProvider:
    """Persist structural breakage issues from HEAD snapshot analysis.

    Walks the HEAD snapshot manifest, builds a set of all known symbols and
    modules via ``parse_symbols()``, then makes a second pass to find import
    records whose referenced names do not exist in that known set.  Each such
    import produces a ``stale_import`` issue.

    No subprocess is spawned.  All data flows from objects stored at push time
    via ``get_backend(owner, slug).get(object_id)``.  Issues are batch-upserted
    into ``musehub_intel_breakage_issues``; the ``issue_id`` is
    ``blob_id(repo_id:ref:file_path:issue_type:description)`` so re-running the
    provider on the same commit is fully idempotent.  Aggregate counts are
    written to ``musehub_intel_breakage_meta`` in a single upsert.

    Parameters
    ----------
    session : AsyncSession
    repo_id : str
    ref : str
    payload : JSONObject

    Returns
    -------
    IntelResults
        ``[("intel.code.breakage", {"total": N, "warnings": W, "errors": E,
        "file_count": F})]`` on success.  ``[]`` when the snapshot manifest
        cannot be resolved.

    Notes
    -----
    The known-symbol set is built from the full manifest before the import-check
    pass so that symbols defined later in the file list are not false-positives.
    Only ``kind="import"`` records from ``parse_symbols()`` are checked; all other
    symbol kinds are silently skipped in the second pass.
    """

Acceptance Criteria

  • 34/34 tests GREEN
  • /gabriel/musehub/intel/breakage returns 200 with real data (195 issues visible)
  • Worker log shows intel.code.breakage done with count > 0
  • No _run_muse / subprocess in BreakageProvider
  • Clean-state panel shows when a repo has 0 issues; empty-state shows when no job has run yet
  • Type and severity filter pills work correctly
  • Stat chips read from meta row (never len(page))
  • All numbers | fmtnum, all user content | e
  • Dashboard card links to /intel/breakage
  • Deployed to staging with CSS rebuild and both containers rebuilt
Activity1
gabriel opened this issue 48 days ago
gabriel 48 days ago

Implementation complete ✓

All 7 phases delivered:

  • Phase 1 — Migration 0017: musehub_intel_breakage_issues + musehub_intel_breakage_meta tables with CASCADE FK and composite index
  • Phase 2BreakageProvider: pure-Python stale-import detection (no subprocess). Two-pass algorithm: (1) build known_symbol_names from all non-import symbols; (2) flag imports whose module path doesn't resolve AND symbol name is unknown. Stable issue_id = blob_id(repo:file:symbol:stale_import) ensures idempotent upserts. Dedup by issue_id before INSERT to prevent CardinalityViolationError on files with repeated imports.
  • Phase 3 — 34 tests across 7 tiers: all GREEN
  • Phase 4intel_breakage_page route at GET /{owner}/{repo_slug}/intel/breakage with ?type and ?top filter params
  • Phase 5intel_breakage.html: 4 stat chips (Total/Warnings/Errors/Files), type+top filter bar, 3-column issue list (Severity | File | Description)
  • Phase 6.bk-* SCSS namespace: components/_breakage.scss (visual) + pages/_breakage.scss (layout)
  • Phase 7 — Dashboard card with warning-count summary and 5-row preview; deployed to staging

Deployed: commit 6140bffa — backfill jobs running for gabriel/muse and gabriel/musehub repos