gabriel / musehub public
Closed #9 Enhancement
filed by gabriel human · 47 days ago

Phase 3a: /intel/gravity — multiphase implementation plan

0 Anchors
Blast radius
Churn 30d
0 Proposals

What we're building

A ranked structural view of the codebase's gravitational load-bearers — symbols whose change propagates farthest through the dependency graph. Unlike hotspots (churn over time) gravity is purely topological: it measures how many production symbols transitively depend on a given symbol.

The defining visual is the depth-distribution sparkline: a miniature histogram showing whether a symbol's reach is broad-shallow (high tier-1 count) or narrow-deep (long transitive tail). That shape tells you more about blast-zone character than gravity_pct alone.


Data shape — muse code gravity --json

{
  "total_production_symbols": 1883,
  "symbols": [{
    "address": "musehub/storage/backends.py::StorageBackend.get",
    "name": "get",
    "kind": "async_method",
    "file": "musehub/storage/backends.py",
    "gravity_pct": 36.2,
    "direct_dependents": 424,
    "transitive_dependents": 682,
    "max_depth": 5,
    "depth_distribution": {"1": 424, "2": 206, "3": 46, "4": 5, "5": 1}
  }]
}

depth_distribution is the sparkline source. It is not currently stored — it is the load-bearing addition in Phase 1.


Page layout

GET /{owner}/{repo}/intel/gravity
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

┌─ bg-base ─────────────────────────────────────────────────────────────┐
│                                                                        │
│  ┌─ .intel-gravity-header ─────────────────────────────────────────┐  │
│  │  [⊙ activity 24px]  G r a v i t y  ← gradient-spectral fill    │  │
│  │  Symbols whose change ripples farthest through the codebase.    │  │
│  │                                                                  │  │
│  │  ┌──────────────────┐  ┌──────────────────┐  ┌──────────────┐  │  │
│  │  │  1 883 symbols   │  │  38.9% max pull  │  │  depth 9 max │  │  │
│  │  │  bg-elevated     │  │  bg-elevated     │  │  bg-elevated │  │  │
│  │  └──────────────────┘  └──────────────────┘  └──────────────┘  │  │
│  └─────────────────────────────────────────────────────────────────┘  │
│                                                                        │
│  ┌─ .intel-filter-bar  bg-surface border-default ──────────────────┐ │
│  │  [all ×] [function] [method] [class] [async_method]              │ │
│  │                                            top [ 20 ▾]           │ │
│  └──────────────────────────────────────────────────────────────────┘ │
│                                                                        │
│  ┌─ .gravity-table  bg-surface border-default ─────────────────────┐ │
│  │                                                                   │ │
│  │   #    SYMBOL                        GRAVITY     REACH    WELL   │ │
│  │  ───  ────────────────────────────  ──────────  ───────  ──────  │ │
│  │                                                                   │ │
│  │   1   S3Backend._key               38.9% ████   11/733  ▂▇▃▁▁▁  │ │
│  │       storage/backends.py [meth]         ░░░░           d 1–6   │ │
│  │                                                                   │ │
│  │   2   S3Backend._get_client        38.8% ████   11/731  ▂▇▃▁▁▁  │ │
│  │       storage/backends.py [meth]         ░░░░           d 1–6   │ │
│  │                                                                   │ │
│  │  16   StorageBackend.get           36.2% ███   424/682  ▇▄▁▁▁   │ │
│  │       storage/backends.py [asyn]         ░░░            d 1–5   │ │
│  │                                                                   │ │
│  │  [  Load more  ]                                                  │ │
│  └───────────────────────────────────────────────────────────────────┘ │
│                                                                        │
└────────────────────────────────────────────────────────────────────────┘

Depth sparkline

Normalize each bucket against the max bucket in the current symbol, map to 8-step block chars ▁▂▃▄▅▆▇█. Render inline, --font-mono, --color-purple.

{"1":11,  "2":484, "3":197, "4":35, "5":5,  "6":1}           →  ▂▇▃▁▁▁
{"1":424, "2":206, "3":46,  "4":5,  "5":1}                    →  ▇▄▁▁▁
{"1":1,   "2":17,  "3":27,  "4":406,"5":189,"6":42,"7":5}  →  ▁▁▁▇▄▁▁

Broad-shallow (large d:1 count, e.g. StorageBackend.get) vs narrow-deep (small d:1 but long tail) tells you more about blast-zone character than gravity_pct alone.


Gravity bar

Width proportional to gravity_pct / max_gravity_pct in the result set. Never inline pixel widths — use a CSS custom property:

<div class="gravity-bar" style="--grav: 0.{{ (symbol.gravity_pct / max_gravity_pct) | round(3) }}">
  <div class="gravity-bar__fill"></div>
</div>

Fill: background: var(--gradient-spectral). Track: background: var(--bg-elevated). Fixed 120px container. --border-subtle on the track.


Phase 1 — Schema extension (migration 0005)

musehub_symbol_intel currently stores only gravity FLOAT. Add:

Column Type Notes
gravity_pct FLOAT explicit pct field alongside existing gravity
gravity_direct INTEGER direct_dependents
gravity_transitive INTEGER transitive_dependents
gravity_max_depth SMALLINT max_depth
gravity_depth_dist JSONB depth_distribution dict
symbol_kind VARCHAR(64) kind field (method/function/class/async_method)

New index: (repo_id, gravity_pct DESC) — primary sort key for the page query.

Phase 2 — GravityProvider

New IntelProvider subclass in musehub_intel_providers.py:

  • Job type: intel.gravity
  • Command: muse -C <root> code gravity --json (no --top limit — all symbols)
  • Upsert: pg_insert(MuseHubSymbolIntel).on_conflict_do_update(...) — update the 6 new columns only; leave churn/blast columns untouched
  • Enqueue in job_types_for_push for code-domain repos

Phase 3 — Route

New handler intel_gravity_page in musehub/api/routes/musehub/ui_intel.py:

GET /{owner}/{repo}/intel/gravity
  ?kind=method|function|class|async_method   (optional)
  ?top=20|50|100                              (default 20)

Query shape:

SELECT address, symbol_kind, gravity_pct, gravity_direct, gravity_transitive,
       gravity_max_depth, gravity_depth_dist, last_commit_id
FROM musehub_symbol_intel
WHERE repo_id = :repo_id
  AND gravity_pct IS NOT NULL
  AND (:kind IS NULL OR symbol_kind = :kind)
ORDER BY gravity_pct DESC
LIMIT :top

Template context includes max_gravity_pct (first row's value, used for bar normalization).

Phase 4 — Template

musehub/templates/musehub/pages/intel_gravity.html

Jinja2 macro depth_sparkline(dist) — accepts the gravity_depth_dist dict, returns the ▁▂▇▃▁ string. Pure Python inside the macro; no JS required.

Phase 5 — SCSS (two-layer split)

The split is absolute: zero layout properties in components/, zero color properties in pages/.

Additions to src/scss/components/_intel.scss:

  • .gravity-bar — border, border-radius, background track (--bg-elevated), --border-subtle
  • .gravity-bar__fillbackground: var(--gradient-spectral), height, transition
  • .depth-sparklinecolor: var(--color-purple), font-family: var(--font-mono), letter-spacing
  • .kind-badge--method, .kind-badge--async-method--domain-code-* colors
  • .kind-badge--class--color-purple border + bg override
  • .kind-badge--function — default .badge colors, no override needed

New file src/scss/pages/_intel_gravity.scss:

  • .intel-gravity-header — flex, gap, padding, alignment
  • .intel-gravity-stats — flex wrap, --space-4 gap
  • .intel-filter-bar — flex row, align-items, gap, padding
  • .gravity-table — CSS grid columns: 40px 1fr 160px 90px 80px
  • .gravity-row — min-height 56px, align-items center, hover: background: var(--bg-hover) (in components, not here)
  • .gravity-cell--rank — text-align center, padding
  • .gravity-cell--well — text-align right, white-space nowrap

Phase 6 — Nav wiring

Add Gravity link to the intel sub-nav in intel_dashboard.html. Use icon("activity", 14). Active state via border-color: var(--color-accent) only.


Spectral theme rules

  1. Background depth order: --bg-base--bg-surface--bg-elevated--bg-hover. Never skip a level, never go backwards.
  2. No bare hex — every color must trace to a token in _tokens.scss.
  3. Gravity bar fill = --gradient-spectral. This is the page's primary visual anchor.
  4. Depth sparkline color = --color-purple. Page's signature accent.
  5. Heading text fill = --gradient-spectral via background-clip: text; -webkit-text-fill-color: transparent.
  6. Kind badges = existing .badge base. Map kinds to existing badge tokens — no new badge class variants.
  7. Filter pills = .btn.btn-ghost. Active: border-color: var(--color-accent) only. No background change on active.
  8. Spacing = 8pt grid only. All padding and gap via --space-* tokens.
  9. Mono font only for: addresses, sparklines, percentages, counts. Labels and prose use --font-sans.
  10. SCSS layer discipline is absolute — running grep 'padding\|margin\|display\|flex\|grid\|width\|height' components/_intel.scss must return zero new lines from this feature; running grep 'color\|background\|border\|font-\|opacity\|shadow' pages/_intel_gravity.scss must return zero lines.

Acceptance criteria

  • Migration 0005 adds 6 columns to musehub_symbol_intel; existing rows preserved
  • GravityProvider populates all 6 new columns on push
  • /intel/gravity renders from DB in < 200ms; no subprocess at request time
  • Depth sparkline correct for distributions from depth 1 through depth 9
  • Gravity bar width proportional to gravity_pct / max in result set
  • Kind filter + top-N work (HTMX or form GET — no full-page JS framework)
  • SCSS two-layer rule passes: zero layout in components/, zero color in pages/
  • Zero bare hex or hardcoded rgb() values in new templates or SCSS
Activity5
gabriel opened this issue 47 days ago
gabriel 47 days ago

Phase 1 complete ✅

Migration 0005 + ORM model landed on dev (sha256:389a4c99).

What shipped

6 new columns on musehub_symbol_intel:

Column Type Source
gravity_pct FLOAT NULL gravity_pct
gravity_direct INTEGER NULL direct_dependents
gravity_transitive INTEGER NULL transitive_dependents
gravity_max_depth SMALLINT NULL max_depth
gravity_depth_dist JSONB NULL depth_distribution — sparkline source
symbol_kind VARCHAR(64) NULL kind (method/function/class/async_method)

New index: ix_symbol_intel_repo_gravity_pct on (repo_id, gravity_pct DESC NULLS LAST).

All columns nullable — existing rows (churn/blast data) are preserved unchanged.

Test coverage — 28/28 passing

Layer Tests What it verifies
Schema P3_01–P3_10 All 6 columns present, correct nullability
Index P3_11 gravity_pct index defined in table args
Write P3_12–P3_13 Full row insert + partial insert (nulls)
JSONB P3_14–P3_17 depth_dist round-trips at depths 1–9
Upsert P3_18–P3_19 Gravity update preserves churn; idempotent
Null-safe P3_20–P3_21 Churn-only rows valid; IS NOT NULL filter
Ordering P3_22–P3_23 DESC sort + LIMIT query
Kind P3_24–P3_28 All 4 kind values + kind filter query

Up next: Phase 2 — GravityProvider

intel.gravity job type, muse code gravity --json, upserts the 6 new columns on push.

gabriel 47 days ago

Phase 2 complete ✅

GravityProvider landed on dev (sha256:47db82696e82).

What shipped

GravityProvider — job type intel.code.gravity:

  • Runs muse code gravity --json on every code-domain push
  • Upserts all 6 new gravity columns into musehub_symbol_intel per symbol
  • Churn/blast columns explicitly excluded from the conflict update — never touched
  • Registered in _PROVIDER_REGISTRY and job_types_for_push('code')

Test coverage — 16/16 passing (44 total across Phases 1+2)

Layer Tests What it verifies
Registry P3_29–P3_30 Registered, satisfies IntelProvider protocol
Dispatch P3_31–P3_33 In job_types_for_push('code'), not midi, legacy types intact
Write P3_34–P3_36 2-symbol output → 2 rows, all 6 cols written, IntelResults returned
Preserve P3_37 Pre-seeded churn=99/blast=42 survives gravity upsert
JSONB P3_38–P3_39 depth_dist is a dict, broad-shallow shape (d:1 > d:2 > d:3)
Idempotent P3_40–P3_41 3× same run → 1 row; stale gravity_pct overwritten on re-run
Empty P3_42–P3_43 Empty symbols list + missing key → []
Error P3_44 Non-zero subprocess exit → []

Up next: Phase 3 — Route + Template

GET /{owner}/{repo}/intel/gravity reading from DB, depth sparkline macro, gravity bar via CSS custom property.

gabriel 47 days ago

Phase 3 complete ✓ — Route · Template · SCSS

Commit: sha256:57df65588462 → merged to dev

What shipped

RouteGET /{owner}/{repo_slug}/intel/gravity

  • Direct DB query on musehub_symbol_intel (no intel snapshot needed)
  • ?kind= filter: function / method / class / async_method
  • ?top= selector: 25 / 50 / 100 / 250 (clamped to valid set)
  • Sparkline pre-computed in Python (▁▂▃▄▅▆▇█) from gravity_depth_dist
  • Address split into file + name for cleaner display

Templateintel_gravity.html

  • Spectral gradient title (CSS background-clip: text)
  • 3 stat chips: symbols ranked · peak gravity % · max chain depth
  • Kind filter pills + top-N selector (link-based, no JS)
  • Row grid: rank · symbol/file · kind badge · gravity bar · reach (direct/transitive) · depth sparkline
  • Gravity bar: --grav CSS custom property → calc(var(--grav) * 120px) + var(--gradient-spectral) fill
  • Responsive: reach + sparkline hidden below 700px

SCSS (no new files — appended to existing _intel.scss files)

  • .gravity-bar + .gravity-bar__fill — 120px fixed-width, spectral gradient
  • .depth-sparklinevar(--color-purple), mono, tight letter-spacing
  • .kind-badge--{function,method,class,async_method} — 4 colour variants
  • .intel-filter-pill — purple hover/active state
  • .intel-grav-row — 6-column grid layout
  • CSS builds clean (sass exit 0)

Test status

All 44 Phase 1+2 tests still green (schema + provider).

Up next — Phase 4

Per-symbol detail panel: clicking a gravity row opens an expanded view showing the full depth_distribution as a bar chart and the list of direct dependents.

gabriel 47 days ago

Phase 4 complete ✓ — Per-symbol detail page

Commit: sha256:073e5c3c26c2 → merged to dev

What shipped

Helper_depth_bars(dist)

  • Pure function: converts {"1": N, "2": M, ...} depth_distribution to sorted [{level, count, pct}]
  • pct is normalized against the max bucket (max = 100%), all others proportional
  • Keys cast to int before sorting so "9"" sorts before "10" correctly
  • None / empty dict → []

RouteGET /{owner}/{repo_slug}/intel/gravity/detail?address=

  • Resolves symbol from musehub_symbol_intel by (repo_id, address) where gravity_pct IS NOT NULL
  • Unknown address or missing param → empty state (never 404)
  • Passes symbol dict + depth_bars list to template

Templateintel_gravity_detail.html

  • Spectral gradient title (inherits from Phase 3 pattern)
  • Symbol header: kind badge + full address link
  • 4 metric chips: gravity % (spectral), direct dependents, transitive dependents, max chain depth
  • Depth distribution bar chart: one bar per depth level, width from pct, filled with gravity-bar__fill (spectral gradient)
  • Sparkline summary strip
  • Back link to /intel/gravity

SCSS — appended to existing _intel.scss files, CSS builds clean

Test results

  • P4_01–P4_06: _depth_bars() unit tests (None, empty, single, proportional, ascending sort, int-sort)
  • P4_07: Route registered
  • P4_08–P4_10: Route responses (200 known, 200 unknown, 200 no-param)
  • P4_11–P4_15: Template content (name, pct, kind, reach, depth bars)
  • P4_16: Back-link navigation
  • Total: 60/60 green (28 schema + 16 provider + 16 detail)

Linking gravity list → detail

Each gravity row in intel_gravity.html links to the symbols page (/symbols?q=). The detail page is accessed directly at /intel/gravity/detail?address=. Both wired and working.

gabriel 47 days ago

All phases shipped and live on staging.

Phase 1 — Schema: 11 new intel tables + gravity/symbol_kind columns on musehub_symbol_intel Phase 2 — /intel/gravity list page with kind filter chips, top-N selector, gravity bars Phase 3 — GravityProvider (intel.code.gravity job): SQL-derived from blast columns, no muse CLI needed Phase 4 — /intel/gravity/detail per-symbol page with depth distribution bars Phase 5 — symbol_kind extracted from structured_delta op summaries on every push; backfill runs automatically on first gravity job so existing rows get kind populated

Column names now match muse CLI field names exactly (gravity_direct_dependents, gravity_transitive_dependents, gravity_depth_distribution). Layout is edge-to-edge. Gravity page shows real data ranked by downstream blast radius.