gabriel / musehub public
agent-model-stale-on-profile.md markdown
65 lines 3.2 KB
Raw
sha256:1dd27e6d5c24550177b01b0a171e8375c07cf046b549e7683364706b6522d3bc docs(#139): mark all 11 checklist items resolved, document … Sonnet 5 12 days ago

Agent identity profile's agent_model is frozen at first-ever provisioning, never updates

Background

Noticed a real discrepancy: https://staging.musehub.ai/gabriel/mists/4dB2qCpPV5vG's provenance correctly shows model_id: "claude-sonnet-5" (accurate, per-object, live), but the linked claude-code agent profile (https://staging.musehub.ai/claude-code) shows model claude-sonnet-4-6 — an older model, from before this session's convention shifted to claude-sonnet-5.

Root cause, traced precisely — not assumed

Two contributing bugs:

  1. ensure_agent_identity (musehub/services/derived_agent_provisioner.py:40-45) is a pure upsert:

    existing = (await session.execute(
        select(MusehubIdentity).where(MusehubIdentity.handle == handle)
    )).scalar_one_or_none()
    if existing is not None:
        return existing
    

    Once an agent's musehub_identities row exists, agent_model is never updated again — regardless of how many later commits use a different model. Every subsequent profile view returns the same frozen row.

  2. The seed value itself came from the wrong commit. The fallback lookup that seeds agent_model when first auto-provisioning a derived agent (musehub/api/routes/musehub/ui_user_profile.py:122-130) orders by MusehubCommit.timestamp ascending, .limit(1) — the oldest commit by that agent_id, not the most recent:

    select(MusehubCommit.agent_id, MusehubCommit.model_id, MusehubCommit.timestamp)
    .where(MusehubCommit.agent_id == handle)
    .order_by(MusehubCommit.timestamp)   # ascending — oldest first
    .limit(1)
    

    So claude-code's identity got permanently seeded with whatever model was used in its very first-ever commit, and has been frozen there since.

Worth noting: musehub/api/routes/musehub/ui_agents.py:278-297 (a different agent-profile-adjacent route, repo-scoped) does this correctly — orders by MusehubCommit.timestamp.desc() and re-derives model_id on every request rather than caching it. That's the pattern to converge on.

Goal (to be decided when picked up — this is a placeholder with real findings, not a full plan yet)

Decide deliberately: should agent_model mean "the model this identity was first seen using" (a genesis/historical marker, in which case the bug is only #1 — the oldest-first ordering was actually intentional and just needs the value to stay frozen, which it already does) or "the model this identity is currently using" (in which case both bugs need fixing: reseed from the newest commit, and refresh on every new commit rather than caching forever)? Given the profile page reads as a live status indicator (no "since" or "first seen" qualifier next to it), current behavior is almost certainly the wrong interpretation for how it's displayed today.

Out of Scope (for now)

  • Any broader redesign of derived-agent auto-provisioning beyond this one field.
  • Retroactively backfilling agent_model for already-provisioned identities — decide whether that's needed once the live-vs-genesis question is resolved.
File History 2 commits
sha256:1dd27e6d5c24550177b01b0a171e8375c07cf046b549e7683364706b6522d3bc docs(#139): mark all 11 checklist items resolved, document … Sonnet 5 12 days ago
sha256:649011bedd713e22f7dca4c4be94bdefbf3b10950d9fc80235928d0b0b823be8 docs: track issue #139 (two-column app-shell refactor) plan… Sonnet 5 12 days ago