Repos are never linked to their marketplace domain — fix + backfill

Background

The proximate trigger

While verifying muse#74 (auto-derived supported_commands) against local musehub, https://localhost:1337/domains showed @gabriel/code with exactly 1 linked repo, despite https://localhost:1337/explore listing roughly 180 code repos. Not a display bug — confirmed directly against Postgres:

SELECT domain_id, marketplace_domain_id, count(*)
FROM musehub_repos GROUP BY domain_id, marketplace_domain_id;

 domain_id | marketplace_domain_id | count
-----------+------------------------+-------
 code      | (null)                 |   179
 identity  | (null)                 |     4
 mist      | (null)                 |     2
 code      | sha256:b5d4...         |     1   ← only a throwaway verification repo

Root cause — verified against source, not assumed

MusehubRepo has two distinct domain columns (see musehub#117 Phase 3 / the list_repos_for_domain fix that already shipped for the read side):

• domain_id — a plain VCS-plugin category string ("code", "mist", "identity"), set from the CLI's --domain flag at repo creation.
• marketplace_domain_id — an FK-shaped link to a specific MusehubDomain.domain_id (a sha256 genesis ID) — the column list_repos_for_domain actually queries.

create_repo() (musehub/services/musehub_repository.py:146) sets domain_id from the caller's domain argument but never touches marketplace_domain_id. The only code path that ever sets it is update_repo_settings()'s explicit --capabilities-style patch (musehub_repository.py:2342) — a repo only gets linked if someone later calls PATCH /api/repos/{id}/settings with an explicit marketplace_domain_id. Nothing in the create flow, the CLI, or any wizard does that today.

This is not just a one-time historical gap: every repo created from now on will keep landing with marketplace_domain_id = NULL unless this is fixed at the source. A backfill alone would only paper over the existing 185 repos and the bug would silently reopen on the next commit.

The ambiguity this issue must resolve — the reason it isn't a one-line fix

domain at repo-creation time is a bare category string ("code"), not a scoped domain ID ("@gabriel/code") — there is no author context to resolve against. Today exactly one non-deprecated MusehubDomain row exists per category (all owned by gabriel), so category → domain is unambiguous. That will not always be true: nothing stops a second author from publishing their own @alice/code tomorrow. Auto-linking must never guess when more than one candidate exists — silently picking one author's domain over another's would be a data-integrity bug, not a fix for one.

Resolution rule (this issue's core design decision): auto-link only when exactly one non-deprecated MusehubDomain matches the category by slug. Zero matches or 2+ matches → leave marketplace_domain_id NULL, unchanged from today's (broken but honest) behavior. An explicit, caller-supplied marketplace_domain_id always overrides auto-resolution and is validated to exist, exactly like update_repo_settings already does — this is the future-proof way for a caller to disambiguate once multiple domains of the same category exist (e.g. eventually a CLI flag --marketplace-domain @alice/code).

Design — the shape to build toward

A single resolution helper, consulted by both the create path (new) and the backfill script (one-time), with the same never-guess contract both places.

musehub/services/musehub_domains.py:

async def resolve_unambiguous_domain_id_by_category(
    session: AsyncSession, category: str
) -> str | None:
    """Return the sole non-deprecated MusehubDomain.domain_id whose slug
    matches *category*, or None if zero or multiple candidates exist.

    Never guesses: ambiguity (2+ live domains sharing a category slug)
    and absence (0 domains) are both honest None, not a fallback.
    """

create_repo() gains an optional marketplace_domain_id: str | None = None parameter:

• Explicit value provided → validated via get_domain_by_id (same ValueError("marketplace_domain_not_found") contract update_repo_settings already uses), then stored as-is.
• Omitted → auto-resolve via resolve_unambiguous_domain_id_by_category(session, domain_id) using the same _domain_id the row is about to store. Result (a real ID or None) is stored — never guessed.
• Whenever the repo ends up with a non-None marketplace_domain_id (explicit or auto-resolved) at creation time, call record_domain_install(session, owner_user_id, marketplace_domain_id) — mirrors what update_repo_settings already does on a later link, so install_count is correct from the moment of creation instead of only reflecting repos that were explicitly re-linked after the fact.

scripts/backfill_marketplace_domain_links.py (new, one-off, following the existing scripts/ convention — see stale_branches.py for style):

• --dry-run (default): reports counts — linked, skipped-ambiguous, skipped-no-match — without writing anything.
• --apply: performs the writes reported by dry-run, calling record_domain_install for each repo it links (real install-count correctness, not just backfilling the FK).
• Idempotent: repos with marketplace_domain_id IS NOT NULL are always skipped, dry-run or apply, re-run or first-run.
• Every skipped-ambiguous and skipped-no-match repo is printed by repo_id — no silent under-reporting of what wasn't fixed.

Goal — definition of done

1. create_repo() never leaves an unambiguously-resolvable repo unlinked — every new repo in a single-domain-per-category world (today's reality) is linked at creation time.
2. Ambiguous or unmatched categories are left NULL, not guessed — verified by a test that deliberately creates two competing domains for one category and confirms no link is made.
3. install_count reflects real installs from creation forward, not only from later explicit re-links.
4. All currently-orphaned repos get backfilled correctly, with a full dry-run report reviewed before any write. (Note: the original "185 local repos" count above is now stale — 181 of those were confirmed ephemeral bench/QA test repos and were deleted outright in a separate cleanup pass, unrelated to this issue's mechanism fix. Current real counts as of Phase 3: 5 unlinked on local — muse, musehub, identity, and 2 mists — and 18 unlinked on staging — muse, musehub, muse-zsh, identity, and 14 mists. All of these are real, not test junk.)
5. /domains on both local and staging shows accurate repo counts per domain after the backfill runs.
6. Every deliverable is TDD'd: red test first, then green.

Phases

Ordered by load-bearing dependency — the resolution helper must exist and be correct before either consumer (create path, backfill script) can use it.

Phase 1 — The resolution helper, in isolation

• [x] MDL_01 — resolve_unambiguous_domain_id_by_category returns the matching domain_id when exactly one non-deprecated MusehubDomain has that slug. Done.
• [x] MDL_02 — Returns None when zero domains match the category. Done.
• [x] MDL_03 — Returns None when 2+ non-deprecated domains share the same category slug — the core ambiguity guard this issue exists to add. Red test: seed two live domains both slugged "code" (different authors), assert resolution is None, not either one. Done.
• [x] MDL_04 — A deprecated domain sharing a slug with a live one does not count as a second candidate — resolves unambiguously to the live domain. Proves deprecation status is part of the candidate filter, not just a display-layer concern. Done.

Exit gate — met. tests/test_domains.py::TestIntegrationResolveUnambiguousDomainByCategory: 4/4 new tests pass, confirmed genuinely red first (ImportError before the function existed). Full file regression check: all 107 tests in test_domains.py still pass (was 103 before this phase). mypy: identical error count on musehub_domains.py before and after (3 — an import-untyped note pair on muse.core.types and one pre-existing no-any-return, both unrelated to this change and both present before my edit; verified by diffing against HEAD content). Implementation: resolve_unambiguous_domain_id_by_category(session, category) in musehub/services/musehub_domains.py, querying MusehubDomain.domain_id where slug == category and is_deprecated.is_(False), returning the single match or None for zero/multiple. Committed on feat/marketplace-domain-link-backfill, not yet merged to dev.

Exit gate: all four resolution-contract cases pass in isolation, with no dependency yet on create_repo or the backfill script.

Phase 2 — Wire into create_repo (stops the bug from recurring)

• [x] MDL_05 — Creating a repo with domain="code" and no explicit marketplace_domain_id, in a fixture with exactly one live "code" domain, auto-links to it. Done.
• [x] MDL_06 — Creating a repo with a category that matches zero marketplace domains leaves marketplace_domain_id NULL — today's honest behavior, unchanged for the no-match case. Done — was already passing before the fix (accidental correctness); kept as a pinned regression guard.
• [x] MDL_07 — An explicit marketplace_domain_id argument to create_repo overrides auto-resolution entirely (even if it disagrees with what auto-resolution would have picked). Done.
• [x] MDL_08 — An explicit marketplace_domain_id that doesn't exist raises ValueError("marketplace_domain_not_found") — same contract update_repo_settings already has, pinned here too so the two call sites can't silently diverge. Done.
• [x] MDL_09 — When a repo ends up linked at creation (explicit or auto-resolved), record_domain_install is called exactly once for owner_user_id — install_count on the target MusehubDomain increments immediately, not only after a later explicit patch. Done.
• [x] MDL_10 — Creating a repo whose category has 2+ live marketplace candidates leaves marketplace_domain_id NULL — the create-path-level regression guard mirroring MDL_03. Done — also already accidentally correct before the fix; pinned now.

Exit gate — met. New class tests/test_repository_service.py::TestMarketplaceDomainLinkAtCreation: 6/6 tests pass. Confirmed genuinely red first by temporarily reverting musehub_repository.py to its pre-Phase-2 HEAD content and re-running — 4/6 failed (MDL_05, MDL_07, MDL_08, MDL_09); MDL_06/MDL_10 passed even against the old code since NULL-on-no-match/ambiguity was already the (accidental) behavior of a create_repo that never touched marketplace_domain_id at all — kept as explicit regression guards now that the field is actively populated. Zero regressions: full test_repository_service.py (78), test_musehub_repos.py (82), and test_domains.py (107) all still pass — 267 total, all green. mypy: identical error count on musehub_repository.py before and after (15, all pre-existing; verified by diffing against HEAD content). Implementation: create_repo gained an optional marketplace_domain_id: str | None = None parameter; resolution happens before the MusehubRepo row is constructed (explicit value validated via get_domain_by_id, else auto-resolved via resolve_unambiguous_domain_id_by_category), and record_domain_install is called once after the row is flushed/refreshed whenever the resolved value is non-None. Committed on feat/marketplace-domain-link-backfill, not yet merged to dev.

Exit gate: every new repo created after this phase lands correctly linked whenever its category is unambiguous; ambiguous/unmatched cases stay honestly NULL; install bookkeeping is correct from creation.

Phase 3 — Backfill existing repos

• [x] MDL_11 — scripts/backfill_marketplace_domain_links.py (dry-run is the default; no --apply flag) reports accurate counts (linked / skipped-ambiguous / skipped-no-match) against a fixture DB, writes nothing. Done.
• [x] MDL_12 — --apply performs exactly the writes the matching dry-run reported, and calls record_domain_install once per linked repo. Done.
• [x] MDL_13 — Repos that already have a non-NULL marketplace_domain_id are left untouched by both dry-run and --apply — idempotent, safe to re-run after Phase 2 ships (new repos created in between are correctly skipped, not re-linked). Done — pinned by two tests: a repo linked before the script ever runs, and a repo linked by a first --apply pass then confirmed untouched (and not double-counted in install_count) by a second --apply pass.
• [x] MDL_14 — Every skipped-ambiguous and skipped-no-match repo is printed by repo_id in both modes — confirms the script never silently under-reports what it left unfixed. Done.

Exit gate — met. New file tests/test_backfill_marketplace_domain_links.py: 6/6 tests pass, covering dry-run (no mutation), apply (writes + install bookkeeping), idempotency (pre-linked repos untouched; safe re-run after apply without double-counting install_count), and full reporting of both skip categories. This is genuinely new code (the script did not exist before this phase — no prior "buggy" behavior to diff against for a red state, unlike Phases 1-2), so red was confirmed the ordinary way: every test failed with ImportError until the module and function existed. Zero regressions: test_domains.py + test_repository_service.py together (185 tests) still pass. mypy: scripts/backfill_marketplace_domain_links.py is clean (0 errors) — one real issue caught and fixed during implementation: MusehubRepo.domain_id is nullable at the DB level (Mapped[str | None]) even though the app always sets it in practice; the backfill now explicitly buckets a None category as skipped-no-match rather than passing None to the resolver. Smoke-tested end-to-end against local musehub's real container (docker exec musehub python3 scripts/backfill_marketplace_domain_links.py, dry-run, no --apply): reported exactly 5 linked / 0 ambiguous / 0 no-match, matching the corrected Phase-3-era count from the Goal section above. Committed on feat/marketplace-domain-link-backfill, not yet merged to dev.

Exit gate: dry-run report reviewed and matches expectations against a full copy of local musehub's real data before --apply is ever run for real (Phase 4).

Phase 4 — Live re-verification

• [x] MDL_15 — Run --dry-run against local musehub's real DB, review the report (expect 5 linked as of Phase 3 — muse, musehub, identity, 2 mists — given today's one-domain-per-category reality), then run --apply. Confirm via GET /api/domains/@gabriel/code (and identity/mist) that install_count/repo listings now reflect the real numbers, and confirm https://localhost:1337/domains in-browser. Done.
• [x] MDL_16 — Repeat against staging once local is confirmed correct. Done — 22 linked (more than the original 18-owned-by-gabriel estimate; the backfill correctly scans every repo server-wide, not just gabriel's own — staging has repos owned by other/test identities muse hub repo list doesn't surface, since that command is scoped to the caller's own + collaborated repos).

Exit gate — met, both local and staging. Local: ran docker exec musehub python3 scripts/backfill_marketplace_domain_links.py (dry-run then --apply) against local musehub's real container/DB: 5 linked / 0 ambiguous / 0 no-match. Verified directly against Postgres — all 5 repos now correctly marketplace_domain_id-linked. Verified live: GET /api/domains/@gabriel/code/repos → total: 2; .../mist/repos → total: 2. Full regression check post-write: test_domains.py + test_repository_service.py + test_backfill_marketplace_domain_links.py — 191/191 pass.

Staging: since scripts/ is never copied into the Docker image (only musehub/, alembic/, docs/, deploy/ — confirmed by reading the Dockerfile's COPY list), the backfill script had to be transferred onto the running musehub-blue container manually — base64-encoded via SSM, written to the host, docker cp'd in as root (the container's default musehub user can't mkdir under /app), then chown'd back to the musehub user before exec. Dry-run reported 22 linked / 0 ambiguous / 0 no-match; --apply produced the identical 22. Verified live: GET /domains?format=json → @gabriel/code repo_count: 6 (includes real collaborator repos like knowtation, gabriel-muse, scooling-lab, not just gabriel's own muse/musehub), @gabriel/mist repo_count: 13, @gabriel/identity repo_count: 0 (private) — confirmed identically in the rendered HTML of both https://staging.musehub.ai/domains and https://staging.musehub.ai/domains/@gabriel/code (stat card now reads 6, matching the live repo list below it). This ad hoc script-transfer process is a one-off for this backfill, not a new standing deploy convention — copying scripts/ into the image permanently, if ever needed again, would be a separate, deliberate Dockerfile change.

Exit gate: Live, in-browser/curl confirmation against both local and staging — not just the test suite. /domains shows real repo counts; no repo is silently mis-linked to the wrong author's domain.

Addendum — a second, more serious bug found during MDL_15 live verification

After applying the local backfill, gabriel reported the domain detail page (/domains/@gabriel/code) still showing "1 repo" despite muse and musehub both being listed underneath. Investigation surfaced two distinct, previously-undetected problems — one about this issue's own verification process, one an unrelated display bug:

1. list_repos_for_domain was never actually fixed on dev. The Background section above claimed the wrong-column bug (MusehubRepo.domain_id vs marketplace_domain_id) "already shipped" on a separate branch, fix/list-repos-for-domain-wrong-column. That was false — the fix was committed but never merged to dev, carried forward incorrectly from an earlier session summary that conflated "committed on a branch" with "merged." Worse: the local musehub container had been running since before this session's work began, with list_repos_for_domain loaded into memory from whatever branch happened to be checked out on disk at container-start time — which, by coincidence, was the fixed branch. Every "live verification" during Phases 1-4 (including MDL_15's initial pass) was silently validated against stale in-memory bytecode from a different, unmerged branch, not the code that would actually ship. A container restart exposed the regression immediately (GET /api/domains/@gabriel/code/repos → total: 0).

Fixed by merging fix/list-repos-for-domain-wrong-column into dev (clean fast-forward, 0 conflicts, 104/104 tests pass), pushing dev to local, deleting the now-merged branch, then merging dev into feat/marketplace-domain-link-backfill (clean 3-way merge, 0 conflicts, 192/192 tests pass). Both containers restarted and confirmed booting clean before re-verifying. Re-verified live post-restart: @gabriel/code/repos → total: 2 (muse, musehub); @gabriel/mist/repos → total: 2 (both mists); @gabriel/identity/repos → total: 0 (correct — that repo is private, correctly excluded by the visibility == "public" filter, not a bug).

Process lesson: this workspace's own CLAUDE.md already documents "ALWAYS restart both containers after code changes" as a hard-won rule from a prior incident — I did not follow it during Phases 1-3 of this issue, and it directly caused a false-positive verification. Restarting before any live check is now non-negotiable for the rest of this issue (and should be treated as such for all future musehub work).

2. Separate, unrelated display bug — now fully fixed on both pages. The "Repositories" stat card on the domain detail page was bound to domain.install_count (a count of distinct users who installed the domain, keyed on (user_id, domain_id)) under a label that reads "Repositories" — a pre-existing mislabel, not caused by this issue's mechanism work. Fixed by swapping the bound value to repos_total (the real, already-computed live repo count) in domain_detail.html.

The /domains listing page had the identical mislabel (domain.install_count rendered as "N repo(s)" per card), but with no equivalent value already computed to swap to — the listing route never queried per-domain repo counts at all. Fixed by adding count_public_repos_by_domain(session, domain_ids) to musehub_domains.py — one grouped-by query batching the count for every domain on the current page (never an N+1 call to list_repos_for_domain per row), matched on marketplace_domain_id exactly like list_repos_for_domain. Wired into domains_page as a new repo_count field alongside (not replacing) install_count — they are genuinely distinct metrics, both worth keeping. 5 new unit tests (TestIntegrationCountPublicReposByDomain) plus one E2E regression test against GET /domains?format=json confirming repo_count and install_count diverge correctly (2 repos, 1 install). Confirmed genuinely red first by reverting both the query wiring and the template binding and re-running the E2E test (KeyError: 'repo_count'), then restored and re-ran green. Zero regressions: full test_domains.py — 114/114 pass (was 107 before this fix, then 113 after the count function, then 114 after the regression test). mypy: identical error count on both musehub_domains.py (2, pre-existing) and ui_domains.py (5, pre-existing) before and after — verified by diffing against HEAD content.

Both containers restarted and confirmed clean boot before live verification. Live confirmation via GET /domains?format=json: @gabriel/code → repo_count: 2, @gabriel/identity → repo_count: 0, @gabriel/mist → repo_count: 2 — install_count on all three independently still 1 (each domain installed once by gabriel, a correct and distinct number). Confirmed in the rendered HTML too: curl https://localhost:1337/domains shows "2 repos" / "0 repos" / "2 repos" for the three domain cards, matching exactly.

Acceptance criteria (whole-issue gate)

• create_repo never leaves an unambiguous-category repo unlinked.
• Ambiguous or unmatched categories are always NULL, never guessed — proven by tests that construct real ambiguity, not just the happy path.
• install_count is correct for repos linked at creation time, not only for repos explicitly re-linked later.
• All existing orphaned repos backfilled where unambiguously resolvable; every unresolved repo explicitly reported, not silently skipped.
• Full TDD coverage: every phase's deliverables have a red-then-green test.
• Verified live on both local and staging, not just unit-tested.

Out of scope (explicit, for future issues)

• Any UI or CLI affordance for a human to manually resolve an ambiguous or no-match repo (e.g. "pick which @author/code this repo belongs to"). This issue makes today's single-domain-per-category reality correct and safe; multi-domain-per-category UX is real future work once a second author actually publishes a competing domain.
• Changing update_repo_settings's existing explicit-link behavior — already correct, already tested; this issue only adds an equivalent auto-resolve path at creation time.
• A CLI flag to pass --marketplace-domain explicitly at muse push / repo-creation time — the service-layer parameter is added in this issue so a future CLI change can wire into it, but exposing it through the CLI itself is a separate, smaller follow-up once the service contract is proven.