fix(bridge): auto-migrate vec0 on dimension change + bind content-hash to provider+model
Why: After PR #203 merged, the first re-index against the persisted blob (1536-dim table from prior OpenAI runs) failed in 1.4s with:
Vector store dimension mismatch: table has 1536, requested 1024. Delete /tmp/.../knowtation_vectors.db and run "knowtation index" ...
The dimension safety check in lib/vector-store-sqlite.mjs:ensureCollection threw BEFORE the new content-hash migration logic could drop+recreate the table. The check is correct for CLI users (an accidental EMBEDDING_PROVIDER swap should surface loudly), but wrong for the bridge: it owns the data lifecycle (download blob -> re-index -> upload blob) and a provider switch can only resolve via a full re-embed of every vault in the DB.
Two fixes shipped together so both regressions close in one deploy / re-index:
1. allow_dimension_migration flag (vector-store-sqlite.mjs) - Opt-in config key that lets ensureCollection drop+recreate when the persisted dimension differs from the requested dimension. - Bridge sets it (getBridgeStoreConfig) so a provider switch unblocks the indexer automatically. - CLI omits it, preserving the existing throw + actionable error message. - Migration always logs a console.warn so the operator can see why every vector in the DB was dropped (auditability).
2. Provider+model in the content-hash prefix (chunk-content-hash.mjs) - Hash format changes from "v1:<32-hex>" to "v1:<provider>:<model>:<32-hex>". - Prevents the same-dimension silent-corruption case the dimension check cannot catch: e.g. DeepInfra BGE-large 1024 -> DeepInfra BGE-m3 1024. With the prior format, every chunk would be a cache hit and stale vectors from the wrong model would silently corrupt Meaning search with no error log. - Provider lowercased + alphanumeric-stripped; model preserves slashes (BAAI/bge-large-en-v1.5) but collapses whitespace. - computeChunkContentHashTagged now requires { provider, model } and throws TypeError if missing — silent fallback would re-introduce the exact bug this prefix prevents. - Cache is empty in production (PR #203's first re-index never completed), so this format change has zero migration cost today.
Tests: 1797 -> 1807 (+10). 0 fail. - chunk-content-hash: format, lowercased provider, slash-preserving model, provider-change invalidates, model-change (same dim) invalidates, missing config throws. - vector-store-sqlite-content-hash: default still throws on dim mismatch (CLI safety regression test), with-flag drop+recreate (warn captured), no-op when dimensions match. - bridge-index-cache-contract: getBridgeStoreConfig sets the flag; hash call uses both args.
Refs: PR #203 (feat/bridge-embed-hash-cache).
0 comments
muse hub commit comment sha256:caf1b416fe0d529a90adf132cd3bf831435d6c59b4edabf672ae310b5320207c --body "your comment"
No comments yet. Be the first to start the discussion.