HUB-API.md
file-level
1
files
1
commits
0
hotspots
0
🧊 dead
0
💥 blast risk
| 1 | # Knowtation Hub API |
| 2 | |
| 3 | This document defines the **Hub REST API contract** and **auth model** for Phase 11. The same contract is implemented by (a) the self-hosted Node server (Docker) and (b) the ICP canister(s). The Hub UI and CLI talk to either deployment using the same routes and JSON shapes. |
| 4 | |
| 5 | **Reference:** [AGENT-INTEGRATION.md](./AGENT-INTEGRATION.md) §4 (proposals), [PROPOSAL-LIFECYCLE.md](./PROPOSAL-LIFECYCLE.md), [SPEC.md](./SPEC.md) §4 (CLI semantics). |
| 6 | |
| 7 | --- |
| 8 | |
| 9 | ## 1. Authentication |
| 10 | |
| 11 | ### 1.1 Model |
| 12 | |
| 13 | - **Login required.** There is no API-key-only path; all Hub API calls require a **JWT** obtained after login. |
| 14 | - **Self-hosted (Docker):** Login via **OAuth 2.0** (Google and/or GitHub). After successful OAuth callback, the server issues a **JWT** (access token). Optional: refresh token for long-lived sessions. |
| 15 | - **Hosted (ICP):** Login via **Internet Identity** (or, if fronted by a gateway that performs OAuth, a JWT trusted by the canister). The canister validates the JWT or II delegation. |
| 16 | |
| 17 | ### 1.2 Obtaining a JWT |
| 18 | |
| 19 | | Deployment | Flow | |
| 20 | |------------|------| |
| 21 | | Self-hosted | User visits `GET /auth/login` (or equivalent); redirect to OAuth provider; callback at `GET /auth/callback`; server issues JWT, sets cookie or returns token in response body. | |
| 22 | | ICP | User signs in with Internet Identity; front-end receives session; subsequent API calls include the II-derived principal or a JWT issued by an auth canister. | |
| 23 | |
| 24 | ### 1.3 Using the JWT |
| 25 | |
| 26 | - **Header:** `Authorization: Bearer <access_token>` |
| 27 | - All Hub API endpoints (except login/callback and public health) require this header. Missing or invalid token → `401 Unauthorized`. |
| 28 | |
| 29 | ### 1.4 Token lifetime and refresh |
| 30 | |
| 31 | - **Access token:** Short-lived (e.g. 15–60 minutes). Document exact lifetime in deployment config. |
| 32 | - **Refresh token (optional):** If supported, store securely; use to obtain a new access token via `POST /auth/refresh` (body: `{ "refresh_token": "..." }`). Refresh tokens are long-lived until revoked. |
| 33 | |
| 34 | ### 1.5 Scopes (optional) |
| 35 | |
| 36 | JWTs may include scopes to distinguish read vs write vs propose: |
| 37 | |
| 38 | - `read` — list notes, get note, search. |
| 39 | - `write` — write note (direct to vault). |
| 40 | - `propose` — create proposal, list own proposals. |
| 41 | - `review` — list all proposals, approve, discard. |
| 42 | |
| 43 | If not implemented in v1, all authenticated users have full access. Document scope semantics when added. |
| 44 | |
| 45 | --- |
| 46 | |
| 47 | ## 2. Base URL and versioning |
| 48 | |
| 49 | - **Base URL:** Self-hosted: `http(s)://<host>:<port>/api` (or no prefix). ICP: `https://<canister-id>.ic0.app/api` (or as deployed). |
| 50 | - **Versioning:** Path prefix `/api/v1` recommended (e.g. `GET /api/v1/notes`). Omit version in this doc for brevity; implementations use a consistent prefix. |
| 51 | - **Vault context (multi-vault / hosted):** Optional header **`X-Vault-Id`** or query param **`vault_id`** to scope requests to a vault. When absent, implementations use a default (e.g. `default` or the single vault). Gateway forwards JWT **`sub`** as **`x-user-id`** to the canister for trusted identity. |
| 52 | |
| 53 | --- |
| 54 | |
| 55 | ## 3. Endpoints (contract) |
| 56 | |
| 57 | Same semantics as CLI where applicable. Request/response JSON matches SPEC §4.2 shapes where noted. |
| 58 | |
| 59 | ### 3.1 Health (no auth) |
| 60 | |
| 61 | - **GET /health** — Returns `200` and `{ "ok": true }` if the Hub is up. No JWT required. |
| 62 | |
| 63 | - **GET /api/v1/auth/providers** (no auth) — Which OAuth providers are configured. **Response:** `{ "google": boolean, "github": boolean }`. The Rich Hub UI uses this to show **Continue with Google** / **Continue with GitHub** only when env vars are set; if both are `false`, the UI explains how to configure OAuth (no separate sign-up — identity is Google or GitHub only). |
| 64 | |
| 65 | ### 3.2 Vault read |
| 66 | |
| 67 | - **GET /notes/facets** — Returns `{ projects: string[], tags: string[], folders: string[] }` for filter dropdowns. JWT required. |
| 68 | |
| 69 | - **GET /vault/folders** — Self-hosted: returns `{ "folders": string[] }` of vault-relative directory prefixes for the active vault (`inbox` first, then other top-level dirs and each `projects/<name>`). Hidden directories (names starting with `.`) are omitted. Used by the Hub **New note** folder picker; includes empty folders. Hosted gateway returns `{ "folders": ["inbox"] }` (no canister filesystem). JWT and vault access required. |
| 70 | |
| 71 | - **GET /notes** — List notes. Query params: `folder`, `project`, `tag`, `since`, `until`, `chain`, `entity`, `episode`, `limit`, `offset`, `order` (`date` \| `date-asc`), `fields` (`path` \| `path+metadata` \| `full`), `count_only`, **`content_scope`** (`all` implicit \| `notes` \| `approval_logs`) — narrow to normal notes vs materialized approval logs under `approvals/` (see approve response). |
| 72 | **Response:** `{ "notes": [ ... ], "total": number }` or `{ "total": number }` if `count_only=true`. Per-note shape per SPEC §4.2 list-notes. |
| 73 | |
| 74 | - **GET /notes/:path** — Get one note by vault-relative path. Path must be URL-encoded. |
| 75 | **Response:** `{ "path": "...", "frontmatter": { ... }, "body": "..." }` per SPEC §4.2 get-note. Hosted gateway responses normalize canister wire frontmatter into a JSON object even when the canister stores it as JSON text. |
| 76 | **404** if not found. |
| 77 | |
| 78 | - **GET /section-source?path=...** — Get body-free SectionSource metadata for one |
| 79 | authorized vault-relative note. JWT required. Hosted gateway only in Phase 1N; no canister |
| 80 | route is added. |
| 81 | **Response:** `knowtation.section_source/v0` with `{ "schema", "path", "title", |
| 82 | "sections", "truncated" }`; each section includes only `{ "section_id", "heading_id", |
| 83 | "level", "heading_path", "heading_text", "child_section_ids", "body_available", |
| 84 | "body_returned": false, "snippet_returned": false }`. |
| 85 | The response excludes note body text, section body text, snippets, full frontmatter, line |
| 86 | ranges, byte offsets, section body lengths, absolute paths, raw canister payloads, |
| 87 | provider payloads, and MCP resource URIs. |
| 88 | **400** if `path` is missing or unsafe; **401** if JWT is missing or invalid; **403** if |
| 89 | vault access or the upstream note read is forbidden; **404** if the note is missing or |
| 90 | outside scoped access; **502** for sanitized upstream failures. |
| 91 | |
| 92 | - **POST /search** — Vault search. Default is **semantic** (vector similarity; requires index on self-hosted; hosted bridge uses per-user vector store). Set **`"mode": "keyword"`** for **keyword** search: case-insensitive match over path, body, and selected frontmatter strings (`title`, `intent`, `tags`, etc.), with the same structural filters as list-notes. Optional **`match`** (keyword only): `"phrase"` (default, whole query as substring) or `"all_terms"` (every whitespace-separated token must appear, AND). Body also supports: `"folder?"`, `"project?"`, `"tag?"`, `"limit?"`, `"since?"`, `"until?"`, `"order?"`, `"fields?"`, `"chain?"`, `"entity?"`, `"episode?"`, `"snippetChars?"`, **`content_scope`** (`notes` \| `approval_logs`), **`count_only`** / **`countOnly`**. |
| 93 | **Response:** `{ "results": [ { "path", "snippet?", "score", "project", "tags" } ], "query": "...", "mode": "semantic" | "keyword" }`; keyword responses may include `"count"` when `count_only` is true. Per SPEC §4.2 search where applicable. |
| 94 | **400** if query missing. |
| 95 | |
| 96 | ### 3.3 Vault write |
| 97 | |
| 98 | - **POST /notes** — Write or update a note. Body: `{ "path": "...", "body?", "frontmatter?", "append?" }`. Path vault-relative. |
| 99 | The Hub **merges server provenance** into frontmatter: `knowtation_editor` (JWT `sub`), `knowtation_edited_at`, `author_kind: human`. Client-supplied values for those keys (and other reserved `knowtation_*` fields) are **ignored**. |
| 100 | **Response:** `{ "path": "...", "written": true }`. |
| 101 | **400** if path invalid; **403** if not allowed. |
| 102 | |
| 103 | - **POST /notes/batch** — Write many notes in one update (ICP canister: single `saveStable()` after all puts). Body: `{ "notes": [ { "path", "body", "frontmatter?" }, ... ] }`. Prefer **`frontmatter` as a JSON object** (same as gateway `POST /notes`). **Max 100** items per request; hosted bridge chunks larger imports. **Response:** `{ "imported": number, "written": true }`. **400** if JSON invalid or over limit. |
| 104 | |
| 105 | - **DELETE /notes/:path** — Remove one note by vault-relative path (URL-encoded, same as GET). **Editor or admin** only (same write gate as `POST /notes`). **Response:** `{ "path": "...", "deleted": true }`. **404** if the note does not exist; **400** if path is invalid. **Hosted semantic search:** the bridge vector index is not updated automatically; after deletes, run **Re-index** so meaning-search does not return stale hits for removed paths (see bridge indexer behavior). |
| 106 | |
| 107 | - **POST /notes/delete-by-prefix** — Bulk delete by **note path string** (vault-relative), **not** by Hub filter metadata. A note’s path is the key used in `POST /notes` (e.g. `inbox/capture.md` or `projects/acme/plan.md`). This endpoint deletes every `.md` file whose path **equals** `path_prefix` or starts with `path_prefix/` after trimming slashes. It does **not** look at frontmatter `project:` or `tags:` — notes that only set **Project** in the Hub UI but live under `inbox/...` are **not** matched by a prefix like `projects/my-slug` unless their paths actually sit under that folder. **Editor or admin** only. Body: `{ "path_prefix": "projects/acme" }` (example: delete everything under the `projects/acme/` folder layout). **Response:** `{ "deleted": number, "paths": string[], "proposals_discarded": number }` (self-hosted Hub also discards **proposed** proposals whose `path` equals one of the deleted paths or was already under the prefix). **400** if `path_prefix` is invalid. On **hosted** (ICP), the same path-string rule applies: there is no filesystem, but each note still has a stored path string. **Semantic search:** run **Re-index** after bulk delete so search does not return stale paths. |
| 108 | |
| 109 | - **POST /notes/delete-by-project** — Bulk delete every markdown note in the current vault whose **effective project slug** matches the request (same rules as `GET /notes?project=` — frontmatter `project` and path inference under `projects/<slug>/` per [SPEC.md](./SPEC.md)). **Editor or admin** only (hosted: **`viewer`** denied). Body: `{ "project": "my-slug" }` (slug normalized like list-notes). **Response:** `{ "deleted": number, "paths": string[], "proposals_discarded": number }`. **Self-hosted:** Node Hub. **Hosted:** **gateway** orchestrates the canister (not a Motoko route). The **Hub** static bundle must include **PR #65** so Settings on hosted issues these POSTs (see [HUB-METADATA-BULK-OPS.md](./HUB-METADATA-BULK-OPS.md)). |
| 110 | |
| 111 | - **POST /notes/rename-project** — Rewrites **frontmatter** `project` from `from` to `to` for every note in the current vault whose effective project slug matches `from` (does not move files on disk / path keys on canister). **Editor or admin** only. Body: `{ "from": "old-slug", "to": "new-slug" }`. **Response:** `{ "updated": number, "paths": string[] }`. **Self-hosted:** Node Hub. **Hosted:** **gateway** orchestrates `POST /notes` per matching path; same **Hub** client requirement as delete-by-project ([HUB-METADATA-BULK-OPS.md](./HUB-METADATA-BULK-OPS.md)). |
| 112 | |
| 113 | - **POST /index** — Re-run the indexer (vault → chunk → embed → vector store). Use after bulk imports or when search should reflect new or changed notes. JWT required. |
| 114 | **Response:** `{ "ok": true, "notesProcessed": number, "chunksIndexed": number }`. |
| 115 | **500** on indexer or config failure. |
| 116 | |
| 117 | - **POST /export** — Export one note to downloadable content (editor/admin). Body: `{ "path": string, "format"?: "md" | "html" }`. |
| 118 | **Response:** `{ "content": string, "filename": string }`. Client may create a blob and trigger download. |
| 119 | **400** if path invalid; **404** if note not found. |
| 120 | **Hosted:** the gateway implements this (fetch note from the canister, then build Markdown/HTML) because the ICP canister’s **`GET /api/v1/export`** is a full-vault JSON export, not a single-file download; a bare **POST** to the canister would return **404**. |
| 121 | |
| 122 | - **POST /notes/copy** — Copy or move a **single** note from one vault to another for the same user/team (editor/admin; **viewer** denied). Body: `{ "from_vault_id": string, "to_vault_id": string, "path": string, "delete_source"?: boolean }`. |
| 123 | **`path`** is vault-relative (same as GET/POST note). **`delete_source: true`** performs a **move**: after a successful write to the target vault, the source note is deleted. |
| 124 | **Access:** `from_vault_id` and `to_vault_id` must both appear in the session **`allowed_vault_ids`** (hosted: bridge `hosted-context`; self-hosted: `hub_vault_access` / defaults). |
| 125 | **Conflicts:** If `path` already exists in the target vault, the operation **overwrites** that note (same semantics as `POST /notes`). |
| 126 | **Response (success):** `{ "ok": true, "path", "from_vault_id", "to_vault_id", "moved": boolean }`. |
| 127 | **400** if vault ids match, path invalid, or body incomplete; **403** if role or vault access fails; **404** if source note not found or outside scope; **502** with **`DELETE_FAILED`** if the copy succeeded but source delete failed (move). |
| 128 | **Hosted:** implemented on the **gateway** (GET source note from canister → POST to target → optional DELETE); not a native canister multi-vault call. After success, the gateway triggers **Re-index** on the bridge for the target vault and, when moving, the source vault (asynchronous; same expectation as other hosted writes for semantic search). |
| 129 | **Self-hosted:** Node Hub reads/writes filesystem vaults on disk (`hub/server.mjs`). |
| 130 | |
| 131 | - **POST /import** — Import from uploaded file or ZIP (editor/admin). Multipart form: `source_type` (required), `file` (required), `project?`, `output_dir?`, `tags?` (comma-separated). Source types include `markdown`, `pdf`, `docx`, `url`, `chatgpt-export`, `claude-export`, `mif`, `mem0-export`, `supabase-memory`, `notion`, `jira-export`, `notebooklm`, `gdrive`, `linear-export`, `audio`, `video`, `wallet-csv` (see `lib/import-source-types.mjs`). If file is a ZIP, it is extracted and the extracted folder is used as input (for folder-based sources like chatgpt-export). For **`pdf`**, upload a single `.pdf` file (not a ZIP). For **`docx`**, upload a single `.docx` file (Office Open XML; not legacy `.doc`). |
| 132 | After import, the Hub runs a **provenance pass** on each imported path (`author_kind: import`, editor `sub`). |
| 133 | **Response:** `{ "imported": [ { "path", "source_id?" } ], "count": number }`. |
| 134 | **400** if file or source_type missing/invalid; **500** on import failure. |
| 135 | |
| 136 | - **POST /import-url** — Import a public **https** URL into the vault (editor/admin). JSON body: `{ "url": string, "mode"?: "auto" | "bookmark" | "extract", "project"?, "output_dir"?, "tags"? }` (`tags` may be a comma-separated string or string array). Server-side fetch with SSRF protections; article extraction when `mode` allows. Same provenance pass and response shape as **POST /import**. **Hosted:** gateway proxies to bridge when `BRIDGE_URL` is set. |
| 137 | |
| 138 | ### 3.3.0 Billing (Phase 16 hosted) |
| 139 | |
| 140 | - **GET /billing/summary** — JWT required. Hosted gateway only. |
| 141 | **Response:** `{ "tier", "period_start?", "period_end?", "monthly_included_cents", "monthly_included_effective_cents", "monthly_used_cents", "addon_cents", "billing_enforced", "stripe_configured", "credit_policy", "monthly_indexing_tokens_included" (number or **null** for beta = unlimited display), "monthly_indexing_tokens_used", "pack_indexing_tokens_balance", "indexing_tokens_policy", "cost_breakdown": [ … ], "usage_chart_status" }`. **Free** tier: `monthly_included_effective_cents` reflects the $0 tier allowance. **`monthly_indexing_tokens_used`** increments after each successful hosted **Re-index** when the bridge returns **`embedding_input_tokens`**. See [HOSTED-CREDITS-DESIGN.md](./HOSTED-CREDITS-DESIGN.md). |
| 142 | |
| 143 | - **POST /billing/webhook** — **Stripe** webhook endpoint; **no JWT**. Expects **raw JSON body** (signature verification). Not used on self-hosted Node Hub unless you expose the same route. |
| 144 | |
| 145 | ### 3.3.1 Settings and vault backup (JWT required) |
| 146 | |
| 147 | - **GET /settings** — Safe config status for the Settings UI. No secrets or full paths. |
| 148 | **Response:** adds **`proposal_policy_stored`** `{ "proposal_evaluation_required", "review_hints_enabled", "enrich_enabled" }` (values saved for the admin checkboxes) and **`proposal_policy_env_locked`** with the same keys (`true` where an explicit host env value overrides the file/prefs). Other fields unchanged: `"role", "user_id", "vault_id", "vault_list": [ { "id", "label?" } ], "allowed_vault_ids", "vault_path_display", "vault_git", "github_connect_available", "github_connected", "embedding_display", "proposal_enrich_enabled", "proposal_evaluation_required", "proposal_review_hints_enabled", "hub_evaluator_may_approve", "proposal_rubric": { "items": [ { "id", "label" } ] } }`. Phase 15: `vault_list` and `allowed_vault_ids` drive the vault switcher; requests use **X-Vault-Id** to scope to a vault. **Proposal LLM + gate:** effective `proposal_*_enabled` / `proposal_evaluation_required` follow [lib/hub-proposal-policy.mjs](../lib/hub-proposal-policy.mjs) on **self-hosted** (env overrides `data/hub_proposal_policy.json`). **Hosted gateway:** same env keys override persisted prefs in `data/hosted_proposal_llm_prefs.json` or Netlify Blob (`hub/gateway/proposal-llm-store.mjs`). `proposal_rubric` is the merged default + optional `data/hub_proposal_rubric.json` (see [PROPOSAL-LIFECYCLE.md](./PROPOSAL-LIFECYCLE.md)). |
| 149 | |
| 150 | - **POST /settings/proposal-policy** — **Admin** only. On the **hosted gateway**, “admin” means JWT from **`HUB_ADMIN_USER_IDS`** **or** bridge **`GET /api/v1/role`** returning **`role: admin`** (Team tab). Body: `{ "proposal_evaluation_required"?: boolean, "review_hints_enabled"?: boolean, "enrich_enabled"?: boolean }`. Merges into `data/hub_proposal_policy.json` (Node Hub) or hosted prefs store (gateway). Fields locked by explicit env on the host are ignored. **Response:** `{ "ok": true }`. |
| 151 | |
| 152 | - **POST /vault/sync** — Run manual vault sync (same as `knowtation vault sync`): git add, commit, push. Use for "Back up now" in Settings. |
| 153 | **Response:** `{ "ok": true, "message": "Synced" | "Nothing to commit" }`. |
| 154 | **400** if vault.git not configured; **500** on git failure. |
| 155 | |
| 156 | To **set the repository**: (1) Use **Settings → Setup** in the Hub to write vault path and Git remote to `data/hub_setup.yaml` (applied immediately). (2) Or edit `config/local.yaml` (see PROVENANCE-AND-GIT.md and How to use → Step 7). **Connect GitHub** (Settings): if the Hub has GitHub OAuth configured, users can click "Connect GitHub" to authorize with `scope=repo`; the Hub stores the token in `data/github_connection.json` and uses it for push so no deploy key is needed. Add callback URL `.../api/v1/auth/callback/github-connect` to your GitHub OAuth App. |
| 157 | |
| 158 | - **GET /setup** — Editable setup (vault_path, vault_git) for the Setup wizard. Returns current values. |
| 159 | - **POST /setup** — Body: `{ vault_path?, vault_git?: { enabled?, remote? } }`. Writes to `data/hub_setup.yaml` and reloads config (no restart). **400** if invalid; **500** on write failure. |
| 160 | |
| 161 | ### 3.3.2 Multi-vault admin (Phase 15; admin only) |
| 162 | |
| 163 | - **GET /vaults** — List vaults (from `data/hub_vaults.yaml` or default single vault). **Response:** `{ "vaults": [ { "id", "path", "label?" } ] }`. |
| 164 | - **POST /vaults** — Body: `{ "vaults": [ { "id", "path", "label?" } ] }`. Writes `data/hub_vaults.yaml`. At least one vault must have id `default`. **400** if invalid. |
| 165 | - **DELETE /vaults/:vaultId** — Permanently remove a **non-default** vault. **Self-hosted (Node Hub):** **admin** only. Deletes the vault directory on disk (must resolve under the project root), removes the entry from `hub_vaults.yaml`, strips `vaultId` from `hub_vault_access.json` and `hub_scope.json`, removes proposals for that vault, and deletes vector-index rows for that `vault_id` (sqlite-vec / Qdrant). **400** if `vaultId` is `default` or unknown, or if two YAML entries share the same path. **403** if the resolved vault path is outside the project root. **Response:** `{ "ok": true, "deleted_vault_id", "proposals_removed", "vectors_purged" }`. **Hosted:** When `BRIDGE_URL` is set, the **gateway** proxies **DELETE** to the **bridge**, which calls the **canister** (Motoko upgrade), then updates team `hub_vault_access` / `hub_scope` and removes the per-user vector blob for that vault. **Editor or admin** (viewer denied); **workspace owner** required when `workspace_owner_id` is set (same as creating a cloud vault). Does **not** delete a linked GitHub repo. **ICP:** Deploy an upgraded canister that implements this route before hosted delete works in production. |
| 166 | - **GET /vault-access** — User → allowed vault IDs. **Response:** `{ "access": { "user_id": [ "vault_id", ... ] } }`. |
| 167 | - **POST /vault-access** — Body: `{ "access": { "user_id": [ "vault_id", ... ] } }`. Writes `data/hub_vault_access.json`. |
| 168 | - **GET /scope** — Per-user per-vault scope (projects/folders). **Response:** `{ "scope": { "user_id": { "vault_id": { "projects": [], "folders": [] } } } }`. |
| 169 | - **POST /scope** — Body: `{ "scope": { ... } }`. Writes `data/hub_scope.json`. |
| 170 | |
| 171 | ### 3.3.3 Hosted workspace owner and delegation (bridge + gateway) |
| 172 | |
| 173 | On **hosted**, vault-access and scope JSON persist in the **bridge** (same shapes as §3.3.2). The gateway proxies these routes when `BRIDGE_URL` is set. **Workspace owner** controls which canister partition is shared with the team: |
| 174 | |
| 175 | - **GET /workspace** — Admin. **Response:** `{ "owner_user_id": string | null }`. |
| 176 | - **POST /workspace** — Admin. Body `{ "owner_user_id": string | null }`. **`null`** disables delegation (each user uses only their own canister id). |
| 177 | |
| 178 | **GET /hosted-context** — JWT. Returns `{ "actor_sub", "workspace_owner_id", "effective_canister_user_id", "delegating", "allowed_vault_ids", "scope": { "projects", "folders" } | null, "role" }` for the current **`X-Vault-Id`** header (default `default`). Used by the gateway and for debugging. |
| 179 | |
| 180 | **Gateway → canister headers:** `X-User-Id` = effective partition owner; **`X-Actor-Id`** = JWT `sub` (human/agent who performed the action). Full semantics: [MULTI-VAULT-AND-SCOPED-ACCESS.md](./MULTI-VAULT-AND-SCOPED-ACCESS.md), [TEAMS-AND-COLLABORATION.md](./TEAMS-AND-COLLABORATION.md). |
| 181 | |
| 182 | ### 3.4 Proposals |
| 183 | |
| 184 | **Variation protocol (Muse-aligned).** Proposals implement a variation lifecycle compatible with [Muse](https://github.com/cgcardona/muse): **identifiers** — `proposal_id` (variation id), `base_state_id` (optional, for optimistic concurrency); **intent** — human- or agent-readable reason for the change; **lifecycle** — propose → review → approve or discard. Default deployments **do not run Muse**; we align our API and payload so we can interoperate or adopt Muse later. Optional `external_ref` (e.g. future Muse commit id) may be added for cross-system references. |
| 185 | |
| 186 | **Lifecycle reference:** [PROPOSAL-LIFECYCLE.md](./PROPOSAL-LIFECYCLE.md) (states, roles, `kn1_` / `base_state_id` semantics). |
| 187 | |
| 188 | **Optional Muse linkage (operators).** A deployment may configure a **read-only** connection to a Muse instance for **lineage / structural history** queries (e.g. Git-replayed history in Muse’s model). That path is **not** required for JWT login, proposal CRUD, vault writes, or search. See [AGENT-INTEGRATION.md](./AGENT-INTEGRATION.md) §4 (*Optional external lineage*) and [MUSE-THIN-BRIDGE.md](./MUSE-THIN-BRIDGE.md) (env, approve behavior, admin proxy). |
| 189 | |
| 190 | - **POST /proposals** — Create a proposal (variation). Body: `{ "path?", "body?", "frontmatter?", "intent?", "base_state_id?", "external_ref?", "labels?" (string[]), "source?" (e.g. agent|human|import) }`. If path omitted, proposal may be a new note (server assigns path or client sends path). |
| 191 | **Response:** `{ "proposal_id": "...", "path": "...", "status": "proposed", ... }`. |
| 192 | **201** (Node Hub) / **200** (some proxies). **400** if invalid. |
| 193 | **Policy + triggers:** [lib/hub-proposal-policy.mjs](../lib/hub-proposal-policy.mjs) and [lib/hub-proposal-review-triggers.mjs](../lib/hub-proposal-review-triggers.mjs) set **`evaluation_status`**, **`review_queue`**, **`review_severity`**, **`auto_flag_reasons`** on create; **`proposal_auto_flagged`** is audited when reasons are non-empty. **Hosted gateway** applies the same rules to the JSON body before the canister. See [PROPOSAL-LIFECYCLE.md](./PROPOSAL-LIFECYCLE.md). |
| 194 | |
| 195 | - **GET /proposals** — List proposals. Query: `status` (e.g. `proposed`, `approved`, `discarded`), `limit`, `offset`, **`label`**, **`source`**, **`path_prefix`**, **`evaluation_status`**, **`review_queue`**, **`review_severity`** (`standard` \| `elevated`). |
| 196 | **Response:** `{ "proposals": [ { …, "review_queue?", "review_severity?", "auto_flag_reasons?" (Node) or "auto_flag_reasons_json" (canister), … } ], "total": number }`. |
| 197 | |
| 198 | - **GET /proposals/:id** — Get one proposal (metadata + proposed content). |
| 199 | **Response:** includes `body`, `frontmatter`, optional **`suggested_labels`**, **`assistant_notes`**, **`assistant_model`**, **`assistant_at`** when enrichment was run; optional **`assistant_suggested_frontmatter`** (object: SPEC-aligned suggested note metadata, normalized server-side; absent or `{}` on older proposals); **human evaluation** fields; optional **`review_hints`**, **`review_hints_at`**, **`review_hints_model`**; **`auto_flag_reasons`** (Node) or **`auto_flag_reasons_json`** (canister). |
| 200 | **404** if not found. |
| 201 | |
| 202 | - **POST /proposals/:id/evaluation** — **Admin** or **evaluator**. Record a human evaluation. Body: `{ "outcome": "pass" | "fail" | "needs_changes", "checklist?": [ { "id", "passed": boolean } ], "grade?": string, "comment?": string }`. Checklist ids should match **`GET /settings`** → **`proposal_rubric.items`**. **Pass** requires every rubric item **`passed: true`** when the rubric is non-empty. **Fail** / **needs_changes** require non-empty **`comment`**. |
| 203 | **Response:** full proposal object (Node). **400** on validation errors; **404** if not found. |
| 204 | |
| 205 | - **POST /proposals/:id/approve** — Apply proposal to vault. **Admin**, or **evaluator** when **`HUB_EVALUATOR_MAY_APPROVE=1`**. Optional body: `{ "base_state_id?", "waiver_reason?", "external_ref?" }`. **`waiver_reason`** (trimmed length ≥ 3) allows approve when **`evaluation_status`** is **`pending`**, **`failed`**, or **`needs_changes`** without a prior pass; stored as **`evaluation_waiver`** and audited. If the effective **`base_state_id`** is non-empty, the **self-hosted Node Hub** recomputes the current note fingerprint (`kn1_` per [PROPOSAL-LIFECYCLE.md](./PROPOSAL-LIFECYCLE.md)) and returns **409** `CONFLICT` when it does not match. Empty `base_state_id` skips the check (backward compatible). |
| 206 | **`external_ref` (optional Muse thin bridge):** When set on approve (non-empty, normalized server-side), it is stored on the approved proposal for cross-system lineage (e.g. Muse commit/branch id). If omitted and **`MUSE_URL`** is set on the server, the Hub **may** perform a **non-blocking** `GET` to `{MUSE_URL}/knowtation/v1/lineage-ref?proposal_id=…&vault_id=…` (Bearer **`MUSE_API_KEY`** when set); the JSON response field **`external_ref`** is used when valid. **Approve never fails** if that request errors or returns nothing; see [MUSE-THIN-BRIDGE.md](./MUSE-THIN-BRIDGE.md). **Hosted:** the gateway merges the resolved value into the JSON forwarded to the canister before approve. |
| 207 | **Response:** full proposal JSON plus **`approval_log_written`** (boolean), optional **`approval_log_path`** (vault-relative `approvals/YYYY-MM-DD-<proposal_id>.md`), and **`approval_log_error`** when the log file could not be written (approve still completes). **Hosted canister:** JSON includes **`external_ref`** on success; also returns `approval_log_path` and `approval_log_written: true` when the second vault put succeeds. |
| 208 | **403** `EVALUATION_REQUIRED` when evaluation blocks approve and waiver is missing/short. **409** if fingerprint mismatch. |
| 209 | |
| 210 | - **GET /operator/muse/proxy** — **Admin only.** Read-only forward to the operator-configured **`MUSE_URL`**. Query: **`path`** = URL-encoded path starting with an allowlisted prefix (default **`/knowtation/v1/`**; override with comma-separated **`MUSE_PROXY_PATH_PREFIXES`**). Returns **404** `NOT_FOUND` when **`MUSE_URL`** is unset (no Muse-specific error text). **Self-hosted Node Hub** and **hosted gateway** both implement this route. See [MUSE-THIN-BRIDGE.md](./MUSE-THIN-BRIDGE.md). |
| 211 | |
| 212 | - **POST /proposals/:id/review-hints** — *(ICP canister)* Internal/async: body `{ "review_hints", "review_hints_model" }` stores non-authoritative hint text when review hints are **enabled** on the gateway (env `KNOWTATION_HUB_PROPOSAL_REVIEW_HINTS` or admin-saved prefs; see **GET /settings**) and the gateway schedules a follow-up after **POST /proposals**. Not a merge gate. |
| 213 | |
| 214 | - **POST /proposals/:id/discard** — Discard proposal (do not apply). **Admin** (Node Hub). |
| 215 | **Response:** `{ "proposal_id", "status": "discarded" }`. |
| 216 | |
| 217 | - **POST /proposals/:id/enrich** — *(Optional Tier 2)* When **enrich** is enabled (env `KNOWTATION_HUB_PROPOSAL_ENRICH` or admin-saved prefs; see **GET /settings**), **editor**, **admin**, or **evaluator** may request an LLM **summary**, **suggested labels**, and **suggested frontmatter** (versioned JSON envelope parsed via [lib/proposal-enrich-llm.mjs](../lib/proposal-enrich-llm.mjs)). **404** if the feature is disabled (`NOT_FOUND` body). **Self-hosted:** Node Hub runs the model and updates local proposal storage. **Hosted:** The **gateway** runs `completeChat` ([lib/llm-complete.mjs](../lib/llm-complete.mjs)) and **POST**s `{ "assistant_notes", "assistant_model", "suggested_labels_json", "assistant_suggested_frontmatter_json" }` to the canister (`assistant_suggested_frontmatter_json` is a JSON **string** of the normalized object, capped like Node); **response** is the same shape as **GET /proposals/:id** from the canister. Chat backends: **OpenAI** (`OPENAI_API_KEY`), else **Anthropic** (`ANTHROPIC_API_KEY`), else **Ollama** (local). **Canister** route stores enrich fields only (trusted caller is the gateway with user headers). **ICP canister:** [hub/icp/src/hub/JsonValidate.mo](../hub/icp/src/hub/JsonValidate.mo) validates that `suggested_labels_json` is a JSON **array** and `assistant_suggested_frontmatter_json` is a JSON **object** before persisting (invalid values are coerced to `[]` / `{}`; **400** if valid JSON but over **4000** / **14000** characters so nothing is truncated mid-token). **GET /proposals/:id** on the canister always splices **valid** JSON fragments for those two fields (legacy bad rows fall back to `[]` / `{}`). Suggestions are **advisory** — they are **not** merged into the vault on approve unless operators copy them manually (or a future product feature adds an explicit apply step). |
| 218 | |
| 219 | ### 3.5 Capture (webhook, no JWT) |
| 220 | |
| 221 | - **POST /api/v1/capture** — Ingest message into vault inbox. Same contract as `scripts/capture-webhook.mjs`. |
| 222 | **Body:** `{ "body": string, "source_id?", "source?", "project?", "tags?" }`. |
| 223 | **Response:** `{ "ok": true, "path": "inbox/..." }`. |
| 224 | **Auth:** If `CAPTURE_WEBHOOK_SECRET` is set, require `X-Webhook-Secret: <secret>` header. Otherwise unauthenticated (local dev). |
| 225 | |
| 226 | ### 3.6 Errors |
| 227 | |
| 228 | - **401** — Missing or invalid JWT. |
| 229 | - **402** — *(Phase 16 hosted, when `BILLING_ENFORCE` is on)* Quota / billing. JSON includes `"code":`: |
| 230 | - **`QUOTA_EXHAUSTED`** — The operation would exceed **both** the **monthly included** pool and **add-on rollover** credits for this period; user should **buy add-on credits**, **upgrade** tier, or wait for period reset. Primary code for “out of credits.” |
| 231 | - **`SUBSCRIPTION_TIER_LIMIT`** — *(Optional / legacy)* Tier does not allow this operation or subscription inactive; upgrade or subscribe. |
| 232 | - **`INSUFFICIENT_CREDITS`** — *(Narrow)* Add-on wallet cannot cover the remainder after monthly pool is exhausted (synonym of exhausted state; prefer **`QUOTA_EXHAUSTED`** for new clients). |
| 233 | |
| 234 | See [HOSTED-CREDITS-DESIGN.md](./HOSTED-CREDITS-DESIGN.md). When enforcement is off (beta default), gateway does not return 402 for billing. |
| 235 | - **403** — Forbidden (e.g. scope or vault permission). |
| 236 | - **404** — Note or proposal not found. |
| 237 | - **409** — Conflict (e.g. base_state_id mismatch on approve). |
| 238 | - **500** — Server error. |
| 239 | |
| 240 | JSON error body: `{ "error": "message", "code": "ERROR_CODE" }` (align with CLI `--json` errors). |
| 241 | |
| 242 | --- |
| 243 | |
| 244 | ## 4. Rich Hub UI (contract for UI) |
| 245 | |
| 246 | The Hub UI consumes the above API. It must provide: |
| 247 | |
| 248 | - **Search bar** — Calls `POST /search` with user query; display results with path, snippet, score. |
| 249 | - **Category / filter picker** — Filter notes by project, tag, or folder using `GET /notes` query params. |
| 250 | - **Quick add** — `POST /notes` from the UI: quick capture (inbox) and full new-note form (path, title, body, project, tags). |
| 251 | - **Browse modes** — **List** (filtered rows), **Calendar** (month grid by note `date`, day drill-down), **Overview** (dashboard cards + charts: by project, tags, month). |
| 252 | - **Filter presets** — Save named filter combos in browser storage; quick filter chips for common project/tag/folder jumps. |
| 253 | - **Task / proposal views:** |
| 254 | - **Suggested tasks** — Proposals with `status=proposed` (need review). |
| 255 | - **In progress** — Proposals recently updated or in review. |
| 256 | - **Problem areas** — Failed/conflicting proposals or notes needing resolution (implementation-defined; e.g. proposals that failed approve due to conflict). |
| 257 | - **State and status** — Every list and detail shows status (draft, proposed, approved, discarded) and, where relevant, base_state_id and intention. |
| 258 | - **Actions** — Approve/discard from proposal detail; open note; edit (if in scope) via write or proposal. |
| 259 | |
| 260 | The UI is a single front-end; it is configured with the Hub base URL (self-hosted or ICP) and uses the same endpoints. |
| 261 | |
| 262 | --- |
| 263 | |
| 264 | ## 5. ICP-specific notes |
| 265 | |
| 266 | - **Internet Identity:** On ICP, the auth canister (or gateway) produces a principal or JWT that the Hub canister(s) trust. Document the exact flow (II login → session/JWT → API calls) in deployment docs. |
| 267 | - **CORS:** Canisters must allow the Hub UI origin; self-hosted Node must set CORS for the UI origin. |
| 268 | - **Storage:** Vault and proposals on ICP are stored in canister state (e.g. Documents/Assets patterns from bornfree-hub). Same API contract; implementation in Motoko (or Rust). |
| 269 | |
| 270 | ### 5.1 Operator full export (ICP hub canister only) |
| 271 | |
| 272 | **Not** implemented on the self-hosted Node Hub. Used for **scheduled logical backups** of **all** tenant user ids without stopping the canister. See [OPERATOR-BACKUP.md](./OPERATOR-BACKUP.md). |
| 273 | |
| 274 | - **`GET /api/v1/operator/export`** |
| 275 | - **Headers:** `X-Operator-Export-Key: <secret>` — must match the value set via `admin_set_operator_export_secret` (see below). |
| 276 | - **Query:** `cursor` (optional, default `0`) — index into the sorted list of user ids; `limit` (optional, default `100`, max `500`) — page size. |
| 277 | - **Response `200`:** JSON `format_version: 3`, `kind: knowtation-operator-user-index`, `user_ids` (array of strings), `next_cursor` (string, empty when `done`), `done` (boolean), `exported_at_ns` (wall clock nanoseconds text). |
| 278 | - **Errors:** `401` if the key is wrong; `503` if the operator secret was never configured (`operator_export_secret` empty on canister). |
| 279 | |
| 280 | After listing user ids, the operator runner calls existing **`GET /api/v1/export`**, **`GET /api/v1/vaults`**, and **`GET /api/v1/proposals`** with **`X-User-Id`** / **`X-Vault-Id`** per user (see [`lib/operator-full-export.mjs`](../lib/operator-full-export.mjs)). |
| 281 | |
| 282 | - **Candid (controllers only):** `admin_set_operator_export_secret(secret: text)` — only the canister’s **controllers** may call this (via `dfx canister call`). Sets the shared secret checked by `X-Operator-Export-Key`. **Do not** expose the secret in client apps or public repos. |
| 283 | |
| 284 | --- |
| 285 | |
| 286 | ## 6. CLI integration |
| 287 | |
| 288 | - **knowtation hub status** — Calls `GET /health` (and optionally an authenticated endpoint) to report whether the Hub at configured URL is reachable and the user is logged in (if token available). |
| 289 | - **knowtation propose --hub \<url\>** — Creates a proposal via `POST /proposals`; requires Hub URL and credentials (token from login flow or env). Document in setup how to obtain and store the token for CLI use. |
| 290 | |
| 291 | See SPEC.md for CLI behavior aligned with Hub routes. |