README.md
file-level
1
files
1
commits
0
hotspots
0
🧊 dead
0
💥 blast risk
| 1 | # Knowtation Hub Gateway |
| 2 | |
| 3 | OAuth (Google/GitHub) + proxy for the **hosted** product. Users log in here; the gateway proxies all `/api/v1/*` requests to the ICP canister with an **X-User-Id** header (proof the canister trusts). Trust model: JWT `sub` → **`x-user-id`** on canister requests — see **[docs/HUB-API.md](../../docs/HUB-API.md)** (auth). For **pre-merge** verification of **bridge-backed** `POST /api/v1/import` (including 4C drop) when **`BRIDGE_URL`** is set, see **[docs/IMPORT-URL-AND-DOCUMENTS-PHASES.md](../../docs/IMPORT-URL-AND-DOCUMENTS-PHASES.md)** §4V. |
| 4 | |
| 5 | ## Routes |
| 6 | |
| 7 | - **GET /health**, **GET /api/v1/health** — Health (no auth). |
| 8 | - **GET /api/v1/auth/providers** — Which OAuth providers are configured (no auth). |
| 9 | - **GET /api/v1/auth/session** — **C7 Session introspection.** Bearer JWT required. Returns verified identity + API scopes: `{ sub, provider, id, name, role, iat, exp, scopes }`. Safe for cross-origin callers (Scooling, external services). Response is derived from the signed token only — no DB call. Scopes are role-derived (`admin` → `[vault:read, vault:write, admin]`; `member` → `[vault:read, vault:write]`); explicit per-user scope management is C4. |
| 10 | - **POST /api/v1/auth/refresh** — Exchange the `ktn_refresh` HttpOnly cookie for a fresh `access_token` Bearer JWT. Rotates the refresh token (reuse detection + family revocation). See `hub/gateway/refresh-token-store.mjs`. |
| 11 | - **POST /api/v1/auth/logout** — Revoke the `ktn_refresh` cookie server-side (burns the token family) and clear the cookie. |
| 12 | - **GET /scooling/note-outline/smoke?path=...** — Local/staging-only Scooling NoteOutline smoke bridge. Disabled unless **`SCOOLING_NOTE_OUTLINE_SMOKE_ENABLED=1`** and **`SCOOLING_NOTE_OUTLINE_SMOKE_ENV=local|staging`**. The gateway owns the upstream bearer token, rejects request credentials from Scooling, validates the upstream body-free `NoteOutline`, and returns only the `knowtation.note_outline/v1` JSON contract. |
| 13 | - **POST /scooling/write-back/smoke** — Staging-only, metadata-only Scooling write-back target smoke check. Disabled unless **`SCOOLING_WRITE_BACK_SMOKE_ENABLED=1`** and **`SCOOLING_WRITE_BACK_SMOKE_ENV=staging`**. It checks canister metadata availability and returns dry-run capability flags only; it never accepts raw credentials or performs live writes. |
| 14 | - **GET /auth/login?provider=google|github** — Redirect to OAuth (plan routes). |
| 15 | - **GET /api/v1/auth/login?provider=...** — Redirects to `/auth/login` for Hub UI compatibility. |
| 16 | - **GET /auth/callback/google**, **GET /auth/callback/github** — OAuth callbacks; on success redirect to `HUB_UI_ORIGIN/?token=<jwt>`. |
| 17 | - **GET /api/v1/billing/summary** — JWT. Hosted billing pools (tier, monthly/add-on cents). See **`hub/gateway/billing-*.mjs`** and **[docs/TOKEN-SAVINGS.md](../../docs/TOKEN-SAVINGS.md)** (billing hooks). |
| 18 | - **POST /api/v1/billing/webhook** — Stripe webhook (**raw JSON body**). No JWT. |
| 19 | - **GET /api/v1/notes/facets** — JWT + **X-Vault-Id**. Aggregates `projects`, `tags`, and `folders` from the canister note list (`hub/gateway/note-facets.mjs`); not proxied as a literal canister path. |
| 20 | - **POST /api/v1/notes/delete-by-project**, **POST /api/v1/notes/rename-project** — JWT + **X-Vault-Id**; **editor/admin/member** (not **viewer**). Gateway orchestrates canister list/delete/write + proposal discards (`hub/gateway/metadata-bulk-canister.mjs`). The **Hub** static bundle must include **PR #65** (`web/hub/hub.js`) so Settings on hosted actually calls these routes. See [HUB-METADATA-BULK-OPS.md](../../docs/HUB-METADATA-BULK-OPS.md). |
| 21 | - **POST /api/v1/notes/copy** — JWT; **editor/admin** (not **viewer**). Copy or **move** one note between vaults (`from_vault_id`, `to_vault_id`, `path`, optional `delete_source`). Gateway-only orchestration (GET → POST → optional DELETE on canister); then bridge **re-index** for affected vaults. Hub UI: note detail **Copy to vault…**. Contract: [HUB-API.md](../../docs/HUB-API.md) §3.3. |
| 22 | - **GET/POST/DELETE /api/v1/*** (other) — Proxied to canister with **X-User-Id** from JWT (e.g. **DELETE /api/v1/notes/:path** removes a note). Returns 401 if no valid token. When **BILLING_ENFORCE** is on, some routes may return **402** (quota). |
| 23 | - **POST /api/v1/roles/evaluator-may-approve** — JWT + **admin** (via `requireAdmin`). Proxied to bridge when **BRIDGE_URL** is set (per-evaluator approve flag); same origin as other Team routes. |
| 24 | - **Proposal approve/discard:** **POST** `/api/v1/proposals/:id/approve` and `…/discard` are checked on the gateway using the **actor** JWT and bridge **role** + **may_approve_proposals** (from **GET /api/v1/role** / **hosted-context**). The canister uses the **effective** workspace user id and does not distinguish actor roles; gateway enforcement is required for correct RBAC. **Discard** requires **admin**. **Approve** requires **admin** or **evaluator** with permission (per-user in Team / bridge blob, else **`HUB_EVALUATOR_MAY_APPROVE=1`** fallback when no per-user entry). |
| 25 | - **LLM proposal helpers:** **Review hints** run on this **gateway** after proposal create when hints are enabled (`KNOWTATION_HUB_PROPOSAL_REVIEW_HINTS` or admin-saved prefs in **`hub/gateway/proposal-llm-store.mjs`** / Netlify Blob; see **`POST /api/v1/settings/proposal-policy`**). **Enrich** runs here when enabled (same pattern; **`POST /api/v1/proposals/:id/enrich`**). Implementation: `proposal-review-hints-async.mjs`, `proposal-enrich-hosted.mjs`. Deploy the **hub** canister from this repo first (stable migration V4 adds enrich fields). See **[docs/HUB-PROPOSAL-LLM-FEATURES.md](../../docs/HUB-PROPOSAL-LLM-FEATURES.md)**. |
| 26 | |
| 27 | ## Canister proxy URL (important) |
| 28 | |
| 29 | The canister proxy runs under **`app.use('/api/v1', …)`**. Express sets **`req.baseUrl` + `req.path`** to the full API path (e.g. `/api/v1/notes`); **`req.originalUrl`** alone can be wrong under Netlify/serverless-http. See **`hub/gateway/request-path.mjs`** (`effectiveRequestPath`, `upstreamPathAndQuery`). |
| 30 | |
| 31 | When the gateway **re-serializes** the JSON body (e.g. provenance merge), it **removes** the incoming **`Content-Length`**, **`Transfer-Encoding`**, and **`Content-Encoding`** before `fetch` to the canister. Keeping the client’s **`Content-Length`** (from the shorter pre-merge body) can cause Undici to **hang** or mis-handle the write relative to the new body, so create/save appears to do nothing; the canister may also see truncated or invalid JSON. |
| 32 | |
| 33 | ## Environment |
| 34 | |
| 35 | | Variable | Required | Description | |
| 36 | |----------|----------|-------------| |
| 37 | | **CANISTER_URL** | Yes | Canister HTTP URL (e.g. `https://<canister-id>.ic0.app`). | |
| 38 | | **SESSION_SECRET** or **HUB_JWT_SECRET** | Yes | Secret to sign JWTs. | |
| 39 | | **HUB_BASE_URL** | Yes (prod) | Public URL of this gateway (for OAuth callback). E.g. `https://knowtation.store` if gateway is same origin. | |
| 40 | | **HUB_UI_ORIGIN** | No | Origin of the Hub UI (for post-login redirect). Defaults to HUB_BASE_URL. E.g. `https://knowtation.store`. | |
| 41 | | **BRIDGE_URL** | No | URL of the Hub Bridge (for Connect GitHub + Back up now). **Must be a full URL with `http://` or `https://`** and the bridge origin only: e.g. `https://knowtation-bridge.netlify.app` — no trailing slash, no path (no `/api/...` or `/auth/...`). A host-only value (no scheme) fails at gateway startup. When set, gateway redirects/proxies `/api/v1/auth/github-connect` and `/api/v1/vault/*` to the bridge so the UI can use one origin. | |
| 42 | | **GOOGLE_CLIENT_ID**, **GOOGLE_CLIENT_SECRET** | No | Google OAuth (enables "Continue with Google"). | |
| 43 | | **GITHUB_CLIENT_ID**, **GITHUB_CLIENT_SECRET** | No | GitHub OAuth (enables "Continue with GitHub"). | |
| 44 | | **GATEWAY_PORT** or **PORT** | No | Port (default 3340). | |
| 45 | | **SCOOLING_NOTE_OUTLINE_SMOKE_ENABLED** | No | Set to `1` only for local/staging structural UX smoke validation. | |
| 46 | | **SCOOLING_NOTE_OUTLINE_SMOKE_ENV** | No | Must be `local` or `staging` for `GET /scooling/note-outline/smoke` to answer; any other value returns 404. | |
| 47 | | **SCOOLING_NOTE_OUTLINE_SMOKE_UPSTREAM** | No | Full HTTP(S) URL for the auth-gated upstream `GET /api/v1/note-outline` endpoint. Must not include credentials, query string, or fragment. | |
| 48 | | **SCOOLING_NOTE_OUTLINE_SMOKE_BEARER_TOKEN** | No | Bearer token held only by the gateway bridge for the upstream NoteOutline request. Scooling must not receive, send, store, or log this token. | |
| 49 | | **SCOOLING_WRITE_BACK_SMOKE_ENABLED** | No | Set to `1` only on staging to expose `POST /scooling/write-back/smoke` for Scooling target validation. | |
| 50 | | **SCOOLING_WRITE_BACK_SMOKE_ENV** | No | Must be `staging` for the Scooling smoke endpoint to answer; any other value returns 404. | |
| 51 | | **HUB_CORS_ORIGIN** | **Yes (prod)** if Hub UI is on another origin | Comma-separated origins, e.g. `https://knowtation.store,https://www.knowtation.store`. Required for credentialed CORS responses — see **`hub/gateway/cors-middleware.mjs`**. | |
| 52 | | **HUB_JWT_EXPIRY** | No | JWT expiry for **issued API tokens** (gateway default in code is **`24h`** unless overridden; match `docs/AGENT-INTEGRATION.md`). | |
| 53 | | **HUB_ADMIN_USER_IDS** | No | Comma-separated user IDs (e.g. `google:123,github:456`) who get role **admin** in the JWT and pass **`requireAdmin`** without a bridge call. When **BRIDGE_URL** is set, **Team admins** (bridge **`GET /api/v1/role`** → `role: admin`) also pass **`requireAdmin`** for gateway-only admin routes (e.g. **`POST /api/v1/settings/proposal-policy`**, workspace, vault-access, invites). Bootstrap operators often list at least one id here; further admins can be granted in Team without redeploying env. See **[docs/PARITY-MATRIX-HOSTED.md](../../docs/PARITY-MATRIX-HOSTED.md)** (hosted admin flows). | |
| 54 | | **HUB_EVALUATOR_MAY_APPROVE** | No | Set to **`1`** so **evaluators** may **approve** when they have **no** explicit row in the bridge blob **`hub_evaluator_may_approve`** (per-user overrides still win). Gateway and bridge honor this; **GET /api/v1/settings** exposes effective **hub_evaluator_may_approve** per user. | |
| 55 | | **BILLING_ENFORCE** | No | Set to `true` to deduct credits and return **402** when monthly + add-on pools are exhausted (default off = beta open usage). | |
| 56 | | **BILLING_SHADOW_LOG** | No | Set to `true` or `1` to emit **structured JSON** (`type: knowtation_billing_shadow`) per billable operation for **usage research** (works even when enforcement is off). | |
| 57 | | **STRIPE_SECRET_KEY** | No | Stripe API key for webhooks and (future) Checkout sessions. | |
| 58 | | **STRIPE_WEBHOOK_SECRET** | No | Signing secret for **POST /api/v1/billing/webhook**. | |
| 59 | | **STRIPE_PRICE_STARTER**, **STRIPE_PRICE_PRO**, **STRIPE_PRICE_TEAM** | No | Stripe Price ids for subscription tiers → included credits/month. | |
| 60 | | **STRIPE_PRICE_PACK_10**, **STRIPE_PRICE_PACK_25**, **STRIPE_PRICE_PACK_50** | No | Stripe Price ids for add-on packs (10 / 25 / 50 credits). | |
| 61 | |
| 62 | **Hub static UI (`web/hub/config.js`, separate from this server’s env):** The browser bundle may set `window.HUB_MCP_PUBLIC_URL` to the **public URL of a persistent gateway** that serves `POST /mcp` (e.g. `https://mcp.example.com/mcp`), so **Settings → Integrations → Copy Hub URL, token & vault** can emit **`KNOWTATION_MCP_URL`** next to `KNOWTATION_HUB_URL`. The **Netlify** site for the REST API does **not** run stateful MCP; operators who split API (Netlify) and MCP (VPS) document both URLs in that copy block. See **`docs/AGENT-INTEGRATION.md`** §2–3. |
| 63 | |
| 64 | **Billing storage:** Local file **`data/hosted_billing.json`** (gitignored with `data/`). On **Netlify**, the gateway function uses Blob store **`gateway-billing`** (see `netlify/functions/gateway.mjs`). |
| 65 | |
| 66 | **Checkout metadata:** Subscription and pack Checkout Sessions should include **`metadata.user_id`** (Hub JWT `sub`). Pack sessions should include **`metadata.credits_cents`** (e.g. `1000` for $10) or use a mapped **Price** id above. |
| 67 | |
| 68 | ## Google OAuth — redirect URI (fixes `redirect_uri_mismatch`) |
| 69 | |
| 70 | Passport uses **`callbackURL = HUB_BASE_URL + '/auth/callback/google'`** (not `/api/v1/...`). The full Node Hub under `hub/server.mjs` uses `/api/v1/auth/callback/google`; the gateway does **not**. |
| 71 | |
| 72 | In **Google Cloud Console** → OAuth client → **Authorized redirect URIs**, add the URI that matches **`HUB_BASE_URL`**: |
| 73 | |
| 74 | | If you run gateway on | Set `HUB_BASE_URL` | Add this exact URI in Google | |
| 75 | |----------------------|--------------------|------------------------------| |
| 76 | | Default (3340) | `http://localhost:3340` | `http://localhost:3340/auth/callback/google` | |
| 77 | | 3333 | `http://localhost:3333` | `http://localhost:3333/auth/callback/google` | |
| 78 | |
| 79 | Production: `https://YOUR-GATEWAY-URL/auth/callback/google` (no trailing slash). |
| 80 | |
| 81 | If Google only has `http://localhost:3333/api/v1/auth/callback/google` (full Hub) and you log in via the **gateway**, the request fails with **Error 400: redirect_uri_mismatch** — add the `/auth/callback/google` URI for the port you use. |
| 82 | |
| 83 | ## Run locally |
| 84 | |
| 85 | ```bash |
| 86 | cd hub/gateway |
| 87 | npm install |
| 88 | export CANISTER_URL=https://<canister-id>.ic0.app |
| 89 | export SESSION_SECRET=your-secret |
| 90 | export HUB_BASE_URL=http://localhost:3340 |
| 91 | export GOOGLE_CLIENT_ID=... GOOGLE_CLIENT_SECRET=... |
| 92 | # optional: GITHUB_CLIENT_ID, GITHUB_CLIENT_SECRET |
| 93 | npm start |
| 94 | ``` |
| 95 | |
| 96 | Point the Hub UI at the same origin as **`HUB_BASE_URL`** (e.g. `window.HUB_API_BASE_URL = 'http://localhost:3340'`). Login will redirect to Google/GitHub and back; then all API calls go through the gateway to the canister with X-User-Id. |
| 97 | |
| 98 | ## Deploy (e.g. Netlify) |
| 99 | |
| 100 | - **This repo:** Production path is `netlify/functions/gateway.mjs` plus root `netlify.toml`. The build runs `scripts/netlify-redirects.mjs` to generate `public/_redirects` (per-site: gateway vs bridge is controlled by `USE_BRIDGE_FUNCTION` on the bridge site only). Do not add a catch-all `[[redirects]]` in root `netlify.toml` when using a second Netlify site for the bridge — see **`deploy/bridge`** packaging and **`hub/bridge/README.md`** (Netlify Blobs). |
| 101 | - **Local / generic Node:** Build is not required when running `npm start` as a normal server. |
| 102 | - For other hosts, use a Node adapter or deploy the Express app as you would any Node service; set **HUB_BASE_URL** and **HUB_UI_ORIGIN** to production URLs. |
| 103 | - Ensure **CANISTER_URL** points to the deployed canister and **SESSION_SECRET** is set in env (no secrets in repo). |
| 104 | |
| 105 | ## Post-deploy verification (GitHub backup + CORS) |
| 106 | |
| 107 | 1. **CORS (Hub UI on knowtation.store / www):** From the repo root, run `npm run check:gateway-cors`. Each listed origin should get a **specific** `Allow-Origin` and `Allow-Credentials: true`. If not, set **`HUB_CORS_ORIGIN`** on this gateway site to both apex and www (`hub/gateway/cors-middleware.mjs`), then redeploy. |
| 108 | |
| 109 | **Hosted “Back up now”:** `POST /api/v1/vault/sync` triggers a CORS **preflight** (`OPTIONS`). The gateway must answer **`OPTIONS` with 204** on that path; it must **not** forward preflight to the bridge (the bridge only implements `POST`). If preflight fails, the Hub shows *Could not reach the API* even when `GET /api/v1/settings` works. |
| 110 | |
| 111 | 2. **`BRIDGE_URL`:** Must be the bridge **origin only** — full URL with `https://`, no path (e.g. `https://knowtation-bridge.netlify.app`). Wrong values produce malformed redirect URLs; see [docs/CONNECT-GITHUB-AND-STORAGE-CHECK.md](../../docs/CONNECT-GITHUB-AND-STORAGE-CHECK.md) §2–3. |
| 112 | |
| 113 | 3. **`SESSION_SECRET` / `HUB_JWT_SECRET`:** The **bridge** site must use the **same** secret as this gateway so JWTs verify on `/api/v1/vault/github-status` and Connect GitHub. Mismatch → Settings shows “Not connected” after OAuth; see [docs/CONNECT-GITHUB-AND-STORAGE-CHECK.md](../../docs/CONNECT-GITHUB-AND-STORAGE-CHECK.md) §6. |
| 114 | |
| 115 | ## Reference |
| 116 | |
| 117 | - [HUB-API.md](../../docs/HUB-API.md) — API contract (auth, proposals, vault headers) |