SECTION-SOURCE-TRANSPORT-PLAN.md
file-level
1
files
1
commits
0
hotspots
0
🧊 dead
0
💥 blast risk
| 1 | # SectionSource Body-Free Transport Plan |
| 2 | |
| 3 | ## Simple Summary |
| 4 | |
| 5 | Phase 1D decides how `SectionSource v0` may later be exposed outside direct library calls. |
| 6 | |
| 7 | The decision is conservative: only body-free section metadata is eligible for transport. |
| 8 | The local CLI and self-hosted MCP surfaces are now implemented. No hosted MCP tool, Hub |
| 9 | endpoint, search mode, Scooling runtime behavior, body text, snippets, summaries, |
| 10 | persistence, PageIndex, OCR, LLM call, or provider routing is added. |
| 11 | |
| 12 | ## Technical Summary |
| 13 | |
| 14 | Phase 1D accepts transport planning only. It does not implement a transport. |
| 15 | |
| 16 | Body-free transport work may proceed only through separately tested implementation phases. |
| 17 | Those implementations must expose the existing `SectionSource v0` shape without adding note |
| 18 | bodies, section bodies, snippets, exact line ranges, byte offsets, section body lengths, raw |
| 19 | frontmatter, vectors, summaries, provider payloads, or persisted section records. |
| 20 | |
| 21 | ## Transport Decision |
| 22 | |
| 23 | The accepted transport decision is: |
| 24 | |
| 25 | - Local library support is already implemented through `readSectionSource`. |
| 26 | - The local CLI surface is implemented for body-free `SectionSource v0` metadata. |
| 27 | - The self-hosted MCP tool is implemented for body-free `SectionSource v0` metadata. |
| 28 | - Hosted MCP, Hub REST, OpenAPI, Hub UI, canister routes, search modes, Scooling runtime |
| 29 | consumption, PageIndex, OCR, LLM calls, external provider routing, section body reads, |
| 30 | and snippets remain blocked. |
| 31 | |
| 32 | This plan does not accept a hosted route, schema extension, hosted role change, ACL change, |
| 33 | or Scooling adapter shape. |
| 34 | |
| 35 | ## Allowed Future Local CLI Shape |
| 36 | |
| 37 | The local CLI phase may add a body-free note-scoped command only if it: |
| 38 | |
| 39 | - accepts exactly one vault-relative note path |
| 40 | - uses existing local config and vault resolution behavior |
| 41 | - reads one note through the existing local note-read path |
| 42 | - returns only the accepted `SectionSource v0` fields |
| 43 | - returns `body_returned: false` |
| 44 | - returns `snippet_returned: false` |
| 45 | - exposes no body text, snippets, raw frontmatter, line ranges, byte offsets, section body |
| 46 | lengths, summaries, vectors, provider payloads, or persistence ids |
| 47 | - includes seven-tier tests for the CLI surface |
| 48 | |
| 49 | ## Allowed Future Self-Hosted MCP Shape |
| 50 | |
| 51 | The self-hosted MCP phase may add a body-free note-scoped read tool only if it: |
| 52 | |
| 53 | - mirrors `get_note_outline`, `get_document_tree`, and `get_metadata_facets` local auth and |
| 54 | path behavior |
| 55 | - accepts exactly one vault-relative note path |
| 56 | - reads one note through the existing local note-read path |
| 57 | - returns only the accepted `SectionSource v0` fields |
| 58 | - returns `body_returned: false` |
| 59 | - returns `snippet_returned: false` |
| 60 | - exposes no MCP resource URI for section bodies, snippets, or section sources |
| 61 | - includes seven-tier tests for the MCP surface |
| 62 | |
| 63 | ## Hosted Transport Block |
| 64 | |
| 65 | Hosted MCP and Hub transport remain blocked. |
| 66 | |
| 67 | Before any hosted SectionSource transport can be proposed, a separate hosted authorization |
| 68 | review must prove: |
| 69 | |
| 70 | - active vault isolation |
| 71 | - effective canister user isolation |
| 72 | - hosted role ACL behavior |
| 73 | - missing and unauthorized note behavior |
| 74 | - no body or snippet logging |
| 75 | - no section body, snippet, line range, byte offset, summary, vector, provider payload, or |
| 76 | persisted section record leakage |
| 77 | - multi-vault regression coverage |
| 78 | |
| 79 | ## Search And Persistence Block |
| 80 | |
| 81 | Transport planning does not approve search or persistence. |
| 82 | |
| 83 | Future search or persistence work requires a separate spec because it changes deletion, |
| 84 | export, stale-data, ranking, logging, and authorization behavior. No transport may create |
| 85 | sidecars, indexes, vectors, memory events, summaries, provider records, or cached section |
| 86 | payloads. |
| 87 | |
| 88 | ## Scooling Boundary |
| 89 | |
| 90 | Scooling remains blocked from live SectionSource runtime consumption until a separate |
| 91 | Scooling phase. |
| 92 | |
| 93 | A future Scooling phase must keep Scooling as a consumer through a Scooling-owned adapter. |
| 94 | Scooling must not parse Markdown as the canonical section parser, derive canonical section |
| 95 | ids, persist section sources as truth, expose private learner sections outside authorized |
| 96 | contexts, or treat section text as model instructions. |
| 97 | |
| 98 | ## Seven-Tier Test Requirements |
| 99 | |
| 100 | ### Unit |
| 101 | |
| 102 | - Transport handlers return the same `SectionSource v0` schema as the library. |
| 103 | - Transport handlers keep `body_returned` and `snippet_returned` false. |
| 104 | - Transport handlers reject unsafe paths before note reads. |
| 105 | |
| 106 | ### Integration |
| 107 | |
| 108 | - CLI and self-hosted MCP surfaces reuse existing local note-read behavior. |
| 109 | - Missing and traversal paths reveal no extra detail beyond existing note reads. |
| 110 | - Future transport output matches the library output for the same note. |
| 111 | |
| 112 | ### End To End |
| 113 | |
| 114 | - Agent flows can inspect section candidates through a body-free surface. |
| 115 | - No agent flow receives section body text or snippets. |
| 116 | - Scooling fallback remains document-level or body-free until a separate Scooling phase. |
| 117 | |
| 118 | ### Stress |
| 119 | |
| 120 | - Large notes remain capped by heading count and input caps. |
| 121 | - Transport output size remains bounded by section count and text caps. |
| 122 | - Repeated transport calls are deterministic for unchanged notes. |
| 123 | |
| 124 | ### Data Integrity |
| 125 | |
| 126 | - Transport calls do not write notes, sidecars, indexes, vectors, memory, summaries, or |
| 127 | canister state. |
| 128 | - Transport calls reflect current note content only. |
| 129 | - Persisted future phases define delete, edit, export, backup, and restore behavior before |
| 130 | implementation. |
| 131 | |
| 132 | ### Performance |
| 133 | |
| 134 | - One-note transport calls do not scan the whole vault. |
| 135 | - No external provider calls occur. |
| 136 | - Hosted future phases must define rate limits and multi-vault isolation before exposure. |
| 137 | |
| 138 | ### Security |
| 139 | |
| 140 | - No note body text in transport output. |
| 141 | - No section body text in transport output. |
| 142 | - No snippets in transport output. |
| 143 | - No full frontmatter in transport output. |
| 144 | - No absolute filesystem paths in transport output. |
| 145 | - No MCP resource URI for section body or snippet data. |
| 146 | - Prompt-injection text remains untrusted source material. |
| 147 | - Hosted, Scooling, classroom, search, and persistence exposure remain blocked until |
| 148 | separately reviewed. |
| 149 | |
| 150 | ## Stop Conditions |
| 151 | |
| 152 | Stop and re-plan if a transport proposal requires: |
| 153 | |
| 154 | - returning note body text |
| 155 | - returning section body text |
| 156 | - returning snippets |
| 157 | - returning exact line ranges |
| 158 | - returning byte offsets |
| 159 | - returning section body lengths |
| 160 | - adding summaries |
| 161 | - adding search, vectors, indexes, persistence, sidecars, or memory events |
| 162 | - adding hosted MCP, Hub REST, OpenAPI, Hub UI, canister routes, or Scooling runtime code |
| 163 | - sending note-derived content to LLMs, OCR, PageIndex, or external providers |
| 164 | - weakening existing note-read authorization behavior |
| 165 | |
| 166 | ## Acceptance Criteria |
| 167 | |
| 168 | Phase 1D is accepted when: |
| 169 | |
| 170 | - This plan limits future transport to body-free SectionSource metadata. |
| 171 | - This plan identifies local CLI and self-hosted MCP as the only eligible future transport |
| 172 | surfaces. |
| 173 | - This plan keeps hosted, Hub, search, persistence, Scooling, body, snippet, provider, OCR, |
| 174 | PageIndex, and LLM behavior blocked. |
| 175 | - Tests prove the current CLI and self-hosted MCP output remain body-free and hosted, Hub, |
| 176 | and Scooling runtime transport remain blocked. |
| 177 | - Tests prove current SectionSource output remains body-free. |
| 178 | |
| 179 | ## Recommendation |
| 180 | |
| 181 | Phase 1E is implemented in `cli/index.mjs`; Phase 1G is implemented in |
| 182 | `mcp/create-server.mjs`. Both are documented in |
| 183 | `docs/SECTION-SOURCE-CLI-IMPLEMENTATION-SPEC.md` and |
| 184 | `docs/SECTION-SOURCE-MCP-IMPLEMENTATION-SPEC.md`. |
| 185 | |
| 186 | The hosted authorization review is accepted in |
| 187 | `docs/SECTION-SOURCE-HOSTED-AUTHORIZATION-REVIEW-SPEC.md`. |
| 188 | |
| 189 | The Scooling adapter planning boundary is accepted in |
| 190 | `docs/SECTION-SOURCE-SCOOLING-ADAPTER-PLANNING-SPEC.md`. |
| 191 | |
| 192 | The Scooling adapter slice is complete in the Scooling repository at commit `ea5421e` |
| 193 | (`Add Scooling SectionSource transport boundary`). |
| 194 | |
| 195 | Proceed next only with a separate hosted implementation spec if hosted SectionSource is |
| 196 | required. Do not implement hosted, Hub, search, persistence, body reads, snippets, |
| 197 | summaries, OCR, PageIndex, LLM calls, provider routing, or write-back behavior without that |
| 198 | separate plan. |