gabriel / musehub public
json_types.py python
271 lines 12.4 KB
Raw
sha256:c73281cffffbc70487969720af074e54b34326d59ec6aff0827fa16512a4e512 docs(mwp-2): mark all acceptance criteria met; add closing … Sonnet 4.6 82 days ago
1 """Canonical type definitions for JSON data structures.
2
3 This module is the **single source of truth for generic JSON types** in
4 MuseHub. Import from here; do not redefine shapes ad hoc.
5
6 ## The Three-Layer Type System
7
8 Python dict invariance is a constant source of friction when building
9 heterogeneous dicts: ``dict[str, str]`` is NOT assignable to
10 ``dict[str, JSONValue]`` even though ``str ⊆ JSONValue``. The correct
11 architectural response is to use three distinct layers:
12
13 | Layer | Type | When to use | Rust analogue |
14 |-------|------|-------------|---------------|
15 | **Building / returning** JSON | ``JSONObject`` (= ``dict[str, JSONValue]``) | Constructing a JSON payload to send or store | ``HashMap<String, Value>`` (owned) |
16 | **Receiving / passing** JSON | ``ReadOnlyJSONObject`` (= ``Mapping[str, JSONValue]``) | Function parameters that only read a JSON dict — covariant, passes the type audit | ``&HashMap<String, Value>`` (borrowed) |
17 | **Known-shape** dicts | Per-page ``TypedDict`` subclass | When the keys are statically known — the long-term goal for all template contexts | ``#[derive(Serialize)] struct PageCtx { ... }`` |
18
19 ### Why ``Mapping[str, JSONValue]`` works at boundaries
20
21 ``Mapping`` (from ``collections.abc``) is **covariant** in its value type.
22 Any ``dict[str, T]`` where ``T ⊆ JSONValue`` is assignable to
23 ``Mapping[str, JSONValue]`` — including ``dict[str, str]``,
24 ``dict[str, int]``, and plain ``dict[str, JSONValue]``. The type audit
25 ``boundary_dict`` pattern fires only on ``dict[str,``; ``mapping_any``
26 fires only on ``Mapping[str, Any]``. ``Mapping[str, JSONValue]`` triggers
27 neither — it is the approved read-only form at function boundaries.
28
29 Do **not** use ``ReadOnlyJSONObject`` for Pydantic ``BaseModel`` fields.
30
31 ## When to use which type
32
33 Use ``JSONValue`` / ``JSONObject`` only when the shape is genuinely unknown
34 (e.g. an arbitrary external payload). For every known structure, use the
35 named TypedDict below.
36
37 Do **not** use ``JSONValue`` or ``JSONObject`` in Pydantic ``BaseModel``
38 fields — Pydantic v2 cannot resolve the recursive forward references and
39 raises ``RecursionError`` at schema generation time. Use
40 ``musehub.muse_contracts.pydantic_types.PydanticJson`` instead.
41
42 ## Conversion helpers
43
44 - ``json_list(items)`` — coerce a ``list[TypedDict]`` to ``list[JSONValue]``
45 at list-insertion boundaries (Python list invariance workaround).
46 - ``jint(v)`` / ``jfloat(v)`` — safe numeric extraction from ``JSONValue``.
47
48 ## Entity catalog
49
50 JSON primitives:
51 JSONScalar — str | int | float | bool | None
52 JSONValue — recursive JSON value (use sparingly; not in Pydantic)
53 JSONObject — dict[str, JSONValue] (mutable; use when building JSON)
54 ReadOnlyJSONObject — Mapping[str, JSONValue] (use at read-only boundaries)
55
56 Protocol introspection aliases:
57 EventJsonSchema — dict[str, JSONValue] (single event JSON Schema)
58 EventSchemaMap — dict[str, EventJsonSchema] (event_type → JSON Schema)
59
60 Utility aliases:
61 StrDict — dict[str, str] (HTTP headers, auth headers, label maps)
62 """
63
64 from collections.abc import Iterable, Mapping
65 from typing import TypedDict, overload
66
67 # ═══════════════════════════════════════════════════════════════════════════════
68 # Generic JSON types — use ONLY when the shape is truly unknown
69 #
70 # PYDANTIC COMPATIBILITY RULE
71 # ───────────────────────────
72 # JSONValue and JSONObject are *mypy-only* type aliases. JSONValue is
73 # recursive: it contains ``list["JSONValue"]`` and ``dict[str, "JSONValue"]``
74 # string forward references. Pydantic v2 must resolve those strings at runtime
75 # against the importing module's namespace, and fails when they cross module
76 # boundaries — producing a ``PydanticUserError: not fully defined`` at
77 # instantiation time.
78 #
79 # Rule: **never use JSONValue or JSONObject in a Pydantic BaseModel field.**
80 #
81 # Where to use each:
82 # JSONValue / JSONObject — TypedDicts, dataclasses, function signatures.
83 # Pure mypy land; Pydantic never sees them.
84 # dict[str, object] — Pydantic BaseModel fields that must hold opaque
85 # external JSON. ``object`` is not ``Any`` — mypy requires
86 # explicit narrowing before use.
87 # ═══════════════════════════════════════════════════════════════════════════════
88
89 JSONScalar = str | int | float | bool | None
90 """A JSON leaf value with no recursive structure."""
91
92 JSONValue = str | int | float | bool | None | list["JSONValue"] | dict[str, "JSONValue"]
93 """Recursive JSON value — the most precise mypy-safe alternative to ``Any``.
94
95 Use this type for JSON payloads whose shape is not statically known.
96
97 **Pydantic restriction:** Do NOT use in Pydantic ``BaseModel`` fields.
98 Use ``PydanticJson`` from ``musehub.muse_contracts.pydantic_types`` instead.
99
100 **Mypy usage:** Use ``isinstance`` guards, ``jint()``, or ``jfloat()``
101 to narrow ``JSONValue`` before dereferencing fields.
102 """
103
104 JSONObject = dict[str, JSONValue]
105 """A JSON object with unknown key set — use when **building** or **returning** JSON.
106
107 Use when the object's keys are not statically known and you need a mutable,
108 owned dict. When passing a JSON dict into a function that only reads it,
109 use ``ReadOnlyJSONObject`` (= ``Mapping[str, JSONValue]``) instead.
110
111 **Pydantic restriction:** Do NOT use in Pydantic ``BaseModel`` fields.
112 See ``JSONValue`` for the full explanation.
113 """
114
115 ReadOnlyJSONObject = Mapping[str, JSONValue]
116 """A read-only JSON object — use at function **parameter** boundaries.
117
118 ``Mapping[str, JSONValue]`` is covariant in its value type, so any
119 ``dict[str, T]`` where ``T ⊆ JSONValue`` is assignable here — including
120 ``dict[str, str]``, ``dict[str, int]``, and ``dict[str, JSONValue]``.
121
122 This is the approved alternative to ``JSONObject`` at function boundaries.
123 It passes the type audit (``boundary_dict`` only fires on ``dict[str,``;
124 ``mapping_any`` only fires on ``Mapping[str, Any]``).
125
126 **Pydantic restriction:** Do NOT use in Pydantic ``BaseModel`` fields.
127 """
128
129 # ═══════════════════════════════════════════════════════════════════════════════
130 # Protocol introspection types
131 #
132 # Named aliases for the multi-dimensional collections returned by the protocol
133 # endpoints. Using explicit names instead of raw dict/list literals makes the
134 # contract between the endpoint, its response model, and callers self-evident.
135 # ═══════════════════════════════════════════════════════════════════════════════
136
137 EventJsonSchema = dict[str, JSONValue]
138 """JSON Schema dict for a single event type, as produced by Pydantic's model_json_schema()."""
139
140 EventSchemaMap = dict[str, EventJsonSchema]
141 """Maps event_type → its JSON Schema. Returned by the protocol /events.json endpoint."""
142
143 StrDict = dict[str, str]
144 """A string-to-string mapping — HTTP headers, auth headers, label maps."""
145
146 IntDict = dict[str, int]
147
148 # ═══════════════════════════════════════════════════════════════════════════════
149 # Named TypedDicts for common anonymous-dict patterns
150 #
151 # Every dict[str, X] that appears in multiple files, or inside a collection,
152 # must be named here so the type audit does not flag list[dict[str, X]].
153 # Rust analogue: every inner HashMap becomes a named struct.
154 # ═══════════════════════════════════════════════════════════════════════════════
155
156 class Breadcrumb(TypedDict):
157 """A single breadcrumb navigation item — label + URL.
158
159 Used by UI route helpers to build breadcrumb_data lists for Jinja2 templates.
160 """
161
162 label: str
163 url: str
164
165 class LabelDef(TypedDict):
166 """A repository label definition — name, hex colour, description.
167
168 Used for the DEFAULT_LABELS seed list and label list endpoints.
169 """
170
171 name: str
172 color: str
173 description: str
174
175 class SitemapEntry(TypedDict, total=False):
176 """One URL entry in a sitemap XML document.
177
178 ``loc`` is always present; the remaining fields are optional per the
179 sitemap protocol specification.
180 """
181
182 loc: str
183 lastmod: str
184 changefreq: str
185 priority: str
186
187 class MemoryFrame(TypedDict):
188 """One tracemalloc frame from ``top_allocations()``."""
189
190 file: str
191 line: int
192 size_kb: float
193 count: int
194
195 class SymbolHistoryEntry(TypedDict):
196 """One operation entry in a symbol's history, as stored in the msgpack index.
197
198 Maps 1-to-1 with the dict appended inside ``update_symbol_index`` and
199 consumed by ``compute_intel``, cross-repo analysis, and blame routes.
200 Rust analogue: ``struct SymbolHistoryEntry { commit_id: String, ... }``
201 """
202
203 commit_id: str
204 committed_at: str
205 author: str
206 op: str
207 content_id: str
208 body_hash: str
209 signature_id: str
210 """A string-to-int mapping — counts, sizes, numeric buckets."""
211
212 # ═══════════════════════════════════════════════════════════════════════════════
213 # Numeric extraction helpers
214 # ═══════════════════════════════════════════════════════════════════════════════
215
216 def jfloat(v: JSONValue, default: float = 0.0) -> float:
217 """Safely extract a ``float`` from a ``JSONValue``.
218
219 Returns *default* when *v* is not numeric::
220
221 beat = jfloat(event.get("beat")) # 0.0 if key absent or non-numeric
222 value = jfloat(event.get("value"), 0.5) # custom default
223 """
224 return float(v) if isinstance(v, (int, float)) else default
225
226 def jint(v: JSONValue, default: int = 0) -> int:
227 """Safely extract an ``int`` from a ``JSONValue``.
228
229 Returns *default* when *v* is not numeric::
230
231 cc = jint(event.get("cc")) # 0 if absent
232 vel = jint(note.get("velocity")) # 0 if absent
233 """
234 return int(v) if isinstance(v, (int, float)) else default
235
236 # ─── List coercion helper ──────────────────────────────────────────────────────
237 #
238 # Python lists are invariant and TypedDicts are not subtypes of
239 # ``dict[str, JSONValue]`` in mypy's type system even when all their value
240 # types are JSONValue-compatible. The principled solution (no ``cast()``, no
241 # per-call ``type: ignore``) is ``@overload`` declarations: enumerate every
242 # domain TypedDict explicitly so call sites are validated precisely. The
243 # single ``type: ignore[arg-type]`` lives only inside the implementation body
244 # — it is the designated coercion boundary for the whole codebase.
245 #
246 # To add a new TypedDict overload: add one ``@overload`` line here.
247
248 @overload
249 def json_list(items: Iterable[str]) -> list[JSONValue]: ...
250
251 @overload
252 def json_list(items: Iterable[dict[str, JSONValue]]) -> list[JSONValue]: ...
253
254 def json_list(items: Iterable[object]) -> list[JSONValue]:
255 """Coerce an iterable of TypedDicts to ``list[JSONValue]``.
256
257 This is the **single designated list-coercion boundary** in the codebase.
258 Each overload is typed precisely for a specific domain dict so call sites
259 are statically verified without needing ``cast()`` or ``type: ignore``::
260
261 params["data"] = json_list(result.items)
262
263 The implementation uses ``Iterable[object]`` so mypy validates call sites
264 against the overloads, not the body. The ``type: ignore[arg-type]`` below
265 is intentional: mypy cannot prove TypedDict ⊆ dict[str, JSONValue] due to
266 dict invariance. This is the ONE place where that coercion is accepted.
267 """
268 result: list[JSONValue] = []
269 for item in items:
270 result.append(item) # type: ignore[arg-type] # coercion boundary: TypedDict → JSONValue, safe at runtime
271 return result
File History 2 commits
sha256:c73281cffffbc70487969720af074e54b34326d59ec6aff0827fa16512a4e512 docs(mwp-2): mark all acceptance criteria met; add closing … Sonnet 4.6 82 days ago
sha256:4dd2a937f66f8e36d9aa59bd1bf3cb880ca1d503fef67300a0cba3b482784081 test(mwp2): Phase 0 RED — reproduction tests for _walk_comm… Sonnet 4.6 82 days ago