Semantic capture contract v1

backend/services/semantic/contracts.py owns the strict semantic schema. It imports neither Torch nor user models. The schema version is independent of graph layout and portable snapshot envelope versions. Live canonical producers and consumers use this schema; legacy captures retain compatibility paths with explicit limitations.

Generate/check the shared extension and browser TypeScript with npm run semantic:generate --prefix extension and npm run semantic:check --prefix extension. Set PYTHON to the project's backend interpreter when needed. The same exporter generates validation contracts without changing their schema. Generated JSON schema checks structural types; Python additionally checks relationships, event ordering and resource budgets.

Resolved execution plans

Browser saved profiles and explicit MCP execution use ResolvedExecutionPlan. Isolated preflight records saved source/configuration/native-file digests, selected interpreter and package facts, exact execution and optional diagnostic recipe, timeout and semantic capture budgets. Factories, checkpoints and input fixtures must have the correct source/native identities. Selected profiles must match the saved profile, target and applicability; explicit recipes record configuration as absent or overridden. Automatic input guesses use CapturePreparation instead.

CapturePreparation is the preceding automatic-input stage. It preserves the exact saved source/environment, configuration status (absent or no_match), bounded original request and limits without claiming a chosen input recipe. The static host resolver rejects dirty source/configuration, malformed configuration and matching model/script profiles that require selection. It shares the plan's source identity, applicability hashing and saved-file/environment rechecks. Browser/MCP automatic requests, including trace-on-save, retain canonical evidence against this preparation. Fake-tensor and non-model legacy paths do not claim this prepared canonical format.

Python owns the canonical applicability fingerprint. It excludes per-attempt IDs, source occurrence IDs and editor document versions, while retaining saved bytes, target, environment, recipe and limits. This is an applicability key, not proof of identical runtime state: actual inputs, weights and RNG state have their separate existing measured identities. Source scope remains explicitly partial.

The host transports the plan in a private 16 MiB-bounded temporary file, with mode 0600 inside a 0700 directory on POSIX and a restricted ACL on Windows. It cleans up on every exit path. The worker verifies saved bytes and environment before importing workspace code; host and worker reconcile again afterward. Stale captures remain available without validating current settings. The MCP setup query pages plan files. Workers enforce independent canonical operation, invocation, value and byte budgets. Canonical runs bypass the legacy operation/reuse stream and folded display graph. A separate diagnostic sidecar retains bounded operation/module/output evidence for numerical and profiling consumers. Library/fake callers and older captures retain their compatibility path; they do not claim canonical semantics.

Preserved observations

• Operations are generic target/schema invocations, never an opcode admission list. Ordered positional and keyword argument trees preserve repeated references, containers, literal types, symbolic expressions and opaque boundaries.

• Integer literals are canonical decimal strings so JavaScript cannot round Python integers. Float tags preserve the float/int distinction and explicitly encode nan, inf, -inf and negative zero (-0). JavaScript JSON retention would otherwise turn numeric negative zero into positive zero. Symbolic expressions are descriptions, never code to evaluate. Unknown objects expose only a type name and omission reason.

• Argument and output paths address the serialized JSON tree. For example, the second tensor in cat([a, b]) is ['args', 0, 'items', 1]. Dictionary entries preserve order and typed keys; a value path includes entries, the entry index, and value. Rendering can give these paths friendly labels without merging them.

• Tensor facts reuse validation's object/storage/version identities and add producer output/effect paths, strides, storage offsets and parameter/buffer references. version is PyTorch's observed counter (nullable for inference tensors), while logicalVersion numbers this object's successive retained observations. A schema-declared write advances a logical version even if dispatch does not advance PyTorch's counter. previousVersion links the preceding value. They contain no tensor payload. Shape equality never establishes aliasing.

• Each operation has explicit effects for writable operands and already observed shared-storage aliases. Effects cite an exact argument path, prior value and new value or opaque limitation. Repeated operands keep distinct paths. A producer's kind selects the operation's output tree or its effects array; effect paths such as [0, 'after'] never invent a returned tensor for a None result. These are conservative schema-declared write boundaries, not measurements of which tensor elements changed. A failed writable kernel remains partial even when the post-failure metadata is available.

• Definitions, runtime module instances, and particular calls have separate IDs. One module can have multiple registered paths and multiple invocations. Entry/exit sequence numbers describe containment, independently of computational dataflow.

• Optional Value.logicalBytes records actual numel() * element_size(), including scalar, empty and strided/view values. It is not allocated or peak memory. Invocation.parameterCount records unique recursive registered parameter objects at call entry, excluding buffers and counting shared registration only once per invocation. Missing legacy facts stay unknown. Optional parameterCountUnavailable distinguishes unsupported metadata and exhaustion of the separate 100,000-item per-capture metadata-read allowance; neither supplies a partial count or changes operation coverage. Browser and MCP summaries use these same facts, with no dtype-to-byte inference or first-output selection.

• Source bindings cite an immutable snapshot and exact file digest plus source/token anchors. Exact, candidate and ambiguous correspondence remain distinct. A binding alone does not grant edit capability. Current captures require saved source; buffer overlays have not been implemented.

• Fake tensors. The recorder observes a FakeTensor (PyTorch's shape-only tensor) exactly like a real tensor: a Value keeps its identity, shape, dtype, device, stride, storage offset, logicalBytes, producer and state references, and the argument trees, operations, invocations and parameter counts are the same. A FakeTensor's .device is a dispatched prim.device call; it is not recorded, as the legacy fold already filtered it. An unsupported FakeTensor is opaque under the type the same program has with real tensors (torch.Tensor or torch.nn.Parameter). Values never exist, so a capture that observed a FakeTensor value or registered parameter/buffer sets valuesUnavailable: "fake_tensor", and the fingerprints of fake tensors are null (stateDigest, and inputDigest for fake inputs). Fingerprinting refuses fake and meta tensors before reading any memory (execution_setup.TensorValuesUnavailable); state_identity reports unavailable: "fake_tensor" or "meta_tensor". Real captures omit the field rather than writing null, so they serialize exactly as before. One known difference: creating nn.Parameter(...) inside a forward dispatches extra aten.detach calls on a FakeTensor. Such dynamic parameters already leave the capture partial.

Coverage and limits

A computational connection must come from a recorded value reference at a specific argument path. Producer outputs must identify that value and occur before its use. No layout edges belong in this contract.

Coverage describes only the executed path. observed, failed, not_observed, capture_limited, static_only, unresolved, and state are separate states. Coverage cannot label a module unobserved when its calls are present. Omitted events require a reason. Unsupported arguments and exhausted budgets prevent a capture from claiming completeness; a failed forward may retain a useful partial prefix. Operations, invocations, values and serialized bytes have independent limits.

Attempt state and retention

RunAttempt records the latest execution outcome separately from capture identity. Its request digest and origin exist even when setup refuses. A preparation can establish source identity while the plan digest remains null; an automatic guess or invalid configuration never receives an invented plan. The worker reports its resolved plan before launch so a timeout can retain that setup without a capture. CaptureState separately names the last complete, last partial and displayed captures. The host ledger uses these records for manual, automatic and MCP runs. A timeout can terminate an attempt without producing a new capture; a projection failure can retain a new capture while displaying an earlier one. Legacy evidence is explicitly partial, even after a successful forward, because its completeness is not established. It cannot fill the complete semantic-capture slot. Captures are immutable JSON copies, bounded to 16 MiB each; only complete/partial/displayed slots remain retained. Oversize captures retain their identity with an explicit retention-limit reason. The latest resolved plan or preparation has a separate 16 MiB bound. These are host-session records, not disk persistence. Planned and prepared automatic captures populate the complete/partial slots after host schema, execution-identity, budget and basic completeness checks. Python remains the owner of full relational validation. Invalid or duplicate canonical payloads cannot replace complete evidence; their legacy result remains explicitly partial with a host-private rejection reason.

The browser's bounded validation.evidence summary carries attempt origin/outcome, source/plan identities, retained slot IDs and displayed-capture attribution. Its optional resolved setup labels contain only profile name, device and mode. Full recipes and interpreter paths remain host-owned. Reconnect rebuilds attribution from the current ledger, and a late capture can update it without replacing a newer refused attempt. Legacy and incomplete canonical captures are labelled incomplete in the result panel. Capture completeness does not certify the legacy canvas projection.

Captured observations are not another execution-input language. Saved-input materialization continues to use services.validation.contracts.Binding; explicit setup/fixture/checkpoint requests continue through the existing execution machinery.

Recorder and identity

services/semantic/identity.py is shared by the capture adapters. It uses weak references and monotonically assigned IDs; producer identity does not require keeping activations alive. Releasing an object clears its lookup entry, while already serialized metadata remains. Input-boundary seeds use those same recorder identities. The canonical recorder and legacy compatibility adapter share this identity implementation.

services/semantic/tensors.py owns the common weak tensor object/storage/version observations used by the validation recorder. That legacy adapter still rejects writes it cannot express using PyTorch counters. The canonical recorder extends the snapshot key with a declared-write epoch and keeps weak per-storage alias indices; no alias inventory keeps activations alive. The semantic argument encoder preserves positional/keyword order, repeated references, typed literals, symbolic descriptions and nested output paths. Its bounded built-in traversal records an opaque subtree for unsupported objects, cycles/depth, large containers and exhausted value budgets; it never evaluates an expression or serializes an object's repr. Workers using resolved plans publish diagnostics.semantic_capture, validated against the canonical contract, alongside the legacy graph. semantic_capture_error is an explicit bounded reason when no canonical payload can be retained. The worker's five-key/version-1 envelope is unchanged. Prepared automatic workers publish the same contract with planDigest=null, a preparationDigest and selected-candidate provenance/index/total attempts. Planned captures have a plan digest and null automatic fields. Exactly one setup identity is required. Only the selected call is retained, and search's reuse of model state is disclosed. Actual input and pre-call state digests remain separate from the preparation's applicability key. Imported source outside the saved snapshot discards canonical attribution with source_mismatch; changes after importing original bytes retain their original capture with a stale current flag. The host rechecks saved setup after worker exit and after asynchronous graph projection. Source/import/configuration/environment changes keep retained captures inspectable but cannot certify the current graph.

The existing dispatcher and module-call hooks feed services/semantic/recorder.py; there is no second dispatch observer. Definitions, registered aliases, actual module instances and calls are separate. Unregistered dynamic modules have an empty paths list, not an invented source path. Parameter/buffer references are observed from the entry inventory. Dynamic state without that identity remains incomplete. Custom operations keep their target and dispatcher schema as atomic operations. Python statements and kernel internals are outside this representation.

Canonical operations and failed invocations share the bounded forward diagnostics. Proven operator shape/dtype constraints survive in the capture; arbitrary exception prose is never parsed for dimensions or copied into diagnostic facts. Exception causality distinguishes a propagated operator failure from a later assertion or explicitly suppressed error. Identical safe facts can share a diagnostic record; operation/invocation identities still distinguish the events. Diagnostic byte/item exhaustion leaves an explicitly incomplete capture and a reserved generic failure link. The recorder does not retain extra exception/traceback objects for this mapping.

The function boundary completes tensor metadata after PyTorch updates autograd and mutation versions. Output/effect ports preserve producer paths and earlier values remain immutable. Non-returned writable values have explicit effect producers. Some custom dispatch kernels do not advance PyTorch's version counter as an ordinary forward would; the capture records that observed counter without inventing a change, and discloses its separate schema-write logical versions in normalization metadata. Potentially partial writes from failed kernels remain incomplete. Escaped mutable storage and unsupported tensor/argument types also prevent completeness. Instrumentation errors do not replace a model's result. No-capture byte errors can occur when even the immutable source/envelope exceeds the requested budget. Conservative reservations bound incremental retention before the final relational/byte validation. Tensor producer paths are charged at their encoded size together with the output or mutation effect that owns them. Event reservations cover only bounded terminal fields and an opaque fallback; unused blanket padding must not discard a graph that fits. A byte stop keeps that reason instead of also claiming the invocation count was exhausted.

Source correspondence

Source cards join exact saved-source captures through SourceCorrespondence: registered paths identify module instances and retained invocations, shared aliases remain explicit, and parameter/buffer values are not described as module calls. Missing calls, capture limits, source-only declarations and unresolved expressions have separate labels. A single shape is summarized only when every retained call has the same direct tensor shape/dtype; multiple outputs and varying calls keep their full trees without a misleading first-tensor summary. Failed calls keep their measured input and no invented output. Joined observations bypass the legacy shape rules and runtime fallback, including an explicit unknown output.

The shared validation loop passes its host attempt ID into plan resolution. That exact ID is used by the plan and canonical capture, including manual, MCP and save-driven attempts. Direct worker callers without a host attempt still receive a fresh ID. The applicability digest continues to exclude per-attempt identity.

Contract tests cover repeated operands, ordered nested outputs, large integers, symbolic values, invocation sharing, aliases, mutation versions, source revisions, partial failures, contradictory coverage, budgets and generated-file parity.

Reversible projection v1

services/semantic/projection.py derives CaptureProjection from a validated capture without importing Torch. Every operation and invocation has its own node; the projection refers back to their captured IDs rather than copying or lowering their meaning again. Ports address exact serialized argument/output/effect paths. Literal, symbolic, opaque and empty-container ports remain inspectable without fake tensor connections. Multiple root input names for the same value remain explicit aliases; Whisper-style distinct input tensors keep separate ports.

Connections exist only for captured tensor argument uses, effect predecessors and returned values. Their source is the actual producer output/effect or an observed input/state/external value. An unused model input is never connected by layout rank. An identity module retains its invocation interfaces without an invented dispatched operation or a claim that returning the same tensor creates a new producer.

Collapse retains every original node, port and connection. Only visibility and display endpoints change. A boundary port cites the exact member port it replaces, including the original destination argument position; internal connections keep their original endpoints while hidden. Expanding regenerates the identical detailed projection. Hidden-operation counts count actual retained records. State/external source nodes are placed at the common scope of their uses so private state can hide inside a collapsed region and shared state remains shared. This placement is a view choice, not a change in parameter or module ownership.

The Python and generated TypeScript projection contracts validate ownership, visibility, hidden counts, value identity, port direction and collapsed endpoints. The producer, live document projection, canvas ports, MCP topology and portable viewer use this contract. Legacy captures retain their original semantic limitations.

The GraphDocument handoff preserves declared ordered input/output port IDs (including empty and unconnected ports) and exact edge handles. Legacy nodes without declarations derive their port IDs from actual connections, falling back to the old x/y defaults only when necessary. Python publishes semantic parent ownership directly, and browser/MCP readers prefer it over legacy canvas copies. The legacy recorder does not gain argument semantics it never captured.

Live captured documents

services/semantic/document.py projects canonical captures directly into the existing GraphDocument envelope, bypassing legacy source/trace edge splicing, rank-based edges and destructive repeat folding. Both single-file and multi-file projection use the same producer. Invalid or foreign captures refuse; they never fall back to legacy wiring. A valid partial/identity capture is usable even when the old visual recorder produced no nodes.

Each rendered record carries CaptureReference (capture/source, kind, semantic ID), ordered DisplayPort records (exact argument paths, value identity, shape, dtype, device), semantic parent ownership and actual descendant operation counts. Occurrences are scoped to their capture, so reused event ordinals in a later run cannot silently become the same selected card. Repeated/shared calls have separate records and readable occurrence labels. Each edge keeps its original ProjectionEdge.

The host retains the full CaptureSnapshot for subsequent inspection. The document has a compact capture summary and source-correspondence table, not a second copy of the environment/setup/argument payload. Its independent 24 MiB decoded budget leaves room for spans and message metadata within the transport's 32 MiB decoded ceiling. Large messages use lossless gzip JSON while the broker's wire limit remains 8 MiB. Overflow refuses with a display-budget message while retained evidence remains available. No records or ports are removed to fit.

The browser keeps original ports/edges intact. Collapsed interfaces use ephemeral boundary:<memberPortId> identities and one display connection per original use. Repeated operands remain distinct. Port pins and both initial/dragged edge anchors share deterministic geometry; card height accounts for the port count. Literal and empty-container ports have no invented tensor edges. Root boundaries display each typed port, including scalar [] and differently typed inputs, rather than a first tensor summary. Expanded calls show their actual retained descendants; no repeat representative replaces another captured occurrence.

Document validation checks references, typed-port consistency and recorded shape summaries without running inference or constructing Torch layers. Its results describe retained evidence, not a new execution or certification of other inputs. Failed calls keep their own error evidence; the propagated terminal failure maps to the canonical operation or root call. Static opening remains unchanged and never imports user code. Source spans on uniquely registered calls permit navigation; they do not grant an edit capability.

Retained capture inspection

The host CaptureInspector reads only the ledger’s validated immutable canonical records. MCP list_captures and inspect_capture expose the independent slots and page the exact JSON tree at each container, preserving argument types/order, repeated references, state aliases/versions, effects and coverage. Container entries carry child counts; leaves keep exact scalars. Pages stay within 64 KiB and never replace a large operation tree with an omitted-record placeholder. Queries require exact capture/source identities and model scope; stale source does not destroy retained evidence, while live focus still checks current revisions. The browser uses this same inspector through private correlated request/reply messages and a temporary Captured details dialog. Ordered argument/output records link to their tensor facts; whole-capture browsing exposes state, coverage and diagnostics.

MCP runs retain the same validated payload under their independent existing 16 MiB-per-run/64 MiB-per-window allowance, 32-run count and 30-minute/session lifetime. Live-slot eviction or model retargeting does not erase an owned retained run. inspect_run uses this reader for its canonical capture, operations and modules sections; legacy runs disclose unavailable semantics. Rejected capture identity cannot be replaced with legacy operation/coverage claims. Run graph snapshots retain their projected capture identity, module instance IDs and every shared registration path. Capture and graph retention failures stay distinct. A second worker callback cannot replace the first captured run evidence, even while projection is pending.

Failure inspection traverses recorded value producers (including nested argument trees and mutation-effect predecessors) from operations sharing the root diagnostic. These are explicit diagnostic candidates: equal safe diagnostic facts can describe multiple caught failures, and do not establish exception causality. No operation is invented for an assertion before dispatcher entry. State/argument-aware comparison shares the canonical host/browser engine across MCP and the portable viewer.

Portable capture v1

PortableCapture versions the detached payload independently from runtime and graph file versions. services/semantic/portable.py validates and copies canonical evidence without importing Torch or workspace code. Inputs, outputs, operations, invocations, values, effects, aliases, module/definition identities, diagnostics and coverage stay exact. Workspace/model/session identities are replaced by portable, editor versions are cleared, and file references (including source anchors) use deterministic aliases. The conventional configuration name remains tensorviz.toml. A digest identifies the original source snapshot without publishing its paths; detached source cannot certify an execution plan from the live workspace.

Literal privacy is lossless-or-refuse: absolute/path-like literals and literals under recognized private/internal argument names refuse export with a fixed message. Nested containers are checked; tensor references and None remain valid under those names because they contain no payload. No operand is removed or replaced. Ordinary model names and literals are computational evidence, so this is not a general secret detector or an assurance that arbitrary user labels contain no personal information. Source bodies, tensor payloads and environment paths are not capture fields.

Shared fixtures and executable corpus checks cover the producer and contract. Version 4 files, live export, offline validation/inspection and canonical comparison consume this payload. Uncertain correspondence remains explicit.

packages/contracts/src/captureReader.ts owns the nonexecuting structural reader shared by host/MCP and the browser's checked generated copy. The offline client accepts an already-validated capture and provides the same bounded pages to Captured details; it has no network or source-edit capability. File import and the viewer use it. UTF-8 byte limits, exact argument reconstruction, foreign-identity refusal and prototype-safe own-property traversal apply identically in both environments.

The version 4 graph-file importer validates the detached payload with precompiled Python-generated JSON-schema validators and explicit offline relationships/privacy checks. Shared Python/browser acceptance cases cover producer/alias/version ordering, ownership, boundaries, coverage, limits and non-returned mutation. Its document gate checks every rendered event, exact argument/effect/result use, producer and value fact against the payload. File layout can reorder display records but cannot omit events or rewire them. The offline viewer uses the shared reader and discloses partial and legacy evidence. The integrated canonical comparison preserves those limitations.

Live export uses the same retained record ownership checks as inspection. The host rechecks displayed target/capture, eviction and connection epoch after Python detaches the payload. Private request/reply transport never broadcasts the capture or retains it in reconnect snapshots. One export can prepare at a time; cancellation aborts the request. Authenticated preparation bounds raw input to 16 MiB and detached output to 6 MiB, with exact computational equality checks in the host and full offline validation in the browser. The browser rechecks current projection and capture before download; layout changes during preparation are preserved. No source is read or model executed.

Verified authored source locations

Canonical definitions, calls and operations can cite exact saved-source bindings. The existing code verifier supplies bounded, digest-checked source before model import; PEP 657 instruction ranges are joined to one AST expression in the same verified code scope. Definition bindings identify a known forward implementation. Coordinates are one-based lines and zero-based UTF-16 columns. No source bodies or live frames are retained in the capture. Unknown ranges/external helpers remain unmapped.

A binding is location evidence, not an edit capability: one expression may produce several operations or repeated calls. Binding metadata has its own bounded share of the overall capture budget. Supported constructor, argument and direct scalar configuration bindings have explicit edit capabilities and shared scope; ambiguous or unsupported recipes refuse.

Stored dictionary outputs

Dictionary subclasses, including Transformers ModelOutput objects, expose their stored entries as dictionary argument trees. Built-in dict/OrderedDict methods read the storage directly, retaining OrderedDict order without calling overridden items, length, iteration, lookup or repr methods. Custom object attributes and computed mapping behavior are not reconstructed; the capture's normalizations record this distinction. Existing key, item, depth and tensor budgets still apply.

Tensor outputs now retain their identities and measured shapes instead of becoming opaque solely because the container is a dictionary subclass. Unsupported nested objects remain explicit: for example, T5's EncoderDecoderCache still yields partial coverage while its encoder and decoder tensor outputs remain visible.