Notebook Documents and Runtime

Status: Implemented in the first rollout. .notebook.md notes are parsed into notebook documents, notebook cells execute through a per-note session runtime, non-DuckDB notebooks use a dedicated worker transport, DuckDB-backed notebooks currently fall back to the in-process runtime path in the browser, scoped notebook execution is available through notebook commands, and notebook results persist as generated state in the app database.

This page describes the markdown-native notebook document format and the runtime model used to execute notebook cells in the browser and PWA host.

The implementation follows the component boundary described in Notebook Plugin and the architectural decision in ADR-002: Notebook Plugin and Markdown Cell Model.

Implementation snapshot

The current rollout includes the following runtime pieces:

• notebook file identity from the .notebook.md path suffix plus optional notebook frontmatter normalization and :::cell{...} parsing inside plugin-notebook
• explicit note-level, current-cell, stale-cell, and inline per-cell notebook execution entry points for notebook-owned markdown notes
• notebook document loading keyed off file identity, with optional frontmatter parsing for runtime config when metadata cache entries are not warm yet
• static dependency analysis for ts/js, sql, dynamic md, and html cells
• per-note session graphs with stale propagation and duplicate/cycle detection
• a notebook execution bridge that runs JavaScript, TypeScript, SQL, and dynamic markdown cells behind notebook-scoped runtime instances, using a dedicated worker transport when a notebook does not depend on DuckDB or live DOM APIs and otherwise falling back to a notebook-local in-process runtime path; the bridge keeps DuckDB-Wasm and the TypeScript compiler behind secondary lazy imports so cold worker/client startup stays smaller for simple notebooks
• static ESM import support in JavaScript and TypeScript notebook cells for configured notebook runtime packages, for bare package imports whose versions are known from the host workspace package manifest, and for vault-scoped relative JavaScript modules stored next to the notebook note; absolute filesystem-like specifiers remain rejected
• DuckDB-Wasm app tables exposed as lapis.notes, lapis.links, and lapis.search_documents
• notebook-owned renderers for table, chart, markdown, HTML, image, and error outputs
• host-neutral execution providers for DuckDB runtimes, notebook vault file reads, and JavaScript-like cell compilation; Electron injects a native DuckDB sidecar provider when its preload capability registry advertises notebook support and registers DuckDB vault-file aliases from validated native vault paths, while browser/PWA execution keeps the existing DuckDB-Wasm service, vault adapter reads, byte-buffer file registration, and lazy TypeScript compiler by default
• plain top-level const|let|var name = view(expression) input cells powered by the notebook-input runtime, with supported controls rendered through @lapis-notes/ui primitives and live controls mounted through notebook-owned structured input outputs and DOM mounts while the remaining upstream compatibility surface is still being retired
• plain top-level const|let|var name = view(expression) input cells powered by the notebook-input runtime, with supported controls rendered through shared notebook-owned input surfaces, including direct Inputs.table(...) support for attachment-backed row selection with sortable headers, and live controls mounted through notebook-owned structured input outputs and DOM mounts while the remaining upstream compatibility surface is still being retired
• htl html and svg helpers for trusted live DOM outputs from TypeScript and JavaScript cells
• markdown directive rendering that mounts notebook cells through the shared markdown component tree in both reading mode and live preview
• live-preview notebook cell widgets suppress direct raw-source activation on in-widget clicks and keep source selection behind the explicit code/edit affordance used by shared markdown block widgets
• inline notebook cell controls allow rerunning cells even when outputs are already present, switch the per-cell play control to an inline stop control only on the actively running cell while queued cells remain visibly scheduled, and restore the prior checkpoint when an inline cancellation interrupts a running cell execution
• the right-side notebook session view derives declared and used variables from the notebook graph, lets those declared-by and used-by cell ids reveal the corresponding notebook cell in the owning note, queries the in-process DuckDB runtime for live schemas and row counts when the active notebook uses DuckDB-backed execution, renders each DuckDB data source as a row-style expandable entry with inline source and column type icons plus row and column counts, and lazily requests small in-panel table previews when the user expands a DuckDB table
• the left-side notebook dependencies view reuses the same notebook graph plus per-cell runtime state to build a view-specific dependency snapshot, renders a Marimo-inspired minimap in notebook order with direct and transitive dependency glyphs, and renders a fit-to-viewport graph tab with vertical or horizontal tree layouts, syntax-highlighted code cards, selectable dependency edges, and popup cell or edge details
• a separate right-side notebook packages view follows the active notebook, groups runtime package descriptors into direct imports, indirect imports, and available imports, resolves transitive package metadata from pinned direct imports when the npm registry provides manifest metadata, lets settings and notebook pane chrome open the package manager directly, and supports adding or removing validated package descriptors without altering the existing dependency graph panel
• plugin settings persisted through the plugin data store now control notebook-level preferences such as default dependency tab, dependency-graph layout, dependency-graph wheel zoom sensitivity, whether notebook sources or dependencies panels auto-open on notebook file open, whether notebook document edits should automatically rerun the current stale-cell set after a short debounce, and the validated runtime dependency registry that the Packages view manages for lazy execution-time loading
• app-database persistence for notebook outputs keyed by note path with file-level mtime and per-cell source fingerprints
• app-database persistence for serializable input values and structured input outputs, while live DOM outputs are treated as ephemeral runtime state and are recreated by rerunning the owning cell
• per-cell runtime metadata for explicit execution state, success or error status, and last-run duration so the notebook UI can show timing and status chrome without storing rendered output in note bytes
• notebook-owned status-bar chrome for the active notebook leaf that reports Safe Mode execution disablement, current runtime transport selection, in-process fallback reasons when worker execution is unavailable, and whether the visible outputs were restored from generated state or recomputed during the current session, with notebook-owned run, rerun-stale, and clear-output actions available from the same surface
• note-level notebook runs publish queued and running session state immediately from the command path, before lazy runtime boot or DuckDB setup completes, so per-cell status bars and inline stop controls can reflect scheduled and running work during cold starts, while notebook stop actions only tear down the runtime that belongs to the targeted note
• workspace Playwright coverage for cursor-scoped execution and notebook-relative CSV loading in the legacy browser vault

The main remaining follow-on work is language and syntax scope rather than basic runtime correctness. NotebookWorkerClient still uses a dedicated worker for non-DuckDB/non-DOM notebooks and host callbacks to fetch app-table snapshots and vault file bytes from the main thread, but DuckDB-backed, live DOM, and host-only provider notebooks execute in process because that path is more reliable in the browser/PWA host today and carries Electron’s nonserializable native DuckDB provider. The shipped session inspector depends on that in-process DuckDB runtime for live schema discovery. The notebook view now exposes that runtime choice directly so users can tell whether a note is on the worker path or on an intentional in-process fallback.

Notebook cell presentation now splits the source preview and the rendered output into separate surfaces. The code surface keeps the rounded notebook card treatment and hosts execution status, duration, and run actions. The output surface renders directly into the document flow without the code-card background, but keeps its own left action rail for collapse, selection highlight, and output-specific actions such as clearing cell outputs. Selection is tracked at the notebook-cell level so only one cell is selected at a time within a note, and the source and output rails for that selected cell switch together from the normal app border color to the interactive accent.

At the notebook level, the active notebook leaf contributes notebook-owned status-bar items so note-wide execution state remains visible without adding sticky notebook chrome to the document body. Those items report when Safe Mode has disabled notebook execution, label worker-backed runs explicitly, spell out the current in-process fallback reason for notebooks that require DuckDB, live DOM helpers, or host-only providers, and distinguish outputs restored from generated state after reload from outputs recomputed in the current session after an explicit run.

Goals

The document model is designed to satisfy the following constraints:

• notebook notes remain markdown files with a dedicated .notebook.md suffix
• multiple executable cells can be mixed with ordinary prose
• the runtime computes a DAG from cell dependencies, marks changed cells plus their rerun closure stale in lazy mode, and reruns the required dependency closure for scoped or stale executions while reusing already-satisfied upstream dependency values so targeted reruns do not rebuild unchanged input cells
• vault data can be loaded in the browser without a server
• app-derived data can be queried from cells
• rich outputs such as tables and charts can be rendered in notebook-aware presentations
• generated outputs stay out of canonical note bytes

Notebook Identity

Notebook behavior is enabled by file identity.

Notebook notes use the .notebook.md suffix. That path-based identity decides which view owns the file and whether notebook parsing/runtime behavior applies.

Notebook notes may also carry optional frontmatter for runtime settings.

---
notebook:
  version: 1
  autorun: false
  mode: lazy
  mounts:
    notes: app://notes
    tags: app://tags
    links: app://links
---

Frontmatter keys:

• version - notebook document model version
• autorun - optional boolean override for stale-cell auto-runs on note edits; when omitted the notebook falls back to the global notebook plugin setting
• mode - reserved runtime mode metadata; the current notebook UI does not auto-run on load and only executes cells from explicit user actions
• mounts - optional aliases for app-backed tables or vault-backed data sources

Notes without a notebook: key still remain notebooks when their file path ends with .notebook.md; they simply fall back to default notebook settings.

Canonical Cell Syntax

The canonical cell unit is a markdown container directive named cell.

This fits the current parser direction because the markdown stack already supports container directives in both CodeMirror parsing and preview rendering.

When the directive omits lang, notebook parsing infers the cell language from a fenced code body’s info string when present. If there is no fenced info string, or it does not map to one of the supported notebook cell languages, the cell still falls back to ts.

Code Cell Example

:::cell{#load-data lang="ts"}

```ts
const activeTag = "project/alpha";
await lapis.duckdb.registerVaultFile("todos_csv", "data/todos.csv");
```

:::

SQL Cell Example

:::cell{#filter-todos lang="sql"}

```sql
SELECT *
FROM read_csv_auto('todos_csv')
WHERE tag = {{activeTag}}
```

:::

Dynamic Markdown Cell Example

:::cell{#summary lang="md"}

## Summary

Current tag: `${activeTag}`
:::

Markdown cells support ${...} expressions evaluated against resolved notebook state. The legacy {{name}} placeholder remains supported as a markdown-native convenience and for SQL scalar interpolation.

HTML Source Cell Example

:::cell{#summary-card lang="html"}

<article><strong>${region}</strong> uses gain <code>${gain}</code>.</article>

:::

HTML source cells render through the notebook HTML output renderer. Static HTML in the cell source is trusted notebook source, while values interpolated through ${...} are escaped before rendering so strings are treated as text by default in both element text and attribute contexts. Use JavaScript-like cells with htl-backed html/svg helpers when a notebook intentionally needs live DOM nodes or richer trusted DOM composition.

Notebook Input Example

Notebook TypeScript and JavaScript cells support a constrained top-level view(...) form inspired by Observable notebooks.

:::cell{#controls lang="ts"}

```ts
const gain = view(Inputs.range([0, 10], { value: 5, step: 1, label: "Gain" }));
const region = view(Inputs.text({ value: "North", label: "Region" }));
```

:::

:::cell{#summary lang="md"}
Selected gain: **{{gain}}**

Selected region: **{{region}}**
:::

The current implementation supports only the explicit top-level shape const|let|var name = view(expression). The expression must evaluate to a DOM-backed input node whose .value property represents the current value and that emits input or change events when it changes. Supported control rendering now lives under @lapis-notes/notebook-input and uses shared @lapis-notes/ui primitives. Full Observable JavaScript syntax remains a follow-on task.

When a user changes an input, the notebook session updates the exported scalar value, finds downstream dependents in the notebook graph, and automatically reruns those dependent cells without rerunning the input-producing cell. Input-triggered execution wakes only affected cell renderers, preserving existing outputs and their live DOM registrations while queued or running states are shown, so unrelated cells do not reload or remount their DOM. Rerunning an input-producing cell clears its previous runtime definitions before evaluation so transformed view(...) declarations can be declared again in the same session.

htl DOM Output Example

TypeScript and JavaScript cells also receive htl-backed html and svg tagged template helpers.

:::cell{#status-card lang="ts"}

```ts
return lapis.html`<div><strong>${region}</strong> uses gain <code>${gain}</code>.</div>`;
```

:::

htl outputs are trusted live DOM outputs, not sanitized strings. htl still escapes interpolated text according to its library semantics, but event handlers and DOM nodes run with the same trust level as the notebook cell code. The notebook output protocol stores only an ephemeral DOM output id; the live node itself is not persisted.

Builtin Runtime Surface

The current notebook cell runtime no longer exposes a public nb namespace. Lapis-provided notebook helpers live canonically under lapis.*:

• lapis.display(value)
• lapis.duckdb.query(sql)
• lapis.duckdb.registerVaultFile(alias, path)
• lapis.vault.readText(path)
• lapis.vault.readBinary(path)
• lapis.Inputs
• lapis.html and lapis.svg
• lapis.md
• lapis.tex and lapis.tex.block
• lapis.invalidation and lapis.now
• lapis.aapl, lapis.alphabet, lapis.cars, lapis.citywages, lapis.diamonds, lapis.flare, lapis.industries, lapis.miserables, lapis.olympians, lapis.penguins, lapis.pizza, and lapis.weather
• lapis.DOM
• lapis.Generators and lapis.Promises
• lapis.table, lapis.chart, lapis.markdown, lapis.text, lapis.image, and lapis.error

Documented bare convenience globals may mirror selected Lapis builtins when that keeps notebook cells terse, for example display, Inputs, html, svg, md, tex, d3, Plot, DOM, Generators, Promises, now, invalidation, and the upstream sample dataset globals aapl, alphabet, cars, citywages, diamonds, flare, industries, miserables, olympians, penguins, pizza, and weather. The current DOM-backed runtime exposes display, Inputs, html, and svg as bare convenience globals while also attaching display, Inputs, html, and svg as lapis.display, lapis.Inputs, lapis.html, and lapis.svg, with lapis.view(name, input) remaining the runtime-owned binding hook behind transformed view(...) declarations. The supported Inputs.* controls are owned inside @lapis-notes/notebook-input, including Inputs.table(...) for serializable row selection with sortable headers, and mounted from shared notebook input surfaces so notebook controls match app-level styling, with reconstructable metadata persisted under source.library: "@lapis-notes/notebook-input". JavaScript-like notebook execution also injects lapis.md, lapis.tex, lapis.DOM, lapis.Generators, lapis.Promises, lapis.now, lapis.invalidation, the bundled sample dataset globals, and the documented bare md, tex, DOM, Generators, Promises, now, invalidation, and sample dataset globals by default from the shared stdlib bootstrap. invalidation resolves when the current cell is rerun or its outputs are explicitly cleared, while now is an execution-time snapshot rather than an ambient auto-refreshing observer. User-installed dependencies are not Lapis builtins and should not be copied under lapis.*; they remain available through configured globals when a notebook runtime package descriptor declares one, or through direct static import syntax for configured packages and host-installed bare package imports.

Package Imports

Notebook JavaScript and TypeScript cells now support top-level static ESM import declarations. The runtime rewrites those declarations before evaluation so cells can keep running inside the notebook execution sandbox instead of relying on browser-native module loading from the raw cell source.

The current import rules are:

• bare imports that match a configured notebook runtime package descriptor resolve through that descriptor’s pinned version and resolver settings
• bare imports that match a package listed in the host workspace package manifest resolve through the package name plus the installed version range known to the workspace app
• absolute URL imports still execute as direct runtime module imports
• vault-scoped relative JavaScript module specifiers such as ./helpers.js and ../shared/math.mjs resolve from the importing notebook or notebook-local module path, reject vault-root escapes, and currently support only .js and .mjs sources
• absolute filesystem-like specifiers remain rejected so notebook modules cannot bypass vault-relative resolution

When a cell uses static import syntax, imported binding names take precedence over configured runtime globals with the same identifier so the legacy global loader does not eagerly fetch the wrong package for that cell.

This keeps the legacy configured-global workflow available while letting notebook cells use ordinary import syntax for locally installed packages such as lodash or configured remote runtime packages.

The internal notebook-input transform compiles constrained const|let|var name = view(expression) declarations to lapis.view(name, expression) so generated notebook cell code stays on the canonical Lapis namespace.

System Guide Acceptance Fixture

The repo now keeps a markdown-native parity anchor at e2e-vault/plugin-notebook/System Guide.notebook.md with supporting attachments under e2e-vault/plugin-notebook/assets/.

That fixture intentionally covers the current Lapis-owned subset of Observable’s System Guide rather than the full Observable surface:

• async JavaScript cells that resolve notebook state before downstream cells render
• explicit display(...) calls from JavaScript program cells, including multiple displays in one cell
• plain view(...) inputs and downstream reactivity into markdown, HTML source, and DOM outputs
• direct Inputs.table(...) row selection with sortable headers backed by attachment-loaded CSV data and downstream reactivity into markdown outputs
• ${...} interpolation in markdown and HTML source cells, with escaped HTML interpolation values
• md and tex tagged template helpers plus now execution snapshots that produce notebook markdown outputs
• trusted html output mounting in JavaScript cells
• vault-backed lapis.FileAttachment() JSON and CSV reads rendered through notebook-aware markdown and table outputs
• bundled Observable sample dataset access from JavaScript cells, including both row-array datasets and the graph-shaped miserables dataset
• vault-scoped relative JavaScript module imports used by notebook cells to share small helper functions between neighboring files

Workspace Playwright acceptance seeds those exact fixture files into the browser test vault and verifies the rendered and interactive behavior end to end. This note is the regression target for the current Observable parity slice and should stay readable enough to explain what the runtime is expected to support.

Compilation And Cancellation

Notebook JavaScript-like cells currently compile through the notebook runtime that owns the cell execution. Worker-eligible notebooks load the TypeScript compiler inside the notebook worker when static imports or TypeScript syntax require it. Browser, PWA, Electron DuckDB-backed, live-DOM-backed, and host-provider-backed notebooks use the in-process execution path and lazily load the same compiler only when a JavaScript-like cell actually needs rewriting or transpilation.

Native TypeScript compilation was evaluated after the Electron DuckDB sidecar landed. The current decision is to keep the existing lazy compiler path and the host-neutral compileProvider seam, without adding an Electron compiler sidecar. Moving only transpilation to Electron main or to a new compiler child would not move notebook evaluation itself: JavaScript-like cells still execute in the renderer/in-process runtime when they need DOM access or nonserializable native providers. It would also not solve cancellation for already-running synchronous user code, because TypeScript transpilation is only one pre-execution step and the current cancellation model restores the notebook checkpoint and terminates the owned worker or DuckDB runtime rather than preempting arbitrary in-process JavaScript.

The compileProvider contract remains a host seam, but native compilation requires a separate proposal backed by renderer memory, cold TypeScript import, and long-compile timing data. Such a proposal also needs its own timeout boundary for compiler requests and a distinct design for cooperative or isolated cancellation of JavaScript-like cell evaluation. In the current architecture, Electron’s native notebook work stays focused on DuckDB query and vault-file execution, where the sidecar removes large data movement and native query work from the renderer.

Mixed Notebook Note Example

# Weekly Review

Ordinary markdown outside `cell` directives stays ordinary prose and is not
itself an executable cell.

:::cell{#load-data lang="ts"}

```ts
const selectedFolder = "Projects";
```

:::

:::cell{#matching-notes lang="sql"}

```sql
SELECT path, name
FROM notes
WHERE path LIKE {{selectedFolder}} || '/%'
ORDER BY name
```

:::

:::cell{#report lang="md"}

## Matching Notes

Folder: `{{selectedFolder}}`
:::

Rich Output Model

Executable cells need a typed output protocol in addition to executable source.

The markdown cell syntax only defines where cells live in the note. marimo-style rich rendering comes from the runtime’s cell result model and from notebook-owned renderers that mount those results into the UI.

Output kinds

The implemented output protocol supports:

• text - plain text
• markdown - rendered markdown fragments
• html - sanitized HTML fragments when explicitly allowed
• input - structured reconstructable input controls for tracked Observable-style notebook inputs
• dom - trusted live DOM nodes produced by Observable inputs, htl, or explicit DOM output helpers; this output kind is ephemeral and is not persisted
• table - structured tabular data for interactive rendering
• chart - chart specs rendered by approved chart libraries
• image - inline images or binary image artifacts
• error - structured runtime and graph errors

An illustrative TypeScript shape:

type NotebookOutput =
  | { kind: "text"; text: string }
  | { kind: "markdown"; markdown: string }
  | { kind: "html"; html: string }
  | {
      kind: "input";
      cellId: string;
      variableName: string;
      inputType: string;
      value: unknown;
      label?: string;
      config?: Record<string, unknown>;
      reconstructable: boolean;
      source?: {
        library?: string;
        factory: string;
        version?: string;
      };
    }
  | {
      kind: "dom";
      outputId: string;
      cellId: string;
      label?: string;
      role: "input" | "html" | "svg" | "dom";
      persist?: false;
    }
  | {
      kind: "table";
      columns: string[];
      rows: unknown[][];
      columnMeta?: {
        key: string;
        label?: string;
        type?:
          | "string"
          | "number"
          | "date"
          | "boolean"
          | "json"
          | "null"
          | "unknown";
        displayType?: string;
        nullable?: boolean;
      }[];
      schema?: {
        rowCount?: number;
        truncated?: boolean;
        queryTimeMs?: number;
        sourceSql?: string;
      };
    }
  | {
      kind: "chart";
      library: "vega-lite" | "plot";
      spec: unknown;
    }
  | { kind: "image"; mime: string; data: string }
  | { kind: "error"; message: string; stack?: string };

Renderer model

Notebook-aware presentations should not treat rich outputs as arbitrary DOM mutations owned by the executing cell.

Instead, the notebook plugin does the following:

• define the output protocol
• keep a renderer registry keyed by output kind and optional library id
• mount notebook-owned Svelte components for each supported output kind
• keep live DOM nodes in a runtime registry and mount them through notebook-owned DOM wrappers

This keeps output rendering consistent across notebook mode, live preview, and reading mode.

First-rollout output priorities

The first rollout should support rich outputs, but in a narrow, explicit way.

Recommended order:

1. interactive table outputs
2. markdown and HTML outputs
3. chart-spec outputs
4. richer interactive widgets such as filters or dataframe explorers

Tables are the most natural first rich output because DuckDB-Wasm and SQL cells already produce tabular results. Charts should be spec-driven rather than imperative so the notebook plugin retains rendering ownership.

The current notebook table renderer now uses a notebook-owned dataframe viewer rather than a plain HTML table: column titles and type metadata stay in the header control row, compact summary blocks render beneath each column header, client-side search, sort, filter, pagination, and column visibility are composed from shared shadcn-style UI primitives, a right-side sheet exposes richer per-column summary detail, and rendered pages switch to virtualized row scrolling when the current page exceeds a couple hundred rows. SQL-backed table outputs now also attach additive schema metadata automatically at execution time, including inferred column types plus rowCount, queryTimeMs, and the executed SQL string when available.

Example table cell

:::cell{#sales-table lang="ts"}

```ts
const result = await lapis.duckdb.query(`
  SELECT region, SUM(amount) AS revenue
  FROM read_csv_auto('sales.csv')
  GROUP BY region
  ORDER BY revenue DESC
`);

lapis.table(result);
```

:::

Example chart cell

:::cell{#sales-chart lang="ts"}

```ts
const result = await lapis.duckdb.query(`
  SELECT region, SUM(amount) AS revenue
  FROM read_csv_auto('sales.csv')
  GROUP BY region
  ORDER BY revenue DESC
`);

lapis.chart("vega-lite", {
  data: { values: result.toArray() },
  mark: "bar",
  encoding: {
    x: { field: "region", type: "nominal" },
    y: { field: "revenue", type: "quantitative" },
  },
});
```

:::

Example dynamic markdown cell

:::cell{#sales-summary lang="md"}

## Revenue Summary

Top region: **{{topRegion}}**

Rows in the table: **{{rowCount}}**
:::

Why Container Directives

The directive-based shape is proposed because it aligns with the existing markdown implementation better than ad hoc fence parsing.

Benefits:

• keeps ordinary fenced code blocks available for documentation snippets
• gives cells explicit identity and metadata through directive attributes
• maps naturally onto block widgets in live preview and reading mode
• reuses parser support already present for markdown directives

Proposed Cell Attributes

The first draft attribute set is:

• #id - stable cell id used for outputs, persistence, and graph diagnostics
• lang - ts, js, sql, or md; py/python cells are not recognized by the current core runtime
• run - optional override: autorun, lazy, or manual
• hide - optional flag to hide source in notebook presentation mode
• folded - optional initial folded state hint

If an authored cell omits an id, the editor should generate one and persist it back into the directive on structural save.

Python Cells Decision

The current decision is to keep Pyodide-backed Python cells out of the core notebook runtime. The supported core language set remains JavaScript, TypeScript, SQL, and dynamic markdown.

Python cells would require a second interpreter lifecycle rather than a small language-mode addition. A viable design would need an optional package or runtime provider boundary, explicit Pyodide asset loading and package policy, cancellation and timeout semantics, value conversion between Python and the JavaScript notebook session graph, table/dataframe exchange with DuckDB outputs, generated-state rules, and browser/Electron parity. Those requirements overlap enough runtime ownership boundaries that they should not be added as an implicit lang="py" branch in the existing JavaScript-like execution path.

Future Python support can be reconsidered as a separate proposal when there is a user-facing workflow that justifies the bundle/runtime cost and when the proposal includes host-neutral tests for parsing, dependency graph behavior, execution isolation, cancellation, output rendering, generated-state hydration, and Electron/browser fallback behavior. Until then, py and python cell languages remain unsupported and continue to normalize to the current TypeScript fallback during parsing.

Dependency Graph Rules

JavaScript and TypeScript cells

The runtime performs static analysis over top-level bindings.

• definitions are top-level exported names, functions, classes, and imports introduced by the cell
• references are top-level names read by the cell but not defined inside it
• _local names are considered cell-local and are not exported to other cells

SQL cells

SQL cells should be notebook-graph aware without requiring a server-side parser.

The proposed first-pass rule is:

• table names introduced by notebook mounts or prior cells are available directly in SQL
• scalar or structured values from other cells are referenced through {{name}} placeholders
• each placeholder is treated as a dependency edge in the notebook graph

This keeps SQL dependency extraction deterministic and browser-safe.

Markdown cells

Dynamic markdown cells do not define names. They participate in the graph only by referencing {{name}} placeholders.

Shared runtime rules

The notebook runtime should adopt marimo-like consistency rules:

• deleting a cell removes its definitions from notebook state
• descendants of a changed cell rerun automatically in autorun mode
• changed cells and the dependency closure required to recompute their stale dependents are marked stale in lazy mode
• mutations are not tracked across cells; new values should be created rather than mutating shared objects elsewhere
• multiply defined exported names are a notebook error
• cycles are notebook errors

Data Access Model

DuckDB-Wasm

DuckDB-Wasm is the proposed query and relational execution engine for browser and PWA notebook sessions.

Why:

• worker-friendly browser runtime
• CSV, Parquet, JSON, and Arrow support
• direct registration of in-memory file buffers from vault reads
• good fit for analytics and notebook-style tabular work

Vault-backed data

Notebook sessions should expose helpers for registering vault files into DuckDB.

Implemented helper shapes:

• lapis.vault.readText(path)
• lapis.vault.readBinary(path)
• lapis.FileAttachment(path).arrayBuffer()
• lapis.FileAttachment(path).text()
• lapis.FileAttachment(path).json()
• lapis.FileAttachment(path).blob(type?)
• lapis.duckdb.registerVaultFile(alias, path)
• lapis.duckdb.query(sql)

The existing vault adapter layer already supports binary reads, so CSV, Parquet, JSON, and SQLite files can be loaded from vault bytes in the browser. Electron desktop sessions keep the same lapis.duckdb.registerVaultFile(alias, path) notebook syntax but register CSV, Parquet, and JSON aliases by sending the resolved vault-relative path and native vault root to Electron main, where the path is validated under the selected vault before the DuckDB sidecar links or copies the file into its temporary working directory.

The current implementation builds the host-neutral lapis.vault and lapis.FileAttachment behavior from @lapis-notes/notebook-stdlib and supplies only the host file-reader callback from the notebook product runtime. DuckDB vault-file registration continues to use @lapis-notes/notebook-duckdb, which shares the same notebook-relative path-normalization rules through the stdlib helper and prefers a native runtime registrar when one exists before falling back to host-read file buffers.

Visualization Helpers

Notebook JavaScript and TypeScript cells expose bundled visualization helpers as Lapis-owned builtins:

• lapis.d3 and bare d3
• lapis.Plot and bare Plot
• lapis.DOM and bare DOM
• lapis.Generators and bare Generators
• lapis.Promises and bare Promises

The current implementation assembles these bundled helpers through the shared @lapis-notes/notebook-stdlib runtime bootstrap when JavaScript-like cells execute, so lapis.d3, lapis.Plot, lapis.DOM, lapis.Generators, lapis.Promises, and the documented bare d3/Plot/DOM/Generators/Promises globals are present by default within that execution scope without per-cell source-triggered loading. Plot references still count as live DOM-backed notebook execution because Plot outputs are DOM/SVG-oriented. Helper-only DOM usages such as DOM.uid remain worker-eligible, while live browser DOM usages such as Inputs, html, svg, viewof, Plot, and browser-owned DOM helpers still force the in-process path.

Dynamic Runtime Dependencies

The dependency registry lets users describe additional JavaScript packages from plugin settings before runtime loading is implemented. A dependency entry includes the package name, pinned version, optional resolver metadata, and optional global name. If the user does not choose a global name, settings validation derives a valid JavaScript identifier from the package name before saving.

Examples:

These globals participate in dependency analysis as reserved external values. The notebook graph should not treat them as cell-local unresolved refs, and notebook authoring should block collisions with lapis, documented bare builtins, and other configured dependency globals.

The current settings UI no longer owns raw JSON editing for runtime dependencies. Instead it links to the dedicated Packages side view, which persists only registries that pass the shared @lapis-notes/notebook-stdlib validation contract and shows the configured descriptors as direct notebook imports, indirect import metadata when available, and all available configured imports. The package-name field now queries the npm registry for autocomplete suggestions, and install actions probe the pinned package metadata before saving while reporting inline progress back to the Packages panel. Notebook execution passes the validated registry into runtime options and JavaScript-like cells lazily resolve only configured dependencies whose globals appear in the executed source. Remote ESM descriptors resolve through dynamic import() from the configured baseUrl, defaulting to https://esm.sh, and expose the module value only as the selected or derived bare global. Serializable remote ESM descriptors travel through the worker protocol for worker-backed notebooks; non-serializable test loaders and future host-specific resolver handles stay on the main-thread path until a host/cached resolver policy exists. Desktop hosts may later add a cached or offline resolver behind the same registry contract.

DuckDB-Wasm Asset Split

DuckDB runtime code should move toward a package boundary where the service accepts DuckDB bundle URLs from a dedicated asset resolver rather than importing WASM and worker files directly from the product plugin package. The intended split is:

• @lapis-notes/notebook-duckdb owns runtime contracts, query normalization, table registration helpers, and inspector-facing catalog helpers.
• @lapis-notes/notebook-duckdb-assets owns DuckDB-Wasm and worker asset URLs, static-copy rules, and bundle selection inputs.
• @lapis-notes/notebook owns the concrete browser/PWA DuckDB-Wasm service lifecycle, query execution, app snapshot collection, and vault reads.

This keeps the product plugin startup-thin and makes DuckDB asset loading explicit for browser, PWA, and future desktop hosts.

Observable Runtime Evaluation

Observable’s standard library is a useful API reference for DOM helpers, FileAttachment, generator helpers, promises, and builtin naming. The @observablehq/runtime compatibility spike concluded that Lapis should keep its own explicit notebook runtime as the core execution engine and borrow Observable-compatible APIs selectively.

The current decision is not to implement full Observable JavaScript compatibility in the core notebook runtime. The supported compatibility boundary is constrained top-level viewof name = expression input declarations, Observable Inputs controls, htl-backed html/svg, vault-backed lapis.FileAttachment, and selected standard-library helper names that can be hosted behind Lapis runtime contracts.

Full Observable semantics stay out of scope for core execution because they conflict with these Lapis-specific requirements:

• explicit note-level, stale-cell, current-cell, and inline execution commands
• generated-state persistence keyed by markdown cell ids and source fingerprints
• TypeScript cells
• SQL and dynamic markdown cells
• worker/main execution selection for DOM, DuckDB, and non-DOM notebooks
• notebook-owned typed output renderers rather than Observable inspectors as the primary output surface

Observable notebook imports, mutable variables, generator cells, implicit invalidation, and observer-driven output lifecycles should not be accepted as ambient JavaScript syntax in .notebook.md cells. Future compatibility work can propose one specific Observable concept at a time only when it preserves explicit Lapis execution commands, cell-id-based persistence, worker/in-process selection, and typed output rendering. There is no standing implementation task to adopt @observablehq/runtime or to pursue wholesale Observable syntax compatibility.

App-backed tables

Notebook sessions should expose read-only app-derived tables through reserved mounts or aliases.

Suggested built-in tables:

• notes
• links
• tags
• properties
• search_docs

For the first rollout, these should be materialized from MetadataCache and AppDatabase snapshots into the notebook runtime rather than treated as direct arbitrary SQL access into the app database.

That choice keeps the current AppDatabase contract honest: it remains a generated-state API for metadata and search persistence, not the notebook runtime’s primary query boundary.

Persistence

Notebook outputs and transient execution state should be generated state, not canonical markdown.

The proposed persistence split is:

• markdown note bytes store frontmatter, prose, and cell source
• generated state stores outputs, stale state, execution errors, and cached session artifacts

Likely persistence targets:

• per-vault generated-state tables in the app database
• IndexedDB fallback when SQLite is unavailable

Generated state should store the typed output payloads rather than only pre-rendered HTML so notebook presentations can choose the correct renderer for tables, charts, markdown, and errors. Each persisted cell state stores a deterministic source fingerprint derived from the cell language and source. When the note file mtime still matches the persisted snapshot, the whole snapshot can hydrate as before. When only the file mtime changed, the notebook can still hydrate cells whose fingerprints match the current parsed cells; cells whose fingerprints changed, plus their graph dependents, reopen with their last output marked stale.

Live DOM outputs are the exception: generated state stores serializable input values plus any structured input outputs, but still omits the DOM output payload itself. Restoring a lazy notebook can seed the input value on the next run. For Observable-style viewof cells whose source fingerprint still matches, reconstructable structured inputs can now rebuild their controls directly from persisted metadata when no live input node is available, without re-executing the owning cell. Non-reconstructable inputs still fall back to rerunning the cell on first render, and other htl or DOM nodes are recreated when the owning cell executes.

Presentation Modes

The notebook syntax should be legible in source mode and richer in notebook-aware presentations.

Source mode

• raw markdown directives and fences remain editable
• cell ids and attributes are visible
• notebook actions are available through gutter or command affordances

Live preview and notebook mode

• cell directives mount as block widgets with toolbar chrome
• code cells keep the live-preview source block visible and mount rendered notebook outputs underneath the block
• markdown cells keep the live-preview directive block visible and mount rendered markdown with placeholder interpolation underneath
• table outputs mount as interactive notebook-owned table components
• chart outputs mount through approved chart-spec renderers
• Observable inputs and htl outputs mount as trusted live DOM outputs through notebook-owned wrappers
• stale and errored cells render explicit state badges

Reading mode

• ordinary prose renders as normal markdown
• code and markdown notebook cells render outputs and, when configured, their source blocks through the shared markdown code renderer, including syntax highlighting and line-number directives

Open Questions

This draft intentionally leaves several details open for later implementation work.

• whether notebook presentation should be a new markdown mode or a dedicated notebook leaf state
• whether SQL cells eventually gain deeper static analysis beyond {{name}} placeholder extraction
• whether notebook outputs should be cached by cell hash, dependency hash, or both
• which initial chart-spec libraries should be treated as first-class notebook renderers
• how much sanitized HTML output should be allowed before it overlaps too much with explicit output kinds