File System, Metadata, and Search
The data flow is file-first. Storage updates propagate into metadata indexing, then into higher-level features such as search and views.
Storage and Vault
The vault is the source of truth for files, folders, reads, writes, renames, and deletes. VaultAdapter is the preferred high-level canonical file interface; it currently extends the existing DataAdapter contract so plugins and older call sites can continue to use DataAdapter while the codebase migrates toward runtime-neutral vault sessions. Saved vault profiles can also carry optional demo metadata that identifies a generated demo fixture (source, fixtureVersion) so chooser and command surfaces can distinguish bundled demo vaults from user-created vaults without overloading the storage kind.
VaultSession is resolved before App construction. A session carries the active runtime, optional browser vault profile, canonical vaultAdapter, and per-vault appDatabase. Browser bootstrap now resolves this session after the user picks OPFS, File System Access, or legacy storage.
The desktop host now resolves the same session boundary through a native bridge. Desktop vault profiles use the desktop-folder kind (legacy tauri-folder profiles are migrated on open), restore a user-selected root path through NativeDesktopVaultAdapter, and keep all generated metadata/search/app state outside the selected folder.
Browser Adapters
| Adapter | API | Use Case |
|---|---|---|
OpfsVaultAdapter | Origin Private File System | Default for new browser-local vaults. No permission prompts. Multiple vaults live under vaults/<vaultId>/. listOpfsVaultIds() and listOrphanOpfsVaultIds() discover OPFS directories missing from the saved profile store so the chooser can offer recovery without deleting stored files. |
BrowserHandleVaultAdapter | File System Access API | User-selected real folders. Requires user gesture for authorization. |
| Lightning FS | IndexedDB-backed FS | Legacy/manual use. Multiple browser-local legacy vaults use per-profile Lightning FS store names; profile id lightningfs-default keeps the original lightning-fs-1 store. |
OPFS and File System Access store canonical file bytes through the vault adapter. Handle-backed browser adapters retry recoverable NotFoundError, NotReadableError, NoModificationAllowedError, InvalidStateError, and ENOENT failures on reads, writes, deletes, and directory listings, and replacement writes use truncating browser writable streams so shorter content cannot leave stale bytes behind. Generated app state is separate from those canonical files. Browser vault profiles remain in IndexedDB-backed profile storage, but metadata and search-derived state now go through AppDatabase.
For normal browser sessions, the app database resolver attempts SqliteWasmAppDatabase first only when OPFS is available and the host is cross-origin isolated, because the current SQLite target uses the OPFS SAH pool VFS. That implementation stores a per-vault SQLite database outside the user-visible vault tree. Browser SQLite execution now lives behind a dedicated worker, with the main thread proxy mirroring metadata/search state for local reads while routing SQLite work through message passing. When the browser exposes the Web Locks API, the resolver now treats lock contention as a coordination state instead of as a fallback condition: it takes an exclusive per-vault lock before opening the SAH pool, starts SQLite only in the lock-owning tab, and returns a coordinated AppDatabase proxy to competing tabs when BroadcastChannel is available. Those secondary tabs forward AppDatabase requests to the owner over a per-vault RPC channel while still listening for heartbeats and retrying lock acquisition with jitter when the owner disappears. If a proxied request is still in flight when the owner disappears, the coordinated database now replays that request locally as soon as the secondary tab finishes promoting itself to owner instead of surfacing a stale RPC timeout back to the app. The blocked session state remains as a fallback when delegation cannot be established. If the prerequisites are missing, Web Locks are unavailable, or SQLite WASM / OPFS startup fails after ownership is acquired, it falls back to IndexedDbAppDatabase, which persists the same AppDatabase state shape in vault-scoped IndexedDB. This fallback remains the compatibility path rather than the normal multi-tab path.
The File System Access adapter works with real folders but requires re-authorization on each session. Its generated app database is still per-vault and must not be written into the selected folder.
Desktop Native Adapter
| Adapter | API | Use Case |
|---|---|---|
NativeDesktopVaultAdapter | NativeDesktopBridge IPC commands | User-selected real folders in the Electron desktop shell. |
The desktop host exposes a narrow bridge for folder selection, canonical file operations scoped to the selected vault root, resource URL conversion, optional native drag starts, optional native file-watch events, and native file actions. NativeDesktopVaultAdapter lives in packages/api so the shared runtime continues to talk to a single vault contract, while the host package owns the actual filesystem work. Electron resource URLs are served through the host-owned lapis-vault-resource:// protocol rather than by copying media bytes through renderer IPC. The protocol resolves only vault-relative paths under the selected root and is consumed through Vault.getResourceUrl() / Vault.revokeResourceUrl() so browser adapters can keep blob URL fallbacks while native hosts can stream images, PDFs, and other read-only embeds from the filesystem. Native file actions resolve vault-relative paths to host paths, open them through the default app, and reveal them in the file manager only after validating that the target remains under the selected vault root.
Desktop generated state now uses NativeDesktopAppDatabase, which persists the serializable AppDatabase state shape in a per-vault native SQLite file under the host user-data directory outside the selected vault folder. The current native backend stores one serialized state payload per vault id inside that SQLite state database and still treats that store as disposable generated state; the follow-on native search schema work owns table-level search/query parity. Search refreshes can now open an explicit search-index batch through AppDatabase, letting the native desktop backend defer per-document native search mirroring and full-state persistence until the batch ends, then flush once with a full native replacement pass. Desktop generated state must never move the primary app database into the selected user vault folder, but metadata cache snapshots are also mirrored to a vault-local generated backup at .lapis/cache/metadata-cache.json. That backup is rebuildable, safe to delete, and exists only to hydrate imported or copied vaults when the host-owned native database is missing. Electron state-load probing returns null when neither SQLite nor legacy JSON state exists instead of creating an empty SQLite file during the probe. Electron now also exposes reveal-folder and per-item reveal/open commands through the native bridge so users can inspect, copy, or open canonical files outside the renderer.
When the active vault is an OPFS-backed browser vault, the app now exposes an import command that prompts for a local File System Access folder, copies that folder tree into the current OPFS vault with overwrite semantics on matching paths, updates the in-memory vault tree and metadata cache, and surfaces progress through app.notifications. Browser-local vaults now also expose the inverse export command, which copies the current vault tree into a user-picked File System Access folder with determinate progress and error reporting. This gives browser and agent sessions a predictable way to move canonical vault files into or out of opaque browser-local storage without depending on a persistent File System Access permission grant for the primary vault itself. The web renderer still presents the progress in-app, while native desktop hosts can mirror durable outcomes through their own notification capability.
App Database
AppDatabase is the per-vault generated-state contract beneath App:
The SQLite schema currently includes schema_meta, files, metadata, links, tags, properties, search_docs, and a search_fts FTS5 virtual table. Canonical note bytes are not stored in SQLite. The database is disposable generated state: it can be deleted and rebuilt from canonical vault files. Electron’s native desktop backend mirrors prepared search documents into host SQLite and uses FTS5 for plain lexical candidate selection. When a platform-compatible sqlite-vec extension is available, Electron also mirrors ready chunk embeddings into a native vec0 table for vector candidate selection; otherwise vector and hybrid search modes fall back to the shared in-memory vector scorer over the same mirrored documents.
Notebook generated state also belongs behind AppDatabase. Notebook cells may persist typed serializable outputs, execution metadata, scalar input values used to seed future runs, and per-cell source fingerprints used to decide whether saved outputs still correspond to the current cell source. Live DOM nodes produced by notebook inputs or htl are runtime state only and must not be written into the generated-state store. Notebook recovery flows therefore keep canonical .notebook.md source separate from generated outputs: users can clear persisted notebook outputs for the active note without deleting notebook source cells.
File history generated by the
History Plugin also belongs behind
AppDatabase. History revisions are per-vault app state, not canonical vault
files, and must stay out of .obsidian/ and user note folders. The first
history rollout stores debounced full-text snapshots for text files only,
excluding .obsidian/, with content-hash deduplication and bounded per-file
retention. Rename, delete, and restore flows are part of this app-database
contract: history follows renames by stable file identity, delete state does not
discard prior revisions, and restores write through the vault before recording a
restore revision.
The browser SQLite implementation currently uses the OPFS SAH pool VFS, so web hosts must emit COOP/COEP response headers to enable the SharedArrayBuffer/Atomics path it requires. Multi-tab coordination now happens at session startup on the main thread: Web Locks provide exclusive ownership, BroadcastChannel heartbeats signal liveness, a per-vault RPC channel forwards AppDatabase calls from secondary tabs to the owner, and the worker is created only after ownership is confirmed. SQLite execution remains isolated behind a dedicated worker so indexing and query work do not run on the UI thread. Search-document upserts now return the worker-prepared document back to the renderer mirror so browser sessions do not rerun the embedding preparation path on the main thread after a successful worker write.
Blob Stores
BlobStoreLike is available as a lower-level byte-store seam for adapters that benefit from it, especially OPFS and native desktop resource handling. It sits below the vault layer and is not a replacement for canonical file operations.
File Tree
The Vault maintains an in-memory file tree: files: Record<string, TAbstractFile>. This is populated by vault.load() which recursively lists the adapter. The tree is the source of truth for path lookups (getAbstractFileByPath, getFileByPath).
Vault Events
| Event | Payload | Trigger |
|---|---|---|
load | — | File tree loaded |
create | file: TAbstractFile | File/folder created |
modify | file: TAbstractFile | File content modified |
delete | file: TAbstractFile | File/folder deleted |
rename | file: TAbstractFile, oldPath: string | File/folder renamed |
all | event: string, file, context | Any event (discriminated) |
File-System Watching
packages/workspace/src/lib/hooks/watch-vault.svelte.ts wires a watcher over the vault adapter.
When an adapter reports nativeWatch: true, DirectoryWatcher now prefers adapter-provided native watch events over the polling fallback. Electron currently satisfies that contract by forwarding preload subscriptions to main-process chokidar watchers and translating host events back into normalized vault-relative paths.
Responsibilities:
- Detect create, modify, delete, and rename activity.
- Close open file views when files are deleted.
- Retarget open file views when files or folders are renamed.
- Emit workspace-level
"file-change"events for active features.
Debouncing: A recentInternalEvents map tracks recent vault operations with a 7.5 s TTL to distinguish internal writes from external file system changes.
File-system synchronization is coordinated by the workspace shell, but effects are applied through runtime objects (views, leaves). Events for the generated metadata backup path .lapis/cache/metadata-cache.json and its generated parent folders are ignored before vault reload handling so periodic backup writes do not re-enter metadata indexing or trigger reload loops.
Metadata Cache
MetadataCache (packages/api/src/lib/cache.svelte.ts) indexes every file’s structural content.
What Gets Indexed
For each file, the cache stores:
- File tracking:
{mtime, size, hash}for change detection. - Parsed metadata:
CachedMetadatakeyed by content hash.
The CachedMetadata structure captures:
- Frontmatter (YAML key-value pairs and position).
- Internal links (
[[link]]) with display text and position. - Embeds (
![[embed]]) with position. - Tags (
#tag) with position. - Headings with level (1-6) and position.
- Sections (code blocks, headings, lists) with type and position.
- List items with optional task state and parent index.
- Footnote definitions.
- Block IDs (
^block-id).
Metadata extraction must tolerate the markdown package’s custom mdast nodes such as wikilinks and embeds during rebuilds. Heading text and original link spans are derived from node text and source offsets rather than relying on a generic mdast serializer that may reject custom node types.
Metadata Processors
Plugins register MetadataProcessor objects for specific file extensions. The markdown plugin registers a worker-backed YAML processor for .md files that runs asynchronously via PromiseWorker.
interface MetadataProcessor {
read(data: string, ctx: { cache; file }): Promise<CachedMetadata>;
write(cache: CachedMetadata): string;
}
Link Resolution
getFirstLinkpathDest(linkpath, sourcePath)— Resolves wikilinks. Tries: exact path →.md→.markdown→ basename match.resolvedLinks: Record<string, Record<string, number>>— Forward link adjacency map.unresolvedLinks: Record<string, Record<string, number>>— Broken link adjacency map.resolveSubpath(cache, subpath)— Resolves#heading,^block,[^footnote]within a file.
Persistence
Caches are persisted primarily through app.appDatabase, keyed by vault ID. MetadataCache keeps the same public API and event surface but writes file tracking, parsed metadata, links, tags, and frontmatter properties into the app database. Metadata snapshots are also periodically mirrored to .lapis/cache/metadata-cache.json using a versioned generated-state envelope. On load, the restore order is app-database snapshot first, portable metadata backup second, legacy ScopedVaultStore metadata snapshot third, and a vault-driven rebuild only when none of those sources exists or validates. Restoring from the portable or legacy snapshot hydrates AppDatabase by saving the snapshot and upserting indexed file records derived from the restored cache.
The database records are derived from canonical vault files. Stale or incompatible metadata can be discarded and rebuilt by rereading files through Vault.
Events: changed(file, data, cache), deleted(file, prevCache), loaded.
Metadata cache events remain per-file and intentionally do not expose a precomputed reverse-dependency graph. Views and plugins that do not own vault-wide derived state are expected to scope their reactions to direct impact instead of reloading on every metadata event. The shared runtime now exposes direct-impact helpers on MetadataCache for this purpose: callers can enumerate a note’s directly resolved references, list the current files that directly reference a given path, and ask whether a changed or deleted path is the same file or a direct reference neighbor of a watched file. That contract is intentionally conservative and first-order only: it covers the active file plus directly linked or embedded targets, but rename remains a vault concern and deeper reverse-dependency expansion is still out of scope.
Metadata cache background load and explicit rebuilds report through app.notifications. Rebuild progress is determinate over the current vault file list and supports cooperative cancellation between files. Workspace command surfaces now group these repair paths under explicit rebuild actions so users can refresh metadata and search generated state without touching canonical vault files. Snapshot writes debounce the primary app-database save and throttle portable backup writes to at most once every 30 seconds, while explicit rebuilds and shell teardown force a backup flush. Loaded snapshots emit loaded before background stale-entry reconciliation, and no-state sessions emit loaded only after the first-open rebuild completes. Metadata events and persistence semantics remain unchanged.
Boot-Time Ordering
Metadata cache now loads immediately after the shell mounts instead of blocking the startup screen. The background load still happens after vault, config, plugins, and layout. This means:
- Layout and plugins exist before metadata is ready.
- Metadata-driven features must tolerate boot-time lag and react to the later
loadedevent. - Features like search rebuild incrementally as metadata arrives.
Metadata Type Tracking
MetadataTypeManager tracks frontmatter property types across all files:
types: Record<string, MetadataTypeDef>— Persisted totypes.json.properties: Record<string, MetadataTypeProperty>— Runtime index:{ name, type, count, files }.- Auto-infers types: Number, Boolean, Tags, Aliases, Date, DateTime, Multitext, Array, Object, Text.
- Plugins register custom
TypeWidgetrenderers for frontmatter editing. trackChanges()listens to metadata cache events and updates properties on file change/delete.normalizeMetadataValue(type, value)in@lapis-notes/apiperforms best-effort write-path coercion against the declared property type. When coercion is unambiguous (for example string"false"on a checkbox column or"42"on a number column), frontmatter writers persist the normalized native value. When coercion is impossible, the original value is written unchanged and the Properties UI may continue to show a type-mismatch warning.
FileManager.processFrontMatter() is the shared mutation entrypoint for note-local frontmatter edits. It now seeds writes from either the current metadata cache or the current file contents, suppresses no-op writes, creates or removes the whole YAML block when the first or last property changes, and retries against a fresh snapshot when the file changes between the initial read and the vault write callback. Frontmatter clients should route note-local property adds, updates, and removals through that API instead of rewriting YAML blocks directly. Shared frontmatter update helpers and Bases inline cell writes call normalizeMetadataValue() against the declared types.json entry (or the caller’s fallback type) before persisting.
MetadataTypeManager now tracks exact nested property paths on a per-file basis instead of only counting top-level property names. That lets property rename operations target only the files that actually contain a given nested path, preserve falsey values such as false and 0, report partial failures when a destination path already exists in a file, and move the persisted types.json definition to the new id only after every targeted file update succeeds. Nested delete and type-change helpers use the same path semantics. For vault-wide top-level maintenance, the manager also exposes exact-key bulk rename, delete, and type-change operations that do not interpret dotted YAML keys as nested paths.
Indexed Metadata Queries
AppDatabase now owns a plugin-neutral indexed metadata query contract alongside metadata persistence and search. The contract returns serializable rows shaped as file records plus the mirrored metadata, property, tag, and link records for each candidate path. Browser SQLite and native SQLite-backed runtimes may use generated-state tables to prefilter, sort, and limit candidate paths before materializing the full rows, while memory and IndexedDB fallback targets preserve the same public contract with deterministic in-process evaluation.
The first consumer is Bases. Its renderer lowers only conservative conjunctive constraints into AppDatabase.queryIndexedMetadata() and still runs PEaQL in the renderer for final semantics. Today that pushdown is intentionally narrower than the full Bases query surface:
- lowered from Bases:
file.inFolder(...),file.hasTag(...),file.hasProperty(...),!hasProperty(...),file.ext = ..., scalar property comparisons (=,!=,>,>=,<,<=), supported file or property sorts, andlimitwhen every active sort lowered. - not currently lowered from Bases:
ORgroups, custom filter strings, non-conjunctive or custom filter lines, file-field predicates outside the subset above, link predicates such asfile.hasLink(...), formulas, group-by, view search, and custom view logic.
Unlowered clauses do not make the query invalid. They simply remain renderer-owned even when AppDatabase has already narrowed the candidate set.
Search Indexing and Retrieval
packages/plugins/plugin-search/src/search-manager.ts and AppDatabase remain the only approved search boundary, but the approved direction now extends that boundary from file-level full-text search into chunk-aware hybrid retrieval. The browser runtime continues to execute SQLite behind the dedicated app-database worker, and the same worker becomes the owner of any browser-local embedding runtime needed for vector search. The main thread and Svelte views do not talk to SQLite, sqlite-vec, or transformer inference directly.
Markdown link sidebars also consume the generated search-document boundary. Backlinks and Outgoing Links unlinked mention sections scan SearchDocumentRecord.content for exact markdown basename and frontmatter-alias matches, use sourceMetadata.frontmatterEndOffset to skip frontmatter, and exclude ranges already covered by metadata links or embeds. The sidebar runtime may read markdown vault files as a temporary fallback for files missing from listSearchDocuments() so the UI remains useful while the generated search index is disabled, rebuilding, or catching up; indexed records remain the preferred source when present.
Search Data Model
The approved target separates file-level search metadata from content-chunk retrieval state:
| Store | Role |
|---|---|
search_docs | One row per source file for path, title, extension, tags, selected metadata text, and file-level change tracking |
search_fts | FTS5 index over file-level lexical fields such as path, title, tags, and selected metadata text |
search_chunks | Derived markdown-aware content chunks keyed by a stable chunk id, with source path, ordinal, offsets, section or heading context, and chunk text |
sqlite-vec virtual table | Chunk embeddings keyed by the same stable chunk id, plus enough metadata to verify model identity and vector dimension |
File-level records remain the source of truth for path and metadata matches. Chunk rows are the source of truth for content snippets, offsets, and vector retrieval.
Worker Ownership and Runtime Model
- Browser sessions continue to host SQLite inside the dedicated app-database worker.
- The same worker owns browser-local embedding inference and model lifecycle for hybrid search.
- The first real browser embedding runtime uses Transformers.js feature extraction with browser-managed model caching; model selection and remote-vs-local loading policy stay in provider configuration persisted through
AppDatabase. - The main thread reaches search only through the existing
AppDatabasebridge. - Secondary tabs continue to delegate through the coordinated proxy path instead of opening independent generated-state stores.
- Search diagnostics such as document count, vector count, model readiness, live model download progress, embedding error state, and long-running indexing progress are part of the approved worker-facing search contract and feed the optional semantic-search status-bar popover.
AppDatabasenow also exposes incremental search-index stats and explicit search-index batch boundaries so UI polling and long-running refresh flows can stay on cheap counters instead of rescanning the full mirrored corpus.
The current implementation now includes both the original provider seam and a real browser-hosted transformer path. Search storage and query plumbing accept a serializable embedding-provider configuration, and the browser worker can host a Transformers.js feature-extraction runtime that downloads and caches model files in the browser while keeping inference off the UI thread. Provider configuration now also starts background warmup so the runtime can surface live download progress before the first semantic query, normalizes the runtime’s remote host and remote path template when the worker bundle leaves them unset, only enables local model resolution when a local model path is explicitly configured, disables the browser Cache API path when the provider runs inside a native desktop renderer so wedged Chromium profile caches cannot stall runtime initialization, and surfaces a runtime message that distinguishes a model that was downloaded in the current session from one that was loaded from existing cache or local files. The deterministic token-hash provider remains available as the no-download fallback and test/runtime compatibility path while sqlite-vec is still pending.
This keeps hybrid search off the UI thread and preserves the existing multi-tab ownership model.
Indexing Behavior
- Metadata cache change events remain the trigger for reactive search updates.
SearchManagerwrites file-level search documents plus the raw content and serializable metadata hints needed to rebuild derived chunk state throughAppDatabase.- The app-database preparation path reuses
CachedMetadata-derived headings, sections, and frontmatter positions to score candidate markdown boundaries, search backward from a target chunk size, and cut at the highest-value nearby breakpoint instead of blindly splitting on every parser section. - Tag hierarchy parsing remains part of file-level indexing so
#parent/childcontinues to contribute searchable path-like tag levels. - Embeddings are generated locally and tracked by model id, vector dimension, and chunk fingerprint so unchanged chunks are not re-embedded unnecessarily. When the runtime cannot load or fetch a model, chunk embedding state records an error and lexical search remains available.
- Search settings that affect chunking or embeddings now trigger a background vault-driven refresh path so the canonical markdown files are reread and re-indexed with the current provider and chunking settings.
- The same vault-driven refresh path now deletes stale generated search documents whose paths are no longer present in the current vault file list, so repair passes can converge after missed delete events or earlier interrupted semantic refresh runs.
- Search-manager refreshes now yield between stale-document deletes, per-file upserts, and late stale cleanup so desktop renderers can keep input and layout responsive while indexing progresses.
- Metadata-cache-driven search updates are now debounced per source path before they reach
AppDatabase, which bounds reactive indexing work during rapid edits and preserves the latest content snapshot for each note. - Search-view presentation settings such as result sorting and header-panel toggles are persisted in the search plugin settings so the sidebar restores the same layout and ordering across reopen and reload.
- Reconfiguring the embedding provider no longer re-embeds every stored chunk inline inside the worker RPC. Existing chunk embeddings are rewritten to a pending state immediately, and the background refresh path performs the actual re-embedding work afterward so plugin startup and metadata persistence do not block on model warmup or downloads.
- When a semantic provider is already configured at startup, chunks whose embedding metadata is missing or stale for the active model are treated as pending and the search plugin quietly schedules a retrying background backfill refresh once provider configuration completes so existing indexed notes converge on the active provider without waiting for a manual rebuild.
- Vault-driven search refreshes tolerate temporarily missing metadata cache entries for markdown files by indexing them with empty source-metadata snapshots instead of skipping the file entirely; later metadata cache change events can enrich the stored search document with headings, sections, tags, and frontmatter once that data becomes available.
AppDatabase.rebuildSearchIndex()becomes the explicit repair path for lexical, chunk, and vector-derived state.- Delete and rename flows must keep file, chunk, and vector state aligned with the owning canonical note.
Search still depends on healthy file content and metadata, and all derived search state remains disposable generated state that must not pollute the user’s canonical vault contents.
Query Behavior
- The shared
packages/apisearch-query parser owns the concrete syntax tree and semantic AST for the Obsidian-style query language.AppDatabase.searchDocuments()evaluates plain literal conjunctions, phrases,OR, parentheses, negation, regex literals, field filters, value-bearing property queries, quoted property names such as["note.status"], nested object and array property queries such as["project.name"]or["items.name"], property null checks, property comparisons, nested property subqueries, case-mode operators, line/block/section scopes, and task scopes through a shared evaluator. Exact top-level property names win first, so a literalnote.statuskey is not treated as a nestednote.statuspath unless no exact key exists. SQLite-backed sessions still use FTS and vector tables for simple candidate selection, but structured or case-sensitive lexical queries route through the same evaluator over the mirrored in-memory search document set so semantics stay consistent across database backends. - Lexical retrieval continues to use SQLite FTS for file-level fields and chunk-level content candidates.
- SQLite-backed runtimes use vec-capable storage when available: browser OPFS sessions load the vec-capable SQLite WASM package, while Electron attempts to load a platform-compatible native sqlite-vec extension in the main process. Ready chunk embeddings are mirrored into the active SQLite vec table, and semantic candidate retrieval queries that table first; if vec lookup is unavailable or incompatible with the active embedding dimensions, the app-database layer falls back to direct chunk-vector scoring so the public search contract remains stable.
- Memory and IndexedDB fallback app-database targets preserve the same public query contract with a deterministic non-SQLite fallback implementation.
- Hybrid retrieval combines lexical and vector candidates with deterministic Reciprocal Rank Fusion over the shared lexical and vector score ranks. Results keep the raw lexical/vector scores plus the RRF ranks and per-signal contributions so callers can explain why a hybrid hit ranked where it did. SQLite-backed paths union semantic candidates with the lexical shortlist instead of dropping vector-only hits whenever FTS already returned some other file.
- The search sidebar persists a view-level toggle for whether structured queries such as field filters,
OR, negation, and comparisons are allowed to use semantic retrieval; when disabled, those queries are forced onto the lexical path while plain literal queries continue using the normal automatic retrieval selection. - The first rollout stops at lexical plus vector fusion. The desktop search contract now includes a serializable Electron-only local query-enhancement request plus diagnostics for unsupported-runtime, strong-signal, and provider-unavailable skips, but actual local LLM expansion and cross-encoder reranking remain blocked until the desktop runtime has a runnable local provider behind that contract. The feature continues to stay disabled by default, settings-gated, Electron-only, and hidden from the UI until that provider exists.
AppDatabase.searchDocuments()returns grouped file results with chunk-level snippets, offsets, section labels, backend provenance, retrieval mode, raw lexical/vector scores, and an explainable fused score breakdown so the search UI can render highlights, explain whether a hit was lexical or semantic, and navigate to matches without a second client-side search engine. Runtime state such as model download progress, embedding readiness, and background indexing progress is surfaced through the optional semantic-search status-bar popover, which continues polling while the embedding runtime reportsdownloadingorembedding.- The search sidebar keeps the query controls in a fixed header above a separate scrolling results region, shows a total-results summary row, and applies user-selected client-side sort modes for filename and file timestamps after AppDatabase returns the ranked hits.
Approved Search Target
The approved target search layer is a SQLite-backed hybrid SearchIndex abstraction over file metadata, chunk-level content, and chunk embeddings. Browser OPFS sessions continue to execute that work behind the app-database worker, and the search UI consumes the returned snippets, ranges, provenance, and progress signals directly. Follow-on slices should build on that single subsystem rather than introducing a second search engine.
Plugin and Config Persistence
| File | Purpose |
|---|---|
/.obsidian/community-plugins.json | Enabled community plugin IDs |
/.obsidian/app.json | Main app configuration defaults |
/.obsidian/types.json | Metadata type definitions |
/.obsidian/plugins/{id}/data.json | Per-plugin persistent data |
/.obsidian/plugins/{id}/manifest.json | Plugin manifest |
/.obsidian/plugins/{id}/main.js | Plugin code |
/.obsidian/plugins/{id}/styles.css | Plugin stylesheet (optional) |
These files form the persisted control plane for workspace behavior.
Generated metadata, backlinks, app-derived indexes, and search state must stay out of .obsidian/ and out of user note folders. .obsidian/* remains canonical only for app/plugin compatibility files such as configuration, layout, plugin manifests/code/data, enablement lists, and types.json.