feat(client): read via per-doc ANCHORED queries; document virtual vs physical wallet

The anchorless union query (`GRAPH ?g`) scanned EVERY named graph in the local
store (the whole shared physical wallet) → O(wallet size), slow/timeouts on a
bloated wallet. Rewrite `readUnion` to run ONE ANCHORED `sparql_query` per by-need
doc (in parallel, per-doc tolerant): an anchored query is restricted to that
repo's graph, so it is O(1) per doc, INDEPENDENT of physical-wallet size. Keep the
ReadCap defense-in-depth gate.

docs/simulation.md: new "Physical wallet vs virtual wallet" section — the physical
shared wallet is a substrate that accumulates and must NEVER be enumerated/scanned;
each user's VIRTUAL wallet (the account's scope index in the shim) is the bounded
thing you enumerate ("list my documents"), then read those docs per-doc anchored.
read-model.md / nextgraph-current-state.md updated to the per-doc anchored rule.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Sylvain Duchesne
2026-07-05 22:50:14 +02:00
parent 6a3501e700
commit ac2b026955
5 changed files with 210 additions and 140 deletions
+17 -8
View File
@@ -191,14 +191,23 @@ from JS today a repo becomes queryable ONLY by being `doc_create`d in this sessi
**Consequence for this lib's mono-wallet polyfill:** every account's documents are
`doc_create`d in the ONE shared wallet within the SAME session, so they are ALL
already in `self.repos` and queryable by the anchorless union **without any per-doc
open step**. `read-model.ts`'s per-doc anchored `ASK` is therefore a *no-op* on a
present repo and an instant `RepoNotFound` skip on an absent one — it never SYNCS
(it cannot). The union query alone spans all same-session repos. At the real
multi-store migration this gap closes: opening a real per-user store repo by cap is
a native broker sync (the `OpenRepo` TODO at `verifier.rs:1423`), and the open step
becomes a real sync. Opening still requires the repo's **NURI + ReadCap** — there is
**no store-level read inheritance** (see § Capability / ReadCap granularity).
already in `self.repos`. `read-model.ts` reads the **bounded, by-need** set of docs
with ONE **anchored** `sparql_query` per doc (`SELECT ?s ?p ?o WHERE { ?s ?p ?o }`,
anchor = the doc NURI): the anchor resolves that same-session repo directly (no
separate open needed) and restricts the query to its graph → O(1) per doc,
independent of the store's size. An absent repo throws `RepoNotFound` on its own
read and is skipped, never aborting the batch.
**Do NOT anchorless-union-scan on the read path.** An anchorless
`SELECT … WHERE { GRAPH ?g { ?s ?p ?o } }` spans EVERY named graph in the store —
O(wallet size). On a **shared / bloated** wallet that accumulates docs across runs
that was O(wallet) and timed out (~90s observed on `readUnion` / probe reads). The
per-doc anchored read makes a non-empty wallet irrelevant. At the real multi-store
migration this is unchanged (the anchored read is native); only bringing a repo into
the session changes: opening a real per-user store repo by cap becomes a native
broker sync (the `OpenRepo` TODO at `verifier.rs:1423`). Opening still requires the
repo's **NURI + ReadCap** — there is **no store-level read inheritance** (see
§ Capability / ReadCap granularity).
### The union is READ-ONLY — writes must target one document
+42 -16
View File
@@ -8,11 +8,23 @@ capability*. The consumer (Festipod) never sees any of this: it asks
`@ng-eventually/client` for its lists **by need** and trusts the answer — the whole
read mechanism lives here, in the polyfill.
> **The rule in one line:** read each by-need doc with its OWN anchored
> `sparql_query`; NEVER run an anchorless union-scan over all graphs. An anchorless
> union spans **every** named graph in the session store — O(wallet size) — and on a
> shared/bloated wallet that accumulates across runs it produced ~90s timeouts. The
> per-doc anchored read is O(1) per doc, INDEPENDENT of wallet size, so a non-empty
> wallet no longer matters.
The governing constraints (all verified in `nextgraph-rs`, cited there):
- One local oxigraph store per session; every opened repo is a **named graph**.
- `sparql_query` with **no anchor** → the **LOCAL UNION** of all opened graphs;
with an anchor → **one** repo. Union is **read-only**.
- `sparql_query` with **no anchor** → the **LOCAL UNION** of all opened graphs
(O(wallet), NOT used on the read path); with a string anchor → restricted to
**one** repo (that repo becomes the query's DEFAULT graph). Union is **read-only**.
- The anchor's one-repo restriction applies only to a **default-graph** body (no
`GRAPH` wrapper); an explicit `GRAPH ?g { … }` body iterates the named graphs
regardless of the anchor (see § probe step 4). The read path therefore uses an
anchored `SELECT ?s ?p ?o WHERE { ?s ?p ?o }` (default-graph body) per doc.
- A repo is queryable **only after it is opened/synced** (needs its NURI + ReadCap;
no store-level read inheritance). **VERIFIED (T03.k):** the current JS SDK exposes
**no primitive that syncs an *unknown* repo**`sparql_query`/`doc_subscribe`/
@@ -55,14 +67,20 @@ notifications — **none** of these is enumerated across accounts. Each is reach
The rule of thumb: **Access ≠ discovery.** You only union-query over graphs you were
already entitled to open.
## Listing = open/sync + ONE union query (never the ORM fan-out)
## Listing = a bounded set of per-doc ANCHORED reads (never a union-scan, never the ORM fan-out)
To produce a list:
To produce a list, take the **bounded, by-need** set of doc NURIs (the index-yielded
event NURIs, my own docs, a connection's shared NURIs) and read **each one with its
OWN anchored `sparql_query`** (`SELECT ?s ?p ?o WHERE { ?s ?p ?o }`, anchor = that
doc NURI, in parallel and tolerant per-doc). The anchor restricts the query to that
one repo's graph, so each read is O(1) in the doc's own size and INDEPENDENT of how
many other graphs the (possibly bloated / shared) session store holds.
1. **Open/sync** the relevant repos (the index-yielded event NURIs, my own docs, a
connection's shared NURIs). This is what puts them in the local union.
2. Run a **single** `sparql_query` with **NO anchor** over the LOCAL UNION, using a
`GRAPH ?g { ... }` body so each result row is attributed to its source graph.
Do **NOT** run an **anchorless union-scan** (`SELECT … WHERE { GRAPH ?g { ?s ?p ?o } }`,
no anchor) over the local union: it iterates **every** named graph in the session
store — O(wallet size) — so on a shared wallet that accumulates across runs it times
out (~90s observed). The read-set is already bounded and known; read exactly those
docs, anchored, and never scan the wallet.
Do **NOT** drive listing through the reactive ORM's per-document fan-out
(`orm_start_graph` over many graphs): a freshly-created or not-yet-synced graph in
@@ -141,11 +159,19 @@ above; a repo absent from `self.repos` throws `RepoNotFound` and is skipped.)
## Implementation — `read-model.ts`
`readModel.readUnion(docs)` implements this: (1) open/sync each doc via a per-doc
anchored `ASK` (tolerant — a doc that can't open is skipped, never aborting the
listing like the ORM fan-out would); (2) run ONE anchorless
`SELECT ?g ?s ?p ?o WHERE { GRAPH ?g { ?s ?p ?o . VALUES ?s { … } } }` over the
local union, constrained to the requested subjects (each entity's subject IRI IS
its own document NURI). Returns the triples grouped per subject; the consumer maps
them to its types (e.g. Festipod's `readEntities`). Reactivity = the consumer
re-calls `readUnion` on its change signal (no reactive union query exists).
`readModel.readUnion(docs)` implements this: for each requested doc NURI (the
bounded by-need set), run — in parallel, tolerant per-doc (a doc that fails is
skipped, never aborting the batch like the ORM fan-out would) ONE **anchored**
`SELECT ?s ?p ?o WHERE { ?s ?p ?o }` with `anchor = docNuri`. The anchor restricts
the query to that doc's graph (default graph), so it returns ONLY that doc's
triples, O(1) per doc, independent of wallet size. There is **NO** anchorless
union-scan. Each entity's subject IRI IS its own document NURI, so the subject is
the anchor doc NURI; the result is grouped per subject (keeping the `UnionSubject[]`
shape: `subject`, `graph`, `props`). A ReadCap gate drops any doc the current user
may not read (defence-in-depth). The consumer maps the result to its types (e.g.
Festipod's `readEntities`). Reactivity = the consumer re-calls `readUnion` on its
change signal (no reactive union query exists).
> The name `readUnion` / `UnionSubject` is historical (it once ran a union query).
> The read is now **per-doc anchored**, bounded to the read-set — the "union" is only
> the logical concatenation of the per-doc results, never an anchorless graph scan.
+29
View File
@@ -21,6 +21,35 @@ application fiction the lib maintains. On top of that one wallet the lib rebuild
by emulation, the per-user stores + capabilities + inbox the consumer codes
against.
## Physical wallet vs virtual wallet — never enumerate the physical one
Because the emulation runs on ONE shared wallet, distinguish two levels:
- **Physical wallet** — the real NextGraph wallet everyone opens. Its local store
holds **every account's documents PLUS the lib's own internals** (the shim index,
the inbox docs, the discovery index) as named graphs. It **accumulates without
bound** across sessions/runs. **Listing / scanning "all documents" of the physical
wallet is meaningless AND O(size)** — it mixes every user's data with lib internals,
and it is exactly what a `sparql_query` with **no anchor** (`GRAPH ?g { … }`) does
(it spans every synced graph). **Never do it.** The physical wallet is a substrate,
not something to enumerate.
- **Virtual wallet** — the lib's emulation of **one user's** wallet: the set of
documents the shim attributes to that account (its per-scope index in
`store-registry.ts`). This is what "the user owns". Over a *virtual* wallet,
"**list my documents**" is meaningful and **bounded** (only that account's docs).
**Consequence for reads (see `read-model.md`):** to list a user's entities you
enumerate the **virtual** wallet — the account's scope index (bounded, O(my docs)),
NOT the physical union — then read those specific documents with a **per-doc anchored**
`sparql_query`. A non-empty / bloated physical wallet then costs nothing, because the
physical union is never scanned. Discovery (all public events) is the one bounded
enumeration hack and goes through the discovery **index**, not a physical scan.
At migration each virtual wallet becomes a real per-user wallet; the physical/virtual
distinction — and the "never enumerate the physical wallet" rule — dissolves into
native per-wallet reads.
## Two axes, never conflate them (store ≠ document)
The single most load-bearing distinction. Two **orthogonal** axes the