feat(client): read via per-doc ANCHORED queries; document virtual vs physical wallet
The anchorless union query (`GRAPH ?g`) scanned EVERY named graph in the local
store (the whole shared physical wallet) → O(wallet size), slow/timeouts on a
bloated wallet. Rewrite `readUnion` to run ONE ANCHORED `sparql_query` per by-need
doc (in parallel, per-doc tolerant): an anchored query is restricted to that
repo's graph, so it is O(1) per doc, INDEPENDENT of physical-wallet size. Keep the
ReadCap defense-in-depth gate.
docs/simulation.md: new "Physical wallet vs virtual wallet" section — the physical
shared wallet is a substrate that accumulates and must NEVER be enumerated/scanned;
each user's VIRTUAL wallet (the account's scope index in the shim) is the bounded
thing you enumerate ("list my documents"), then read those docs per-doc anchored.
read-model.md / nextgraph-current-state.md updated to the per-doc anchored rule.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -191,14 +191,23 @@ from JS today a repo becomes queryable ONLY by being `doc_create`d in this sessi
|
||||
|
||||
**Consequence for this lib's mono-wallet polyfill:** every account's documents are
|
||||
`doc_create`d in the ONE shared wallet within the SAME session, so they are ALL
|
||||
already in `self.repos` and queryable by the anchorless union **without any per-doc
|
||||
open step**. `read-model.ts`'s per-doc anchored `ASK` is therefore a *no-op* on a
|
||||
present repo and an instant `RepoNotFound` skip on an absent one — it never SYNCS
|
||||
(it cannot). The union query alone spans all same-session repos. At the real
|
||||
multi-store migration this gap closes: opening a real per-user store repo by cap is
|
||||
a native broker sync (the `OpenRepo` TODO at `verifier.rs:1423`), and the open step
|
||||
becomes a real sync. Opening still requires the repo's **NURI + ReadCap** — there is
|
||||
**no store-level read inheritance** (see § Capability / ReadCap granularity).
|
||||
already in `self.repos`. `read-model.ts` reads the **bounded, by-need** set of docs
|
||||
with ONE **anchored** `sparql_query` per doc (`SELECT ?s ?p ?o WHERE { ?s ?p ?o }`,
|
||||
anchor = the doc NURI): the anchor resolves that same-session repo directly (no
|
||||
separate open needed) and restricts the query to its graph → O(1) per doc,
|
||||
independent of the store's size. An absent repo throws `RepoNotFound` on its own
|
||||
read and is skipped, never aborting the batch.
|
||||
|
||||
**Do NOT anchorless-union-scan on the read path.** An anchorless
|
||||
`SELECT … WHERE { GRAPH ?g { ?s ?p ?o } }` spans EVERY named graph in the store —
|
||||
O(wallet size). On a **shared / bloated** wallet that accumulates docs across runs
|
||||
that was O(wallet) and timed out (~90s observed on `readUnion` / probe reads). The
|
||||
per-doc anchored read makes a non-empty wallet irrelevant. At the real multi-store
|
||||
migration this is unchanged (the anchored read is native); only bringing a repo into
|
||||
the session changes: opening a real per-user store repo by cap becomes a native
|
||||
broker sync (the `OpenRepo` TODO at `verifier.rs:1423`). Opening still requires the
|
||||
repo's **NURI + ReadCap** — there is **no store-level read inheritance** (see
|
||||
§ Capability / ReadCap granularity).
|
||||
|
||||
### The union is READ-ONLY — writes must target one document
|
||||
|
||||
|
||||
+42
-16
@@ -8,11 +8,23 @@ capability*. The consumer (Festipod) never sees any of this: it asks
|
||||
`@ng-eventually/client` for its lists **by need** and trusts the answer — the whole
|
||||
read mechanism lives here, in the polyfill.
|
||||
|
||||
> **The rule in one line:** read each by-need doc with its OWN anchored
|
||||
> `sparql_query`; NEVER run an anchorless union-scan over all graphs. An anchorless
|
||||
> union spans **every** named graph in the session store — O(wallet size) — and on a
|
||||
> shared/bloated wallet that accumulates across runs it produced ~90s timeouts. The
|
||||
> per-doc anchored read is O(1) per doc, INDEPENDENT of wallet size, so a non-empty
|
||||
> wallet no longer matters.
|
||||
|
||||
The governing constraints (all verified in `nextgraph-rs`, cited there):
|
||||
|
||||
- One local oxigraph store per session; every opened repo is a **named graph**.
|
||||
- `sparql_query` with **no anchor** → the **LOCAL UNION** of all opened graphs;
|
||||
with an anchor → **one** repo. Union is **read-only**.
|
||||
- `sparql_query` with **no anchor** → the **LOCAL UNION** of all opened graphs
|
||||
(O(wallet), NOT used on the read path); with a string anchor → restricted to
|
||||
**one** repo (that repo becomes the query's DEFAULT graph). Union is **read-only**.
|
||||
- The anchor's one-repo restriction applies only to a **default-graph** body (no
|
||||
`GRAPH` wrapper); an explicit `GRAPH ?g { … }` body iterates the named graphs
|
||||
regardless of the anchor (see § probe step 4). The read path therefore uses an
|
||||
anchored `SELECT ?s ?p ?o WHERE { ?s ?p ?o }` (default-graph body) per doc.
|
||||
- A repo is queryable **only after it is opened/synced** (needs its NURI + ReadCap;
|
||||
no store-level read inheritance). **VERIFIED (T03.k):** the current JS SDK exposes
|
||||
**no primitive that syncs an *unknown* repo** — `sparql_query`/`doc_subscribe`/
|
||||
@@ -55,14 +67,20 @@ notifications — **none** of these is enumerated across accounts. Each is reach
|
||||
The rule of thumb: **Access ≠ discovery.** You only union-query over graphs you were
|
||||
already entitled to open.
|
||||
|
||||
## Listing = open/sync + ONE union query (never the ORM fan-out)
|
||||
## Listing = a bounded set of per-doc ANCHORED reads (never a union-scan, never the ORM fan-out)
|
||||
|
||||
To produce a list:
|
||||
To produce a list, take the **bounded, by-need** set of doc NURIs (the index-yielded
|
||||
event NURIs, my own docs, a connection's shared NURIs) and read **each one with its
|
||||
OWN anchored `sparql_query`** (`SELECT ?s ?p ?o WHERE { ?s ?p ?o }`, anchor = that
|
||||
doc NURI, in parallel and tolerant per-doc). The anchor restricts the query to that
|
||||
one repo's graph, so each read is O(1) in the doc's own size and INDEPENDENT of how
|
||||
many other graphs the (possibly bloated / shared) session store holds.
|
||||
|
||||
1. **Open/sync** the relevant repos (the index-yielded event NURIs, my own docs, a
|
||||
connection's shared NURIs). This is what puts them in the local union.
|
||||
2. Run a **single** `sparql_query` with **NO anchor** over the LOCAL UNION, using a
|
||||
`GRAPH ?g { ... }` body so each result row is attributed to its source graph.
|
||||
Do **NOT** run an **anchorless union-scan** (`SELECT … WHERE { GRAPH ?g { ?s ?p ?o } }`,
|
||||
no anchor) over the local union: it iterates **every** named graph in the session
|
||||
store — O(wallet size) — so on a shared wallet that accumulates across runs it times
|
||||
out (~90s observed). The read-set is already bounded and known; read exactly those
|
||||
docs, anchored, and never scan the wallet.
|
||||
|
||||
Do **NOT** drive listing through the reactive ORM's per-document fan-out
|
||||
(`orm_start_graph` over many graphs): a freshly-created or not-yet-synced graph in
|
||||
@@ -141,11 +159,19 @@ above; a repo absent from `self.repos` throws `RepoNotFound` and is skipped.)
|
||||
|
||||
## Implementation — `read-model.ts`
|
||||
|
||||
`readModel.readUnion(docs)` implements this: (1) open/sync each doc via a per-doc
|
||||
anchored `ASK` (tolerant — a doc that can't open is skipped, never aborting the
|
||||
listing like the ORM fan-out would); (2) run ONE anchorless
|
||||
`SELECT ?g ?s ?p ?o WHERE { GRAPH ?g { ?s ?p ?o . VALUES ?s { … } } }` over the
|
||||
local union, constrained to the requested subjects (each entity's subject IRI IS
|
||||
its own document NURI). Returns the triples grouped per subject; the consumer maps
|
||||
them to its types (e.g. Festipod's `readEntities`). Reactivity = the consumer
|
||||
re-calls `readUnion` on its change signal (no reactive union query exists).
|
||||
`readModel.readUnion(docs)` implements this: for each requested doc NURI (the
|
||||
bounded by-need set), run — in parallel, tolerant per-doc (a doc that fails is
|
||||
skipped, never aborting the batch like the ORM fan-out would) — ONE **anchored**
|
||||
`SELECT ?s ?p ?o WHERE { ?s ?p ?o }` with `anchor = docNuri`. The anchor restricts
|
||||
the query to that doc's graph (default graph), so it returns ONLY that doc's
|
||||
triples, O(1) per doc, independent of wallet size. There is **NO** anchorless
|
||||
union-scan. Each entity's subject IRI IS its own document NURI, so the subject is
|
||||
the anchor doc NURI; the result is grouped per subject (keeping the `UnionSubject[]`
|
||||
shape: `subject`, `graph`, `props`). A ReadCap gate drops any doc the current user
|
||||
may not read (defence-in-depth). The consumer maps the result to its types (e.g.
|
||||
Festipod's `readEntities`). Reactivity = the consumer re-calls `readUnion` on its
|
||||
change signal (no reactive union query exists).
|
||||
|
||||
> The name `readUnion` / `UnionSubject` is historical (it once ran a union query).
|
||||
> The read is now **per-doc anchored**, bounded to the read-set — the "union" is only
|
||||
> the logical concatenation of the per-doc results, never an anchorless graph scan.
|
||||
|
||||
@@ -21,6 +21,35 @@ application fiction the lib maintains. On top of that one wallet the lib rebuild
|
||||
by emulation, the per-user stores + capabilities + inbox the consumer codes
|
||||
against.
|
||||
|
||||
## Physical wallet vs virtual wallet — never enumerate the physical one
|
||||
|
||||
Because the emulation runs on ONE shared wallet, distinguish two levels:
|
||||
|
||||
- **Physical wallet** — the real NextGraph wallet everyone opens. Its local store
|
||||
holds **every account's documents PLUS the lib's own internals** (the shim index,
|
||||
the inbox docs, the discovery index) as named graphs. It **accumulates without
|
||||
bound** across sessions/runs. **Listing / scanning "all documents" of the physical
|
||||
wallet is meaningless AND O(size)** — it mixes every user's data with lib internals,
|
||||
and it is exactly what a `sparql_query` with **no anchor** (`GRAPH ?g { … }`) does
|
||||
(it spans every synced graph). **Never do it.** The physical wallet is a substrate,
|
||||
not something to enumerate.
|
||||
|
||||
- **Virtual wallet** — the lib's emulation of **one user's** wallet: the set of
|
||||
documents the shim attributes to that account (its per-scope index in
|
||||
`store-registry.ts`). This is what "the user owns". Over a *virtual* wallet,
|
||||
"**list my documents**" is meaningful and **bounded** (only that account's docs).
|
||||
|
||||
**Consequence for reads (see `read-model.md`):** to list a user's entities you
|
||||
enumerate the **virtual** wallet — the account's scope index (bounded, O(my docs)),
|
||||
NOT the physical union — then read those specific documents with a **per-doc anchored**
|
||||
`sparql_query`. A non-empty / bloated physical wallet then costs nothing, because the
|
||||
physical union is never scanned. Discovery (all public events) is the one bounded
|
||||
enumeration hack and goes through the discovery **index**, not a physical scan.
|
||||
|
||||
At migration each virtual wallet becomes a real per-user wallet; the physical/virtual
|
||||
distinction — and the "never enumerate the physical wallet" rule — dissolves into
|
||||
native per-wallet reads.
|
||||
|
||||
## Two axes, never conflate them (store ≠ document)
|
||||
|
||||
The single most load-bearing distinction. Two **orthogonal** axes the
|
||||
|
||||
Reference in New Issue
Block a user