feat(client): read via per-doc ANCHORED queries; document virtual vs physical wallet
The anchorless union query (`GRAPH ?g`) scanned EVERY named graph in the local
store (the whole shared physical wallet) → O(wallet size), slow/timeouts on a
bloated wallet. Rewrite `readUnion` to run ONE ANCHORED `sparql_query` per by-need
doc (in parallel, per-doc tolerant): an anchored query is restricted to that
repo's graph, so it is O(1) per doc, INDEPENDENT of physical-wallet size. Keep the
ReadCap defense-in-depth gate.
docs/simulation.md: new "Physical wallet vs virtual wallet" section — the physical
shared wallet is a substrate that accumulates and must NEVER be enumerated/scanned;
each user's VIRTUAL wallet (the account's scope index in the shim) is the bounded
thing you enumerate ("list my documents"), then read those docs per-doc anchored.
read-model.md / nextgraph-current-state.md updated to the per-doc anchored rule.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -191,14 +191,23 @@ from JS today a repo becomes queryable ONLY by being `doc_create`d in this sessi
|
|||||||
|
|
||||||
**Consequence for this lib's mono-wallet polyfill:** every account's documents are
|
**Consequence for this lib's mono-wallet polyfill:** every account's documents are
|
||||||
`doc_create`d in the ONE shared wallet within the SAME session, so they are ALL
|
`doc_create`d in the ONE shared wallet within the SAME session, so they are ALL
|
||||||
already in `self.repos` and queryable by the anchorless union **without any per-doc
|
already in `self.repos`. `read-model.ts` reads the **bounded, by-need** set of docs
|
||||||
open step**. `read-model.ts`'s per-doc anchored `ASK` is therefore a *no-op* on a
|
with ONE **anchored** `sparql_query` per doc (`SELECT ?s ?p ?o WHERE { ?s ?p ?o }`,
|
||||||
present repo and an instant `RepoNotFound` skip on an absent one — it never SYNCS
|
anchor = the doc NURI): the anchor resolves that same-session repo directly (no
|
||||||
(it cannot). The union query alone spans all same-session repos. At the real
|
separate open needed) and restricts the query to its graph → O(1) per doc,
|
||||||
multi-store migration this gap closes: opening a real per-user store repo by cap is
|
independent of the store's size. An absent repo throws `RepoNotFound` on its own
|
||||||
a native broker sync (the `OpenRepo` TODO at `verifier.rs:1423`), and the open step
|
read and is skipped, never aborting the batch.
|
||||||
becomes a real sync. Opening still requires the repo's **NURI + ReadCap** — there is
|
|
||||||
**no store-level read inheritance** (see § Capability / ReadCap granularity).
|
**Do NOT anchorless-union-scan on the read path.** An anchorless
|
||||||
|
`SELECT … WHERE { GRAPH ?g { ?s ?p ?o } }` spans EVERY named graph in the store —
|
||||||
|
O(wallet size). On a **shared / bloated** wallet that accumulates docs across runs
|
||||||
|
that was O(wallet) and timed out (~90s observed on `readUnion` / probe reads). The
|
||||||
|
per-doc anchored read makes a non-empty wallet irrelevant. At the real multi-store
|
||||||
|
migration this is unchanged (the anchored read is native); only bringing a repo into
|
||||||
|
the session changes: opening a real per-user store repo by cap becomes a native
|
||||||
|
broker sync (the `OpenRepo` TODO at `verifier.rs:1423`). Opening still requires the
|
||||||
|
repo's **NURI + ReadCap** — there is **no store-level read inheritance** (see
|
||||||
|
§ Capability / ReadCap granularity).
|
||||||
|
|
||||||
### The union is READ-ONLY — writes must target one document
|
### The union is READ-ONLY — writes must target one document
|
||||||
|
|
||||||
|
|||||||
+42
-16
@@ -8,11 +8,23 @@ capability*. The consumer (Festipod) never sees any of this: it asks
|
|||||||
`@ng-eventually/client` for its lists **by need** and trusts the answer — the whole
|
`@ng-eventually/client` for its lists **by need** and trusts the answer — the whole
|
||||||
read mechanism lives here, in the polyfill.
|
read mechanism lives here, in the polyfill.
|
||||||
|
|
||||||
|
> **The rule in one line:** read each by-need doc with its OWN anchored
|
||||||
|
> `sparql_query`; NEVER run an anchorless union-scan over all graphs. An anchorless
|
||||||
|
> union spans **every** named graph in the session store — O(wallet size) — and on a
|
||||||
|
> shared/bloated wallet that accumulates across runs it produced ~90s timeouts. The
|
||||||
|
> per-doc anchored read is O(1) per doc, INDEPENDENT of wallet size, so a non-empty
|
||||||
|
> wallet no longer matters.
|
||||||
|
|
||||||
The governing constraints (all verified in `nextgraph-rs`, cited there):
|
The governing constraints (all verified in `nextgraph-rs`, cited there):
|
||||||
|
|
||||||
- One local oxigraph store per session; every opened repo is a **named graph**.
|
- One local oxigraph store per session; every opened repo is a **named graph**.
|
||||||
- `sparql_query` with **no anchor** → the **LOCAL UNION** of all opened graphs;
|
- `sparql_query` with **no anchor** → the **LOCAL UNION** of all opened graphs
|
||||||
with an anchor → **one** repo. Union is **read-only**.
|
(O(wallet), NOT used on the read path); with a string anchor → restricted to
|
||||||
|
**one** repo (that repo becomes the query's DEFAULT graph). Union is **read-only**.
|
||||||
|
- The anchor's one-repo restriction applies only to a **default-graph** body (no
|
||||||
|
`GRAPH` wrapper); an explicit `GRAPH ?g { … }` body iterates the named graphs
|
||||||
|
regardless of the anchor (see § probe step 4). The read path therefore uses an
|
||||||
|
anchored `SELECT ?s ?p ?o WHERE { ?s ?p ?o }` (default-graph body) per doc.
|
||||||
- A repo is queryable **only after it is opened/synced** (needs its NURI + ReadCap;
|
- A repo is queryable **only after it is opened/synced** (needs its NURI + ReadCap;
|
||||||
no store-level read inheritance). **VERIFIED (T03.k):** the current JS SDK exposes
|
no store-level read inheritance). **VERIFIED (T03.k):** the current JS SDK exposes
|
||||||
**no primitive that syncs an *unknown* repo** — `sparql_query`/`doc_subscribe`/
|
**no primitive that syncs an *unknown* repo** — `sparql_query`/`doc_subscribe`/
|
||||||
@@ -55,14 +67,20 @@ notifications — **none** of these is enumerated across accounts. Each is reach
|
|||||||
The rule of thumb: **Access ≠ discovery.** You only union-query over graphs you were
|
The rule of thumb: **Access ≠ discovery.** You only union-query over graphs you were
|
||||||
already entitled to open.
|
already entitled to open.
|
||||||
|
|
||||||
## Listing = open/sync + ONE union query (never the ORM fan-out)
|
## Listing = a bounded set of per-doc ANCHORED reads (never a union-scan, never the ORM fan-out)
|
||||||
|
|
||||||
To produce a list:
|
To produce a list, take the **bounded, by-need** set of doc NURIs (the index-yielded
|
||||||
|
event NURIs, my own docs, a connection's shared NURIs) and read **each one with its
|
||||||
|
OWN anchored `sparql_query`** (`SELECT ?s ?p ?o WHERE { ?s ?p ?o }`, anchor = that
|
||||||
|
doc NURI, in parallel and tolerant per-doc). The anchor restricts the query to that
|
||||||
|
one repo's graph, so each read is O(1) in the doc's own size and INDEPENDENT of how
|
||||||
|
many other graphs the (possibly bloated / shared) session store holds.
|
||||||
|
|
||||||
1. **Open/sync** the relevant repos (the index-yielded event NURIs, my own docs, a
|
Do **NOT** run an **anchorless union-scan** (`SELECT … WHERE { GRAPH ?g { ?s ?p ?o } }`,
|
||||||
connection's shared NURIs). This is what puts them in the local union.
|
no anchor) over the local union: it iterates **every** named graph in the session
|
||||||
2. Run a **single** `sparql_query` with **NO anchor** over the LOCAL UNION, using a
|
store — O(wallet size) — so on a shared wallet that accumulates across runs it times
|
||||||
`GRAPH ?g { ... }` body so each result row is attributed to its source graph.
|
out (~90s observed). The read-set is already bounded and known; read exactly those
|
||||||
|
docs, anchored, and never scan the wallet.
|
||||||
|
|
||||||
Do **NOT** drive listing through the reactive ORM's per-document fan-out
|
Do **NOT** drive listing through the reactive ORM's per-document fan-out
|
||||||
(`orm_start_graph` over many graphs): a freshly-created or not-yet-synced graph in
|
(`orm_start_graph` over many graphs): a freshly-created or not-yet-synced graph in
|
||||||
@@ -141,11 +159,19 @@ above; a repo absent from `self.repos` throws `RepoNotFound` and is skipped.)
|
|||||||
|
|
||||||
## Implementation — `read-model.ts`
|
## Implementation — `read-model.ts`
|
||||||
|
|
||||||
`readModel.readUnion(docs)` implements this: (1) open/sync each doc via a per-doc
|
`readModel.readUnion(docs)` implements this: for each requested doc NURI (the
|
||||||
anchored `ASK` (tolerant — a doc that can't open is skipped, never aborting the
|
bounded by-need set), run — in parallel, tolerant per-doc (a doc that fails is
|
||||||
listing like the ORM fan-out would); (2) run ONE anchorless
|
skipped, never aborting the batch like the ORM fan-out would) — ONE **anchored**
|
||||||
`SELECT ?g ?s ?p ?o WHERE { GRAPH ?g { ?s ?p ?o . VALUES ?s { … } } }` over the
|
`SELECT ?s ?p ?o WHERE { ?s ?p ?o }` with `anchor = docNuri`. The anchor restricts
|
||||||
local union, constrained to the requested subjects (each entity's subject IRI IS
|
the query to that doc's graph (default graph), so it returns ONLY that doc's
|
||||||
its own document NURI). Returns the triples grouped per subject; the consumer maps
|
triples, O(1) per doc, independent of wallet size. There is **NO** anchorless
|
||||||
them to its types (e.g. Festipod's `readEntities`). Reactivity = the consumer
|
union-scan. Each entity's subject IRI IS its own document NURI, so the subject is
|
||||||
re-calls `readUnion` on its change signal (no reactive union query exists).
|
the anchor doc NURI; the result is grouped per subject (keeping the `UnionSubject[]`
|
||||||
|
shape: `subject`, `graph`, `props`). A ReadCap gate drops any doc the current user
|
||||||
|
may not read (defence-in-depth). The consumer maps the result to its types (e.g.
|
||||||
|
Festipod's `readEntities`). Reactivity = the consumer re-calls `readUnion` on its
|
||||||
|
change signal (no reactive union query exists).
|
||||||
|
|
||||||
|
> The name `readUnion` / `UnionSubject` is historical (it once ran a union query).
|
||||||
|
> The read is now **per-doc anchored**, bounded to the read-set — the "union" is only
|
||||||
|
> the logical concatenation of the per-doc results, never an anchorless graph scan.
|
||||||
|
|||||||
@@ -21,6 +21,35 @@ application fiction the lib maintains. On top of that one wallet the lib rebuild
|
|||||||
by emulation, the per-user stores + capabilities + inbox the consumer codes
|
by emulation, the per-user stores + capabilities + inbox the consumer codes
|
||||||
against.
|
against.
|
||||||
|
|
||||||
|
## Physical wallet vs virtual wallet — never enumerate the physical one
|
||||||
|
|
||||||
|
Because the emulation runs on ONE shared wallet, distinguish two levels:
|
||||||
|
|
||||||
|
- **Physical wallet** — the real NextGraph wallet everyone opens. Its local store
|
||||||
|
holds **every account's documents PLUS the lib's own internals** (the shim index,
|
||||||
|
the inbox docs, the discovery index) as named graphs. It **accumulates without
|
||||||
|
bound** across sessions/runs. **Listing / scanning "all documents" of the physical
|
||||||
|
wallet is meaningless AND O(size)** — it mixes every user's data with lib internals,
|
||||||
|
and it is exactly what a `sparql_query` with **no anchor** (`GRAPH ?g { … }`) does
|
||||||
|
(it spans every synced graph). **Never do it.** The physical wallet is a substrate,
|
||||||
|
not something to enumerate.
|
||||||
|
|
||||||
|
- **Virtual wallet** — the lib's emulation of **one user's** wallet: the set of
|
||||||
|
documents the shim attributes to that account (its per-scope index in
|
||||||
|
`store-registry.ts`). This is what "the user owns". Over a *virtual* wallet,
|
||||||
|
"**list my documents**" is meaningful and **bounded** (only that account's docs).
|
||||||
|
|
||||||
|
**Consequence for reads (see `read-model.md`):** to list a user's entities you
|
||||||
|
enumerate the **virtual** wallet — the account's scope index (bounded, O(my docs)),
|
||||||
|
NOT the physical union — then read those specific documents with a **per-doc anchored**
|
||||||
|
`sparql_query`. A non-empty / bloated physical wallet then costs nothing, because the
|
||||||
|
physical union is never scanned. Discovery (all public events) is the one bounded
|
||||||
|
enumeration hack and goes through the discovery **index**, not a physical scan.
|
||||||
|
|
||||||
|
At migration each virtual wallet becomes a real per-user wallet; the physical/virtual
|
||||||
|
distinction — and the "never enumerate the physical wallet" rule — dissolves into
|
||||||
|
native per-wallet reads.
|
||||||
|
|
||||||
## Two axes, never conflate them (store ≠ document)
|
## Two axes, never conflate them (store ≠ document)
|
||||||
|
|
||||||
The single most load-bearing distinction. Two **orthogonal** axes the
|
The single most load-bearing distinction. Two **orthogonal** axes the
|
||||||
|
|||||||
@@ -1,40 +1,48 @@
|
|||||||
/**
|
/**
|
||||||
* read-model — the LISTING primitive of the polyfill: open/sync a set of
|
* read-model — the LISTING primitive of the polyfill: read a BOUNDED, by-need set
|
||||||
* documents, then run ONE anchorless union `sparql_query` over the LOCAL UNION
|
* of documents, each with its OWN anchored `sparql_query`, and return the triples
|
||||||
* and return the triples grouped per subject. This is the mechanism documented
|
* grouped per subject. This is the mechanism documented in docs/read-model.md.
|
||||||
* in docs/read-model.md, verified against the real broker (T03.k probe): a
|
*
|
||||||
* `GRAPH ?g { ?s ?p ?o }` body with NO anchor spans every graph currently opened
|
* ── WHY per-doc ANCHORED, never an anchorless union-scan ───────────────────
|
||||||
* in the session store.
|
* An ANCHORED `sparql_query(sid, "SELECT ?s ?p ?o WHERE { ?s ?p ?o }", base, doc)`
|
||||||
|
* is restricted to the anchor repo's graph: `resolve_target_for_sparql(Repo)` →
|
||||||
|
* `Some(repo_graph_name)`, which becomes the query's DEFAULT graph. A body with NO
|
||||||
|
* `GRAPH` wrapper reads only that default graph → ONLY that doc's triples, O(1) per
|
||||||
|
* doc, INDEPENDENT of how many other graphs the local store holds.
|
||||||
|
*
|
||||||
|
* An ANCHORLESS query (`anchor` undefined → `UserSite` → `set_default_graph_as_union`)
|
||||||
|
* spans EVERY named graph currently in the session store. On a SHARED / bloated
|
||||||
|
* wallet that accumulates across runs, that is O(wallet size) → the observed ~90s
|
||||||
|
* timeouts. So the read path must NEVER union-scan all graphs: it reads exactly the
|
||||||
|
* bounded by-need set, one anchored query per doc.
|
||||||
|
*
|
||||||
|
* NB (verified, docs/read-model.md § probe step 4): an explicit `GRAPH ?g { … }`
|
||||||
|
* body iterates the NAMED graphs regardless of the default graph, so an anchor does
|
||||||
|
* NOT bound such a body. The per-doc read therefore uses a DEFAULT-GRAPH body (no
|
||||||
|
* `GRAPH` wrapper) so the anchor's one-repo restriction actually applies.
|
||||||
*
|
*
|
||||||
* ── WHY not the reactive ORM fan-out ──────────────────────────────────────
|
* ── WHY not the reactive ORM fan-out ──────────────────────────────────────
|
||||||
* `useShape({ graphs: […manyDocs] })` drives `orm_start_graph` over a fan-out of
|
* `useShape({ graphs: […manyDocs] })` drives `orm_start_graph` over a fan-out of
|
||||||
* per-entity graphs; a freshly-created / not-yet-synced doc in that fan-out makes
|
* per-entity graphs; a freshly-created / not-yet-synced doc in that fan-out makes
|
||||||
* `RepoNotFound` abort the whole subscription → the readyPromise never resolves →
|
* `RepoNotFound` abort the whole subscription → the readyPromise never resolves →
|
||||||
* the ~75s hang (docs/nextgraph-current-state.md § The ORM fan-out hang). Listing
|
* the ~75s hang (docs/nextgraph-current-state.md § The ORM fan-out hang). Listing
|
||||||
* MUST instead be a one-shot union `sparql_query`. There is no reactive union
|
* MUST instead be a set of one-shot anchored `sparql_query`s. There is no reactive
|
||||||
* query, so reactivity is assembled by RE-QUERYING on a change signal.
|
* union query, so reactivity is assembled by RE-QUERYING on a change signal.
|
||||||
*
|
|
||||||
* ── The two steps ─────────────────────────────────────────────────────────
|
|
||||||
* 1. OPEN/SYNC the docs. A repo is only in the union once opened into the
|
|
||||||
* session's oxigraph store. Opening is done PER-DOC via an anchored probe
|
|
||||||
* query (`resolveTargetForSparql(anchor)` opens that one repo) — per-doc, so a
|
|
||||||
* single unreachable doc fails in isolation instead of aborting a fan-out.
|
|
||||||
* 2. UNION QUERY. One anchorless `SELECT ?g ?s ?p ?o WHERE { GRAPH ?g { ?s ?p ?o } }`
|
|
||||||
* over the local union; group rows by subject.
|
|
||||||
*
|
*
|
||||||
* ── GENERIC by construction ───────────────────────────────────────────────
|
* ── GENERIC by construction ───────────────────────────────────────────────
|
||||||
* Zero application domain here: the consumer passes the doc NURIs to open (from
|
* Zero application domain here: the consumer passes the doc NURIs to read (from
|
||||||
* the discovery index for public events, or its own scope docs for my-entities)
|
* the discovery index for public events, or its own scope docs for my-entities)
|
||||||
* and interprets the returned per-subject property bags. All NextGraph I/O routes
|
* and interprets the returned per-subject property bags. All NextGraph I/O routes
|
||||||
* through the T01.a `docs` primitives (the REAL injected `ng`), so this module
|
* through the T01.a `docs` primitives (the REAL injected `ng`), so this module
|
||||||
* imports no `@ng-org` package.
|
* imports no `@ng-org` package.
|
||||||
*
|
*
|
||||||
* At migration the union query is unchanged (native SPARQL union over the wallet);
|
* At the real multi-store migration the per-doc anchored read is unchanged (native
|
||||||
* only the "open/sync" step swaps to opening real per-user store repos by cap.
|
* SPARQL, anchored to one repo); only bringing a repo into the session (open by cap)
|
||||||
|
* changes — the anchored query already resolves a same-session repo directly.
|
||||||
*/
|
*/
|
||||||
|
|
||||||
import { docCreate, sparqlUpdate, sparqlQuery } from "./docs";
|
import { docCreate, sparqlUpdate, sparqlQuery } from "./docs";
|
||||||
import { getStoreRegistryDeps, getCaps, getCurrentUser } from "./polyfill";
|
import { getCaps, getCurrentUser, getStoreRegistryDeps } from "./polyfill";
|
||||||
import { assertNuri } from "./sparql";
|
import { assertNuri } from "./sparql";
|
||||||
import type { Nuri } from "./types";
|
import type { Nuri } from "./types";
|
||||||
|
|
||||||
@@ -44,11 +52,11 @@ import type { Nuri } from "./types";
|
|||||||
void docCreate;
|
void docCreate;
|
||||||
void sparqlUpdate;
|
void sparqlUpdate;
|
||||||
|
|
||||||
/** One subject read from the union, with its properties (predicate → values). */
|
/** One subject read from a doc, with its properties (predicate → values). */
|
||||||
export interface UnionSubject {
|
export interface UnionSubject {
|
||||||
/** The subject IRI (`?s`). */
|
/** The subject IRI (`?s`) — in the polyfill, the doc's own NURI. */
|
||||||
subject: string;
|
subject: string;
|
||||||
/** The graph the subject was read from (`?g` — the repo graph name). */
|
/** The graph (doc NURI) the subject was read from. */
|
||||||
graph: string;
|
graph: string;
|
||||||
/** predicate IRI → the list of object values (literals or IRIs) for it. */
|
/** predicate IRI → the list of object values (literals or IRIs) for it. */
|
||||||
props: Record<string, string[]>;
|
props: Record<string, string[]>;
|
||||||
@@ -72,93 +80,92 @@ async function sessionId(): Promise<string> {
|
|||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Touch ONE doc with a cheap anchored `ASK`, tolerant per-doc.
|
* Read ONE doc with an ANCHORED default-graph query, tolerant per-doc.
|
||||||
*
|
*
|
||||||
* VERIFIED (T03.k, see docs/nextgraph-current-state.md § *A repo is only queryable
|
* The anchor (`doc` NURI) restricts the query to that repo's graph as the DEFAULT
|
||||||
* once OPENED/synced*): the current JS SDK exposes **no primitive that SYNCS an
|
* graph (`resolve_target_for_sparql(Repo)` → `Some(repo_graph_name)`); a body with
|
||||||
* unknown repo**. An anchored `sparql_query` resolves via `resolve_target_for_sparql`
|
* NO `GRAPH` wrapper reads exactly that default graph → ONLY this doc's triples.
|
||||||
* → `self.repos.get(id).ok_or(RepoNotFound)` — it never PULLS an absent repo (and
|
* This is O(1) in the doc's own size and INDEPENDENT of the rest of the (possibly
|
||||||
* neither do `doc_subscribe` nor `orm_start_graph`; the real loader
|
* bloated / shared) session store — it never iterates other graphs.
|
||||||
* `load_repo_from_read_cap` is `pub(crate)`, unexposed). So this `ASK` cannot sync:
|
*
|
||||||
* on a repo already in `self.repos` (every doc `doc_create`d this session — which,
|
* All same-session repos (every doc `doc_create`d this session — in the mono-wallet
|
||||||
* in the mono-wallet polyfill, is ALL of them) it is a no-op that confirms presence;
|
* polyfill, ALL of them) are already in `self.repos`, so the anchored query resolves
|
||||||
* on a genuinely-absent repo it throws `RepoNotFound` HERE, in isolation, and the
|
* the repo directly with no separate open. A genuinely-absent repo throws
|
||||||
* doc is skipped — never aborting the listing of the others (the ORM fan-out's
|
* `RepoNotFound` HERE, in isolation, and the doc is skipped — never aborting the
|
||||||
* failure mode). Returns whether the touch succeeded.
|
* others (unlike the ORM fan-out). Returns the doc's rows, or `[]` on failure.
|
||||||
*
|
*
|
||||||
* At the real multi-store migration this becomes a real sync: opening a per-user
|
* At the real multi-store migration this becomes a real sync: opening a per-user
|
||||||
* store repo by cap is a native broker fetch (`verifier.rs:1423` `OpenRepo` TODO).
|
* store repo by cap is a native broker fetch (`verifier.rs:1423` `OpenRepo` TODO).
|
||||||
*/
|
*/
|
||||||
async function openDoc(sid: string, doc: Nuri): Promise<boolean> {
|
async function readDoc(
|
||||||
|
sid: string,
|
||||||
|
doc: Nuri,
|
||||||
|
): Promise<Array<Record<string, { value: string } | undefined>>> {
|
||||||
try {
|
try {
|
||||||
const nuri = assertNuri(doc);
|
const nuri = assertNuri(doc);
|
||||||
await sparqlQuery(sid, "ASK { ?s ?p ?o }", undefined, nuri);
|
// Anchored to `nuri` → default graph = this repo. NO `GRAPH ?g` wrapper, so
|
||||||
return true;
|
// the anchor's one-repo restriction applies (an explicit `GRAPH ?g` body would
|
||||||
|
// iterate all named graphs regardless of the anchor — see docs § probe step 4).
|
||||||
|
const res = await sparqlQuery(
|
||||||
|
sid,
|
||||||
|
"SELECT ?s ?p ?o WHERE { ?s ?p ?o }",
|
||||||
|
undefined,
|
||||||
|
nuri,
|
||||||
|
);
|
||||||
|
return bindings(res);
|
||||||
} catch (error) {
|
} catch (error) {
|
||||||
console.error("[read-model] open failed for", doc, error);
|
console.error("[read-model] read failed for", doc, error);
|
||||||
return false;
|
return [];
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* OPEN/SYNC a set of docs, then read the LOCAL UNION with ONE anchorless
|
* Read a BOUNDED, by-need set of docs — each with its OWN anchored query — and
|
||||||
* `sparql_query`, grouped per subject. `docs` are the NURIs to bring into the
|
* return the triples grouped per subject. `docs` are the NURIs to read (the
|
||||||
* union (the consumer resolves them by need — index for public, own scope docs
|
* consumer resolves them by need — index for public, own scope docs for mine).
|
||||||
* for mine). Docs that fail to open are skipped (see {@link openDoc}).
|
* Docs that fail are skipped (see {@link readDoc}); a failing doc never aborts the
|
||||||
|
* batch.
|
||||||
*
|
*
|
||||||
* The union query is anchorless (`anchor` undefined → LOCAL UNION) with an
|
* NEVER an anchorless union-scan over all graphs (which is O(wallet size) and wrong
|
||||||
* explicit `GRAPH ?g { ?s ?p ?o }` body, so it spans every opened graph and
|
* on a shared / bloated wallet). Each doc is read with an anchored default-graph
|
||||||
* attributes each row to its source graph.
|
* query, O(1) per doc, independent of wallet size — a non-empty wallet no longer
|
||||||
|
* matters. Reads run in parallel via `Promise.all`.
|
||||||
*/
|
*/
|
||||||
export async function readUnion(docs: Nuri[]): Promise<UnionSubject[]> {
|
export async function readUnion(docs: Nuri[]): Promise<UnionSubject[]> {
|
||||||
const sid = await sessionId();
|
const sid = await sessionId();
|
||||||
const unique = [...new Set(docs.filter(Boolean))];
|
const unique = [...new Set(docs.filter(Boolean))];
|
||||||
// 1. OPEN/SYNC — per-doc, tolerant (a bad doc never aborts the whole list).
|
|
||||||
await Promise.all(unique.map((d) => openDoc(sid, d)));
|
|
||||||
|
|
||||||
if (unique.length === 0) return [];
|
if (unique.length === 0) return [];
|
||||||
|
|
||||||
// 2. UNION QUERY — ONE anchorless query over the local union. The union spans
|
// One anchored query per doc, in parallel, tolerant (a bad doc yields []).
|
||||||
// ALL opened graphs (including the shim/inbox internals), so constrain it to
|
const perDoc = await Promise.all(
|
||||||
// the requested subjects with a VALUES block. In the polyfill each entity's
|
unique.map(async (d) => ({ doc: assertNuri(d), rows: await readDoc(sid, d) })),
|
||||||
// subject IRI IS its own document NURI (writeEntity uses the doc NURI as the
|
);
|
||||||
// subject), so the requested doc NURIs ARE the subjects to read — precise and
|
|
||||||
// independent of the repo_graph_name overlay suffix on ?g.
|
|
||||||
const values = unique.map((d) => `<${assertNuri(d)}>`).join(" ");
|
|
||||||
const query =
|
|
||||||
`SELECT ?g ?s ?p ?o WHERE { GRAPH ?g { ?s ?p ?o . VALUES ?s { ${values} } } }`;
|
|
||||||
let rows: Array<Record<string, { value: string } | undefined>> = [];
|
|
||||||
try {
|
|
||||||
rows = bindings(await sparqlQuery(sid, query, undefined, undefined));
|
|
||||||
} catch (error) {
|
|
||||||
console.error("[read-model] union query failed:", error);
|
|
||||||
return [];
|
|
||||||
}
|
|
||||||
|
|
||||||
// Cap gate (defence-in-depth). The union spans every OPENED graph, so a subject
|
// Cap gate (defence-in-depth). A doc whose read policy the current user may not
|
||||||
// whose DOCUMENT is under a read policy the current user may not satisfy must be
|
// satisfy is dropped. Isolation holds BY CONSTRUCTION (the app only resolves docs
|
||||||
// dropped — isolation holds BY CONSTRUCTION (the app only resolves docs it is
|
// it is entitled to) AND BY FILTER here. Generic: the lib owns the cap registry;
|
||||||
// entitled to) AND BY FILTER here. Generic: the lib owns the cap registry; a doc
|
// a doc under no policy (`!governsRead`) flows through unchanged. In this polyfill
|
||||||
// under no policy (`!governsRead`) flows through unchanged. In this polyfill each
|
// each subject IRI IS its own document NURI, so the cap key is the doc NURI.
|
||||||
// subject IRI IS its own document NURI, so the cap key is the subject.
|
|
||||||
const caps = getCaps();
|
const caps = getCaps();
|
||||||
const user = getCurrentUser();
|
const user = getCurrentUser();
|
||||||
const denied = (doc: string): boolean => caps.governsRead(doc) && !caps.canRead(doc, user);
|
|
||||||
|
|
||||||
const bySubject = new Map<string, UnionSubject>();
|
const bySubject = new Map<string, UnionSubject>();
|
||||||
|
for (const { doc, rows } of perDoc) {
|
||||||
|
if (caps.governsRead(doc) && !caps.canRead(doc, user)) continue;
|
||||||
|
// Anchored to `doc`, so every row belongs to `doc`; the subject IS the doc NURI
|
||||||
|
// (writeEntity invariant). Pin subject/graph to the doc NURI (the anchor), which
|
||||||
|
// is stable regardless of the repo_graph_name overlay suffix the store carries.
|
||||||
for (const row of rows) {
|
for (const row of rows) {
|
||||||
const s = row.s?.value;
|
|
||||||
const p = row.p?.value;
|
const p = row.p?.value;
|
||||||
const o = row.o?.value;
|
const o = row.o?.value;
|
||||||
const g = row.g?.value ?? "";
|
if (!p || o === undefined) continue;
|
||||||
if (!s || !p || o === undefined) continue;
|
let entry = bySubject.get(doc);
|
||||||
if (denied(s)) continue;
|
|
||||||
let entry = bySubject.get(s);
|
|
||||||
if (!entry) {
|
if (!entry) {
|
||||||
entry = { subject: s, graph: g, props: {} };
|
entry = { subject: doc, graph: doc, props: {} };
|
||||||
bySubject.set(s, entry);
|
bySubject.set(doc, entry);
|
||||||
}
|
}
|
||||||
(entry.props[p] ??= []).push(o);
|
(entry.props[p] ??= []).push(o);
|
||||||
}
|
}
|
||||||
|
}
|
||||||
return [...bySubject.values()];
|
return [...bySubject.values()];
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -2,32 +2,28 @@ import { test, expect, mock } from "bun:test";
|
|||||||
import { readUnion } from "../src/read-model";
|
import { readUnion } from "../src/read-model";
|
||||||
import { configure, configureStoreRegistry } from "../src/polyfill";
|
import { configure, configureStoreRegistry } from "../src/polyfill";
|
||||||
|
|
||||||
// A fake `ng` whose sparql_query answers the ANCHORLESS union query with the
|
// A fake `ng` whose sparql_query answers the ANCHORED per-doc query (SELECT ?s ?p ?o
|
||||||
// triples of the requested subjects, and the anchored ASK (open step) with an
|
// WHERE { ?s ?p ?o }, anchor = the doc NURI) with ONLY that doc's triples. There is
|
||||||
// empty result. Each entity subject IRI IS its own document NURI (writeEntity
|
// NO anchorless union scan: each doc is read independently by its own anchor. Each
|
||||||
// convention), so the fixture keys triples by the doc NURI.
|
// entity subject IRI IS its own document NURI (writeEntity convention), so the
|
||||||
|
// fixture keys triples by the doc NURI and returns them for the matching anchor.
|
||||||
function fakeNgWith(triplesByDoc: Record<string, Array<[string, string]>>) {
|
function fakeNgWith(triplesByDoc: Record<string, Array<[string, string]>>) {
|
||||||
return {
|
return {
|
||||||
doc_create: mock(async () => "did:ng:o:new"),
|
doc_create: mock(async () => "did:ng:o:new"),
|
||||||
sparql_update: mock(async () => undefined),
|
sparql_update: mock(async () => undefined),
|
||||||
sparql_query: mock(async (_sid: string, query: string, _base: unknown, anchor: unknown) => {
|
sparql_query: mock(async (_sid: string, _query: string, _base: unknown, anchor: unknown) => {
|
||||||
// The open step is `ASK { ?s ?p ?o }` with an anchor → return a truthy ASK.
|
// Every read is ANCHORED to one doc NURI — never anchorless.
|
||||||
if (query.startsWith("ASK")) return { boolean: true };
|
if (anchor === undefined) {
|
||||||
// The union query is anchorless (anchor undefined) with a VALUES ?s block.
|
throw new Error("read-model must NEVER run an anchorless (union) query");
|
||||||
if (anchor !== undefined) return { results: { bindings: [] } };
|
}
|
||||||
const bindings: Array<Record<string, { value: string }>> = [];
|
const doc = anchor as string;
|
||||||
for (const [doc, triples] of Object.entries(triplesByDoc)) {
|
const triples = triplesByDoc[doc];
|
||||||
// Only surface docs whose NURI is named in the VALUES block.
|
if (!triples) return { results: { bindings: [] } };
|
||||||
if (!query.includes(`<${doc}>`)) continue;
|
const bindings = triples.map(([p, o]) => ({
|
||||||
for (const [p, o] of triples) {
|
|
||||||
bindings.push({
|
|
||||||
g: { value: `${doc}:graph` },
|
|
||||||
s: { value: doc },
|
s: { value: doc },
|
||||||
p: { value: p },
|
p: { value: p },
|
||||||
o: { value: o },
|
o: { value: o },
|
||||||
});
|
}));
|
||||||
}
|
|
||||||
}
|
|
||||||
return { results: { bindings } };
|
return { results: { bindings } };
|
||||||
}),
|
}),
|
||||||
};
|
};
|
||||||
@@ -46,23 +42,26 @@ function inject(triplesByDoc: Record<string, Array<[string, string]>>) {
|
|||||||
const TYPE = "http://www.w3.org/1999/02/22-rdf-syntax-ns#type";
|
const TYPE = "http://www.w3.org/1999/02/22-rdf-syntax-ns#type";
|
||||||
const FP = "http://festipod.org/";
|
const FP = "http://festipod.org/";
|
||||||
|
|
||||||
test("readUnion opens each doc then runs ONE anchorless union query", async () => {
|
test("readUnion reads each doc with its OWN anchored query (never anchorless)", async () => {
|
||||||
const ng = inject({
|
const ng = inject({
|
||||||
"did:ng:o:a": [[TYPE, `${FP}Event`], [`${FP}title`, "A"]],
|
"did:ng:o:a": [[TYPE, `${FP}Event`], [`${FP}title`, "A"]],
|
||||||
"did:ng:o:b": [[TYPE, `${FP}Event`], [`${FP}title`, "B"]],
|
"did:ng:o:b": [[TYPE, `${FP}Event`], [`${FP}title`, "B"]],
|
||||||
});
|
});
|
||||||
const subjects = await readUnion(["did:ng:o:a", "did:ng:o:b"]);
|
const subjects = await readUnion(["did:ng:o:a", "did:ng:o:b"]);
|
||||||
|
|
||||||
// Two per-doc ASK opens + one anchorless union query = 3 sparql_query calls.
|
// One anchored query per doc = 2 sparql_query calls, each anchored (c[3] set).
|
||||||
expect(ng.sparql_query).toHaveBeenCalledTimes(3);
|
expect(ng.sparql_query).toHaveBeenCalledTimes(2);
|
||||||
const anchorless = ng.sparql_query.mock.calls.filter(
|
const anchored = ng.sparql_query.mock.calls.filter((c: unknown[]) => c[3] !== undefined);
|
||||||
(c: unknown[]) => !String(c[1]).startsWith("ASK") && c[3] === undefined,
|
expect(anchored.length).toBe(2);
|
||||||
|
// The anchors are exactly the requested doc NURIs.
|
||||||
|
expect(new Set(anchored.map((c: unknown[]) => c[3]))).toEqual(
|
||||||
|
new Set(["did:ng:o:a", "did:ng:o:b"]),
|
||||||
);
|
);
|
||||||
expect(anchorless.length).toBe(1);
|
|
||||||
|
|
||||||
expect(subjects.length).toBe(2);
|
expect(subjects.length).toBe(2);
|
||||||
const a = subjects.find((s) => s.subject === "did:ng:o:a")!;
|
const a = subjects.find((s) => s.subject === "did:ng:o:a")!;
|
||||||
expect(a.props[`${FP}title`]).toEqual(["A"]);
|
expect(a.props[`${FP}title`]).toEqual(["A"]);
|
||||||
|
expect(a.graph).toBe("did:ng:o:a");
|
||||||
});
|
});
|
||||||
|
|
||||||
test("readUnion groups predicates per subject", async () => {
|
test("readUnion groups predicates per subject", async () => {
|
||||||
@@ -86,12 +85,12 @@ test("readUnion returns [] for an empty doc set (no query)", async () => {
|
|||||||
expect(ng.sparql_query).toHaveBeenCalledTimes(0);
|
expect(ng.sparql_query).toHaveBeenCalledTimes(0);
|
||||||
});
|
});
|
||||||
|
|
||||||
test("a doc that fails to open is skipped, not aborting the union", async () => {
|
test("a doc that fails to read is skipped, not aborting the batch", async () => {
|
||||||
const ng = fakeNgWith({ "did:ng:o:ok": [[TYPE, `${FP}Event`], [`${FP}title`, "ok"]] });
|
const ng = fakeNgWith({ "did:ng:o:ok": [[TYPE, `${FP}Event`], [`${FP}title`, "ok"]] });
|
||||||
// Make the OPEN (ASK) throw for the bad doc only.
|
|
||||||
const orig = ng.sparql_query;
|
const orig = ng.sparql_query;
|
||||||
|
// Make the anchored read throw for the bad doc only.
|
||||||
ng.sparql_query = mock(async (sid: string, query: string, base: unknown, anchor: unknown) => {
|
ng.sparql_query = mock(async (sid: string, query: string, base: unknown, anchor: unknown) => {
|
||||||
if (query.startsWith("ASK") && anchor === "did:ng:o:bad") throw new Error("RepoNotFound");
|
if (anchor === "did:ng:o:bad") throw new Error("RepoNotFound");
|
||||||
return orig(sid, query, base, anchor);
|
return orig(sid, query, base, anchor);
|
||||||
}) as any;
|
}) as any;
|
||||||
configure({ ng: ng as any, useShape: (() => {}) as any });
|
configure({ ng: ng as any, useShape: (() => {}) as any });
|
||||||
@@ -101,6 +100,6 @@ test("a doc that fails to open is skipped, not aborting the union", async () =>
|
|||||||
});
|
});
|
||||||
|
|
||||||
const subjects = await readUnion(["did:ng:o:ok", "did:ng:o:bad"]);
|
const subjects = await readUnion(["did:ng:o:ok", "did:ng:o:bad"]);
|
||||||
// The bad doc opened-failed but the good one still lists.
|
// The bad doc failed its read but the good one still lists.
|
||||||
expect(subjects.map((s) => s.subject)).toEqual(["did:ng:o:ok"]);
|
expect(subjects.map((s) => s.subject)).toEqual(["did:ng:o:ok"]);
|
||||||
});
|
});
|
||||||
|
|||||||
Reference in New Issue
Block a user