docs: query capability (local-union sparql_query) + the read model
Source-verified against nextgraph-rs: - nextgraph-current-state.md: NextGraph keeps ONE local oxigraph store per session; each synced repo is a named graph. sparql_query with NO anchor (UserSite/None) queries the UNION of all synced graphs (set_default_graph_as _union); with an anchor it is restricted to one repo. Union is read-only (updates need a doc anchor). No reactive SPARQL (one-shot). Root cause of the ORM fan-out hang: orm_start_graph opens every graph in scope; a fresh/unsynced per-entity doc → RepoNotFound aborts the subscription → the 75s never-fires. - read-model.md (new): the read model — events via the global index (the one enumeration hack); everything else by following a shared graph, opened/synced, then listed via a single anchorless union sparql_query (never the ORM per-doc fan-out); reactivity via re-query on a doc_subscribe/ORM change signal. Plus the minimal broker probe to confirm the union behavior. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Sylvain Duchesne
@@ -129,6 +129,97 @@ path NOT taken). A related exposed primitive: `social_query_start` (a federated
|
||||
query via inbox up to `degree` hops) exists but is limited to **contacts** — it
|
||||
does not cover an anonymous notification to a non-connected host.
|
||||
|
||||
## The query capability — ONE local store, named graphs, union queries
|
||||
|
||||
The single fact that makes read-time *listing* possible on the shared wallet, and
|
||||
the reason the reactive ORM must **not** be used as the listing primitive. Verified
|
||||
directly in `nextgraph-rs`.
|
||||
|
||||
### One oxigraph Store per session; each repo is a NAMED GRAPH
|
||||
|
||||
- The verifier keeps **ONE** local oxigraph `Store` per session:
|
||||
`graph_dataset: Option<Store>` (`engine/verifier/src/verifier.rs:94`).
|
||||
- Every synced repo's data is inserted into that one store as a **distinct NAMED
|
||||
GRAPH keyed by the repo** — `update_graph` writes each repo's main-branch triples
|
||||
under `NuriV0::repo_graph_name(&repo_id, &overlay_id)`
|
||||
(`engine/verifier/src/commits/transaction.rs:646-701`, the `ov_graphname` /
|
||||
`repo_graph_name` around line 669). So all opened repos coexist as named graphs in
|
||||
one dataset — a `GRAPH ?g { ... }` body can span them.
|
||||
|
||||
### The NURI target decides scope: one repo vs the LOCAL UNION
|
||||
|
||||
`sparql_query`'s scope is resolved from the NURI target by
|
||||
`resolve_target_for_sparql` (`engine/verifier/src/request_processor.rs:256-285`):
|
||||
|
||||
- `Repo(repo_id)` / `PrivateStore` → `Some(repo_graph_name)` = **ONE repo's graph**.
|
||||
- `UserSite` / `None` → `Ok(None)` = the **UNION of all named graphs**. That `None`
|
||||
is passed to oxigraph's `store.query(parsed, default_graph)`
|
||||
(`engine/oxigraph/src/oxigraph/store.rs:200`) as the default graph, and
|
||||
`sparql_query` first calls `dataset.set_default_graph_as_union()` when the query
|
||||
has no explicit dataset (`request_processor.rs:656-675`, the
|
||||
`if dataset.has_no_default_dataset()` block). Union = "every named graph currently
|
||||
in the store".
|
||||
|
||||
### The wasm binding defaults the target to UserSite when no `nuri` is passed
|
||||
|
||||
The binding (`sdk/js/lib-wasm/src/lib.rs` — both the nodejs variant at ~`350-405`
|
||||
and the web variant at ~`553-610`) reads the target from the `nuri` arg: a string
|
||||
`nuri` is parsed; an **absent** `nuri` falls back to
|
||||
`NuriV0::new_entire_user_site()` = `UserSite`
|
||||
(`engine/net/src/app_protocol.rs:488-490`). Therefore:
|
||||
|
||||
> **`sparql_query(sid, query, base, /*anchor*/ undefined)` queries the LOCAL UNION
|
||||
> across all synced/opened graphs; with a string anchor it is restricted to that one
|
||||
> repo.** A `GRAPH ?g { ... }` body then spans/attributes across the local graphs.
|
||||
|
||||
### A repo is only queryable once OPENED/synced into the store
|
||||
|
||||
A repo's triples enter `graph_dataset` (hence the union) only after the repo is
|
||||
opened/synced into `self.repos` **and** its commits applied via `update_graph`.
|
||||
Paths that open a repo: `doc_create` (own docs), bootstrap /
|
||||
`load_repo_from_read_cap`, or being followed from a store's `ldp:contains` / a
|
||||
shared cap / an inbox. Opening requires possessing the repo's **NURI + ReadCap** —
|
||||
there is **no store-level read inheritance** (see § Capability / ReadCap
|
||||
granularity). *(INFERRED: that "following a graph reference makes a previously
|
||||
unknown repo known/openable" is the one step not read verbatim in source; the
|
||||
open-requires-cap and no-inheritance facts around it ARE verified.)*
|
||||
|
||||
### The union is READ-ONLY — writes must target one document
|
||||
|
||||
`resolve_target_for_sparql(update=true)` returns `InvalidTarget` for `UserSite` /
|
||||
`None` (`request_processor.rs:275-282`). So `sparql_update` cannot write "to the
|
||||
union": every write must name **one** document's `@graph` — exactly what the
|
||||
polyfill's `docs.sparqlUpdate` already does.
|
||||
|
||||
### No reactive SPARQL — `sparql_query` is one-shot
|
||||
|
||||
`sparql_query` is non-streamed: it computes a `QueryResults` and returns once
|
||||
(`lib-wasm/src/lib.rs:352-405` / `553-610`). There is **no** "subscribe to a union
|
||||
query". The only reactive primitives are the streamed ones: `orm_start_graph`,
|
||||
`orm_start_discrete`, `doc_subscribe`, `app_request_stream`.
|
||||
|
||||
### The ORM fan-out hang — verified root cause
|
||||
|
||||
The reactive ORM is structurally unfit for a fan-out of per-entity / not-yet-synced
|
||||
graphs, and this is *why* subscribing such a fan-out hangs:
|
||||
|
||||
- `OrmStartGraph` first loops over EVERY graph in the requested scope and calls
|
||||
`open_for_target(&nuri.target, /*publisher*/ true)` on each
|
||||
(`request_processor.rs:53-66`), and `orm/graph/initialize.rs` does the same
|
||||
fan-out again for the graphs the ORM discovers (~`125-128`).
|
||||
- `open_for_target` → `resolve_target` → `self.repos.get(repo_id).ok_or(RepoNotFound)`
|
||||
(`request_processor.rs:286-294` calling `resolve_target` at `:147`, the
|
||||
`RepoNotFound` at `:155/:163`).
|
||||
- A **freshly-created per-entity doc**, or any **not-yet-synced other-account doc**,
|
||||
is absent from `self.repos` → `RepoNotFound` propagates through the `?` and
|
||||
**aborts the whole `orm_start_graph`**. The subscription never emits its initial →
|
||||
the ORM `readyPromise` never resolves → the multi-second hang observed (≈75s)
|
||||
when subscribing a fan-out of per-entity graphs.
|
||||
|
||||
**Consequence:** passing per-entity / unsynced graphs to the reactive ORM is broken.
|
||||
Listing must go through a one-shot union `sparql_query` instead — see
|
||||
[`read-model.md`](./read-model.md).
|
||||
|
||||
## JS SDK limits (`@ng-org/web`)
|
||||
|
||||
`@ng-org/web` (verified `0.1.2-alpha.13` = `upstream/main` at 2026-05-21, the
|
||||
|
||||
@@ -0,0 +1,114 @@
|
||||
# The READ MODEL the polyfill implements
|
||||
|
||||
How the polyfill turns "give me my lists" into concrete NextGraph reads on the
|
||||
shared wallet. This is a **design decision**, grounded entirely in the query
|
||||
capability documented in
|
||||
[`nextgraph-current-state.md`](./nextgraph-current-state.md) § *The query
|
||||
capability*. The consumer (Festipod) never sees any of this: it asks
|
||||
`@ng-eventually/client` for its lists **by need** and trusts the answer — the whole
|
||||
read mechanism lives here, in the polyfill.
|
||||
|
||||
The governing constraints (all verified in `nextgraph-rs`, cited there):
|
||||
|
||||
- One local oxigraph store per session; every opened repo is a **named graph**.
|
||||
- `sparql_query` with **no anchor** → the **LOCAL UNION** of all opened graphs;
|
||||
with an anchor → **one** repo. Union is **read-only**.
|
||||
- A repo is queryable **only after it is opened/synced** (needs its NURI + ReadCap;
|
||||
no store-level read inheritance).
|
||||
- **No reactive union query**, and the reactive ORM **hangs** if handed a per-entity
|
||||
/ unsynced graph fan-out (`RepoNotFound` aborts `orm_start_graph`).
|
||||
|
||||
## Two read regimes — enumerate vs follow
|
||||
|
||||
There is **no cross-wallet read** in current NextGraph, so nothing is globally
|
||||
enumerable "for free". The polyfill splits every list into one of two regimes:
|
||||
|
||||
### Events (all public) = the GLOBAL INDEX — the ONE enumeration hack
|
||||
|
||||
Public events are the only thing enumerated across accounts, via the emulated
|
||||
discovery index (`discovery.readIndex`, see
|
||||
[`simulation.md`](./simulation.md) § *Emulated discovery index*). This is the ONE
|
||||
"hack", and it is justified precisely because P2P has no cross-wallet read: without
|
||||
a shared index a client could never learn that another account's public event-doc
|
||||
**exists**. `readIndex` yields the event-doc **NURIs** to open/sync; those repos
|
||||
then enter the local union and become union-queryable.
|
||||
|
||||
### Everything else = FOLLOW a graph, never enumerate across accounts
|
||||
|
||||
My participations / my profile, a connection's shared protected data, my
|
||||
notifications — **none** of these is enumerated across accounts. Each is reached by
|
||||
**what is already reachable to me**:
|
||||
|
||||
- **my own docs** (always in `self.repos`);
|
||||
- docs reachable via a **connection's shared cap** (a bilateral connection surfaces
|
||||
the peer's protected NURIs — see the bilateral connection registry in
|
||||
[`simulation.md`](./simulation.md));
|
||||
- my **inbox** (deposits addressed to me).
|
||||
|
||||
The rule of thumb: **Access ≠ discovery.** You only union-query over graphs you were
|
||||
already entitled to open.
|
||||
|
||||
## Listing = open/sync + ONE union query (never the ORM fan-out)
|
||||
|
||||
To produce a list:
|
||||
|
||||
1. **Open/sync** the relevant repos (the index-yielded event NURIs, my own docs, a
|
||||
connection's shared NURIs). This is what puts them in the local union.
|
||||
2. Run a **single** `sparql_query` with **NO anchor** over the LOCAL UNION, using a
|
||||
`GRAPH ?g { ... }` body so each result row is attributed to its source graph.
|
||||
|
||||
Do **NOT** drive listing through the reactive ORM's per-document fan-out
|
||||
(`orm_start_graph` over many graphs): a freshly-created or not-yet-synced graph in
|
||||
the fan-out makes `RepoNotFound` abort the whole subscription → the readyPromise
|
||||
never resolves → the ~75s hang (root cause verified in
|
||||
[`nextgraph-current-state.md`](./nextgraph-current-state.md) § *The ORM fan-out
|
||||
hang*).
|
||||
|
||||
## Reactivity = re-query on a change signal (no reactive union)
|
||||
|
||||
There is **no reactive union query**. So reactivity is assembled:
|
||||
|
||||
- keep a lightweight reactive subscription — `doc_subscribe`, or the ORM on an
|
||||
**already-opened single store** (never a per-entity fan-out) — on the synced docs;
|
||||
- on its change signal, **re-run** the one-shot union `sparql_query`.
|
||||
|
||||
Keep the reactive ORM strictly to already-opened single stores; it is a change
|
||||
*signal* source here, not the list source.
|
||||
|
||||
## The boundary with the consumer
|
||||
|
||||
Festipod asks the SDK for its lists by need (`listMyMeetingPoints()`,
|
||||
`listEvents()`, …) and trusts the returned set. It never constructs a NURI, never
|
||||
picks the union-vs-anchor mode, never touches the ORM. Open/sync + union-query +
|
||||
re-query-on-signal all live in the polyfill.
|
||||
|
||||
## Minimal broker probe (confirms the union behaviour)
|
||||
|
||||
The one experiment that pins down union vs anchor, to run against a real broker:
|
||||
|
||||
1. `doc_create` two docs **A** and **B** (own docs → both opened into the session
|
||||
store).
|
||||
2. `sparql_update` a **distinct** triple into each (target A's `@graph`, then B's).
|
||||
3. **No anchor** — expect BOTH graphs:
|
||||
|
||||
```
|
||||
sparql_query(
|
||||
sid,
|
||||
"SELECT ?g ?s ?p ?o WHERE { GRAPH ?g { ?s ?p ?o } }",
|
||||
undefined /* base */,
|
||||
undefined /* anchor → UserSite → LOCAL UNION */
|
||||
)
|
||||
// → rows from BOTH A's and B's graphs
|
||||
```
|
||||
|
||||
4. **Anchor = A** — expect only A:
|
||||
|
||||
```
|
||||
sparql_query(sid, "SELECT ?g ?s ?p ?o WHERE { GRAPH ?g { ?s ?p ?o } }",
|
||||
undefined, A /* string NURI → one repo */)
|
||||
// → rows from A's graph only
|
||||
```
|
||||
|
||||
If (3) returns both and (4) returns only A, the union read model above holds as
|
||||
implemented in `resolve_target_for_sparql` /
|
||||
`set_default_graph_as_union`.
|
||||
Reference in New Issue
Block a user