diff --git a/README.md b/README.md index 1dd7cc9..05e7c8c 100644 --- a/README.md +++ b/README.md @@ -29,6 +29,10 @@ Docs (this library's own engineering doctrine, under [`docs/`](./docs/)): behaviour on ONE shared wallet (shim, per-document ReadCaps, emulated inbox+curator, write guard, faux login, the two axes, the double-proxy constraint). +- [`docs/read-model.md`](./docs/read-model.md) — the READ MODEL the polyfill + implements: events via the global index, everything else by following a shared + graph; listing via a one-shot union `sparql_query` (never the ORM fan-out, which + hangs); reactivity via re-query on a change signal. - [`docs/decisions/`](./docs/decisions/) — historical current-SDK ADRs (private-store scope, SPARQL delete, shared-wallet login, discovery mechanism). - [`docs/fork-inbox-fallback.md`](./docs/fork-inbox-fallback.md) — the Rust-patch / diff --git a/docs/nextgraph-current-state.md b/docs/nextgraph-current-state.md index f1a75d2..4790263 100644 --- a/docs/nextgraph-current-state.md +++ b/docs/nextgraph-current-state.md @@ -129,6 +129,97 @@ path NOT taken). A related exposed primitive: `social_query_start` (a federated query via inbox up to `degree` hops) exists but is limited to **contacts** — it does not cover an anonymous notification to a non-connected host. +## The query capability — ONE local store, named graphs, union queries + +The single fact that makes read-time *listing* possible on the shared wallet, and +the reason the reactive ORM must **not** be used as the listing primitive. Verified +directly in `nextgraph-rs`. + +### One oxigraph Store per session; each repo is a NAMED GRAPH + +- The verifier keeps **ONE** local oxigraph `Store` per session: + `graph_dataset: Option` (`engine/verifier/src/verifier.rs:94`). +- Every synced repo's data is inserted into that one store as a **distinct NAMED + GRAPH keyed by the repo** — `update_graph` writes each repo's main-branch triples + under `NuriV0::repo_graph_name(&repo_id, &overlay_id)` + (`engine/verifier/src/commits/transaction.rs:646-701`, the `ov_graphname` / + `repo_graph_name` around line 669). So all opened repos coexist as named graphs in + one dataset — a `GRAPH ?g { ... }` body can span them. + +### The NURI target decides scope: one repo vs the LOCAL UNION + +`sparql_query`'s scope is resolved from the NURI target by +`resolve_target_for_sparql` (`engine/verifier/src/request_processor.rs:256-285`): + +- `Repo(repo_id)` / `PrivateStore` → `Some(repo_graph_name)` = **ONE repo's graph**. +- `UserSite` / `None` → `Ok(None)` = the **UNION of all named graphs**. That `None` + is passed to oxigraph's `store.query(parsed, default_graph)` + (`engine/oxigraph/src/oxigraph/store.rs:200`) as the default graph, and + `sparql_query` first calls `dataset.set_default_graph_as_union()` when the query + has no explicit dataset (`request_processor.rs:656-675`, the + `if dataset.has_no_default_dataset()` block). Union = "every named graph currently + in the store". + +### The wasm binding defaults the target to UserSite when no `nuri` is passed + +The binding (`sdk/js/lib-wasm/src/lib.rs` — both the nodejs variant at ~`350-405` +and the web variant at ~`553-610`) reads the target from the `nuri` arg: a string +`nuri` is parsed; an **absent** `nuri` falls back to +`NuriV0::new_entire_user_site()` = `UserSite` +(`engine/net/src/app_protocol.rs:488-490`). Therefore: + +> **`sparql_query(sid, query, base, /*anchor*/ undefined)` queries the LOCAL UNION +> across all synced/opened graphs; with a string anchor it is restricted to that one +> repo.** A `GRAPH ?g { ... }` body then spans/attributes across the local graphs. + +### A repo is only queryable once OPENED/synced into the store + +A repo's triples enter `graph_dataset` (hence the union) only after the repo is +opened/synced into `self.repos` **and** its commits applied via `update_graph`. +Paths that open a repo: `doc_create` (own docs), bootstrap / +`load_repo_from_read_cap`, or being followed from a store's `ldp:contains` / a +shared cap / an inbox. Opening requires possessing the repo's **NURI + ReadCap** — +there is **no store-level read inheritance** (see § Capability / ReadCap +granularity). *(INFERRED: that "following a graph reference makes a previously +unknown repo known/openable" is the one step not read verbatim in source; the +open-requires-cap and no-inheritance facts around it ARE verified.)* + +### The union is READ-ONLY — writes must target one document + +`resolve_target_for_sparql(update=true)` returns `InvalidTarget` for `UserSite` / +`None` (`request_processor.rs:275-282`). So `sparql_update` cannot write "to the +union": every write must name **one** document's `@graph` — exactly what the +polyfill's `docs.sparqlUpdate` already does. + +### No reactive SPARQL — `sparql_query` is one-shot + +`sparql_query` is non-streamed: it computes a `QueryResults` and returns once +(`lib-wasm/src/lib.rs:352-405` / `553-610`). There is **no** "subscribe to a union +query". The only reactive primitives are the streamed ones: `orm_start_graph`, +`orm_start_discrete`, `doc_subscribe`, `app_request_stream`. + +### The ORM fan-out hang — verified root cause + +The reactive ORM is structurally unfit for a fan-out of per-entity / not-yet-synced +graphs, and this is *why* subscribing such a fan-out hangs: + +- `OrmStartGraph` first loops over EVERY graph in the requested scope and calls + `open_for_target(&nuri.target, /*publisher*/ true)` on each + (`request_processor.rs:53-66`), and `orm/graph/initialize.rs` does the same + fan-out again for the graphs the ORM discovers (~`125-128`). +- `open_for_target` → `resolve_target` → `self.repos.get(repo_id).ok_or(RepoNotFound)` + (`request_processor.rs:286-294` calling `resolve_target` at `:147`, the + `RepoNotFound` at `:155/:163`). +- A **freshly-created per-entity doc**, or any **not-yet-synced other-account doc**, + is absent from `self.repos` → `RepoNotFound` propagates through the `?` and + **aborts the whole `orm_start_graph`**. The subscription never emits its initial → + the ORM `readyPromise` never resolves → the multi-second hang observed (≈75s) + when subscribing a fan-out of per-entity graphs. + +**Consequence:** passing per-entity / unsynced graphs to the reactive ORM is broken. +Listing must go through a one-shot union `sparql_query` instead — see +[`read-model.md`](./read-model.md). + ## JS SDK limits (`@ng-org/web`) `@ng-org/web` (verified `0.1.2-alpha.13` = `upstream/main` at 2026-05-21, the diff --git a/docs/read-model.md b/docs/read-model.md new file mode 100644 index 0000000..fb56f91 --- /dev/null +++ b/docs/read-model.md @@ -0,0 +1,114 @@ +# The READ MODEL the polyfill implements + +How the polyfill turns "give me my lists" into concrete NextGraph reads on the +shared wallet. This is a **design decision**, grounded entirely in the query +capability documented in +[`nextgraph-current-state.md`](./nextgraph-current-state.md) § *The query +capability*. The consumer (Festipod) never sees any of this: it asks +`@ng-eventually/client` for its lists **by need** and trusts the answer — the whole +read mechanism lives here, in the polyfill. + +The governing constraints (all verified in `nextgraph-rs`, cited there): + +- One local oxigraph store per session; every opened repo is a **named graph**. +- `sparql_query` with **no anchor** → the **LOCAL UNION** of all opened graphs; + with an anchor → **one** repo. Union is **read-only**. +- A repo is queryable **only after it is opened/synced** (needs its NURI + ReadCap; + no store-level read inheritance). +- **No reactive union query**, and the reactive ORM **hangs** if handed a per-entity + / unsynced graph fan-out (`RepoNotFound` aborts `orm_start_graph`). + +## Two read regimes — enumerate vs follow + +There is **no cross-wallet read** in current NextGraph, so nothing is globally +enumerable "for free". The polyfill splits every list into one of two regimes: + +### Events (all public) = the GLOBAL INDEX — the ONE enumeration hack + +Public events are the only thing enumerated across accounts, via the emulated +discovery index (`discovery.readIndex`, see +[`simulation.md`](./simulation.md) § *Emulated discovery index*). This is the ONE +"hack", and it is justified precisely because P2P has no cross-wallet read: without +a shared index a client could never learn that another account's public event-doc +**exists**. `readIndex` yields the event-doc **NURIs** to open/sync; those repos +then enter the local union and become union-queryable. + +### Everything else = FOLLOW a graph, never enumerate across accounts + +My participations / my profile, a connection's shared protected data, my +notifications — **none** of these is enumerated across accounts. Each is reached by +**what is already reachable to me**: + +- **my own docs** (always in `self.repos`); +- docs reachable via a **connection's shared cap** (a bilateral connection surfaces + the peer's protected NURIs — see the bilateral connection registry in + [`simulation.md`](./simulation.md)); +- my **inbox** (deposits addressed to me). + +The rule of thumb: **Access ≠ discovery.** You only union-query over graphs you were +already entitled to open. + +## Listing = open/sync + ONE union query (never the ORM fan-out) + +To produce a list: + +1. **Open/sync** the relevant repos (the index-yielded event NURIs, my own docs, a + connection's shared NURIs). This is what puts them in the local union. +2. Run a **single** `sparql_query` with **NO anchor** over the LOCAL UNION, using a + `GRAPH ?g { ... }` body so each result row is attributed to its source graph. + +Do **NOT** drive listing through the reactive ORM's per-document fan-out +(`orm_start_graph` over many graphs): a freshly-created or not-yet-synced graph in +the fan-out makes `RepoNotFound` abort the whole subscription → the readyPromise +never resolves → the ~75s hang (root cause verified in +[`nextgraph-current-state.md`](./nextgraph-current-state.md) § *The ORM fan-out +hang*). + +## Reactivity = re-query on a change signal (no reactive union) + +There is **no reactive union query**. So reactivity is assembled: + +- keep a lightweight reactive subscription — `doc_subscribe`, or the ORM on an + **already-opened single store** (never a per-entity fan-out) — on the synced docs; +- on its change signal, **re-run** the one-shot union `sparql_query`. + +Keep the reactive ORM strictly to already-opened single stores; it is a change +*signal* source here, not the list source. + +## The boundary with the consumer + +Festipod asks the SDK for its lists by need (`listMyMeetingPoints()`, +`listEvents()`, …) and trusts the returned set. It never constructs a NURI, never +picks the union-vs-anchor mode, never touches the ORM. Open/sync + union-query + +re-query-on-signal all live in the polyfill. + +## Minimal broker probe (confirms the union behaviour) + +The one experiment that pins down union vs anchor, to run against a real broker: + +1. `doc_create` two docs **A** and **B** (own docs → both opened into the session + store). +2. `sparql_update` a **distinct** triple into each (target A's `@graph`, then B's). +3. **No anchor** — expect BOTH graphs: + + ``` + sparql_query( + sid, + "SELECT ?g ?s ?p ?o WHERE { GRAPH ?g { ?s ?p ?o } }", + undefined /* base */, + undefined /* anchor → UserSite → LOCAL UNION */ + ) + // → rows from BOTH A's and B's graphs + ``` + +4. **Anchor = A** — expect only A: + + ``` + sparql_query(sid, "SELECT ?g ?s ?p ?o WHERE { GRAPH ?g { ?s ?p ?o } }", + undefined, A /* string NURI → one repo */) + // → rows from A's graph only + ``` + +If (3) returns both and (4) returns only A, the union read model above holds as +implemented in `resolve_target_for_sparql` / +`set_default_graph_as_union`.