diff --git a/packages/client/docs/sdk-reference.md b/packages/client/docs/sdk-reference.md new file mode 100644 index 0000000..47fed41 --- /dev/null +++ b/packages/client/docs/sdk-reference.md @@ -0,0 +1,283 @@ +# SDK reference — reading data with `@ng-eventually/client` + +**Audience:** anyone using `@ng-eventually/client` (the app that consumes it, and +the lib itself when honoring the contract). This is the reference on the SDK's +**read/reactivity surface** — how you read data and how a read stays live. + +`@ng-eventually/client` is written and consumed as if NextGraph were a **finished, +mature SDK**: documents per entity placed by scope, capabilities, inboxes, and a +**reactive ORM**. This file documents that finished-SDK contract. Where today's +emulation does not yet deliver it, that is called out in one clearly-separated +section at the end ([§ Current emulation status](#current-emulation-status)) and in +[`nextgraph-current-state.md`](../../../docs/nextgraph-current-state.md) — that is +an emulation gap to close, **not** the SDK's design. Read the reference itself as the +target contract. + +The ground truth for the finished contract is the real NextGraph platform +(`nextgraph-rs`, sibling clone at `../nextgraph-rs`); the reactive primitives are +cited by `file:symbol` throughout so a future agent can re-verify cheaply. + +--- + +## TL;DR — the canonical read is reactive + +> **Read data with the reactive ORM hook `useShape` — subscribe to a shape over a +> scope, get the current value, and re-render on every change (yours or a remote +> peer's, synced through the broker). Subscription/push, never polling. One-shot +> reads are the exception, not the rule.** + +```ts +import { useShape } from "@ng-eventually/client"; +import { EventShapeType } from "…/shapes/orm/…"; + +function EventList() { + // A live, reactive set. Re-renders whenever any Event doc in scope changes — + // locally or from a remote peer synced by the broker. No polling, no refetch. + const events = useShape(EventShapeType, { graphs: [scopeNuri] }); + return <>{[...events].map((e) => )}; +} +``` + +--- + +## The reactivity model — subscription/push, never polling + +NextGraph's philosophy is **subscription-based push**. You do not poll for changes; +you **subscribe once** and the platform **pushes** an update to every subscriber the +moment a document changes. A change is a new **commit** on a document's branch, and +it reaches subscribers whether it was applied **locally** (your own write) or +**delivered from a remote peer** and synced through the broker. + +The load-bearing fact — verified in `nextgraph-rs` — is that both origins converge +on a single push point in the verifier: + +- **Local commit** (your own SPARQL update / ORM write): the write path builds + `BranchUpdateInfo`s and calls `Verifier::update_graph` + (`engine/verifier/src/commits/transaction.rs:646`). +- **Remote commit** (another session/peer, synced via the broker): the broker hands + the event to `Verifier::deliver` (`engine/verifier/src/verifier.rs:1718`) → + `verify_commit` → `verify_async_transaction` + (`engine/verifier/src/commits/transaction.rs:295`), which calls the **same** + `update_graph` (`transaction.rs:327`). +- **The single choke point:** `update_graph` pushes an `AppResponseV0::Patch` to the + branch's subscribers via `Verifier::push_app_response` + (`engine/verifier/src/verifier.rs:252`) — it looks up the branch in + `branch_subscriptions` and `sender.send(response).await` — **and** fans out to the + reactive ORM via `orm_backend_update` + (`engine/verifier/src/orm/graph/handle_backend_update.rs:48`), which sends an + `AppResponseV0::GraphOrmUpdate` to each ORM subscription whose scope was touched. + +So: **one document, one commit, every subscriber pushed** — the same code path for a +local edit and for a remote peer's edit arriving over the network. That is what makes +a `useShape` read reactive *across peers* with no polling. + +--- + +## The recommended read path — the reactive ORM hook `useShape` + +`useShape` is **the** way to read. It subscribes to a **shape** (a typed view — the +SHEX/ORM shape) over a **scope** (one or more document NURIs / a subject set), returns +the current materialized set immediately, and **re-renders the component on every +change** to any document in scope. + +### Signature + +```ts +useShape( + shape: ShapeType, + scope: Scope | string | undefined, +): DeepSignalSet +``` + +- `shape` — the ORM shape type (generated from a SHEX shape). Names the entity type + and the properties to materialize. +- `scope` — where to read: a `{ graphs, subjects }` scope object or a NURI string. + `undefined` yields an empty read. +- **Returns** a `DeepSignalSet` — a **live reactive set**. Iterate it like a set; + the component re-renders whenever the set changes. + +Verified surface in `nextgraph-rs`: +`sdk/js/orm/src/frontendAdapters/react/useShape.ts` (`useShape`, line 86) → +`OrmSubscription` (`sdk/js/orm/src/connector/GraphOrmSubscription.ts`), which calls +`ng.orm_start_graph(...)` with a callback, applies the initial materialized objects +and every subsequent patch to a `DeepSignalSet` +(`applyPatchesToDeepSignal`), and drives React re-render via +`useDeepSignal` (`@ng-org/alien-deepsignals/react`). Vue and Svelte adapters exist +alongside the React one (`sdk/js/orm/src/frontendAdapters/{vue,svelte}/`). + +`@ng-eventually/client` re-exports `useShape` from +[`../src/use-shape.ts`](../src/use-shape.ts); import it from the SDK +(`@ng-eventually/client`), never from `@ng-org/orm` directly. + +### What you get, in order + +1. **An initial value.** On subscribe, the ORM materializes the current objects in + scope and delivers them first (`AppResponseV0::GraphOrmInitial`, + `engine/verifier/src/orm/graph/initialize.rs:113`). The hook returns them as the + initial `DeepSignalSet`. +2. **A stream of updates.** On every subsequent commit affecting the scope — local or + remote — the ORM pushes a patch (`AppResponseV0::GraphOrmUpdate`), the connector + applies it to the `DeepSignalSet`, and the component re-renders. No refetch, no + interval. + +### Under the hood — the streamed primitives + +`useShape` is built on NextGraph's streamed request primitives. A consumer never +calls these directly, but they define the contract: + +- `orm_start_graph(shape, scope, …, callback)` — the reactive **graph ORM** + subscription (`sdk/js/lib-wasm/src/lib.rs:1951`). Returns `GraphOrmInitial` then a + stream of `GraphOrmUpdate`. This is what `useShape` uses. +- `orm_start_discrete(nuri, …, callback)` — the reactive **discrete** (Yjs/Automerge + document) ORM (`lib-wasm/src/lib.rs:1929`); `DiscreteOrmInitial` then + `DiscreteOrmUpdate`. +- `doc_subscribe(nuri, …, callback)` — a lower-level **document** subscription + (`lib-wasm/src/lib.rs:1908`; verifier `Verifier::create_branch_subscription`, + `verifier.rs:352`). Delivers an initial `TabInfo` + `State` (heads, full graph, + discrete state, files — `verifier.rs:470`/`:476`) then a stream of `Patch` on each + commit. This is the raw reactive read; `useShape` is the typed, ergonomic layer on + top. +- All of these are **streamed** (marked by `AppRequestCommandV0::is_stream()`, + `engine/net/src/app_protocol.rs:762`) and delivered through the one generic streamed + binding `app_request_stream_` (`lib-wasm/src/lib.rs:1385`), which invokes a JS + callback per `AppResponse` and returns a cancel function. The reactive surface is + **callback-based** at the wasm boundary; `useShape` hides that behind a reactive + signal. + +> **Rule of thumb:** to read, `useShape`. It subscribes, gives you the value now, and +> keeps it live. Reach for a one-shot read only when you explicitly do *not* want to +> stay subscribed. + +--- + +## The one-shot read — the exception + +Sometimes you want the current value **once**, with no live subscription (a batch, a +guard, a migration). NextGraph's one-shot read is a plain SPARQL query — non-streamed, +computes a result and returns once (`sparql_query`, +`sdk/js/lib-wasm/src/lib.rs:352`/`553`; no "subscribe to a query" exists — +`sparql_query` is not reactive). + +In `@ng-eventually/client` the one-shot read is exposed as: + +- **`docs.sparqlQuery(sid, query, base?, anchor?)`** — a raw anchored SPARQL query + ([`../src/docs.ts`](../src/docs.ts)). `anchor` = the document NURI to read; the + anchor restricts the query to that one repo's graph. +- **`readModel.readUnion(docs)`** — read a **bounded, by-need set** of document NURIs, + each with its own anchored query, grouped per subject + ([`../src/read-model.ts`](../src/read-model.ts)). This is the polyfill's listing + primitive (see [§ Current emulation status](#current-emulation-status) and + [`read-model.md`](../../../docs/read-model.md)). + +One-shot reads do not re-render on change. To stay live over a one-shot read you must +re-run it on a change **signal** (e.g. re-call `readUnion` when a `doc_subscribe` +fires) — a manual assembly that exists only because of the emulation gap below; the +finished contract is `useShape`. + +--- + +## The write surface (at a glance) + +You do not need the write internals to read, but reads and writes share the same +document model, so briefly: + +- **Create a document:** `docs.docCreate(sid, crdt, cls, dest, store?)` + ([`../src/docs.ts`](../src/docs.ts)) — mirrors `ng.doc_create`. **One document = one + repo** (`did:ng:o:`); there is no separate `Document` type. +- **Write into it:** `docs.sparqlUpdate(sid, query, anchor)` — a SPARQL + `INSERT/DELETE` scoped to the anchor document's graph. Or, at the ORM layer, the ORM + update primitives (`graph_orm_update`). A write is a **commit** on the document's + branch — which is exactly what every `useShape` subscriber over that document is + pushed. +- **Writes target one document**, never "the union": a SPARQL update must name one + document's graph (`resolve_target_for_sparql(update=true)` returns `InvalidTarget` + for the union, `engine/verifier/src/request_processor.rs:275`). + +--- + +## Identity & scope (what a consumer needs) + +Data is isolated **per document (repo)**, and each document lives in a **scope**: + +| Scope | Read | Write | +|---|---|---| +| **Private** | Owner only | Owner only | +| **Protected** | Owner + explicit grant holders | Owner + permissioned collaborators | +| **Public** | Everyone (no capability needed) | **Owner only** | + +Consequences a consumer must internalize: + +- **Isolation is per-document, not per-store.** Holding a store's cap does **not** + grant read on the documents it contains — each document has its own ReadCap. Fine- + grained isolation therefore means **one document per entity** + (`engine/repo/src/types.rs`, ReadCap granularity; see + [`nextgraph-current-state.md`](../../../docs/nextgraph-current-state.md) § + *Capability / ReadCap granularity*). +- **Read isolation is cryptographic.** A reactive/union read over a repo you hold no + cap for simply returns nothing (the repo is never decrypted); a *targeted* read of + an unheld repo raises `RepoNotFound`. +- **Public means everyone reads, only the owner writes.** There is **no** primitive by + which a non-owner appends to a document (public or otherwise): a write commit + requires repo **membership** plus a matching write permission, gated by + `Repo::verify_permission` (`engine/repo/src/repo.rs:584` — a non-member author is + `PermissionDenied`) and cryptographically bound to the repo's write-cap secret. The + permission enum (`engine/repo/src/types.rs:1729`, `PermissionV0`) has `WriteAsync`/ + `WriteSync` but **no** add-only/append permission and **no** public-writable grant. + To surface data to others without a shared write, use the **inbox** (any identity — + even anonymous — can deposit into a document's native inbox; the owner materializes + deposits) or make the document **public-readable** and let each identity own its own + document. + +The consumer asks the SDK for what it needs and trusts the result; it does not +construct NURIs, pick union-vs-anchor, or reason about caps. The domain-shaped list +helpers live in the consumer app; the SDK exposes the generic reactive/by-need read. + +--- + +## Current emulation status + +> **This section is about where today's polyfill does NOT yet deliver the reactive +> contract above.** It is an **emulation gap to close**, not the SDK's design. The +> reference above is the target; the finished SDK reads reactively via `useShape` +> everywhere. Full detail: +> [`nextgraph-current-state.md`](../../../docs/nextgraph-current-state.md), +> [`read-model.md`](../../../docs/read-model.md), +> [`simulation.md`](../../../docs/simulation.md). + +Today, on a single shared wallet emulating the mature platform, three gaps diverge +from the reactive contract: + +1. **Entity-list reads are one-shot, not reactive.** The reactive ORM cannot be used + as the listing primitive because the ORM fan-out over a set of per-entity / + not-yet-synced document graphs **hangs**: a freshly-created or unsynced graph makes + `RepoNotFound` abort the whole `orm_start_graph`, so the subscription never emits + its initial and never resolves (root cause verified — + `engine/verifier/src/request_processor.rs` `resolve_target` → + `self.repos.get(...).ok_or(RepoNotFound)`; see + [`nextgraph-current-state.md`](../../../docs/nextgraph-current-state.md) § *The ORM + fan-out hang*). So the lib reads entity lists with **`readModel.readUnion`** — a + bounded set of one-shot anchored `sparql_query`s + ([`read-model.md`](../../../docs/read-model.md)) — and reassembles reactivity by + **re-querying on a change signal** (a lightweight `doc_subscribe` / single-store ORM + used only as a *signal* source, then re-run `readUnion`). `useShape` remains valid + for a **single already-opened document**; it is the per-entity **fan-out** that is + unfit today. + +2. **Inbox and discovery index use polling watchers.** The inbox is emulated + (`AppRequestCommandV0::InboxPost` has no verifier arm today; no wasm helper seals a + deposit), so `inbox.watch` ([`../src/inbox.ts`](../src/inbox.ts)) and + `discovery.watchIndex` ([`../src/discovery.ts`](../src/discovery.ts)) **poll** via + `setInterval` (default 1s) instead of subscribing. The finished contract is push + (the broker already routes the inbox natively); these become subscriptions when the + sealed-inbox path (`inbox_post_link`) lands. + +3. **No cross-wallet / on-demand repo open.** There is no JS primitive to sync an + *unknown* repo by NURI+ReadCap today (`load_repo_from_read_cap` is `pub(crate)`, + unexposed; the `OpenRepo` broker path is a TODO at + `engine/verifier/src/verifier.rs:1423`). The mono-wallet polyfill sidesteps this: + every account's docs are `doc_create`d in the same session, so they are already + queryable. At the multi-store migration, opening a repo by cap becomes a native + broker sync and the anchored read is unchanged. + +When these gaps close, the read path collapses to the reference above: `useShape` +everywhere, push everywhere, no polling and no re-query-on-signal assembly.