ng-eventually/packages/client/docs/sdk-reference.md

# SDK reference — reading data with `@ng-eventually/client`

**Audience:** anyone using `@ng-eventually/client` (the app that consumes it, and
the lib itself when honoring the contract). This is the reference on the SDK's
**read/reactivity surface** — how you read data and how a read stays live.

`@ng-eventually/client` is written and consumed as if NextGraph were a **finished,
mature SDK**: documents per entity placed by scope, capabilities, inboxes, and a
**reactive ORM**. This file documents that finished-SDK contract. Where today's
emulation does not yet deliver it, that is called out in one clearly-separated
section at the end ([§ Current emulation status](#current-emulation-status)) and in
[`nextgraph-current-state.md`](../../../docs/nextgraph-current-state.md) — that is
an emulation gap to close, **not** the SDK's design. Read the reference itself as the
target contract.

The ground truth for the finished contract is the real NextGraph platform
(`nextgraph-rs`, sibling clone at `../nextgraph-rs`); the reactive primitives are
cited by `file:symbol` throughout so a future agent can re-verify cheaply.

---

## TL;DR — the canonical read is reactive

> **Read data with the reactive ORM hook `useShape` — subscribe to a shape over a
> scope, get the current value, and re-render on every change (yours or a remote
> peer's, synced through the broker). Subscription/push, never polling. One-shot
> reads are the exception, not the rule.**

```ts
import { useShape } from "@ng-eventually/client";
import { EventShapeType } from "…/shapes/orm/…";

function EventList() {
  // A live, reactive set. Re-renders whenever any Event doc in scope changes —
  // locally or from a remote peer synced by the broker. No polling, no refetch.
  const events = useShape(EventShapeType, { graphs: [scopeNuri] });
  return <>{[...events].map((e) => <Row key={e["@id"]} event={e} />)}</>;
}
```

---

## The reactivity model — subscription/push, never polling

NextGraph's philosophy is **subscription-based push**. You do not poll for changes;
you **subscribe once** and the platform **pushes** an update to every subscriber the
moment a document changes. A change is a new **commit** on a document's branch, and
it reaches subscribers whether it was applied **locally** (your own write) or
**delivered from a remote peer** and synced through the broker.

The load-bearing fact — verified in `nextgraph-rs` — is that both origins converge
on a single push point in the verifier:

- **Local commit** (your own SPARQL update / ORM write): the write path builds
  `BranchUpdateInfo`s and calls `Verifier::update_graph`
  (`engine/verifier/src/commits/transaction.rs:646`).
- **Remote commit** (another session/peer, synced via the broker): the broker hands
  the event to `Verifier::deliver` (`engine/verifier/src/verifier.rs:1718`) →
  `verify_commit` → `verify_async_transaction`
  (`engine/verifier/src/commits/transaction.rs:295`), which calls the **same**
  `update_graph` (`transaction.rs:327`).
- **The single choke point:** `update_graph` pushes an `AppResponseV0::Patch` to the
  branch's subscribers via `Verifier::push_app_response`
  (`engine/verifier/src/verifier.rs:252`) — it looks up the branch in
  `branch_subscriptions` and `sender.send(response).await` — **and** fans out to the
  reactive ORM via `orm_backend_update`
  (`engine/verifier/src/orm/graph/handle_backend_update.rs:48`), which sends an
  `AppResponseV0::GraphOrmUpdate` to each ORM subscription whose scope was touched.

So: **one document, one commit, every subscriber pushed** — the same code path for a
local edit and for a remote peer's edit arriving over the network. That is what makes
a `useShape` read reactive *across peers* with no polling.

---

## The recommended read path — the reactive ORM hook `useShape`

`useShape` is **the** way to read. It subscribes to a **shape** (a typed view — the
SHEX/ORM shape) over a **scope** (one or more document NURIs / a subject set), returns
the current materialized set immediately, and **re-renders the component on every
change** to any document in scope.

### Signature

```ts
useShape<T extends BaseType>(
  shape: ShapeType<T>,
  scope: Scope | string | undefined,
): DeepSignalSet<T>
```

- `shape` — the ORM shape type (generated from a SHEX shape). Names the entity type
  and the properties to materialize.
- `scope` — where to read: a `{ graphs, subjects }` scope object or a NURI string.
  **Not to be confused with this library's `Scope`** (`types.ts`), which is the
  literal union `public | protected | private` naming a store. Same word, two
  meanings: the ORM's is a read target, ours is a placement.
  `undefined` yields an empty read.
- **Returns** a `DeepSignalSet<T>` — a **live reactive set**. Iterate it like a set;
  the component re-renders whenever the set changes.

Verified surface in `nextgraph-rs`:
`sdk/js/orm/src/frontendAdapters/react/useShape.ts` (`useShape`, line 86) →
`OrmSubscription` (`sdk/js/orm/src/connector/GraphOrmSubscription.ts`), which calls
`ng.orm_start_graph(...)` with a callback, applies the initial materialized objects
and every subsequent patch to a `DeepSignalSet`
(`applyPatchesToDeepSignal`), and drives React re-render via
`useDeepSignal` (`@ng-org/alien-deepsignals/react`). Vue and Svelte adapters exist
alongside the React one (`sdk/js/orm/src/frontendAdapters/{vue,svelte}/`).

`@ng-eventually/client` re-exports `useShape` from
[`../src/use-shape.ts`](../src/use-shape.ts); import it from the SDK
(`@ng-eventually/client`), never from `@ng-org/orm` directly.

### What you get, in order

1. **An initial value.** On subscribe, the ORM materializes the current objects in
   scope and delivers them first (`AppResponseV0::GraphOrmInitial`,
   `engine/verifier/src/orm/graph/initialize.rs:113`). The hook returns them as the
   initial `DeepSignalSet`.
2. **A stream of updates.** On every subsequent commit affecting the scope — local or
   remote — the ORM pushes a patch (`AppResponseV0::GraphOrmUpdate`), the connector
   applies it to the `DeepSignalSet`, and the component re-renders. No refetch, no
   interval.

### Under the hood — the streamed primitives

`useShape` is built on NextGraph's streamed request primitives. A consumer never
calls these directly, but they define the contract:

- `orm_start_graph(shape, scope, …, callback)` — the reactive **graph ORM**
  subscription (`sdk/js/lib-wasm/src/lib.rs:1951`). Returns `GraphOrmInitial` then a
  stream of `GraphOrmUpdate`. This is what `useShape` uses.
- `orm_start_discrete(nuri, …, callback)` — the reactive **discrete** (Yjs/Automerge
  document) ORM (`lib-wasm/src/lib.rs:1929`); `DiscreteOrmInitial` then
  `DiscreteOrmUpdate`.
- `doc_subscribe(nuri, …, callback)` — a lower-level **document** subscription
  (`lib-wasm/src/lib.rs:1908`; verifier `Verifier::create_branch_subscription`,
  `verifier.rs:352`). Delivers an initial `TabInfo` + `State` (heads, full graph,
  discrete state, files — `verifier.rs:470`/`:476`) then a stream of `Patch` on each
  commit. This is the raw reactive read; `useShape` is the typed, ergonomic layer on
  top.
- All of these are **streamed** (marked by `AppRequestCommandV0::is_stream()`,
  `engine/net/src/app_protocol.rs:762`) and delivered through the one generic streamed
  binding `app_request_stream_` (`lib-wasm/src/lib.rs:1385`), which invokes a JS
  callback per `AppResponse` and returns a cancel function. The reactive surface is
  **callback-based** at the wasm boundary; `useShape` hides that behind a reactive
  signal.

> **Rule of thumb:** to read, `useShape`. It subscribes, gives you the value now, and
> keeps it live. Reach for a one-shot read only when you explicitly do *not* want to
> stay subscribed.

---

## The one-shot read — the exception

Sometimes you want the current value **once**, with no live subscription (a batch, a
guard, a migration). NextGraph's one-shot read is a plain SPARQL query — non-streamed,
computes a result and returns once (`sparql_query`,
`sdk/js/lib-wasm/src/lib.rs:352`/`553`; no "subscribe to a query" exists —
`sparql_query` is not reactive).

In `@ng-eventually/client` the one-shot read is exposed as:

- **`docs.sparqlQuery(sid, query, base?, anchor?)`** — a raw anchored SPARQL query
  ([`../src/docs.ts`](../src/docs.ts)). `anchor` = the document NURI to read; the
  anchor restricts the query to that one repo's graph.
- **`readModel.readUnion(docs)`** — read a **bounded, by-need set** of document NURIs,
  each with its own anchored query, grouped per subject
  ([`../src/read-model.ts`](../src/read-model.ts)). This is the polyfill's listing
  primitive (see [§ Current emulation status](#current-emulation-status) and
  [`read-model.md`](../../../docs/read-model.md)).

One-shot reads do not re-render on change. To stay live over a one-shot read you must
re-run it on a change **signal** (e.g. re-call `readUnion` when a `doc_subscribe`
fires) — a manual assembly that exists only because of the emulation gap below; the
finished contract is `useShape`.

---

## The write surface (at a glance)

You do not need the write internals to read, but reads and writes share the same
document model, so briefly:

- **Create a document:** `docs.docCreate(sid, crdt, cls, dest, store?)`
  ([`../src/docs.ts`](../src/docs.ts)) — mirrors `ng.doc_create`. **One document = one
  repo** (`did:ng:o:<RepoID>`); there is no separate `Document` type.
- **Write into it:** `docs.sparqlUpdate(sid, query, anchor)` — a SPARQL
  `INSERT/DELETE` scoped to the anchor document's graph. Or, at the ORM layer, the ORM
  update primitives (`graph_orm_update`). A write is a **commit** on the document's
  branch — which is exactly what every `useShape` subscriber over that document is
  pushed.
- **Writes target one document**, never "the union": a SPARQL update must name one
  document's graph (`resolve_target_for_sparql(update=true)` returns `InvalidTarget`
  for the union, `engine/verifier/src/request_processor.rs:275`).

---

## Identity & scope (what a consumer needs)

Data is isolated **per document (repo)**, and each document lives in a **scope**:

| Scope | Read | Write |
|---|---|---|
| **Private** | Owner only | Owner only |
| **Protected** | Owner + whoever the owner delivered the cap to | Owner + permissioned collaborators |
| **Public** | Whoever has the URL (the repo link) | **Owner only** |

Consequences a consumer must internalize:

- **Reading is key possession, never an authorization list.** You hold a document's
  `ReadCap` (`…:r:{cap}`) or you do not read it — there is no "may X read Y?" to ask,
  here or upstream. A cap-less `did:ng:o:…` **names** a document without granting
  anything, which is what lets public content point at private content without
  disclosing it. Caps reach you two ways: creating a document files its own, and
  someone delivering one to your inbox (`shareCap`). Nothing derives a cap from a
  bare reference.
- **Isolation is per-document, not per-store.** Holding a store's cap does **not**
  grant read on the documents it contains — each document has its own ReadCap. Fine-
  grained isolation therefore means **one document per entity**
  (`engine/repo/src/types.rs`, ReadCap granularity; see
  [`nextgraph-current-state.md`](../../../docs/nextgraph-current-state.md) §
  *Capability / ReadCap granularity*).
- **Read isolation is cryptographic.** A reactive/union read over a repo you hold no
  cap for simply returns nothing (the repo is never decrypted); a *targeted* read of
  an unheld repo raises `RepoNotFound`.
- **Public means everyone reads, only the owner writes.** There is **no** primitive by
  which a non-owner appends to a document (public or otherwise): a write commit
  requires repo **membership** plus a matching write permission, gated by
  `Repo::verify_permission` (`engine/repo/src/repo.rs:584` — a non-member author is
  `PermissionDenied`) and cryptographically bound to the repo's write-cap secret. The
  permission enum (`engine/repo/src/types.rs:1729`, `PermissionV0`) has `WriteAsync`/
  `WriteSync` but **no** add-only/append permission and **no** public-writable grant.
  To surface data to others without a shared write, use the **inbox** (any identity —
  even anonymous — can deposit; only the owner reads back) or make the document
  **public-readable** and let each identity own its own document. *Per-document inboxes
  are this library's, not the engine's: upstream only the public and protected store
  repos carry one (`engine/verifier/src/site.rs:128,149`).*

  Depositing into a document you do not own is **one** call, and it names the document:

  ```ts
  await inbox.postToDocument(doc, { payload: { signingUp: true } });
  ```

  You need the **document** (its cap), nothing else — the address rides on it. It
  **throws** if the document has no inbox: its owner opens one with
  `storeRegistry.openDocumentInbox(doc)` for documents meant to receive, so a fresh
  document has none. When "no inbox" is an expected case, check first with
  `storeRegistry.documentInboxAddress(doc)` (→ `Nuri | undefined`).

  A deposit carries no target document, deliberately — one inbox belongs to one
  document, so the address already identifies it, exactly as upstream (`inboxes:
  PubKey → RepoId`). Do not encode the document in your payload; you would have to
  unlearn it. Reading that inbox is a different right, and it stays the owner's
  (`inbox.read` refuses otherwise).

The consumer asks the SDK for what it needs and trusts the result; it does not
construct NURIs, pick union-vs-anchor, or reason about caps. The domain-shaped list
helpers live in the consumer app; the SDK exposes the generic reactive/by-need read.

---

## Current emulation status

> **This section is about where today's polyfill does NOT yet deliver the reactive
> contract above.** It is an **emulation gap to close**, not the SDK's design. The
> reference above is the target; the finished SDK reads reactively via `useShape`
> everywhere. Full detail:
> [`nextgraph-current-state.md`](../../../docs/nextgraph-current-state.md),
> [`read-model.md`](../../../docs/read-model.md),
> [`simulation.md`](../../../docs/simulation.md).

Today, on a single shared wallet emulating the mature platform, four gaps diverge
from the reactive contract:

1. **Entity-list reads are one-shot, not reactive.** The reactive ORM cannot be used
   as the listing primitive because the ORM fan-out over a set of per-entity /
   not-yet-synced document graphs **hangs**: a freshly-created or unsynced graph makes
   `RepoNotFound` abort the whole `orm_start_graph`, so the subscription never emits
   its initial and never resolves (root cause verified —
   `engine/verifier/src/request_processor.rs` `resolve_target` →
   `self.repos.get(...).ok_or(RepoNotFound)`; see
   [`nextgraph-current-state.md`](../../../docs/nextgraph-current-state.md) § *The ORM
   fan-out hang*). So the lib reads entity lists with **`readModel.readUnion`** — a
   bounded set of one-shot anchored `sparql_query`s
   ([`read-model.md`](../../../docs/read-model.md)) — and reassembles reactivity by
   **re-querying on a change signal** (a lightweight `doc_subscribe` / single-store ORM
   used only as a *signal* source, then re-run `readUnion`). `useShape` remains valid
   for a **single already-opened document**; it is the per-entity **fan-out** that is
   unfit today.

2. **The inbox uses a polling watcher.** The inbox is emulated
   (`AppRequestCommandV0::InboxPost` has no verifier arm today; no wasm helper seals a
   deposit), so `inbox.watch` ([`../src/inbox.ts`](../src/inbox.ts)) **polls** via
   `setInterval` (default 1s) instead of subscribing. The finished contract is push
   (the broker already routes the inbox natively); these become subscriptions when the
   sealed-inbox path is exposed to JS (no such method exists today).

3. **No cross-wallet / on-demand repo open.** There is no JS primitive to sync an
   *unknown* repo by NURI+ReadCap today (`load_repo_from_read_cap` is `pub(crate)`,
   unexposed; the `OpenRepo` broker path is a TODO at
   `engine/verifier/src/verifier.rs:1423`). The mono-wallet polyfill sidesteps this:
   every account's docs are `doc_create`d in the same session, so they are already
   queryable. At the multi-store migration, opening a repo by cap becomes a native
   broker sync and the anchored read is unchanged.

4. **The subscription may not echo the writer's OWN local commit — HYPOTHESIS
   (high-confidence), confirmation in progress (2026-07-18); NOT confirmed, NOT
   fixed.** Unlike gaps 1–3 (designed emulation stopgaps), this is a suspected
   defect in the polyfill's own reactive assembly. When a client does a local
   `sparqlUpdate` on a doc it is itself subscribed to (`subscribeDoc` /
   `ng.doc_subscribe`), the subscription callback appears NOT to fire for its OWN
   local commit in the same session — so the reactive re-read chain
   ([`../src/watch-shape.ts`](../src/watch-shape.ts) `watchShape` → `reread` →
   [`../src/read-model.ts`](../src/read-model.ts) `readUnion`) never runs, and
   consumers keep the STALE value until the next connection delivers a fresh
   initial `State`. **Remote** commits DO push correctly (verified: cross-browser
   reactive update works). A code review verified the consumer wiring is correct,
   the doc IS in the subscribed set, and a triggered re-read WOULD return the new
   value — leaving the self-commit echo as the only suspect link. That link is
   **INFERRED**, not observed: the real `ng.doc_subscribe` runtime is not readable
   from source, and [`../src/subscribe.ts`](../src/subscribe.ts)'s own doc-comment
   CLAIMS local writes push a `Patch` — contradicted by the observation. (This
   also sits in tension with § *The reactivity model* above, which documents the
   target contract — one commit, every subscriber pushed, local or remote.) The
   requirement at stake is multi-user: a value change (e.g. a participant count)
   must propagate reactively to ALL viewers — other viewers (remote push, which
   works) AND the writer's own view (this suspect link). **Treatment (PLANNED,
   not done):** confirm first via the temporary instrumentation just added
   ([`../src/subscribe.ts`](../src/subscribe.ts) ≈`:119` logs
   `doc_subscribe FIRE <nuri> (State|Patch)`;
   [`../src/watch-shape.ts`](../src/watch-shape.ts) ≈`:341` logs
   `reread TRIGGER by <nuri>` — line numbers volatile, grep the log strings);
   then, IF confirmed, fix **polyfill-side** — a
   local commit should notify the doc's active `subscribeDoc` callbacks.
   Consumers must not compensate. Short entry:
   [`nextgraph-current-state.md`](../../../docs/nextgraph-current-state.md) §
   *Known open issues*.

When these gaps close, the read path collapses to the reference above: `useShape`
everywhere, push everywhere, no polling and no re-query-on-signal assembly.