ng-eventually/docs/nextgraph-current-state.md

# Current-state NextGraph — what the SDK/broker do and do NOT expose

**Owner:** this library. `@ng-eventually/client` exists because the *current*
NextGraph JS SDK is immature. This file is the authoritative reference on what
today's SDK/broker actually give us — the ground truth every polyfill in this
lib compensates for. Read [`simulation.md`](./simulation.md) for how we emulate
the mature behaviour on top of these limits, and
[`migration-guide.md`](./migration-guide.md) for what changes when they lift.

Verified against `nextgraph-rs` (local clone at `../nextgraph-rs`, sibling of
this repo) and the installed npm packages. Store/permission facts cross-checked
with the official docs ([Documents & Stores](https://docs.nextgraph.org/en/documents/),
[Getting started](https://docs.nextgraph.org/en/getting-started/)).

## Source pointers (`nextgraph-rs`)

Where the ground truth lives, so future re-verification is cheap:

- `sdk/js/lib-wasm/src/lib.rs` — the wasm API actually exposed to JS.
- `engine/net/src/app_protocol.rs` — `AppRequestCommandV0` enum, `NuriV0` formats.
- `engine/verifier/src/request_processor.rs` — the effective `app_request`
  dispatch (the truth on what is actually *processed*).
- `engine/net/src/types.rs` — inbox types (`InboxPost`, `InboxMsg`, `InboxMsgContent`).
- `engine/verifier/src/inbox_processor.rs` — inbox message handling.
- `engine/verifier/src/verifier.rs:1423` — the `OpenRepo` TODO (cross-wallet read).
- `engine/repo/src/types.rs` — `RootBranchV0.store: StoreOverlay` (repo → its store).

## The 5 store types

Every wallet has the **3 default stores** out of the box (session fields
`private_store_id`, `protected_store_id`, `public_store_id`). Group and Dialog
are created on demand.

| Store | Read | Write | Creation |
|---|---|---|---|
| **Private** | Owner only | Owner only | Default |
| **Protected** | Owner + link+permission holders | Owner + permissioned collaborators | Default |
| **Public** | Everyone, no capability | Owner only | Default |
| **Group** | Group members | Group members (collaborative) | On demand |
| **Dialog** | The two users only | The two users only | On demand |

Doc citations (verbatim): Private — *"only you have access to … not possible to
share"*; Protected — *"share … but they will need a special link and permission"*;
Public — *"equivalent to your website … without the need for special permissions"*;
Group — *"each Group is a separate Store … documents inherit the permissions of
the store"*; Dialog — *"hold all the data you exchange with another user (and only
with that other user) … You cannot add more users"*.

## Document = repo (there is no `Document` type)

*"A Repo is the equivalent of an E2EE group for one and only one Document."*
**1 document = 1 repo** (commits + permissions). Identifier: `did:ng:o:<RepoID>`.

There is **no `Document` type** in `nextgraph-rs` (verified 2026-06-29): a
"document" is simply *any repo*. A **store is a special repo** (`is_store=true`,
with `Store`/`Overlay`/`User` branches) — so *a store is a document, but a
document is not necessarily a store*.

**Containment (store → repos) is by REFERENCE, not by a list.** A store does not
hold a `Vec<RepoId>`: it references its repos through an **RDF graph** in its
Overlay/User branch. Conversely each repo names its parent store via
`RootBranchV0.store: StoreOverlay` — **a repo belongs to exactly one store**.

## Capability / ReadCap granularity — the load-bearing fact for this lib

`ReadCap = ObjectRef`. Granularity is at the **repo AND branch** level (each
branch has its own `read_cap`), down to the **block** (`ObjectKey`/ChaCha20 key).
Write is managed at the **document (repo)** level.

**No automatic read inheritance.** Holding a **store's** ReadCap does **not**
grant the repos it contains — **you need each repo's own ReadCap**. The optional
`inherit_perms_users_and_quorum_from_store: Option<ReadCap>` shares only
users/quorum (write/permissions), **not** read-cap possession. (Repos of a
`private_store` inherit implicitly.)

> Consequence for this lib's emulation (see [`simulation.md`](./simulation.md)):
> the read access unit is the repo = each item's `@graph` — a per-document
> filter, never per-store and never per-item. This is exactly what
> `caps.ts` (`CapRegistry`) and `read-filter.ts` model: no store-level
> inheritance, purely per-document caps. In a mono-store layout (all items in one
> repo) the filter is therefore all-or-nothing on that document — which *is* the
> native behaviour, and why fine-grained isolation requires one document per
> entity. Read isolation is cryptographic in the target: with no cap for a repo, a
> union / reactive read returns empty (the repo is never decrypted), while a
> targeted read of an unheld repo returns `RepoNotFound`. There is no
> cap-introspection API — the polyfill's `canRead` / `governsRead` are
> emulation-only, with no NextGraph API behind them.

### Store ↔ document confusion (recurring)

The isolation axis is the **document (repo/`@graph`)**, never the **store**: a
store *contains* several documents and does not share their read caps. See the
two-axes warning in [`simulation.md`](./simulation.md): "multi-store" in this
lib's emulation means **multiple DOCUMENTS in one shared store**, not multiple
stores.

## Capability sharing / NURI

Sharing transmits a **NURI** embedding the crypto capability (read and/or write).
No central ACL: holding the NURI *is* the right. *"adding permissions can be done
offline"*; *"removing permissions … requires a SyncSignature"* (synchronous).

## Inbox

Every document has a native inbox. A non-editor can deposit a link (DID
cap) into it without being invited as an editor; the owner moderates. NURI:
`did:ng:d:<inbox_id>`. Content: the `InboxMsgContent` enum (`ContactDetails`,
`DialogRequest`, `Link`, `Patch`, `ServiceRequest`, `ExtRequest`,
`RemoteQuery`, `SocialQuery`…). Messages are sealed (`crypto_box::seal`) to
the inbox pubkey, so only the owner decrypts. The `from` field is optional, so an
anonymous sender is possible. This is the "identified if known, anonymous
otherwise" behaviour native to the protocol.

The recipient's own verifier unseals each queued message and applies it inline when
it processes its inbox — there is no separate curator or materialization service.

### The inbox is not usable from the JS SDK

- `app_request(request)` is exposed, and `AppRequestCommandV0::InboxPost` +
  `AppRequest::inbox_post()` exist, but the verifier's `request_processor`
  has no `InboxPost` arm (arms actually handled: `OrmStart(Discrete)`,
  `Fetch`, `FileGet`, `OrmUpdate`, `OrmDiscreteUpdate`, `SocialQueryStart`,
  `QrCodeProfile(Import)`, `Header`, `Create`, `FilePut`). Sending an `InboxPost`
  triggers nothing.
- Building an `InboxPost` requires crypto sealing on the Rust side; no wasm
  helper exposes it. A high-level `inbox_post_link` is a proposed/future API, not
  yet present.
- Inbox deposit is only triggered internally by `QrCodeProfileImport`
  (`post_to_inbox(new_contact_details)`) and `social_query_start` (contact
  propagation via inbox).

**Consequence:** there is no clean way to "drop a Link" into an arbitrary
document's inbox from the JS SDK today. This lib emulates the inbox instead of
patching the broker — see [`simulation.md`](./simulation.md) (emulated inbox) and
[`fork-inbox-fallback.md`](./fork-inbox-fallback.md) (the Rust-patch path not taken).
A related exposed primitive: `social_query_start` (a federated query via inbox up to
`degree` hops) exists but is limited to contacts — it does not cover an anonymous
notification to a non-connected host.

## The query capability — ONE local store, named graphs, union queries

The single fact that makes read-time *listing* possible on the shared wallet, and
the reason the reactive ORM must **not** be used as the listing primitive. Verified
directly in `nextgraph-rs`.

### One oxigraph Store per session; each repo is a NAMED GRAPH

- The verifier keeps **ONE** local oxigraph `Store` per session:
  `graph_dataset: Option<Store>` (`engine/verifier/src/verifier.rs:94`).
- Every synced repo's data is inserted into that one store as a **distinct NAMED
  GRAPH keyed by the repo** — `update_graph` writes each repo's main-branch triples
  under `NuriV0::repo_graph_name(&repo_id, &overlay_id)`
  (`engine/verifier/src/commits/transaction.rs:646-701`, the `ov_graphname` /
  `repo_graph_name` around line 669). So all opened repos coexist as named graphs in
  one dataset — a `GRAPH ?g { ... }` body can span them.

### The NURI target decides scope: one repo vs the LOCAL UNION

`sparql_query`'s scope is resolved from the NURI target by
`resolve_target_for_sparql` (`engine/verifier/src/request_processor.rs:256-285`):

- `Repo(repo_id)` / `PrivateStore` → `Some(repo_graph_name)` = **ONE repo's graph**.
- `UserSite` / `None` → `Ok(None)` = the **UNION of all named graphs**. That `None`
  is passed to oxigraph's `store.query(parsed, default_graph)`
  (`engine/oxigraph/src/oxigraph/store.rs:200`) as the default graph, and
  `sparql_query` first calls `dataset.set_default_graph_as_union()` when the query
  has no explicit dataset (`request_processor.rs:656-675`, the
  `if dataset.has_no_default_dataset()` block). Union = "every named graph currently
  in the store".

### The wasm binding defaults the target to UserSite when no `nuri` is passed

The binding (`sdk/js/lib-wasm/src/lib.rs` — both the nodejs variant at ~`350-405`
and the web variant at ~`553-610`) reads the target from the `nuri` arg: a string
`nuri` is parsed; an **absent** `nuri` falls back to
`NuriV0::new_entire_user_site()` = `UserSite`
(`engine/net/src/app_protocol.rs:488-490`). Therefore:

> **`sparql_query(sid, query, base, /*anchor*/ undefined)` queries the LOCAL UNION
> across all synced/opened graphs; with a string anchor it is restricted to that one
> repo.** A `GRAPH ?g { ... }` body then spans/attributes across the local graphs.

### A repo is only queryable once OPENED/synced into the store

A repo's triples enter `graph_dataset` (hence the union) only after the repo is
opened/synced into `self.repos` **and** its commits applied via `update_graph`.

**VERIFIED (T03.k) — there is NO JS primitive to sync an *unknown* repo.** Every
JS entry point that could "open" a repo — `sparql_query` anchored,
`doc_subscribe` (`Fetch::Subscribe`), `orm_start_graph` — resolves its target via
`resolve_target`/`resolve_target_for_sparql`, which does
`self.repos.get(repo_id).ok_or(RepoNotFound)` (`request_processor.rs:155/163/264/269`).
None of them PULLS a repo that is absent from `self.repos`; they only touch a repo
already there. The primitive that actually loads a repo from its ReadCap,
`Verifier::load_repo_from_read_cap` (`verifier.rs:2237`), is **`pub(crate)` —
unexposed to JS**; it is only reached internally (bootstrap, inbox processing). So
from JS today a repo becomes queryable ONLY by being `doc_create`d in this session
(own docs) or synced by an internal path — never on demand by NURI+ReadCap.

**Consequence for this lib's mono-wallet polyfill:** every account's documents are
`doc_create`d in the one shared wallet within the same session, so they are all
already in `self.repos`. `read-model.ts` reads the bounded, by-need set of docs
with one anchored `sparql_query` per doc (`SELECT ?s ?p ?o WHERE { ?s ?p ?o }`,
anchor = the doc NURI): the anchor resolves that same-session repo directly (no
separate open needed) and restricts the query to its graph, so it is O(1) per doc,
independent of the store's size. An absent repo throws `RepoNotFound` on its own
read and is skipped, never aborting the batch.

The read path avoids an anchorless union-scan. An anchorless
`SELECT … WHERE { GRAPH ?g { ?s ?p ?o } }` spans every named graph in the store —
O(wallet size). On a shared wallet that accumulates docs across runs that cost grows
with the whole wallet, which is why the read path is per-doc anchored: the anchored
read makes a non-empty wallet irrelevant. At the real multi-store
migration this is unchanged (the anchored read is native); only bringing a repo into
the session changes: opening a real per-user store repo by cap becomes a native
broker sync (the `OpenRepo` TODO at `verifier.rs:1423`). Opening still requires the
repo's NURI + ReadCap — there is no store-level read inheritance (see
§ Capability / ReadCap granularity).

### The union is read-only — writes must target one document

`resolve_target_for_sparql(update=true)` returns `InvalidTarget` for `UserSite` /
`None` (`request_processor.rs:275-282`). So `sparql_update` cannot write "to the
union": every write must name one document's `@graph` — exactly what the
polyfill's `docs.sparqlUpdate` already does.

### No reactive SPARQL — `sparql_query` is one-shot

`sparql_query` is non-streamed: it computes a `QueryResults` and returns once
(`lib-wasm/src/lib.rs:352-405` / `553-610`). There is no "subscribe to a union
query". The only reactive primitives are the streamed ones: `orm_start_graph`,
`orm_start_discrete`, `doc_subscribe`, `app_request_stream`.

### The ORM fan-out hang — verified root cause

The reactive ORM is structurally unfit for a fan-out of per-entity / not-yet-synced
graphs, and this is *why* subscribing such a fan-out hangs:

- `OrmStartGraph` first loops over every graph in the requested scope and calls
  `open_for_target(&nuri.target, /*publisher*/ true)` on each
  (`request_processor.rs:53-66`), and `orm/graph/initialize.rs` does the same
  fan-out again for the graphs the ORM discovers (~`125-128`).
- `open_for_target` → `resolve_target` → `self.repos.get(repo_id).ok_or(RepoNotFound)`
  (`request_processor.rs:286-294` calling `resolve_target` at `:147`, the
  `RepoNotFound` at `:155/:163`).
- A freshly-created per-entity doc, or any not-yet-synced other-account doc,
  is absent from `self.repos`, so `RepoNotFound` propagates through the `?` and
  aborts the whole `orm_start_graph`. The subscription never emits its initial, so
  the ORM `readyPromise` never resolves and the subscription hangs when a fan-out of
  per-entity graphs is passed in.

**Consequence:** passing per-entity / unsynced graphs to the reactive ORM is broken.
Listing must go through a one-shot union `sparql_query` instead — see
[`read-model.md`](./read-model.md).

## JS SDK limits (`@ng-org/web`)

`@ng-org/web` (verified `0.1.2-alpha.13` = `upstream/main` at 2026-05-21, the
installed version) **does NOT expose**: Group/Dialog store creation; capability
sharing (a NURI with rights); permission manipulation; inbox deposit/read.

Available JS methods: `doc_create`, `doc_subscribe`, `sparql_query`,
`sparql_update`, `orm_start_graph`, `orm_start_discrete`, `graph_orm_update`,
`discrete_orm_update`, `file_get`, `app_request_stream`. The docs announce *"An
API will be provided for permission manipulation"* (no date).

## Integration & deployment model

NextGraph is consumed via an **iframe proxy** (`@ng-org/web`): the third-party
app contains no engine, it delegates to a hosted ng-app (default `nextgraph.net`)
that runs the engine in an iframe.

### The JS packages

- **`@ng-org/web`** — **published**. Lightweight postMessage proxy (no wasm
  embedded). **The** third-party integration path; `@ng-org/orm` and every
  example depend on it. This lib wraps it.
- **`@ng-org/api-web`** — **private** (unpublished). Full in-browser engine
  (loads `@ng-org/lib-wasm` in a Web Worker). Consumed only by `app/nextgraph`
  (the ng-app frontend) — **not** a third-party integration target.
- **`@ng-org/lib-wasm`** — the compiled wasm engine (contains the verifier).
  Source `sdk/js/lib-wasm/`.
- **`nextgraph`** (npm) — the NodeJS API (`pkg-node` build).
- **`@ng-org/orm`** — reactive ORM (`useShape`…), built on `@ng-org/web`.

### Where the verifier runs

In the standard web model, the verifier runs **in the iframe**: `app/nextgraph`
loads `api-web` → `lib-wasm` in a Web Worker, browser-side. The broker (`ngd`)
only does **transport and storage**.

**Consequence:** changing verifier logic (`request_processor`,
`inbox_processor`) means rebuilding the **ng-app**, not the broker.

### iframe model & build-time retargeting

`@ng-org/web` redirects to the hosted ng-app, which reloads the third-party app
in an iframe after auth, then relays over `postMessage`. **Retargetable at build
time** (`sdk/js/web/src/index.ts`, `import.meta.env`):

| Variable | Target |
|---|---|
| `NG_REDIR_SERVER` | default `nextgraph.net` |
| `NG_DEV3` | `127.0.0.1:3033` |
| `NG_DEV` | `localhost:14402`/`14404` |
| `NG_DEV_LOCAL_BROKER` | `localhost:1421` |

**No runtime override** — `init()` takes no broker URL. To point at a
self-hosted ng-app: **rebuild `@ng-org/web`** (pure TS, no wasm → trivial build).

### Proxy ↔ iframe ↔ worker plumbing (generic)

The call path is **entirely generic** (no allowlist): `@ng-org/web` is a JS
`Proxy` relaying *any* method name over `postMessage`; `app/nextgraph` dispatches
via `Reflect.apply(ng[method], …)`. So a new wasm function in simple
request/response form is *reachable* without touching the JS — but that's an
untyped **hack** (quick test, not a plan). The **streamed** case needs an entry
on both sides (`E` in `@ng-org/web` + `streamed_api` in api-web; current streamed
methods: `doc_subscribe`, `orm_start_graph`, `orm_start_discrete`, `file_get`,
`app_request_stream`).

> This is exactly why `docs.ts` in this lib calls the **real injected `ng`**
> directly and never layers our own `Proxy` on top of `@ng-org/web`'s
> iframe-RPC proxy — see the `DataCloneError` double-proxy constraint in
> [`simulation.md`](./simulation.md).

### The broker (ngd)

- Already supports the inbox natively (`inbox_post`, `inbox_register`,
  `inbox_pop_for_user` in `engine/net/src/server_broker.rs`) — a standard `ngd`
  would route the inbox, **no broker patch needed**. The gap is in the
  verifier/SDK layer, not the broker.
- **WebSocket** daemon (`async-tungstenite`), **stateful**: RocksDB under
  `--base-path`, persisted PeerId (critical volume).
- CLI: `--local PORT`, `--domain DOMAIN:PORT,LOCAL_PORT` (behind a TLS-terminated
  reverse proxy — Traefik/Coolify).
- **Serves no static assets**: the ng-app frontend is a separate static deploy
  (`pnpm webfilebuild`). First boot is **interactive** (admin-wallet invitation
  link). Official Dockerfiles are **broken**.

## Apps & services: mono-user, no global data

NextGraph's app/service execution model — important because it **invalidates**
the idea of "a service with its own wallet sharing global data".

- **Apps AND services are mono-user.** They see only **what the user makes
  available** to them. There is **no global data** natively, and no central
  service holding shared data.
- **Local settings document.** Every app — even a singleton — and every service
  has a **settings document** the user configures it through.
- **Multi-instance apps.** A **non-singleton** app can be **instantiated several
  times** (e.g. a text editor, once per open file).
- **Singleton apps.** Also **mono-user**, but **bound to a particular user (the
  developer)**. A singleton app **can hold a global document**, administered by
  that user.

**Consequence for a "global document" (e.g. a discovery index):** the only path
glimpsed is a singleton app whose global document is administered by the
developer-user — though this is not implemented and not guaranteed (simpler
paths may exist; to explore later). The model that does exist is this
singleton-app one; a dedicated service with its own wallet sharing a
freely-readable index is not a NextGraph shape (a service is mono-user, no global
data). This is why a global-index package is deferred in this lib (see the top-level
README).

## Third-party wallet auto-import constraint

Verified empirically (2026-06-17): with the **hosted** broker (`nextgraph.net`),
a third-party web app **cannot** provision/import a wallet programmatically. A
wallet must **pre-exist** in the browser before the auth redirect can succeed.

Mechanism (from `@ng-org/web`'s `ngweb.js` dist):

- **`init()` top-level REDIRECTS**: when `window.self === window.top` it does
  `window.location.href = https://nextgraph.net/redir/#/?o=<url>`. The app's code
  stops running.
- **Every `ng.*` method is relayed** by `parent.postMessage` to `nextgraph.net`,
  and the handler **throws `"you must call init() first"` until a session is
  established** (internal `d !== false` guard). This includes
  `wallet_import_from_code`, `add_in_memory_wallet`, `session_in_memory_start`.
- The third-party app runs **inside the iframe only AFTER** the broker has opened
  a wallet and established the session. There is **no window** where our code runs
  *before* the broker's wallet gate → **nothing to hook an auto-import onto**.

Of the wallet-import methods offered on `nextgraph.eu`, only the **wallet FILE**
(`.ngw`) is a static, reusable export; TextCode/QR are temporary device↔device
transfers (5 min, both devices online, single use) — unusable to embed. The only
real way to eliminate the cross-origin round-trip is to self-host/fork the ng-app
(see [`fork-inbox-fallback.md`](./fork-inbox-fallback.md)).

## Login is not programmable

NextGraph login is a web redirect to the broker page (`nextgraph.net`). There
is no way to open a wallet silently — at least one broker-redirect pass per device
is required. Session persistence: the wallet is remembered iframe-side
(`localStorage` long-term + `sessionStorage` for the active session); on reload,
`init()` recovers the session without re-triggering the redirect while the
broker session exists (`sdk/js/web/src/index.ts`, `sdk/js/api-web/main.ts`). A
full browser restart (losing `sessionStorage`) can re-trigger the gate. A real
logout is exposed (`ng.session_stop()`, `ng.user_disconnect()`,
`ng.wallet_close()` in `sdk/js/lib-wasm/src/lib.rs`) but forces a new
redirect afterwards. This lib's identity store sidesteps all of it — the identity
id is set at wallet-import time and relayed to the lib, without a separate login;
see the identity store in [`simulation.md`](./simulation.md).