ng-eventually/docs/nextgraph-current-state.md

# Current-state NextGraph — what the SDK/broker do and do NOT expose

**Owner:** this library. `@ng-eventually/client` exists because the *current*
NextGraph JS SDK is immature. This file is the authoritative reference on what
today's SDK/broker actually give us — the ground truth every polyfill in this
lib compensates for. Read [`simulation.md`](./simulation.md) for how we emulate
the mature behaviour on top of these limits, and
[`migration-guide.md`](./migration-guide.md) for what changes when they lift.

Verified against `nextgraph-rs` (local clone at `../nextgraph-rs`, sibling of
this repo) and the installed npm packages. Store/permission facts cross-checked
with the official docs ([Documents & Stores](https://docs.nextgraph.org/en/documents/),
[Getting started](https://docs.nextgraph.org/en/getting-started/)).

## Source pointers (`nextgraph-rs`)

Where the ground truth lives, so future re-verification is cheap:

- `sdk/js/lib-wasm/src/lib.rs` — the wasm API actually exposed to JS.
- `engine/net/src/app_protocol.rs` — `AppRequestCommandV0` enum, `NuriV0` formats.
- `engine/verifier/src/request_processor.rs` — the effective `app_request`
  dispatch (the truth on what is actually *processed*).
- `engine/net/src/types.rs` — inbox types (`InboxPost`, `InboxMsg`, `InboxMsgContent`).
- `engine/verifier/src/inbox_processor.rs` — inbox message handling.
- `engine/verifier/src/verifier.rs:1423` — the `OpenRepo` TODO (cross-wallet read).
- `engine/repo/src/types.rs` — `RootBranchV0.store: StoreOverlay` (repo → its store).

## The 5 store types

Every wallet has the **3 default stores** out of the box (session fields
`private_store_id`, `protected_store_id`, `public_store_id`). Group and Dialog
are created on demand.

| Store | Read | Write | Creation |
|---|---|---|---|
| **Private** | Owner only | Owner only | Default |
| **Protected** | Owner + link+permission holders | Owner + permissioned collaborators | Default |
| **Public** | Everyone, no capability | Owner only | Default |
| **Group** | Group members | Group members (collaborative) | On demand |
| **Dialog** | The two users only | The two users only | On demand |

Doc citations (verbatim): Private — *"only you have access to … not possible to
share"*; Protected — *"share … but they will need a special link and permission"*;
Public — *"equivalent to your website … without the need for special permissions"*;
Group — *"each Group is a separate Store … documents inherit the permissions of
the store"*; Dialog — *"hold all the data you exchange with another user (and only
with that other user) … You cannot add more users"*.

## Document = repo (there is no `Document` type)

*"A Repo is the equivalent of an E2EE group for one and only one Document."*
**1 document = 1 repo** (commits + permissions). Identifier: `did:ng:o:<RepoID>`.

There is **no `Document` type** in `nextgraph-rs` (verified 2026-06-29): a
"document" is simply *any repo*. A **store is a special repo** (`is_store=true`,
with `Store`/`Overlay`/`User` branches) — so *a store is a document, but a
document is not necessarily a store*.

**Containment (store → repos) is by REFERENCE, not by a list.** A store does not
hold a `Vec<RepoId>`: it references its repos through an **RDF graph** in its
Overlay/User branch. Conversely each repo names its parent store via
`RootBranchV0.store: StoreOverlay` — **a repo belongs to exactly one store**.

## Capability / ReadCap granularity — the load-bearing fact for this lib

`ReadCap = ObjectRef`. Granularity is at the **repo AND branch** level (each
branch has its own `read_cap`), down to the **block** (`ObjectKey`/ChaCha20 key).
Write is managed at the **document (repo)** level.

**No automatic read inheritance.** Holding a **store's** ReadCap does **not**
grant the repos it contains — **you need each repo's own ReadCap**. The optional
`inherit_perms_users_and_quorum_from_store: Option<ReadCap>` shares only
users/quorum (write/permissions), **not** read-cap possession. (Repos of a
`private_store` inherit implicitly.)

> **Consequence for this lib's emulation (see [`simulation.md`](./simulation.md)):**
> the read access UNIT is the **repo = each item's `@graph`** — a per-DOCUMENT
> filter, never per-store and never per-item. This is exactly what
> `caps.ts` (`CapRegistry`) and `read-filter.ts` model: no store-level
> inheritance, purely per-document caps. In a mono-store layout (all items in one
> repo) the filter is therefore all-or-nothing on that document — which *is* the
> native behaviour, and why fine-grained isolation requires one document per
> entity.

### Store ↔ document confusion (recurring)

The isolation axis is the **document (repo/`@graph`)**, never the **store**: a
store *contains* several documents and does not share their read caps. See the
two-axes warning in [`simulation.md`](./simulation.md): "multi-store" in this
lib's emulation means **multiple DOCUMENTS in one shared store**, not multiple
stores.

## Capability sharing / NURI

Sharing transmits a **NURI** embedding the crypto capability (read and/or write).
No central ACL: holding the NURI *is* the right. *"adding permissions can be done
offline"*; *"removing permissions … requires a SyncSignature"* (synchronous).

## Inbox

**Every document has a native inbox.** A non-editor can **deposit a link (DID
cap)** into it without being invited as an editor; the owner **moderates**. NURI:
`did:ng:d:<inbox_id>`. Content: the `InboxMsgContent` enum (`ContactDetails`,
`DialogRequest`, **`Link`**, `Patch`, `ServiceRequest`, `ExtRequest`,
`RemoteQuery`, `SocialQuery`…). Messages are **sealed** (`crypto_box::seal`) to
the inbox pubkey → only the owner decrypts. The `from` field is **optional** → an
**anonymous** sender is possible. This is the "identified if known, anonymous
otherwise" behaviour native to the protocol.

### The inbox is NOT usable from the JS SDK

- `app_request(request)` is exposed, and `AppRequestCommandV0::InboxPost` +
  `AppRequest::inbox_post()` exist. **BUT** the verifier's `request_processor`
  has **no `InboxPost` arm** (arms actually handled: `OrmStart(Discrete)`,
  `Fetch`, `FileGet`, `OrmUpdate`, `OrmDiscreteUpdate`, `SocialQueryStart`,
  `QrCodeProfile(Import)`, `Header`, `Create`, `FilePut`). Sending an `InboxPost`
  triggers nothing.
- Building an `InboxPost` requires crypto sealing on the Rust side; **no wasm
  helper** exposes it.
- Inbox deposit is only triggered **internally** by `QrCodeProfileImport`
  (`post_to_inbox(new_contact_details)`) and `social_query_start` (contact
  propagation via inbox).

**Consequence:** there is no clean way to "drop a Link" into an arbitrary
document's inbox from the JS SDK today. This lib emulates the inbox instead of
patching the broker — see [`simulation.md`](./simulation.md) (emulated inbox +
curator) and [`fork-inbox-fallback.md`](./fork-inbox-fallback.md) (the Rust-patch
path NOT taken). A related exposed primitive: `social_query_start` (a federated
query via inbox up to `degree` hops) exists but is limited to **contacts** — it
does not cover an anonymous notification to a non-connected host.

## The query capability — ONE local store, named graphs, union queries

The single fact that makes read-time *listing* possible on the shared wallet, and
the reason the reactive ORM must **not** be used as the listing primitive. Verified
directly in `nextgraph-rs`.

### One oxigraph Store per session; each repo is a NAMED GRAPH

- The verifier keeps **ONE** local oxigraph `Store` per session:
  `graph_dataset: Option<Store>` (`engine/verifier/src/verifier.rs:94`).
- Every synced repo's data is inserted into that one store as a **distinct NAMED
  GRAPH keyed by the repo** — `update_graph` writes each repo's main-branch triples
  under `NuriV0::repo_graph_name(&repo_id, &overlay_id)`
  (`engine/verifier/src/commits/transaction.rs:646-701`, the `ov_graphname` /
  `repo_graph_name` around line 669). So all opened repos coexist as named graphs in
  one dataset — a `GRAPH ?g { ... }` body can span them.

### The NURI target decides scope: one repo vs the LOCAL UNION

`sparql_query`'s scope is resolved from the NURI target by
`resolve_target_for_sparql` (`engine/verifier/src/request_processor.rs:256-285`):

- `Repo(repo_id)` / `PrivateStore` → `Some(repo_graph_name)` = **ONE repo's graph**.
- `UserSite` / `None` → `Ok(None)` = the **UNION of all named graphs**. That `None`
  is passed to oxigraph's `store.query(parsed, default_graph)`
  (`engine/oxigraph/src/oxigraph/store.rs:200`) as the default graph, and
  `sparql_query` first calls `dataset.set_default_graph_as_union()` when the query
  has no explicit dataset (`request_processor.rs:656-675`, the
  `if dataset.has_no_default_dataset()` block). Union = "every named graph currently
  in the store".

### The wasm binding defaults the target to UserSite when no `nuri` is passed

The binding (`sdk/js/lib-wasm/src/lib.rs` — both the nodejs variant at ~`350-405`
and the web variant at ~`553-610`) reads the target from the `nuri` arg: a string
`nuri` is parsed; an **absent** `nuri` falls back to
`NuriV0::new_entire_user_site()` = `UserSite`
(`engine/net/src/app_protocol.rs:488-490`). Therefore:

> **`sparql_query(sid, query, base, /*anchor*/ undefined)` queries the LOCAL UNION
> across all synced/opened graphs; with a string anchor it is restricted to that one
> repo.** A `GRAPH ?g { ... }` body then spans/attributes across the local graphs.

### A repo is only queryable once OPENED/synced into the store

A repo's triples enter `graph_dataset` (hence the union) only after the repo is
opened/synced into `self.repos` **and** its commits applied via `update_graph`.
Paths that open a repo: `doc_create` (own docs), bootstrap /
`load_repo_from_read_cap`, or being followed from a store's `ldp:contains` / a
shared cap / an inbox. Opening requires possessing the repo's **NURI + ReadCap** —
there is **no store-level read inheritance** (see § Capability / ReadCap
granularity). *(INFERRED: that "following a graph reference makes a previously
unknown repo known/openable" is the one step not read verbatim in source; the
open-requires-cap and no-inheritance facts around it ARE verified.)*

### The union is READ-ONLY — writes must target one document

`resolve_target_for_sparql(update=true)` returns `InvalidTarget` for `UserSite` /
`None` (`request_processor.rs:275-282`). So `sparql_update` cannot write "to the
union": every write must name **one** document's `@graph` — exactly what the
polyfill's `docs.sparqlUpdate` already does.

### No reactive SPARQL — `sparql_query` is one-shot

`sparql_query` is non-streamed: it computes a `QueryResults` and returns once
(`lib-wasm/src/lib.rs:352-405` / `553-610`). There is **no** "subscribe to a union
query". The only reactive primitives are the streamed ones: `orm_start_graph`,
`orm_start_discrete`, `doc_subscribe`, `app_request_stream`.

### The ORM fan-out hang — verified root cause

The reactive ORM is structurally unfit for a fan-out of per-entity / not-yet-synced
graphs, and this is *why* subscribing such a fan-out hangs:

- `OrmStartGraph` first loops over EVERY graph in the requested scope and calls
  `open_for_target(&nuri.target, /*publisher*/ true)` on each
  (`request_processor.rs:53-66`), and `orm/graph/initialize.rs` does the same
  fan-out again for the graphs the ORM discovers (~`125-128`).
- `open_for_target` → `resolve_target` → `self.repos.get(repo_id).ok_or(RepoNotFound)`
  (`request_processor.rs:286-294` calling `resolve_target` at `:147`, the
  `RepoNotFound` at `:155/:163`).
- A **freshly-created per-entity doc**, or any **not-yet-synced other-account doc**,
  is absent from `self.repos` → `RepoNotFound` propagates through the `?` and
  **aborts the whole `orm_start_graph`**. The subscription never emits its initial →
  the ORM `readyPromise` never resolves → the multi-second hang observed (≈75s)
  when subscribing a fan-out of per-entity graphs.

**Consequence:** passing per-entity / unsynced graphs to the reactive ORM is broken.
Listing must go through a one-shot union `sparql_query` instead — see
[`read-model.md`](./read-model.md).

## JS SDK limits (`@ng-org/web`)

`@ng-org/web` (verified `0.1.2-alpha.13` = `upstream/main` at 2026-05-21, the
installed version) **does NOT expose**: Group/Dialog store creation; capability
sharing (a NURI with rights); permission manipulation; inbox deposit/read.

Available JS methods: `doc_create`, `doc_subscribe`, `sparql_query`,
`sparql_update`, `orm_start_graph`, `orm_start_discrete`, `graph_orm_update`,
`discrete_orm_update`, `file_get`, `app_request_stream`. The docs announce *"An
API will be provided for permission manipulation"* (no date).

## Integration & deployment model

NextGraph is consumed via an **iframe proxy** (`@ng-org/web`): the third-party
app contains no engine, it delegates to a hosted ng-app (default `nextgraph.net`)
that runs the engine in an iframe.

### The JS packages

- **`@ng-org/web`** — **published**. Lightweight postMessage proxy (no wasm
  embedded). **The** third-party integration path; `@ng-org/orm` and every
  example depend on it. This lib wraps it.
- **`@ng-org/api-web`** — **private** (unpublished). Full in-browser engine
  (loads `@ng-org/lib-wasm` in a Web Worker). Consumed only by `app/nextgraph`
  (the ng-app frontend) — **not** a third-party integration target.
- **`@ng-org/lib-wasm`** — the compiled wasm engine (contains the verifier).
  Source `sdk/js/lib-wasm/`.
- **`nextgraph`** (npm) — the NodeJS API (`pkg-node` build).
- **`@ng-org/orm`** — reactive ORM (`useShape`…), built on `@ng-org/web`.

### Where the verifier runs

In the standard web model, the verifier runs **in the iframe**: `app/nextgraph`
loads `api-web` → `lib-wasm` in a Web Worker, browser-side. The broker (`ngd`)
only does **transport and storage**.

**Consequence:** changing verifier logic (`request_processor`,
`inbox_processor`) means rebuilding the **ng-app**, not the broker.

### iframe model & build-time retargeting

`@ng-org/web` redirects to the hosted ng-app, which reloads the third-party app
in an iframe after auth, then relays over `postMessage`. **Retargetable at build
time** (`sdk/js/web/src/index.ts`, `import.meta.env`):

| Variable | Target |
|---|---|
| `NG_REDIR_SERVER` | default `nextgraph.net` |
| `NG_DEV3` | `127.0.0.1:3033` |
| `NG_DEV` | `localhost:14402`/`14404` |
| `NG_DEV_LOCAL_BROKER` | `localhost:1421` |

**No runtime override** — `init()` takes no broker URL. To point at a
self-hosted ng-app: **rebuild `@ng-org/web`** (pure TS, no wasm → trivial build).

### Proxy ↔ iframe ↔ worker plumbing (generic)

The call path is **entirely generic** (no allowlist): `@ng-org/web` is a JS
`Proxy` relaying *any* method name over `postMessage`; `app/nextgraph` dispatches
via `Reflect.apply(ng[method], …)`. So a new wasm function in simple
request/response form is *reachable* without touching the JS — but that's an
untyped **hack** (quick test, not a plan). The **streamed** case needs an entry
on both sides (`E` in `@ng-org/web` + `streamed_api` in api-web; current streamed
methods: `doc_subscribe`, `orm_start_graph`, `orm_start_discrete`, `file_get`,
`app_request_stream`).

> This is exactly why `docs.ts` in this lib calls the **real injected `ng`**
> directly and never layers our own `Proxy` on top of `@ng-org/web`'s
> iframe-RPC proxy — see the `DataCloneError` double-proxy constraint in
> [`simulation.md`](./simulation.md).

### The broker (ngd)

- Already supports the inbox natively (`inbox_post`, `inbox_register`,
  `inbox_pop_for_user` in `engine/net/src/server_broker.rs`) — a standard `ngd`
  would route the inbox, **no broker patch needed**. The gap is in the
  verifier/SDK layer, not the broker.
- **WebSocket** daemon (`async-tungstenite`), **stateful**: RocksDB under
  `--base-path`, persisted PeerId (critical volume).
- CLI: `--local PORT`, `--domain DOMAIN:PORT,LOCAL_PORT` (behind a TLS-terminated
  reverse proxy — Traefik/Coolify).
- **Serves no static assets**: the ng-app frontend is a separate static deploy
  (`pnpm webfilebuild`). First boot is **interactive** (admin-wallet invitation
  link). Official Dockerfiles are **broken**.

## Apps & services: mono-user, no global data

NextGraph's app/service execution model — important because it **invalidates**
the idea of "a service with its own wallet sharing global data".

- **Apps AND services are mono-user.** They see only **what the user makes
  available** to them. There is **no global data** natively, and no central
  service holding shared data.
- **Local settings document.** Every app — even a singleton — and every service
  has a **settings document** the user configures it through.
- **Multi-instance apps.** A **non-singleton** app can be **instantiated several
  times** (e.g. a text editor, once per open file).
- **Singleton apps.** Also **mono-user**, but **bound to a particular user (the
  developer)**. A singleton app **can hold a global document**, administered by
  that user.

**Consequence for a "global document" (e.g. a discovery index):** the only path
glimpsed is a **singleton app** whose global document is administered by the
developer-user — **but this is not implemented and not guaranteed** (simpler
paths may exist; to explore later). The **incorrect** model to avoid: "a
dedicated service with its own wallet sharing a freely-readable index" — that
does not exist in NextGraph (a service is mono-user, no global data). This is why
a global-index curator package is **deferred** in this lib (see the top-level
README).

## Third-party wallet auto-import constraint

Verified empirically (2026-06-17): with the **hosted** broker (`nextgraph.net`),
a third-party web app **cannot** provision/import a wallet programmatically. A
wallet must **pre-exist** in the browser before the auth redirect can succeed.

Mechanism (from `@ng-org/web`'s `ngweb.js` dist):

- **`init()` top-level REDIRECTS**: when `window.self === window.top` it does
  `window.location.href = https://nextgraph.net/redir/#/?o=<url>`. The app's code
  stops running.
- **Every `ng.*` method is relayed** by `parent.postMessage` to `nextgraph.net`,
  and the handler **throws `"you must call init() first"` until a session is
  established** (internal `d !== false` guard). This includes
  `wallet_import_from_code`, `add_in_memory_wallet`, `session_in_memory_start`.
- The third-party app runs **inside the iframe only AFTER** the broker has opened
  a wallet and established the session. There is **no window** where our code runs
  *before* the broker's wallet gate → **nothing to hook an auto-import onto**.

Of the wallet-import methods offered on `nextgraph.eu`, only the **wallet FILE**
(`.ngw`) is a static, reusable export; TextCode/QR are temporary device↔device
transfers (5 min, both devices online, single use) — unusable to embed. The only
real way to eliminate the cross-origin round-trip is to self-host/fork the ng-app
(see [`fork-inbox-fallback.md`](./fork-inbox-fallback.md)).

## Login is not programmable

NextGraph login is a **web redirect** to the broker page (`nextgraph.net`). There
is no way to open a wallet silently — at least one broker-redirect pass per device
is required. Session persistence: the wallet is remembered iframe-side
(`localStorage` long-term + `sessionStorage` for the active session); on reload,
`init()` recovers the session **without re-triggering the redirect** while the
broker session exists (`sdk/js/web/src/index.ts`, `sdk/js/api-web/main.ts`). A
full browser restart (losing `sessionStorage`) can re-trigger the gate. A real
logout IS exposed (`ng.session_stop()`, `ng.user_disconnect()`,
`ng.wallet_close()` in `sdk/js/lib-wasm/src/lib.rs`) but **forces a new
redirect** afterwards. This lib's faux login sidesteps all of it — see the faux
login in [`simulation.md`](./simulation.md).