ng-eventually/docs/nextgraph-current-state.md

Current-state NextGraph — what the SDK/broker do and do NOT expose

Owner: this library. @ng-eventually/client exists because the current NextGraph JS SDK is immature. This file is the authoritative reference on what today's SDK/broker actually give us — the ground truth every polyfill in this lib compensates for. Read simulation.md for how we emulate the mature behaviour on top of these limits, and migration-guide.md for what changes when they lift.

Verified against nextgraph-rs (local clone at ../nextgraph-rs, sibling of this repo) and the installed npm packages. Store/permission facts cross-checked with the official docs (Documents & Stores, Getting started).

Source pointers (nextgraph-rs)

Where the ground truth lives, so future re-verification is cheap:

• sdk/js/lib-wasm/src/lib.rs — the wasm API actually exposed to JS.
• engine/net/src/app_protocol.rs — AppRequestCommandV0 enum, NuriV0 formats.
• engine/verifier/src/request_processor.rs — the effective app_request dispatch (the truth on what is actually processed).
• engine/net/src/types.rs — inbox types (InboxPost, InboxMsg, InboxMsgContent).
• engine/verifier/src/inbox_processor.rs — inbox message handling.
• engine/verifier/src/verifier.rs:1423 — the OpenRepo TODO (cross-wallet read).
• engine/repo/src/types.rs — RootBranchV0.store: StoreOverlay (repo → its store).

The 5 store types

Every wallet has the 3 default stores out of the box (session fields private_store_id, protected_store_id, public_store_id). Group and Dialog are created on demand.

Store	Read	Write	Creation
Private	Owner only	Owner only	Default
Protected	Owner + link+permission holders	Owner + permissioned collaborators	Default
Public	Everyone, no capability	Owner only	Default
Group	Group members	Group members (collaborative)	On demand
Dialog	The two users only	The two users only	On demand

Doc citations (verbatim): Private — "only you have access to … not possible to share"; Protected — "share … but they will need a special link and permission"; Public — "equivalent to your website … without the need for special permissions"; Group — "each Group is a separate Store … documents inherit the permissions of the store"; Dialog — "hold all the data you exchange with another user (and only with that other user) … You cannot add more users".

Document = repo (there is no Document type)

"A Repo is the equivalent of an E2EE group for one and only one Document." 1 document = 1 repo (commits + permissions). Identifier: did:ng:o:<RepoID>.

There is no Document type in nextgraph-rs (verified 2026-06-29): a "document" is simply any repo. A store is a special repo (is_store=true, with Store/Overlay/User branches) — so a store is a document, but a document is not necessarily a store.

Containment (store → repos) is by REFERENCE, not by a list. A store does not hold a Vec<RepoId>: it references its repos through an RDF graph in its Overlay/User branch. Conversely each repo names its parent store via RootBranchV0.store: StoreOverlay — a repo belongs to exactly one store.

Capability / ReadCap granularity — the load-bearing fact for this lib

ReadCap = ObjectRef. Granularity is at the repo AND branch level (each branch has its own read_cap), down to the block (ObjectKey/ChaCha20 key). Write is managed at the document (repo) level.

No automatic read inheritance. Holding a store's ReadCap does not grant the repos it contains — you need each repo's own ReadCap. The optional inherit_perms_users_and_quorum_from_store: Option<ReadCap> shares only users/quorum (write/permissions), not read-cap possession. (Repos of a private_store inherit implicitly.)

Consequence for this lib's emulation (see simulation.md): the read access unit is the repo = each item's @graph — a per-document filter, never per-store and never per-item. This is exactly what caps.ts (CapRegistry) and read-filter.ts model: no store-level inheritance, purely per-document caps. In a mono-store layout (all items in one repo) the filter is therefore all-or-nothing on that document — which is the native behaviour, and why fine-grained isolation requires one document per entity. Read isolation is cryptographic in the target: with no cap for a repo, a union / reactive read returns empty (the repo is never decrypted), while a targeted read of an unheld repo returns RepoNotFound. There is no cap-introspection API — the polyfill's canRead / governsRead are emulation-only, with no NextGraph API behind them.

Store ↔ document confusion (recurring)

The isolation axis is the document (repo/@graph), never the store: a store contains several documents and does not share their read caps. See the two-axes warning in simulation.md: "multi-store" in this lib's emulation means multiple DOCUMENTS in one shared store, not multiple stores.

Capability sharing / NURI

Sharing transmits a NURI embedding the crypto capability (read and/or write). No central ACL: holding the NURI is the right. "adding permissions can be done offline"; "removing permissions … requires a SyncSignature" (synchronous).

Inbox

Every document has a native inbox. A non-editor can deposit a link (DID cap) into it without being invited as an editor; the owner moderates. NURI: did:ng:d:<inbox_id>. Content: the InboxMsgContent enum (ContactDetails, DialogRequest, Link, Patch, ServiceRequest, ExtRequest, RemoteQuery, SocialQuery…). Messages are sealed (crypto_box::seal) to the inbox pubkey, so only the owner decrypts. The from field is optional, so an anonymous sender is possible. This is the "identified if known, anonymous otherwise" behaviour native to the protocol.

The recipient's own verifier unseals each queued message and applies it inline when it processes its inbox — there is no separate curator or materialization service.

The inbox is not usable from the JS SDK

• app_request(request) is exposed, and AppRequestCommandV0::InboxPost + AppRequest::inbox_post() exist, but the verifier's request_processor has no InboxPost arm (arms actually handled: OrmStart(Discrete), Fetch, FileGet, OrmUpdate, OrmDiscreteUpdate, SocialQueryStart, QrCodeProfile(Import), Header, Create, FilePut). Sending an InboxPost triggers nothing.
• Building an InboxPost requires crypto sealing on the Rust side; no wasm helper exposes it. A high-level inbox_post_link is a proposed/future API, not yet present.
• Inbox deposit is only triggered internally by QrCodeProfileImport (post_to_inbox(new_contact_details)) and social_query_start (contact propagation via inbox).

Consequence: there is no clean way to "drop a Link" into an arbitrary document's inbox from the JS SDK today. This lib emulates the inbox instead of patching the broker — see simulation.md (emulated inbox) and fork-inbox-fallback.md (the Rust-patch path not taken). A related exposed primitive: social_query_start (a federated query via inbox up to degree hops) exists but is limited to contacts — it does not cover an anonymous notification to a non-connected host.

The query capability — ONE local store, named graphs, union queries

The single fact that makes read-time listing possible on the shared wallet, and the reason the reactive ORM must not be used as the listing primitive. Verified directly in nextgraph-rs.

One oxigraph Store per session; each repo is a NAMED GRAPH

• The verifier keeps ONE local oxigraph Store per session: graph_dataset: Option<Store> (engine/verifier/src/verifier.rs:94).
• Every synced repo's data is inserted into that one store as a distinct NAMED GRAPH keyed by the repo — update_graph writes each repo's main-branch triples under NuriV0::repo_graph_name(&repo_id, &overlay_id) (engine/verifier/src/commits/transaction.rs:646-701, the ov_graphname / repo_graph_name around line 669). So all opened repos coexist as named graphs in one dataset — a GRAPH ?g { ... } body can span them.

The NURI target decides scope: one repo vs the LOCAL UNION

sparql_query's scope is resolved from the NURI target by resolve_target_for_sparql (engine/verifier/src/request_processor.rs:256-285):

• Repo(repo_id) / PrivateStore → Some(repo_graph_name) = ONE repo's graph.
• UserSite / None → Ok(None) = the UNION of all named graphs. That None is passed to oxigraph's store.query(parsed, default_graph) (engine/oxigraph/src/oxigraph/store.rs:200) as the default graph, and sparql_query first calls dataset.set_default_graph_as_union() when the query has no explicit dataset (request_processor.rs:656-675, the if dataset.has_no_default_dataset() block). Union = "every named graph currently in the store".

The wasm binding defaults the target to UserSite when no nuri is passed

The binding (sdk/js/lib-wasm/src/lib.rs — both the nodejs variant at ~350-405 and the web variant at ~553-610) reads the target from the nuri arg: a string nuri is parsed; an absent nuri falls back to NuriV0::new_entire_user_site() = UserSite (engine/net/src/app_protocol.rs:488-490). Therefore:

sparql_query(sid, query, base, /*anchor*/ undefined) queries the LOCAL UNION across all synced/opened graphs; with a string anchor it is restricted to that one repo. A GRAPH ?g { ... } body then spans/attributes across the local graphs.

A repo is only queryable once OPENED/synced into the store

A repo's triples enter graph_dataset (hence the union) only after the repo is opened/synced into self.repos and its commits applied via update_graph.

VERIFIED (T03.k) — there is NO JS primitive to sync an unknown repo. Every JS entry point that could "open" a repo — sparql_query anchored, doc_subscribe (Fetch::Subscribe), orm_start_graph — resolves its target via resolve_target/resolve_target_for_sparql, which does self.repos.get(repo_id).ok_or(RepoNotFound) (request_processor.rs:155/163/264/269). None of them PULLS a repo that is absent from self.repos; they only touch a repo already there. The primitive that actually loads a repo from its ReadCap, Verifier::load_repo_from_read_cap (verifier.rs:2237), is pub(crate) — unexposed to JS; it is only reached internally (bootstrap, inbox processing). So from JS today a repo becomes queryable ONLY by being doc_created in this session (own docs) or synced by an internal path — never on demand by NURI+ReadCap.

Consequence for this lib's mono-wallet polyfill: every account's documents are doc_created in the one shared wallet within the same session, so they are all already in self.repos. read-model.ts reads the bounded, by-need set of docs with one anchored sparql_query per doc (SELECT ?s ?p ?o WHERE { ?s ?p ?o }, anchor = the doc NURI): the anchor resolves that same-session repo directly (no separate open needed) and restricts the query to its graph, so it is O(1) per doc, independent of the store's size. An absent repo throws RepoNotFound on its own read and is skipped, never aborting the batch.

The read path avoids an anchorless union-scan. An anchorless SELECT … WHERE { GRAPH ?g { ?s ?p ?o } } spans every named graph in the store — O(wallet size). On a shared wallet that accumulates docs across runs that cost grows with the whole wallet, which is why the read path is per-doc anchored: the anchored read makes a non-empty wallet irrelevant. At the real multi-store migration this is unchanged (the anchored read is native); only bringing a repo into the session changes: opening a real per-user store repo by cap becomes a native broker sync (the OpenRepo TODO at verifier.rs:1423). Opening still requires the repo's NURI + ReadCap — there is no store-level read inheritance (see § Capability / ReadCap granularity).

Findable-without-lookup vs subscribable (first-State barrier) — DISJOINT

Two properties a fresh session might want from a document, and no single document has both:

• Findable without a lookup. The ONLY NURI a fresh session can NAME with nothing but the session in hand is the store-root, did:ng:${privateStoreId} (from session.private_store_id). Any per-document repo is did:ng:o:<RepoID> with a random RepoID minted by doc_create — not derivable, so it must be looked up somewhere first.
• Subscribable with a sync BARRIER. doc_subscribe(nuri) delivers TabInfo then an initial State (verifier.rs:470/:476); that first State is the sync barrier — after it, presence is guaranteed and absence is definitive (pinned empirically by CONTRACT 3 in packages/client/e2e/). But this barrier exists only for a repo doc_subscribe can open, i.e. a did:ng:o:<RepoID> repo. A store-root has no first-State barrier: an anchored read on it can return 0 rows during sync-lag with no signal distinguishing "still syncing" from "genuinely empty".

These two are mutually exclusive: the guessable target (store-root) is not barrier-authoritative, and the barrier-authoritative target (o: repo) is not guessable. Consequence: you cannot build a lookup table that is BOTH reachable cold (findable) AND authoritative on a cold read (barrier). A cold "0 rows" read of a store-root graph is therefore fundamentally ambiguous — which is the trap the shim's account map fell into (see next section).

The pointer → doc-shim indirection (how the polyfill shim resolves accounts)

store-registry.ts keeps a map identifier → {docPublic, docProtected, docPrivate} (the "shim", the account→document trust root). It must be reachable by a fresh reconnecting session (findable) AND authoritative on a cold read (so a fresh page does not mistake sync-lag for "account absent" and PROVISION a fork). Since no single document is both (previous section), the shim uses an indirection:

1. doc-shim — a doc_created graph document (did:ng:o:..., hence a first-State barrier). All AccountRecords live inside it. Because it is subscribable, an anchored read behind its ensureRepoOpen barrier is authoritative: a cold 0 means the account is genuinely absent.
2. pointer — a single well-known, write-once triple in the store-root graph, <urn:ng-eventually:shim:root> <urn:ng-eventually:shim:shimDoc> <docShimNuri>. The store-root is findable-without-lookup, so a fresh session can always read it; the pointer being the OLDEST, write-once triple in that graph, it is near-always already synced on a cold read.

Resolution (resolveShimDoc): read the pointer from the store-root → open the named doc-shim through its barrier (ensureRepoOpen) → read the account AUTHORITATIVELY. First login (no pointer): doc_create the doc-shim, publish the pointer, done. A pointer fork (two devices each writing a pointer before either synced) is reconciled to the lexicographically-smallest doc-shim NURI (content-addressed, so every device converges on the same doc-shim).

The account-level retry is GONE. Before this indirection the shim lived directly in the store-root graph, so an account read had no barrier and a cold 0 was ambiguous; the lib compensated with a bounded account-level retry (provisionRetry / resolveAccountReliably) that re-read the account several times before concluding "new". Moving the records behind the doc-shim barrier makes the account read authoritative on the FIRST read, so that retry was removed — a barrier is deterministic where a retry only guessed. The only residual bounded guard is a small re-read of the pointer itself (pointerGuard, one write-once triple): it can NEVER re-provision or fork an account — at worst it takes a couple extra reads to see a pointer that is still landing.

No legacy migration. A wallet written under the OLD scheme (accounts directly in the store-root graph, no pointer) is NOT recovered: opening it under the current scheme simply provisions a fresh doc-shim, and the pre-indirection store-root records are ignored. This is deliberate — the only such wallets are dev data — so the resolution path carries no legacy-migration step; it always reads the account authoritatively from the doc-shim.

The union is read-only — writes must target one document

resolve_target_for_sparql(update=true) returns InvalidTarget for UserSite / None (request_processor.rs:275-282). So sparql_update cannot write "to the union": every write must name one document's @graph — exactly what the polyfill's docs.sparqlUpdate already does.

No reactive SPARQL — sparql_query is one-shot

sparql_query is non-streamed: it computes a QueryResults and returns once (lib-wasm/src/lib.rs:352-405 / 553-610). There is no "subscribe to a union query". The only reactive primitives are the streamed ones: orm_start_graph, orm_start_discrete, doc_subscribe, app_request_stream.

The ORM fan-out hang — verified root cause

The reactive ORM is structurally unfit for a fan-out of per-entity / not-yet-synced graphs, and this is why subscribing such a fan-out hangs:

• OrmStartGraph first loops over every graph in the requested scope and calls open_for_target(&nuri.target, /*publisher*/ true) on each (request_processor.rs:53-66), and orm/graph/initialize.rs does the same fan-out again for the graphs the ORM discovers (~125-128).
• open_for_target → resolve_target → self.repos.get(repo_id).ok_or(RepoNotFound) (request_processor.rs:286-294 calling resolve_target at :147, the RepoNotFound at :155/:163).
• A freshly-created per-entity doc, or any not-yet-synced other-account doc, is absent from self.repos, so RepoNotFound propagates through the ? and aborts the whole orm_start_graph. The subscription never emits its initial, so the ORM readyPromise never resolves and the subscription hangs when a fan-out of per-entity graphs is passed in.

Consequence: passing per-entity / unsynced graphs to the reactive ORM is broken. Listing must go through a one-shot union sparql_query instead — see read-model.md.

JS SDK limits (@ng-org/web)

@ng-org/web (verified 0.1.2-alpha.13 = upstream/main at 2026-05-21, the installed version) does NOT expose: Group/Dialog store creation; capability sharing (a NURI with rights); permission manipulation; inbox deposit/read.

Available JS methods: doc_create, doc_subscribe, sparql_query, sparql_update, orm_start_graph, orm_start_discrete, graph_orm_update, discrete_orm_update, file_get, app_request_stream. The docs announce "An API will be provided for permission manipulation" (no date).

Integration & deployment model

NextGraph is consumed via an iframe proxy (@ng-org/web): the third-party app contains no engine, it delegates to a hosted ng-app (default nextgraph.net) that runs the engine in an iframe.

The JS packages

• @ng-org/web — published. Lightweight postMessage proxy (no wasm embedded). The third-party integration path; @ng-org/orm and every example depend on it. This lib wraps it.
• @ng-org/api-web — private (unpublished). Full in-browser engine (loads @ng-org/lib-wasm in a Web Worker). Consumed only by app/nextgraph (the ng-app frontend) — not a third-party integration target.
• @ng-org/lib-wasm — the compiled wasm engine (contains the verifier). Source sdk/js/lib-wasm/.
• nextgraph (npm) — the NodeJS API (pkg-node build).
• @ng-org/orm — reactive ORM (useShape…), built on @ng-org/web.

Where the verifier runs

In the standard web model, the verifier runs in the iframe: app/nextgraph loads api-web → lib-wasm in a Web Worker, browser-side. The broker (ngd) only does transport and storage.

Consequence: changing verifier logic (request_processor, inbox_processor) means rebuilding the ng-app, not the broker.

iframe model & build-time retargeting

@ng-org/web redirects to the hosted ng-app, which reloads the third-party app in an iframe after auth, then relays over postMessage. Retargetable at build time (sdk/js/web/src/index.ts, import.meta.env):

Variable	Target
NG_REDIR_SERVER	default nextgraph.net
NG_DEV3	127.0.0.1:3033
NG_DEV	localhost:14402/14404
NG_DEV_LOCAL_BROKER	localhost:1421

No runtime override — init() takes no broker URL. To point at a self-hosted ng-app: rebuild @ng-org/web (pure TS, no wasm → trivial build).

Proxy ↔ iframe ↔ worker plumbing (generic)

The call path is entirely generic (no allowlist): @ng-org/web is a JS Proxy relaying any method name over postMessage; app/nextgraph dispatches via Reflect.apply(ng[method], …). So a new wasm function in simple request/response form is reachable without touching the JS — but that's an untyped hack (quick test, not a plan). The streamed case needs an entry on both sides (E in @ng-org/web + streamed_api in api-web; current streamed methods: doc_subscribe, orm_start_graph, orm_start_discrete, file_get, app_request_stream).

This is exactly why docs.ts in this lib calls the real injected ng directly and never layers our own Proxy on top of @ng-org/web's iframe-RPC proxy — see the DataCloneError double-proxy constraint in simulation.md.

The broker (ngd)

• Already supports the inbox natively (inbox_post, inbox_register, inbox_pop_for_user in engine/net/src/server_broker.rs) — a standard ngd would route the inbox, no broker patch needed. The gap is in the verifier/SDK layer, not the broker.
• WebSocket daemon (async-tungstenite), stateful: RocksDB under --base-path, persisted PeerId (critical volume).
• CLI: --local PORT, --domain DOMAIN:PORT,LOCAL_PORT (behind a TLS-terminated reverse proxy — Traefik/Coolify).
• Serves no static assets: the ng-app frontend is a separate static deploy (pnpm webfilebuild). First boot is interactive (admin-wallet invitation link). Official Dockerfiles are broken.

Apps & services: mono-user, no global data

NextGraph's app/service execution model — important because it invalidates the idea of "a service with its own wallet sharing global data".

• Apps AND services are mono-user. They see only what the user makes available to them. There is no global data natively, and no central service holding shared data.
• Local settings document. Every app — even a singleton — and every service has a settings document the user configures it through.
• Multi-instance apps. A non-singleton app can be instantiated several times (e.g. a text editor, once per open file).
• Singleton apps. Also mono-user, but bound to a particular user (the developer). A singleton app can hold a global document, administered by that user.

Consequence for a "global document" (e.g. a discovery index): the only path glimpsed is a singleton app whose global document is administered by the developer-user — though this is not implemented and not guaranteed (simpler paths may exist; to explore later). The model that does exist is this singleton-app one; a dedicated service with its own wallet sharing a freely-readable index is not a NextGraph shape (a service is mono-user, no global data). This is why a global-index package is deferred in this lib (see the top-level README).

Third-party wallet auto-import constraint

Verified empirically (2026-06-17): with the hosted broker (nextgraph.net), a third-party web app cannot provision/import a wallet programmatically. A wallet must pre-exist in the browser before the auth redirect can succeed.

Mechanism (from @ng-org/web's ngweb.js dist):

• init() top-level REDIRECTS: when window.self === window.top it does window.location.href = https://nextgraph.net/redir/#/?o=<url>. The app's code stops running.
• Every ng.* method is relayed by parent.postMessage to nextgraph.net, and the handler throws "you must call init() first" until a session is established (internal d !== false guard). This includes wallet_import_from_code, add_in_memory_wallet, session_in_memory_start.
• The third-party app runs inside the iframe only AFTER the broker has opened a wallet and established the session. There is no window where our code runs before the broker's wallet gate → nothing to hook an auto-import onto.

Of the wallet-import methods offered on nextgraph.eu, only the wallet FILE (.ngw) is a static, reusable export; TextCode/QR are temporary device↔device transfers (5 min, both devices online, single use) — unusable to embed. The only real way to eliminate the cross-origin round-trip is to self-host/fork the ng-app (see fork-inbox-fallback.md).

Login is not programmable

NextGraph login is a web redirect to the broker page (nextgraph.net). There is no way to open a wallet silently — at least one broker-redirect pass per device is required. Session persistence: the wallet is remembered iframe-side (localStorage long-term + sessionStorage for the active session); on reload, init() recovers the session without re-triggering the redirect while the broker session exists (sdk/js/web/src/index.ts, sdk/js/api-web/main.ts). A full browser restart (losing sessionStorage) can re-trigger the gate. A real logout is exposed (ng.session_stop(), ng.user_disconnect(), ng.wallet_close() in sdk/js/lib-wasm/src/lib.rs) but forces a new redirect afterwards. This lib's identity store sidesteps all of it — the identity id is set at wallet-import time and relayed to the lib, without a separate login; see the identity store in simulation.md.