docs: own the current-NextGraph-state knowledge + boundary (lib side)

This library presents a mature-NextGraph SDK face to consumers while compensating for the current SDK's gaps via a shared-wallet simulation. It therefore OWNS all current-state + simulation knowledge — moved here out of the Festipod app repo, which must treat this library as a finished SDK. New docs/: - nextgraph-current-state.md — what the current SDK/broker do and don't expose (5 store types, document=repo, per-document ReadCap, inbox not exposed, iframe RPC proxy, mono-user/no-global-data, wallet import constraint). Keeps the nextgraph-rs source pointers. - simulation.md — how the lib emulates the mature behaviour on one shared wallet (shim, store!=document two axes, docCreate→private store, RepoNotFound scope rule, @ng-org double-proxy DataCloneError, emulated ReadCap/inbox/curator). - decisions/ — the current-SDK ADRs (private-store-nuri-scope, sparql-delete, shared-wallet-login, discovery mechanism). - fork-inbox-fallback.md — the Rust-patch/self-host route not taken. - migration-guide.md — the checklist for when real NextGraph matures. README: boundary framing from the lib's side + docs/ index; replaced the stale "scaffold/stubbed" status with the actually-implemented mechanisms per source. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-03 23:23:23 +02:00
parent d804a436d7
commit bea9f51d91
9 changed files with 1021 additions and 4 deletions
@@ -0,0 +1,279 @@
+# Current-state NextGraph — what the SDK/broker do and do NOT expose
+
+**Owner:** this library. `@ng-eventually/client` exists because the *current*
+NextGraph JS SDK is immature. This file is the authoritative reference on what
+today's SDK/broker actually give us — the ground truth every polyfill in this
+lib compensates for. Read [`simulation.md`](./simulation.md) for how we emulate
+the mature behaviour on top of these limits, and
+[`migration-guide.md`](./migration-guide.md) for what changes when they lift.
+
+Verified against `nextgraph-rs` (local clone at `../nextgraph-rs`, sibling of
+this repo) and the installed npm packages. Store/permission facts cross-checked
+with the official docs ([Documents & Stores](https://docs.nextgraph.org/en/documents/),
+[Getting started](https://docs.nextgraph.org/en/getting-started/)).
+
+## Source pointers (`nextgraph-rs`)
+
+Where the ground truth lives, so future re-verification is cheap:
+
+- `sdk/js/lib-wasm/src/lib.rs` — the wasm API actually exposed to JS.
+- `engine/net/src/app_protocol.rs` — `AppRequestCommandV0` enum, `NuriV0` formats.
+- `engine/verifier/src/request_processor.rs` — the effective `app_request`
+  dispatch (the truth on what is actually *processed*).
+- `engine/net/src/types.rs` — inbox types (`InboxPost`, `InboxMsg`, `InboxMsgContent`).
+- `engine/verifier/src/inbox_processor.rs` — inbox message handling.
+- `engine/verifier/src/verifier.rs:1423` — the `OpenRepo` TODO (cross-wallet read).
+- `engine/repo/src/types.rs` — `RootBranchV0.store: StoreOverlay` (repo → its store).
+
+## The 5 store types
+
+Every wallet has the **3 default stores** out of the box (session fields
+`private_store_id`, `protected_store_id`, `public_store_id`). Group and Dialog
+are created on demand.
+
+| Store | Read | Write | Creation |
+|---|---|---|---|
+| **Private** | Owner only | Owner only | Default |
+| **Protected** | Owner + link+permission holders | Owner + permissioned collaborators | Default |
+| **Public** | Everyone, no capability | Owner only | Default |
+| **Group** | Group members | Group members (collaborative) | On demand |
+| **Dialog** | The two users only | The two users only | On demand |
+
+Doc citations (verbatim): Private — *"only you have access to … not possible to
+share"*; Protected — *"share … but they will need a special link and permission"*;
+Public — *"equivalent to your website … without the need for special permissions"*;
+Group — *"each Group is a separate Store … documents inherit the permissions of
+the store"*; Dialog — *"hold all the data you exchange with another user (and only
+with that other user) … You cannot add more users"*.
+
+## Document = repo (there is no `Document` type)
+
+*"A Repo is the equivalent of an E2EE group for one and only one Document."*
+**1 document = 1 repo** (commits + permissions). Identifier: `did:ng:o:<RepoID>`.
+
+There is **no `Document` type** in `nextgraph-rs` (verified 2026-06-29): a
+"document" is simply *any repo*. A **store is a special repo** (`is_store=true`,
+with `Store`/`Overlay`/`User` branches) — so *a store is a document, but a
+document is not necessarily a store*.
+
+**Containment (store → repos) is by REFERENCE, not by a list.** A store does not
+hold a `Vec<RepoId>`: it references its repos through an **RDF graph** in its
+Overlay/User branch. Conversely each repo names its parent store via
+`RootBranchV0.store: StoreOverlay` — **a repo belongs to exactly one store**.
+
+## Capability / ReadCap granularity — the load-bearing fact for this lib
+
+`ReadCap = ObjectRef`. Granularity is at the **repo AND branch** level (each
+branch has its own `read_cap`), down to the **block** (`ObjectKey`/ChaCha20 key).
+Write is managed at the **document (repo)** level.
+
+**No automatic read inheritance.** Holding a **store's** ReadCap does **not**
+grant the repos it contains — **you need each repo's own ReadCap**. The optional
+`inherit_perms_users_and_quorum_from_store: Option<ReadCap>` shares only
+users/quorum (write/permissions), **not** read-cap possession. (Repos of a
+`private_store` inherit implicitly.)
+
+> **Consequence for this lib's emulation (see [`simulation.md`](./simulation.md)):**
+> the read access UNIT is the **repo = each item's `@graph`** — a per-DOCUMENT
+> filter, never per-store and never per-item. This is exactly what
+> `caps.ts` (`CapRegistry`) and `read-filter.ts` model: no store-level
+> inheritance, purely per-document caps. In a mono-store layout (all items in one
+> repo) the filter is therefore all-or-nothing on that document — which *is* the
+> native behaviour, and why fine-grained isolation requires one document per
+> entity.
+
+### Store ↔ document confusion (recurring)
+
+The isolation axis is the **document (repo/`@graph`)**, never the **store**: a
+store *contains* several documents and does not share their read caps. See the
+two-axes warning in [`simulation.md`](./simulation.md): "multi-store" in this
+lib's emulation means **multiple DOCUMENTS in one shared store**, not multiple
+stores.
+
+## Capability sharing / NURI
+
+Sharing transmits a **NURI** embedding the crypto capability (read and/or write).
+No central ACL: holding the NURI *is* the right. *"adding permissions can be done
+offline"*; *"removing permissions … requires a SyncSignature"* (synchronous).
+
+## Inbox
+
+**Every document has a native inbox.** A non-editor can **deposit a link (DID
+cap)** into it without being invited as an editor; the owner **moderates**. NURI:
+`did:ng:d:<inbox_id>`. Content: the `InboxMsgContent` enum (`ContactDetails`,
+`DialogRequest`, **`Link`**, `Patch`, `ServiceRequest`, `ExtRequest`,
+`RemoteQuery`, `SocialQuery`…). Messages are **sealed** (`crypto_box::seal`) to
+the inbox pubkey → only the owner decrypts. The `from` field is **optional** → an
+**anonymous** sender is possible. This is the "identified if known, anonymous
+otherwise" behaviour native to the protocol.
+
+### The inbox is NOT usable from the JS SDK
+
+- `app_request(request)` is exposed, and `AppRequestCommandV0::InboxPost` +
+  `AppRequest::inbox_post()` exist. **BUT** the verifier's `request_processor`
+  has **no `InboxPost` arm** (arms actually handled: `OrmStart(Discrete)`,
+  `Fetch`, `FileGet`, `OrmUpdate`, `OrmDiscreteUpdate`, `SocialQueryStart`,
+  `QrCodeProfile(Import)`, `Header`, `Create`, `FilePut`). Sending an `InboxPost`
+  triggers nothing.
+- Building an `InboxPost` requires crypto sealing on the Rust side; **no wasm
+  helper** exposes it.
+- Inbox deposit is only triggered **internally** by `QrCodeProfileImport`
+  (`post_to_inbox(new_contact_details)`) and `social_query_start` (contact
+  propagation via inbox).
+
+**Consequence:** there is no clean way to "drop a Link" into an arbitrary
+document's inbox from the JS SDK today. This lib emulates the inbox instead of
+patching the broker — see [`simulation.md`](./simulation.md) (emulated inbox +
+curator) and [`fork-inbox-fallback.md`](./fork-inbox-fallback.md) (the Rust-patch
+path NOT taken). A related exposed primitive: `social_query_start` (a federated
+query via inbox up to `degree` hops) exists but is limited to **contacts** — it
+does not cover an anonymous notification to a non-connected host.
+
+## JS SDK limits (`@ng-org/web`)
+
+`@ng-org/web` (verified `0.1.2-alpha.13` = `upstream/main` at 2026-05-21, the
+installed version) **does NOT expose**: Group/Dialog store creation; capability
+sharing (a NURI with rights); permission manipulation; inbox deposit/read.
+
+Available JS methods: `doc_create`, `doc_subscribe`, `sparql_query`,
+`sparql_update`, `orm_start_graph`, `orm_start_discrete`, `graph_orm_update`,
+`discrete_orm_update`, `file_get`, `app_request_stream`. The docs announce *"An
+API will be provided for permission manipulation"* (no date).
+
+## Integration & deployment model
+
+NextGraph is consumed via an **iframe proxy** (`@ng-org/web`): the third-party
+app contains no engine, it delegates to a hosted ng-app (default `nextgraph.net`)
+that runs the engine in an iframe.
+
+### The JS packages
+
+- **`@ng-org/web`** — **published**. Lightweight postMessage proxy (no wasm
+  embedded). **The** third-party integration path; `@ng-org/orm` and every
+  example depend on it. This lib wraps it.
+- **`@ng-org/api-web`** — **private** (unpublished). Full in-browser engine
+  (loads `@ng-org/lib-wasm` in a Web Worker). Consumed only by `app/nextgraph`
+  (the ng-app frontend) — **not** a third-party integration target.
+- **`@ng-org/lib-wasm`** — the compiled wasm engine (contains the verifier).
+  Source `sdk/js/lib-wasm/`.
+- **`nextgraph`** (npm) — the NodeJS API (`pkg-node` build).
+- **`@ng-org/orm`** — reactive ORM (`useShape`…), built on `@ng-org/web`.
+
+### Where the verifier runs
+
+In the standard web model, the verifier runs **in the iframe**: `app/nextgraph`
+loads `api-web` → `lib-wasm` in a Web Worker, browser-side. The broker (`ngd`)
+only does **transport and storage**.
+
+**Consequence:** changing verifier logic (`request_processor`,
+`inbox_processor`) means rebuilding the **ng-app**, not the broker.
+
+### iframe model & build-time retargeting
+
+`@ng-org/web` redirects to the hosted ng-app, which reloads the third-party app
+in an iframe after auth, then relays over `postMessage`. **Retargetable at build
+time** (`sdk/js/web/src/index.ts`, `import.meta.env`):
+
+| Variable | Target |
+|---|---|
+| `NG_REDIR_SERVER` | default `nextgraph.net` |
+| `NG_DEV3` | `127.0.0.1:3033` |
+| `NG_DEV` | `localhost:14402`/`14404` |
+| `NG_DEV_LOCAL_BROKER` | `localhost:1421` |
+
+**No runtime override** — `init()` takes no broker URL. To point at a
+self-hosted ng-app: **rebuild `@ng-org/web`** (pure TS, no wasm → trivial build).
+
+### Proxy ↔ iframe ↔ worker plumbing (generic)
+
+The call path is **entirely generic** (no allowlist): `@ng-org/web` is a JS
+`Proxy` relaying *any* method name over `postMessage`; `app/nextgraph` dispatches
+via `Reflect.apply(ng[method], …)`. So a new wasm function in simple
+request/response form is *reachable* without touching the JS — but that's an
+untyped **hack** (quick test, not a plan). The **streamed** case needs an entry
+on both sides (`E` in `@ng-org/web` + `streamed_api` in api-web; current streamed
+methods: `doc_subscribe`, `orm_start_graph`, `orm_start_discrete`, `file_get`,
+`app_request_stream`).
+
+> This is exactly why `docs.ts` in this lib calls the **real injected `ng`**
+> directly and never layers our own `Proxy` on top of `@ng-org/web`'s
+> iframe-RPC proxy — see the `DataCloneError` double-proxy constraint in
+> [`simulation.md`](./simulation.md).
+
+### The broker (ngd)
+
+- Already supports the inbox natively (`inbox_post`, `inbox_register`,
+  `inbox_pop_for_user` in `engine/net/src/server_broker.rs`) — a standard `ngd`
+  would route the inbox, **no broker patch needed**. The gap is in the
+  verifier/SDK layer, not the broker.
+- **WebSocket** daemon (`async-tungstenite`), **stateful**: RocksDB under
+  `--base-path`, persisted PeerId (critical volume).
+- CLI: `--local PORT`, `--domain DOMAIN:PORT,LOCAL_PORT` (behind a TLS-terminated
+  reverse proxy — Traefik/Coolify).
+- **Serves no static assets**: the ng-app frontend is a separate static deploy
+  (`pnpm webfilebuild`). First boot is **interactive** (admin-wallet invitation
+  link). Official Dockerfiles are **broken**.
+
+## Apps & services: mono-user, no global data
+
+NextGraph's app/service execution model — important because it **invalidates**
+the idea of "a service with its own wallet sharing global data".
+
+- **Apps AND services are mono-user.** They see only **what the user makes
+  available** to them. There is **no global data** natively, and no central
+  service holding shared data.
+- **Local settings document.** Every app — even a singleton — and every service
+  has a **settings document** the user configures it through.
+- **Multi-instance apps.** A **non-singleton** app can be **instantiated several
+  times** (e.g. a text editor, once per open file).
+- **Singleton apps.** Also **mono-user**, but **bound to a particular user (the
+  developer)**. A singleton app **can hold a global document**, administered by
+  that user.
+
+**Consequence for a "global document" (e.g. a discovery index):** the only path
+glimpsed is a **singleton app** whose global document is administered by the
+developer-user — **but this is not implemented and not guaranteed** (simpler
+paths may exist; to explore later). The **incorrect** model to avoid: "a
+dedicated service with its own wallet sharing a freely-readable index" — that
+does not exist in NextGraph (a service is mono-user, no global data). This is why
+a global-index curator package is **deferred** in this lib (see the top-level
+README).
+
+## Third-party wallet auto-import constraint
+
+Verified empirically (2026-06-17): with the **hosted** broker (`nextgraph.net`),
+a third-party web app **cannot** provision/import a wallet programmatically. A
+wallet must **pre-exist** in the browser before the auth redirect can succeed.
+
+Mechanism (from `@ng-org/web`'s `ngweb.js` dist):
+
+- **`init()` top-level REDIRECTS**: when `window.self === window.top` it does
+  `window.location.href = https://nextgraph.net/redir/#/?o=<url>`. The app's code
+  stops running.
+- **Every `ng.*` method is relayed** by `parent.postMessage` to `nextgraph.net`,
+  and the handler **throws `"you must call init() first"` until a session is
+  established** (internal `d !== false` guard). This includes
+  `wallet_import_from_code`, `add_in_memory_wallet`, `session_in_memory_start`.
+- The third-party app runs **inside the iframe only AFTER** the broker has opened
+  a wallet and established the session. There is **no window** where our code runs
+  *before* the broker's wallet gate → **nothing to hook an auto-import onto**.
+
+Of the wallet-import methods offered on `nextgraph.eu`, only the **wallet FILE**
+(`.ngw`) is a static, reusable export; TextCode/QR are temporary device↔device
+transfers (5 min, both devices online, single use) — unusable to embed. The only
+real way to eliminate the cross-origin round-trip is to self-host/fork the ng-app
+(see [`fork-inbox-fallback.md`](./fork-inbox-fallback.md)).
+
+## Login is not programmable
+
+NextGraph login is a **web redirect** to the broker page (`nextgraph.net`). There
+is no way to open a wallet silently — at least one broker-redirect pass per device
+is required. Session persistence: the wallet is remembered iframe-side
+(`localStorage` long-term + `sessionStorage` for the active session); on reload,
+`init()` recovers the session **without re-triggering the redirect** while the
+broker session exists (`sdk/js/web/src/index.ts`, `sdk/js/api-web/main.ts`). A
+full browser restart (losing `sessionStorage`) can re-trigger the gate. A real
+logout IS exposed (`ng.session_stop()`, `ng.user_disconnect()`,
+`ng.wallet_close()` in `sdk/js/lib-wasm/src/lib.rs`) but **forces a new
+redirect** afterwards. This lib's faux login sidesteps all of it — see the faux
+login in [`simulation.md`](./simulation.md).