docs+refactor(client): fidelity pass — id identity, drop connections, no faux-login, accurate NextGraph framing

Align the polyfill's surface and docs with the verified NextGraph reality and
remove application-level concepts:

- Identity is an ID, not a username: AccountRecord.id, shim predicate shim:id,
  normalizeId; accounts core becomes IdentityStore (set/clear/get) — the faux
  login/logout framing is gone (identity is set at wallet-import time).
- Relationship/connection is an application concept, not a platform primitive
  (NextGraph has no bilateral-connection primitive: grantee is unpersisted
  scaffolding, cap-send is unimplemented). Remove connections.ts; caps exposes
  only a directed grantRead(doc, granteeId) + a read-only protectedDocsOf(owner).
  Delete the now-dead isolation.ts social-visibility axis.
- Inbox docs: NextGraph has no separate curator — the recipient's own verifier
  unseals and applies each queued sealed message inline (process_inbox);
  inbox_post_link is a proposed/future API. Stop attributing the emulated
  curator to the platform.
- Read isolation reframed around the outcome: no cap -> empty union read;
  targeted read of an unheld repo -> RepoNotFound; cap introspection
  (canRead/governsRead) is emulation-only with no NextGraph API behind it.
- read-model.md corrected: the listing path is per-doc ANCHORED default-graph
  queries, never the anchorless GRAPH ?g union (that is O(wallet)); the probe
  section no longer claims the opposite.
- README recap table restructured (target | current NextGraph status | current
  emulation); INDEX_ACCOUNT documented as reservedAccount("index") in the
  sentinel namespace; de-domained generic-layer comments; softened tone.

Consumer application (Festipod) rewired separately to own the relationship
concept and feed the lib an id. Lib gates: bun test 83 pass / 0 fail, tsc clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Sylvain Duchesne
2026-07-06 14:02:16 +02:00
parent d39b12885a
commit 63ecfeeff8
31 changed files with 1059 additions and 1396 deletions
+85 -91
View File
@@ -1,128 +1,122 @@
# ng-eventually
A **generic polyfill layer** over the [NextGraph](https://nextgraph.org) JS SDK.
A generic polyfill layer over the [NextGraph](https://nextgraph.org) JS SDK.
NextGraph's JS SDK does not yet expose cross-wallet reads, capabilities, inboxes
or group stores. `ng-eventually` lets an app behave as if those existed today, by
emulating them on top of a **single shared wallet / broker**. It is **generic**:
it contains **no application domain** — the consumer injects its shapes and the
*acts* of granting access.
or group stores. `ng-eventually` lets a consumer application behave as if those
existed today, by emulating them on top of a single shared wallet / broker. It is
generic: it contains no application domain — the consumer application injects its
shapes and the acts of granting access.
The name: *eventually* NextGraph will ship these features; until then this layer
fills the gap (and nods at eventual consistency / events).
## The boundary — mature face out, compensation in
The asymmetry is the whole point. **Consumers write SDK-shaped code as if
NextGraph were finished**: per-entity documents in public/protected/private
stores, capabilities, inboxes. This library **owns all the current-state
NextGraph knowledge and the simulation** that fabricates that mature face — a
**shared-wallet** emulation — so the application never sees it. When NextGraph
matures, **only this library changes**; the consumer's code does not.
The asymmetry is the point. The consumer application writes SDK-shaped code as if
NextGraph were finished: per-entity documents in public/protected/private stores,
capabilities, inboxes. This library owns the current-state NextGraph knowledge and
the simulation that fabricates that mature face — a shared-wallet emulation — so
the application never sees it. As NextGraph matures, this library changes; the
consumer application's code does not.
Docs (this library's own engineering doctrine, under [`docs/`](./docs/)):
- [`docs/nextgraph-current-state.md`](./docs/nextgraph-current-state.md) — the
authoritative reference on what the CURRENT SDK/broker do and do NOT expose
(the ground truth every polyfill compensates for).
authoritative reference on what the current SDK/broker do and do not expose
(the ground truth each polyfill compensates for).
- [`docs/simulation.md`](./docs/simulation.md) — how this lib emulates the mature
behaviour on ONE shared wallet (shim, per-document ReadCaps, emulated
inbox+curator, write guard, faux login, the two axes, the double-proxy
constraint).
- [`docs/read-model.md`](./docs/read-model.md) — the READ MODEL the polyfill
behaviour on one shared wallet (shim, per-document ReadCaps, emulated inbox,
write guard, the two axes, the double-proxy constraint).
- [`docs/read-model.md`](./docs/read-model.md) — the read model the polyfill
implements: events via the global index, everything else by following a shared
graph; listing via a bounded set of **per-doc anchored** `sparql_query`s (never
an anchorless union-scan of the physical wallet, never the ORM fan-out — both
hang/time out); reactivity via re-query on a change signal.
- [`docs/decisions/`](./docs/decisions/) — historical current-SDK ADRs
(private-store scope, SPARQL delete, shared-wallet login, discovery mechanism).
graph; listing via a bounded set of per-doc anchored `sparql_query`s; reactivity
via re-query on a change signal.
- [`docs/decisions/`](./docs/decisions/) — current-SDK ADRs (private-store scope,
SPARQL delete, shared-wallet identity, discovery mechanism).
- [`docs/fork-inbox-fallback.md`](./docs/fork-inbox-fallback.md) — the Rust-patch /
self-host inbox path NOT taken (kept as fallback).
self-host inbox path not taken (kept as a fallback).
- [`docs/migration-guide.md`](./docs/migration-guide.md) — the checklist for when
real NextGraph matures.
## What is emulated (and how it goes away)
## What is emulated
**Nothing in this library is a real NextGraph feature.** Every behaviour below is
**emulated** — a stopgap fabricated on top of the *current, immature* NextGraph
(one shared wallet, everything physically readable). Each has a **real NextGraph
target**, and the switch to it is a **lib-only swap: the consumer's SDK-shaped code
does not change** (see [`docs/migration-guide.md`](./docs/migration-guide.md)). The
consumer always sees the mature SDK face; the emulation lives entirely here.
Nothing in this library is a real NextGraph feature. Each behaviour below is
emulated — a stopgap fabricated on top of the current, immature NextGraph (one
shared wallet, everything physically readable). The consumer application always
sees the mature SDK face; the emulation lives entirely here.
| Behavior | What the consumer sees (SDK-shaped API) | How it's emulated today (on one shared wallet) | Real NextGraph target | Migration (what changes; consumer unchanged) |
The table reads: what the consumer application does, the real NextGraph target it
is written against, the current NextGraph implementation status (why a workaround
is needed), and how this lib emulates it today.
| Capability | What the consumer application does | Real NextGraph target | Current NextGraph status (why a workaround) | Current emulation |
|---|---|---|---|---|
| **Multi-user / per-user wallet** | Each username is its own identity with its own documents | **One shared wallet** everyone opens; "users" are **virtual wallets** — shim accounts keyed by a **virtual-wallet id**, each mapped to its documents in `store-registry.ts` ([`simulation.md`](./docs/simulation.md#physical-wallet-vs-virtual-wallet--never-enumerate-the-physical-one)) | One real per-user wallet each; native cross-wallet reads | Each virtual wallet → a real wallet; drop the shim; the physical/virtual split dissolves |
| **3 native stores per user** | `public` / `protected` / `private` scopes | **3 emulated scope-index documents** per account — each "store" is an index doc listing its entity-doc NURIs; all physically live in the ONE shared private store (`docCreate(..., undefined)`), scope is a **logical label** ([`simulation.md`](./docs/simulation.md#a-virtual-wallets-structure--the-three-emulated-stores)) | The user's 3 real native stores hold the entity documents | The 3 index docs become the 3 real stores; the logical scope label becomes real placement |
| **Per-document read isolation** | `getCaps().open(doc, scope, owner)` the *acts* of granting | Emulated **`CapRegistry`** (`caps.ts`, per-document ReadCap) + **read filter** (`read-filter.ts`, defence-in-depth view) + read-set construction in `read-model.ts`; owner/connections injected as principals ([`simulation.md`](./docs/simulation.md#emulated-readcap--per-document-capsts--read-filterts)) | Broker/verifier only delivers documents the wallet holds a ReadCap for; `useShape` already returns an authorized subset | Translate the registry to real caps; **delete** the read filter (dead code) — access unit is already the document (`@graph`) |
| **Bilateral connections** | `declareConnections(peers)` — declare your own side | Emulated **connection registry** (`connections.ts`): directed assertions, a link is live only when **both** sides assert (two-sided) → drives protected read grants ([`simulation.md`](./docs/simulation.md#making-the-readcap-active--current-user--connection-driven-grants)) | Mutual capability exchange between two wallets | Real mutual caps replace the materialized link; drop the registry |
| **Inbox (registration notifications)** | `inbox.post` / `read` / `watch` | Emulated **deposits via SPARQL** into an inbox document + **in-client materialization** (curator played inline); no native `inbox_post` exists in the JS SDK ([`simulation.md`](./docs/simulation.md#emulated-inbox--curator-inboxts), [`nextgraph-current-state.md`](./docs/nextgraph-current-state.md#inbox)) | Native per-document inbox: `inbox_post_link` seals a deposit; a separate curator materializes it | `post` → native `inbox_post_link`; read side → a **separate curator package** (deferred) |
| **Discovery of all public events** | `submitToIndex(ref)` / `readIndex()` | Emulated **global index** = a document **owned by a reserved special account** (`@index`), fed via **its inbox** + inline **curator** (dedup); a **stable NURI** every client resolves. **NOT a physical-wallet scan** ([`simulation.md`](./docs/simulation.md#emulated-discovery-index--special-account-discoveryts), [`read-model.md`](./docs/read-model.md)) | A real owned global document (undecided owner — singleton-app path only glimpsed), fed via its native inbox, materialized by a curator | Special account disappears; ownership moves to the decided owner; `submitToIndex` → native inbox post; curator queries the real index |
| **Reads / listing** | `listMyMeetingPoints()`, `listEvents()`, … by need | **Per-doc ANCHORED `sparql_query`** over the **virtual** wallet's by-need doc set — **never an anchorless scan of the physical wallet** (O(wallet), ~90s timeout) and never the ORM fan-out (~75s hang) ([`read-model.md`](./docs/read-model.md)) | Native per-wallet reads over real per-user stores | The anchored read is already native; only *bringing a repo into the session* becomes a real broker sync (the `OpenRepo` TODO) |
| **Reactivity** | Lists update on change | **Re-query** the bounded per-doc anchored set on a lightweight change signal (`doc_subscribe` / ORM on an already-opened single store) — there is **no reactive union query** ([`read-model.md`](./docs/read-model.md#reactivity--re-query-on-a-change-signal-no-reactive-union)) | Native reactive reads | Re-query pattern collapses onto native reactive primitives |
| **Writes** | Write an entity to its scope | **Per-entity documents** via **direct SPARQL** (`docs.sparqlUpdate` on the real injected `ng`); `doc_create` can only target the **private** store today (`StoreRepo` not JS-constructible) ([`simulation.md`](./docs/simulation.md#reponotfound-and-the-orm_start_graph-scope-rule)) | Writes land in the entity's real store via native primitives | `docCreate` targets the real per-scope store once the SDK lets you construct one |
| **Login** | `login(username)` / `logout()` | **Faux `localStorage` identity** (`accounts.ts`): declarative username, no password, no NextGraph call — the shared-wallet broker gate stays open underneath ([`simulation.md`](./docs/simulation.md#faux-login-accountsts)) | Opening your **own** wallet at the broker gate IS the login | Remove faux login; the broker redirect becomes the real per-user login (flow shape unchanged) |
| **Write-guard** | Writes refused without the write cap | **Best-effort**: the guard (`ng-proxy.ts`) fires only on the public proxy, but real write paths call the injected `ng` directly (the `DataCloneError` constraint)**not guarded today** ([`simulation.md`](./docs/simulation.md#write-guard-coverage-honest-scope)) | Broker/verifier enforces the write cap natively | Native enforcement replaces the guard; delete it (dead code) |
| Multi-identity / per-identity wallet | Treats each identity id as its own wallet with its own documents | Each identity opens its own real wallet; native cross-wallet reads | Not-yet-implemented: the JS SDK exposes no cross-wallet read, so one session cannot read another identity's wallet | One shared wallet everyone opens; "identities" are virtual wallets — shim accounts keyed by an id, each mapped to its documents in `store-registry.ts` |
| Three native stores per identity | Places entities by scope `public` / `protected` / `private` | The identity's three real native stores hold the entity documents | Not-yet-implemented: `doc_create`/ORM can target only the private (and protected) native store today; a `public`/arbitrary `StoreRepo` is not JS-constructible | Three emulated scope-index documents per account — each "store" is an index doc listing its entity-doc NURIs; all physically live in the one shared private store, and scope is a logical label |
| Per-document read isolation | Declares a document's read policy via `getCaps().open(doc, scope, owner)`, then issues directed read grants (`grantRead(doc, granteeId)`) | The broker/verifier delivers only documents the wallet holds a ReadCap for; accessing a document without the cap yields an empty result in a union read (a targeted read of an unheld repo errors with `RepoNotFound`) | Bug/gap for emulation purposes: there is no cap-introspection API — a client cannot ask "may this identity read this doc?", so the polyfill cannot mirror the broker's decision from NextGraph itself | An emulated `CapRegistry` (`caps.ts`, per-document read/write caps) + a read filter (`read-filter.ts`, a defence-in-depth view) that keep only documents the current identity may read; `canRead`/`governsRead` are emulation-only, with no NextGraph API behind them |
| Directed read sharing | Owns the relationship concept ("who is connected to whom") itself, and for each relationship issues directed read grants on the owner's protected documents | A native per-document ReadCap issued to a specific identity — but note this target is itself not-yet-built in nextgraph-rs today, not merely unexposed in JS: `AccessGrantV0{grantee}` is unpersisted scaffolding and cap-send is `unimplemented!()`, so directing a grant to another identity has no working platform primitive yet | Not-yet-implemented: sending a cap to another identity is `unimplemented!()`, and no relationship/mutuality primitive exists — relationship is an application concept, not a platform one | The app selects the owner's protected documents via `getCaps().protectedDocsOf(owner)` and calls `grantRead(doc, granteeId)` per grantee; the lib records the per-document grant |
| Inbox (registration notifications) | `inbox.post` / `read` / `watch` | A message is sealed to the recipient's key and queued in their inbox; the recipient's own verifier unseals and applies each queued message inline while processing the inbox | Not-yet-implemented: the sender-side seal-into-inbox call (`inbox_post_link`) is proposed/future, not exposed in the JS SDK | Deposits written as RDF into an inbox document via SPARQL; `read`/`watch` read the deposits back — an in-lib stand-in for the recipient's own inbox processing |
| Discovery of all public events | `submitToIndex(ref)` / `readIndex()` | A real owned global document (owner undecided — a singleton-app path), fed via its native inbox, read as a materialized index | Not-yet-implemented / undecided: an identity's apps and services see only what it shares, so there is no global backend index yet | A global index document owned by a reserved special account (`@index`), fed via its inbox, read with dedup; a stable NURI every client resolves |
| Reads / listing | Lists the documents it needs, by scope, and reads them | Native per-wallet reads over the real per-identity stores | Bug/perf: an anchorless union query spans every named graph in the session store, which on a shared / accumulating wallet is O(wallet size) and stalls | A bounded, by-need set of per-doc anchored `sparql_query`s (each anchored to one repo's default graph), independent of wallet size |
| Reactivity | Lists update on change | Native reactive reads | Not-yet-implemented: there is no reactive union query across graphs | Re-query the bounded per-doc anchored set on a lightweight change signal (`doc_subscribe` / ORM on an already-opened single store) |
| Writes | Writes an entity to its scope | Writes land in the entity's real store via native primitives | Not-yet-implemented: `doc_create` can target only the private/protected store today (`StoreRepo` not JS-constructible) | Per-entity documents via direct SPARQL (`docs.sparqlUpdate` on the real injected `ng`) |
| Current identity | Sets the current identity id (established at wallet import) via the SDK's current-identity call | Opening one's own wallet at the broker gate establishes the session identity | Not-yet-implemented for the shared-wallet case: everyone shares one wallet, so the broker cannot distinguish identities | A relayed id (`accounts.ts` `IdentityStore` persists it); the read filter and inbox `from` read it |
| Write-guard | Writes refused without the write cap | The broker/verifier enforces the write cap natively | Partial: the guard fires only on the public proxy, but the real write paths call the injected `ng` directly (the `DataCloneError` constraint), so it is best-effort today | A `sparql_update` override (`ng-proxy.ts`) checking the emulated write cap |
## Packages
| Package | Role | At migration |
|---|---|---|
| **`@ng-eventually/client`** | **SDK-identical** wrapper the app imports instead of `@ng-org/web` / `@ng-org/orm`. Adds the polyfills the broker/verifier will do natively (shared-wallet login, capability enforcement, anticipated cap/inbox methods). | Disappears: the app points back at the real SDK (build alias removed). |
| Package | Role |
|---|---|
| `@ng-eventually/client` | The SDK-identical wrapper the app imports instead of `@ng-org/web` / `@ng-org/orm`. It adds the polyfills the broker/verifier will do natively (shared-wallet identity, capability enforcement, anticipated cap/inbox methods). As NextGraph matures, the app points back at the real SDK (build alias removed) and this package falls away. |
> **A global-index curator package is deferred.** NextGraph is **mono-user with
> no global data** (apps/services see only what the user shares; there is no
> multi-user backend). A global index would come from a **singleton app**
> (a global document administered by the developer) — **not implemented and
> uncertain**, and simpler paths may exist. So no second package for now; it will
> be (re)introduced once the global-index mechanism is decided. The curator must
> never be bundled in the client → it will be a separate package when it lands.
A global-index package is deferred. In NextGraph an identity's apps and services
see only what it shares, so there is no multi-identity backend. A global index
would come from a singleton app (a global document administered by the developer),
which is not implemented and undecided; simpler paths may exist. So there is no
second package for now it will be introduced once the global-index mechanism is
decided, and it will be separate from the client.
## Design principle
The application code is written **as if the target NextGraph existed**. All
compensation lives here, *beside* the app. Migration = remove this layer; the
app code (SDK-shaped) is unchanged.
The application code is written as if the target NextGraph existed. All
compensation lives here, beside the app. As NextGraph matures, this layer falls
away; the app code (SDK-shaped) is unchanged.
- **SDK-identical surface**: the client wraps the real `ng` (a Proxy that
forwards everything and overrides only what must be emulated) and `useShape`.
The real SDK is **injected** via `configure()` (no hard import → build-alias
safe + testable).
- **Authorization = emulated capabilities**: documents carry grants; the client
enforces them generically (read filter + write guard). The app *attaches*
grants via cap operations — same as it will in the target. No policy is
injected.
- **Inbox**: the client `inbox` namespace deposits (`post`) and, in the
shared-wallet emulation, also plays the curator inline (`read` / `materialize`
/ `watch`). At migration the read side moves to a separate curator package,
deferred with the global-index mechanism — see the note above.
- **Tests** of the polyfill (against a real broker) live **in this repo**, so
the consuming app can test its features against a mocked, clean API.
- SDK-identical surface: the client wraps the real `ng` (a Proxy that forwards
everything and overrides only what must be emulated) and `useShape`. The real
SDK is injected via `configure()` (no hard import → build-alias safe and
testable).
- Authorization is emulated capabilities: documents carry grants; the client
enforces them generically (read filter + write guard). The app declares a
document's read policy and issues directed grants — the same acts it will
perform in the target. No policy is injected.
- Inbox: the client `inbox` namespace deposits (`post`) and, in the shared-wallet
emulation, reads the deposits back (`read` / `materialize` / `watch`) in place
of the recipient's own inbox processing.
- Tests of the polyfill (against a real broker) live in this repo, so a consuming
app can test its features against a clean, mocked API.
## Status
**Implemented.** The polyfill mechanisms are wired against a real broker, not
stubbed:
Implemented. The polyfill mechanisms are wired against a real broker, not stubbed:
- **Shared-wallet shim**`store-registry.ts` (`(account, scope) → document
NURI`, `createEntityDoc` / `listEntityDocs` + per-scope index, cross-device via
the RDF shim anchored in the private store).
- **Document / SPARQL primitive** — `docs.ts`, calling the real injected `ng`
directly (avoids the `@ng-org` double-proxy `DataCloneError`).
- **Emulated ReadCaps** — `caps.ts` (`CapRegistry`, per-document) + read filter
`read-filter.ts` (reactive-set `Proxy` view), applied by `use-shape.ts` only
when a policy is declared.
- **Write guard** — `ng-proxy.ts` (`sparql_update` override, emulated write cap).
- **Inbox** — `inbox.ts` (`post` / `read` / `materialize` / `watch`, emulated
curator inline).
- **Isolation** — `isolation.ts` (pure social-visibility matrix, distinct axis
from ReadCaps).
- **Accounts** — `accounts.ts` (faux username login, injected storage).
- **SPARQL hardening** — `sparql.ts` (`escapeLiteral` / `escapeIri` / `assertNuri`).
- Shared-wallet shim — `store-registry.ts` (`(account, scope) → document NURI`,
`createEntityDoc` / `listEntityDocs` + per-scope index, cross-device via the RDF
shim anchored in the private store).
- Document / SPARQL primitive — `docs.ts`, calling the real injected `ng` directly
(avoids the `@ng-org` double-proxy `DataCloneError`).
- Emulated ReadCaps — `caps.ts` (`CapRegistry`, per-document, directed grants) +
read filter `read-filter.ts` (reactive-set `Proxy` view), applied by
`use-shape.ts` only when a policy is declared.
- Write guard — `ng-proxy.ts` (`sparql_update` override, emulated write cap).
- Inbox — `inbox.ts` (`post` / `read` / `materialize` / `watch`).
- Identity — `accounts.ts` (`IdentityStore`, injected storage).
- SPARQL hardening — `sparql.ts` (`escapeLiteral` / `escapeIri` / `assertNuri`).
Remaining `TODO` markers are narrow: the shared-wallet credential passthrough in
the `login`/`session_start` proxy branch, and the anticipated cap/inbox SDK
signatures to reconcile if the official API differs. See
The remaining `TODO` markers are narrow: the shared-wallet credential passthrough
in the `session_start` proxy branch, and the anticipated cap/inbox SDK signatures
to reconcile if the official API differs. See
[`docs/simulation.md`](./docs/simulation.md) for what each piece does and
[`docs/migration-guide.md`](./docs/migration-guide.md) for what is removed at
migration.
[`docs/migration-guide.md`](./docs/migration-guide.md) for what changes as
NextGraph matures.