Sylvain Duchesne c0498a6ebc feat(client): per-document reactive subscription (doc_subscribe), drop polling
Expose subscribeDoc(nuri, onChange) / subscribeDocs(nuris, onChange) wrapping the
real ng.doc_subscribe — per-document, event-driven (initial State + a Patch per
commit, local or broker-synced from a remote peer), returning a sync unsubscribe.
subscribeDocs isolates per doc (a failing/unsynced doc never aborts the others),
so it sidesteps the ORM fan-out hang (never orm_start_graph(graphs:[…])).

Replace the setInterval polling in inbox.watch() and discovery.watchIndex() with
doc_subscribe — same public contract, now push not poll. NextGraph is subscription-
first; no polling remains.

Verified against the real broker (@data harness spike): the doc_subscribe callback
marshals across the iframe RPC (@ng-org/web strips the callback and drives it via a
MessagePort — no DataCloneError) and fires on the initial push and on a real write.
Lib bun test 91 pass, tsc clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 23:04:32 +02:00

ng-eventually

A generic polyfill layer over the NextGraph JS SDK.

NextGraph's JS SDK does not yet expose cross-wallet reads, capabilities, inboxes or group stores. ng-eventually lets a consumer application behave as if those existed today, by emulating them on top of a single shared wallet / broker. It is generic: it contains no application domain — the consumer application injects its shapes and the acts of granting access.

The name: eventually NextGraph will ship these features; until then this layer fills the gap (and nods at eventual consistency / events).

The boundary — mature face out, compensation in

The asymmetry is the point. The consumer application writes SDK-shaped code as if NextGraph were finished: per-entity documents in public/protected/private stores, capabilities, inboxes. This library owns the current-state NextGraph knowledge and the simulation that fabricates that mature face — a shared-wallet emulation — so the application never sees it. As NextGraph matures, this library changes; the consumer application's code does not.

Docs (this library's own engineering doctrine, under docs/):

  • docs/nextgraph-current-state.md — the authoritative reference on what the current SDK/broker do and do not expose (the ground truth each polyfill compensates for).
  • docs/simulation.md — how this lib emulates the mature behaviour on one shared wallet (shim, per-document ReadCaps, emulated inbox, write guard, the two axes, the double-proxy constraint).
  • docs/read-model.md — the read model the polyfill implements: events via the global index, everything else by following a shared graph; listing via a bounded set of per-doc anchored sparql_querys; reactivity via re-query on a change signal.
  • docs/decisions/ — current-SDK ADRs (private-store scope, SPARQL delete, shared-wallet identity, discovery mechanism).
  • docs/fork-inbox-fallback.md — the Rust-patch / self-host inbox path not taken (kept as a fallback).
  • docs/migration-guide.md — the checklist for when real NextGraph matures.

What is emulated

Nothing in this library is a real NextGraph feature. Each behaviour below is emulated — a stopgap fabricated on top of the current, immature NextGraph (one shared wallet, everything physically readable). The consumer application always sees the mature SDK face; the emulation lives entirely here.

The table reads: what the consumer application does, the real NextGraph target it is written against, the current NextGraph implementation status (why a workaround is needed), and how this lib emulates it today.

Capability What the consumer application does Real NextGraph target Current NextGraph status (why a workaround) Current emulation
Multi-identity / per-identity wallet Treats each identity id as its own wallet with its own documents Each identity opens its own real wallet; native cross-wallet reads Not-yet-implemented: the JS SDK exposes no cross-wallet read, so one session cannot read another identity's wallet One shared wallet everyone opens; "identities" are virtual wallets — shim accounts keyed by an id, each mapped to its documents in store-registry.ts
Three native stores per identity Places entities by scope public / protected / private The identity's three real native stores hold the entity documents Not-yet-implemented: doc_create/ORM can target only the private (and protected) native store today; a public/arbitrary StoreRepo is not JS-constructible Three emulated scope-index documents per account — each "store" is an index doc listing its entity-doc NURIs; all physically live in the one shared private store, and scope is a logical label
Per-document read isolation Declares a document's read policy via getCaps().open(doc, scope, owner), then issues directed read grants (grantRead(doc, granteeId)) The broker/verifier delivers only documents the wallet holds a ReadCap for; accessing a document without the cap yields an empty result in a union read (a targeted read of an unheld repo errors with RepoNotFound) Bug/gap for emulation purposes: there is no cap-introspection API — a client cannot ask "may this identity read this doc?", so the polyfill cannot mirror the broker's decision from NextGraph itself An emulated CapRegistry (caps.ts, per-document read/write caps) + a read filter (read-filter.ts, a defence-in-depth view) that keep only documents the current identity may read; canRead/governsRead are emulation-only, with no NextGraph API behind them
Directed read sharing Owns the relationship concept ("who is connected to whom") itself, and for each relationship issues directed read grants on the owner's protected documents A native per-document ReadCap issued to a specific identity — but note this target is itself not-yet-built in nextgraph-rs today, not merely unexposed in JS: AccessGrantV0{grantee} is unpersisted scaffolding and cap-send is unimplemented!(), so directing a grant to another identity has no working platform primitive yet Not-yet-implemented: sending a cap to another identity is unimplemented!(), and no relationship/mutuality primitive exists — relationship is an application concept, not a platform one The app selects the owner's protected documents via getCaps().protectedDocsOf(owner) and calls grantRead(doc, granteeId) per grantee; the lib records the per-document grant
Inbox (registration notifications) inbox.post / read / watch A message is sealed to the recipient's key and queued in their inbox; the recipient's own verifier unseals and applies each queued message inline while processing the inbox Not-yet-implemented: the sender-side seal-into-inbox call (inbox_post_link) is proposed/future, not exposed in the JS SDK Deposits written as RDF into an inbox document via SPARQL; read/watch read the deposits back — an in-lib stand-in for the recipient's own inbox processing
Discovery of all public events submitToIndex(ref) / readIndex() A real owned global document (owner undecided — a singleton-app path), fed via its native inbox, read as a materialized index Not-yet-implemented / undecided: an identity's apps and services see only what it shares, so there is no global backend index yet A global index document owned by a reserved special account (@index), fed via its inbox, read with dedup; a stable NURI every client resolves
Reads / listing Lists the documents it needs, by scope, and reads them Native per-wallet reads over the real per-identity stores Bug/perf: an anchorless union query spans every named graph in the session store, which on a shared / accumulating wallet is O(wallet size) and stalls A bounded, by-need set of per-doc anchored sparql_querys (each anchored to one repo's default graph), independent of wallet size
Reactivity Lists update on change Native reactive reads Not-yet-implemented: there is no reactive union query across graphs Re-query the bounded per-doc anchored set on a lightweight change signal (doc_subscribe / ORM on an already-opened single store)
Writes Writes an entity to its scope Writes land in the entity's real store via native primitives Not-yet-implemented: doc_create can target only the private/protected store today (StoreRepo not JS-constructible) Per-entity documents via direct SPARQL (docs.sparqlUpdate on the real injected ng)
Current identity Sets the current identity id (established at wallet import) via the SDK's current-identity call Opening one's own wallet at the broker gate establishes the session identity Not-yet-implemented for the shared-wallet case: everyone shares one wallet, so the broker cannot distinguish identities A relayed id (accounts.ts IdentityStore persists it); the read filter and inbox from read it
Write-guard Writes refused without the write cap The broker/verifier enforces the write cap natively Partial: the guard fires only on the public proxy, but the real write paths call the injected ng directly (the DataCloneError constraint), so it is best-effort today A sparql_update override (ng-proxy.ts) checking the emulated write cap

Packages

Package Role
@ng-eventually/client The SDK-identical wrapper the app imports instead of @ng-org/web / @ng-org/orm. It adds the polyfills the broker/verifier will do natively (shared-wallet identity, capability enforcement, anticipated cap/inbox methods). As NextGraph matures, the app points back at the real SDK (build alias removed) and this package falls away.

A global-index package is deferred. In NextGraph an identity's apps and services see only what it shares, so there is no multi-identity backend. A global index would come from a singleton app (a global document administered by the developer), which is not implemented and undecided; simpler paths may exist. So there is no second package for now — it will be introduced once the global-index mechanism is decided, and it will be separate from the client.

Design principle

The application code is written as if the target NextGraph existed. All compensation lives here, beside the app. As NextGraph matures, this layer falls away; the app code (SDK-shaped) is unchanged.

  • SDK-identical surface: the client wraps the real ng (a Proxy that forwards everything and overrides only what must be emulated) and useShape. The real SDK is injected via configure() (no hard import → build-alias safe and testable).
  • Authorization is emulated capabilities: documents carry grants; the client enforces them generically (read filter + write guard). The app declares a document's read policy and issues directed grants — the same acts it will perform in the target. No policy is injected.
  • Inbox: the client inbox namespace deposits (post) and, in the shared-wallet emulation, reads the deposits back (read / materialize / watch) in place of the recipient's own inbox processing.
  • Tests of the polyfill (against a real broker) live in this repo, so a consuming app can test its features against a clean, mocked API.

Status

Implemented. The polyfill mechanisms are wired against a real broker, not stubbed:

  • Shared-wallet shim — store-registry.ts ((account, scope) → document NURI, createEntityDoc / listEntityDocs + per-scope index, cross-device via the RDF shim anchored in the private store).
  • Document / SPARQL primitive — docs.ts, calling the real injected ng directly (avoids the @ng-org double-proxy DataCloneError).
  • Emulated ReadCaps — caps.ts (CapRegistry, per-document, directed grants) + read filter read-filter.ts (reactive-set Proxy view), applied by use-shape.ts only when a policy is declared.
  • Write guard — ng-proxy.ts (sparql_update override, emulated write cap).
  • Inbox — inbox.ts (post / read / materialize / watch).
  • Identity — accounts.ts (IdentityStore, injected storage).
  • SPARQL hardening — sparql.ts (escapeLiteral / escapeIri / assertNuri).

The remaining TODO markers are narrow: the shared-wallet credential passthrough in the session_start proxy branch, and the anticipated cap/inbox SDK signatures to reconcile if the official API differs. See docs/simulation.md for what each piece does and docs/migration-guide.md for what changes as NextGraph matures.

S
Description
No description provided
Readme 375 KiB
Languages
TypeScript 100%