ng-eventually/docs/simulation.md

How this library emulates mature NextGraph on one shared wallet

Everything in this file is emulation. None of the behaviours described here is a real NextGraph feature: each is a stopgap the lib fabricates on top of the current, immature NextGraph (the exact gaps it compensates for are in nextgraph-current-state.md). Every piece has a real target and goes away when NextGraph matures — the swap is lib-only, and the consumer application's code is unchanged. The per-behaviour recap table lives in the top-level README.md (What is emulated (and how it goes away)); the removal checklist is migration-guide.md. Read this file for how each emulation works; read those two for what is fake and what replaces it.

The consumer application writes against @ng-eventually/client as if NextGraph already shipped per-entity documents in public/protected/private stores, capabilities and inboxes. It hasn't (see nextgraph-current-state.md). This file is the lib's own engineering doctrine on how it fabricates that mature face on top of one single shared wallet / broker. Everything here is polyfill-era and disappears at migration (migration-guide.md).

The premise: one shared wallet, everything readable

Current NextGraph has no cross-wallet read (OpenRepo is a TODO at engine/verifier/src/verifier.rs:1423; a foreign NURI raises RepoNotFound; a session only holds its own 3 stores in self.repos). So "each user their own wallet" is blocked at the root — no data ever crosses the boundary between two wallets.

The lib's answer: everyone opens the same wallet. NextGraph sees a single identity, so everything is physically readable. "Multi-user" becomes an application fiction the lib maintains. On top of that one wallet the lib rebuilds, by emulation, the per-user stores + capabilities + inbox the consumer application codes against.

Physical wallet vs virtual wallet — never enumerate the physical one

Because the emulation runs on ONE shared wallet, distinguish two levels:

• Physical wallet — the real NextGraph wallet everyone opens. Its local store holds every account's documents plus the lib's own internals (the shim index, the inbox docs, the discovery index) as named graphs. It accumulates without bound across sessions/runs. Listing or scanning "all documents" of the physical wallet is meaningless and O(size) — it mixes every user's data with lib internals, and it is exactly what a sparql_query with no anchor (GRAPH ?g { … }) does (it spans every synced graph). The physical wallet is a substrate, not something to enumerate.

• Virtual wallet — the lib's emulation of one user's wallet: the set of documents the shim attributes to that account (its per-scope index in store-registry.ts). This is what "the user owns". Over a virtual wallet, "list my documents" is meaningful and bounded (only that account's docs).

Consequence for reads (see read-model.md): to list a user's entities you enumerate the virtual wallet — the account's scope index (bounded, O(my docs)), not the physical union — then read those specific documents with a per-doc anchored sparql_query. A non-empty / bloated physical wallet then costs nothing, because the physical union is never scanned. Discovery (all public events) is the one bounded enumeration hack and goes through the discovery index, not a physical scan.

At migration each virtual wallet becomes a real per-user wallet; the physical/virtual distinction — and the "never enumerate the physical wallet" rule — dissolves into native per-wallet reads.

Two axes, never conflate them (store ≠ document)

The single most load-bearing distinction. Two orthogonal axes the terminology historically fused:

• Axis A — which native store? A wallet has 3: private_store_id, protected_store_id, public_store_id. Historic origin of "mono-store / multi-store" (use 1 store vs the 3).
• Axis B — how many documents in a store? A store contains documents; the document (= repo = @graph) is the sharing + rights boundary. The ReadCap — hence isolation — is per-document.

docCreate(sessionId, "Graph", "data:graph", "store", undefined) targets the shared wallet's private store. The trailing store arg left undefined targets the private store (this is what store-registry.ts's createDoc() does). So every document the shim creates physically lives in one store (private), and the public|protected|private scope is a logical label tracked in RDF by the shim — not a NextGraph store. Therefore what a consumer application's "multi-store" flag switches on is really multi-document with logical scope labels, never multi-store. Do not read Scope (types.ts) as a physical store — it is the logical label the registry attaches.

Why undefined and not a real store? Because doc_create cannot target a non-private native store today: StoreRepo is not JS-constructible (verified — see the parked getNativeStore note in migration-guide.md). The private store is reachable because it opens without RepoNotFound.

The shared-wallet shim (store-registry.ts)

Emulates the target infrastructure — where each user owns their own public/protected/private stores — on top of one shared wallet.

• One document per (account × scope) inside the shared wallet, created via the docs.docCreate primitive. The scope (public|protected|private) is a logical attribute tracked here, not a physical store.
• The sharedWalletShim is the mapping account → its 3 scope-document NURIs. It is persisted as RDF, but not directly in the store-root graph — it lives in a subscribable doc-shim reached through a write-once pointer in the store-root, an indirection forced by a NextGraph fact: "findable-without-lookup" (store-root) and "subscribable / cold-read-authoritative" (did:ng:o: repo with a first-State barrier) are DISJOINT. The pointer (findable) names the doc-shim (authoritative); resolution reads the pointer from the store-root, opens the doc-shim through its barrier, and reads the account authoritatively — so a fresh reconnecting session never mistakes sync-lag for "account absent" (which would provision a FORK). Full rationale — including why the old account-level retry (provisionRetry) is removed (pre-indirection store-root records are NOT recovered; such wallets are dev data and simply get a fresh doc-shim) — is in nextgraph-current-state.md §§ Findable vs subscribable / The pointer → doc-shim indirection. This map is the account→document trust root, which is why every untrusted value that reaches its SPARQL is escaped (see SPARQL hardening below). It makes identity resolution cross-device: another device opening the same wallet reads the same pointer → the same doc-shim → the same accounts.
• Per-entity documents + per-scope index. createEntityDoc(id, scope) makes a dedicated document for one entity (mirrors the target, where each entity is its own document/repo with a future inbox) and appends its NURI to the account's scope index document — the index doc plays the role of the future store-container (it lists the entity-document NURIs "in" that scope). listEntityDocs(scope) unions the contained NURIs across all accounts. This is a fallback / test-only path, not the read path: enumerating every account and handing the NURIs to useShape({ graphs }) opens/syncs other accounts' possibly- unsynced docs and hangs (the ORM fan-out — see read-model.md). The real read path is readModel.readUnion(docs), which reads the by-need doc set with one per-doc anchored sparql_query, never an anchorless union-scan of the physical wallet (see read-model.md). The consumer application resolves the by-need doc set from the discovery index (public events) and listMyEntityDocs(id, scope) (its own account, bounded — no cross-account fan-out).
• Generic by construction. The registry knows only the three native scopes, zero application entity kind. The consumer application maps its entities to a scope and injects the session + identity-id normalization via configureStoreRegistry({ getSession, normalizeId }) (polyfill.ts).

The store≠document two axes materialize here directly: the registry moves along axis B (more documents = more isolation), never axis A (it always writes into the one private store via docCreate(..., undefined)).

A virtual wallet's structure — the three emulated stores

A virtual wallet = one account in the shim, keyed by its virtual-wallet id (the technical identifier the consumer application sets when the physical wallet is opened; it identifies which virtual wallet, and is an id rather than a human-friendly handle). Its structure mirrors the target "1 user = 1 wallet with 3 native stores":

Virtual wallet (id)
├── public store    = docPublic  index → [ entity doc NURI, entity doc NURI, … ]
├── protected store = docProtected index → [ record doc NURI, record doc NURI, … ]
└── private store   = docPrivate index → [ record doc NURI, … ]

So the 3 native stores (public/protected/private) are present, but emulated: each "store" is an index document (AccountRecord.{docPublic,docProtected,docPrivate}) that lists the NURIs of the per-entity documents in that scope. It is not a physical native store.

Everything is physical in one place: the 3 index documents, every per-entity document, and the shim anchor itself all live in the shared physical wallet's private store (docCreate(..., undefined)). The 3-store structure is the per-account logical layer the lib maintains on top.

Physical wallet (shared, one) → private_store (physical) holds everything:
   • the shim anchor:  virtual-wallet-id → { docPublic, docProtected, docPrivate }
   • every account's 3 scope-index docs + all per-entity docs + inbox + discovery index

At migration each virtual wallet's 3 index documents become the user's 3 real native stores, the entity documents move into them physically, and the virtual/physical distinction dissolves (see migration-guide.md).

SDK-shaped scope resolvers — the consumer application holds no store-id

The consumer application must never construct a did:ng:${store_id} NURI itself: physical placement is the lib's job (the whole point of the SDK boundary). Two resolvers turn a logical scope into an opaque graph NURI without exposing any store-id:

• resolveScopeGraph(scope) — the graph where the current session writes entities of scope, and whose repo useShape subscribes to read them back. Use the returned value as BOTH the read scope (useShape(shape, nuri)) and the @graph write target. Placement lives HERE (Axis A): private → the private native store; public + protected → the protected native store, because doc_create/ORM cannot target a non-private/protected native store today (SDK blocker, migration-guide.md). At migration each scope resolves to the user's real per-scope store — the change is in this function, and the consumer application is unchanged.
• resolveInboxAnchor() — the anchor where emulated inbox deposits land: a dedicated inbox document (a reserved account's public scope document, from docCreate — a real repo NURI, stable across clients), not the shared wallet's private-store root. Why dedicated: the shim (the account→document trust root) lives in the private-store graph and is scanned on every loadShim; routing every inbox deposit into that same graph bloats it without bound (thousands of deposit triples across sessions), turning loadShim into a multi-second full-graph scan. A separate inbox document keeps the shim graph small and the deposits isolated. At migration it becomes the host's native inbox NURI.

Both resolve the native store ids from the injected session (RegistrySession.protectedStoreId / publicStoreId, alongside the existing privateStoreId anchor). The consumer application hands the whole session to the lib at the one injection point (configureStoreRegistry({ getSession })) — that is wiring, not placement logic; everything else in the consumer application speaks only in scopes. If the session omits protectedStoreId, the non-private scopes fall back to the private store rather than emit a broken NURI.

RepoNotFound and the orm_start_graph scope rule

A hard constraint inherited from the SDK: to read and write entities through the ORM, the store's repo must be explicitly opened in the verifier's self.repos HashMap. orm_start_graph with a store's NURI opens that repo; without it, orm_frontend_update fails with RepoNotFound.

• Scope for useShape: the store NURI, e.g. did:ng:${privateStoreId} (or, in the consumer application, a per-user store once that migration happens).
• @graph (write target): the same store NURI.
• Never use did:ng:i as a scope: it subscribes to the user's whole site via a special code path (NuriTargetV0::UserSite) that does not open individual repos, breaking every write with RepoNotFound.

Both the private and the protected native stores were verified to open the same way for ORM+SPARQL (round-trip probe, no RepoNotFound). The original arbitration is preserved in decisions/private-store-nuri-scope.md.

The @ng-org double-proxy DataCloneError constraint

A validated hard constraint, not a style choice: docs.ts calls the real injected ng (getConfig().ng) directly, never the public ng proxy (makeNg in ng-proxy.ts).

@ng-org/web's ng is already an iframe-RPC proxy (postMessage marshaling, see nextgraph-current-state.md § integration). Wrapping it in the lib's own JS Proxy (double proxy) breaks doc_create's postMessage marshaling with DataCloneError: function ... could not be cloned. Reaching the real ng held in the config avoids the double-proxy. This was verified: routing the shim's doc_create/SPARQL through the public proxy turned 4 multistore scenarios red, so it was reverted. The integration boundary is:

• Through the lib's public proxy (validated): useShape (ORM + ReadCap filter), init/initNg, login.
• Through the real injected ng (docs.ts primitives): doc_create + all shim/inbox SPARQL.

docs.ts therefore imports no @ng-org package and must not import from ./ng-proxy.

Emulated ReadCap — per document (caps.ts + read-filter.ts)

In the target the broker only delivers documents the wallet holds a ReadCap for, so useShape already returns an authorized subset. Here (single shared wallet, everything readable) the lib reproduces that with a read-filtered view:

• CapRegistry (caps.ts) models ReadCaps as faithfully as a data layer can. The access unit is the document = repo NURI (an item's @graph), never the item — because in nextgraph-rs a store is just a container repo and holding its cap does not grant the repos it references (no store-level read inheritance; verified). So the registry is purely per-document: grantRead(doc, granteeId) issues a directed read grant to one identity, alongside grantWrite / makePublic / open(doc, scope, owner) / canRead / canWrite / governsRead / hasReadPolicy, plus the read-only accessor protectedDocsOf(owner) the consumer application uses to pick which protected docs to grant. The consumer application performs the acts of granting (create-public, grant a specific doc to a specific identity…) exactly as it will in the target; the lib injects no policy.
• read-filter.ts — makeReadFilteredView wraps the reactive set in a Proxy: iteration / size / forEach are filtered by caps.canRead(item['@graph'], user); everything else (add, delete, has, getById…) forwards to the target, preserving writes and reactivity. An item with no @graph, or in a document under no cap policy, is kept (the filter only restricts documents that declare a cap — no regression on ungoverned data). filterReadable is the pure variant.
• useShape (use-shape.ts) applies the view only if caps.hasReadPolicy() — otherwise it passes the real set through unchanged (no regression when the consumer application declares no caps).

In a mono-store layout (every item in one repo) this is all-or-nothing on that document — exactly the native behaviour, and why fine-grained isolation requires one document per entity (axis B).

Making the ReadCap active — current identity + directed grants

The filter only discriminates once the consumer application (a) tells the SDK who is reading and (b) declares the access policy on the documents. Both are plain SDK calls; the consumer application never touches the registry internals:

• setCurrentUser(id) (polyfill.ts) — the SDK's "current identity" call. useShape's filtered view reads it lazily, so the delivered subset always reflects the identity in effect at read time. Until it is set, the filter has no principal and (per canRead(doc, null)) only public documents pass — which is why isolation stays dormant until the consumer application makes this call.
• getCaps().open(doc, scope, owner) — declares a document's policy when the consumer application creates it: public → world-readable; protected/private → owner reads, owner holds the write cap. open also remembers (scope, owner) per document so protectedDocsOf(owner) can later enumerate the protected ones.
• grantRead(doc, granteeId) (caps.ts, exposed via getCaps()) — the one relationship-shaped sharing act the lib exposes: a directed per-document read grant issued to a specific identity. Public docs stay world-readable; private docs stay owner-only; a protected doc becomes readable by granteeId once the owner grants it. The consumer application passes a document NURI and a grantee id — no store id.

The relationship concept — who is "connected" to whom, and therefore which of their protected docs to grant — is owned by the consumer application, not the lib. A connection or friendship is not a NextGraph primitive; the only platform-mappable primitive is the directed per-document read grant above. So the consumer application decides a relationship exists and, for each protected doc it wants to share, calls grantRead(doc, granteeId) — typically iterating protectedDocsOf(owner) to pick the owner's protected docs. The intended target of such a directed grant is a native per-document ReadCap issued to that identity — but that target is itself scaffolding-only in nextgraph-rs today, not merely unexposed in JS: AccessGrantV0 {grantee} is unpersisted and cap-send is unimplemented!(), so directing a grant to another identity is not-yet-built at the platform level. There is no bilateral capability exchange to mirror, only (eventually) individual directed grants.

The result is the target's discrimination reproduced end-to-end: private → owner; protected → owner + whoever the owner has directly granted; public → all. Proven in test/isolation-active.test.ts: an unconnected principal is denied a protected document, granted it after the owner issues a directed grantRead, and reads the public document throughout.

This discrimination is only observable because each entity is its own document (the consumer application creates per-entity docs via createEntityDoc and opens each) — in a mono-store layout the per-document ReadCap is all-or-nothing.

Write-guard coverage (honest scope)

The emulated write guard (ng-proxy.ts, sparql_update override) enforces the per-document write cap on the public ng proxy only. In practice the consumer application's write paths (docs.sparqlUpdate, ORM ngSet) call the real injected ng directly — never the public proxy — for the validated DataCloneError reason above. So the guard is best-effort: it fires for any write routed through the public proxy, but the consumer application's real write paths bypass it and are not guarded today. This is a deliberate, recorded limitation of the emulation (the write guard becomes effective only when the broker/verifier enforces caps natively at migration); the read side is what makes isolation observably active.

The per-document ReadCap is the isolation path (item-level filter retired)

Isolation is enforced by the per-document ReadCap (caps.ts + read-filter.ts) alone: the access unit is the document (@graph = repo), and grants are explicit (open / grantRead / makePublic) — for protected, the owner issues a directed grantRead(doc, granteeId) per identity it wants to share with. Because the consumer application now writes one document per entity (createEntityDoc + open per entity), the per-document cap discriminates at entity granularity — the target's behaviour.

The old item-level application-visibility filter (isolation.ts applyIsolation, a Set-of-records filter keyed on owner+scope) is retired from the consumer path: the application carries no access logic — it declares its identity and issues directed grants, and trusts the SDK. Its matrix functions are dead scaffolding kept for reference and removed at migration. There is no longer a second, coexisting app-layer filter to reconcile — the single axis is the per-document cap, exactly as in the target.

Emulated inbox (inbox.ts)

Current NextGraph does not expose the inbox to the JS SDK (verifier has no InboxPost arm; no wasm sealing helper — see nextgraph-current-state.md § Inbox). Rather than fork the broker (fork-inbox-fallback.md), the lib emulates the inbox on the shared wallet:

• Target vs polyfill. In the target, post seals a reference into the owner's native inbox (inbox_post_link(...), a proposed/future API) and the recipient's own verifier unseals each queued message and applies it inline when it processes its inbox — there is no separate curator or materialization process. Here, everything is readable, so the lib emulates the read side in-lib.
• post(targetInbox, opts) appends a deposit { from, payload, ts } as RDF into the inbox document (in the shared wallet) via docs.sparqlUpdate. Each deposit is a unique RDF subject, so concurrent deposits don't collide. from is bound to the current identity (getCurrentUser) — it is authenticated, not caller-supplied: omit it to stamp the current user, pass null to deposit anonymously, and a from naming another principal is rejected as a spoof. This reproduces the protocol's "identified if known, anonymous otherwise" and the target's guarantee that a client cannot forge another's sender identity (in the target the broker seals from from the wallet's own key; here the check closes the spoof the shared wallet would otherwise allow). The emulation stores from = null as absence of a triple, so it does not provide the target's crypto anonymity (from = None sealed), which only a native inbox would. Proven in test/inbox.test.ts case (c).
• read / materialize (alias) emulate the recipient-side read: they read the deposits back via docs.sparqlQuery, JSON-parse each payload, sort by ts.
• watch(targetInbox, onDeposits, { intervalMs }) is the emulated watcher: it polls read and fires when the deposit count changes (the polyfill has no reactive inbox subscription). Fires once immediately; returns an unsubscribe.

The module knows no domain — the consumer application supplies the inbox document NURI and interprets payload. At migration post becomes the native inbox_post_link (proposed/future) and the read side is served by the recipient's own verifier unsealing queued messages inline (see the deferred global-index note in the top-level README and decisions/discovery-model.md). The inbox + watcher is the one deposit/read mechanism a consumer reuses for its own purposes — e.g. a registration/deposit in one consumer app and submission to a discovery index — same post API, same watcher.

Emulated discovery index + special account (discovery.ts)

Discovery is a surface on top of the inbox, not a new primitive. Access is not the same as discovery: a public entity is world-readable with its NURI; the discovery index is how a client learns that NURI exists without holding a relationship to its creator (see decisions/discovery-model.md). The model is: one global index = an owned document (public read), fed via its inbox. Nobody writes the index directly — a creator deposits a reference into the index's inbox, and the index is built up from those deposits. That build-up step is the natural dedup / moderation point.

• The special account (polyfill owner). "Who owns the global index" is undecided in the target (NextGraph is mono-user with no global data — a singleton app is the only glimpsed path). So the polyfill parks ownership on a reserved special account in the shim — INDEX_ACCOUNT = reservedAccount("index"). This is NOT the key "index" / "@index": reservedAccount mints a sentinel-prefixed key in the shim's reserved namespace (e.g. " reserved:index") that normalizeId can never produce, so no user id — not even one typed as "index" or "@index", which normalizes to the disjoint key "index" — can collide with or hijack the index account (asserted in discovery.test.ts). It is a normal shim account (so its 3 scope documents are created on first sight like any other), but never a real user; it only hosts the index document. Its public scope document is the index document, and its inbox receives the deposits — a stable NURI: every client opening the same shared wallet resolves the same account, hence the same document, so all clients read/write one shared index.
• submitToIndex(ref, opts?) — the SDK act "make this discoverable". Deposits ref into the index document's inbox via inbox.post. from follows the inbox convention (bound to the current identity; anonymous when null). ref is opaque here — the consumer application serializes whatever locates the entity (e.g. an entity document NURI + discovery metadata). Public-only guard: when opts.doc names the document being surfaced, a document under a non-public (protected/private) read policy is refused (caps.governsRead(doc) && !caps.canRead(doc, null)) — the global index is world-readable, so admitting a governed doc's NURI would leak it past its scope. Proven in test/discovery.test.ts case (d).
• readIndex() — the emulated read side. Reads every submission, dedups by serialized ref (the moderation point: a duplicate submission surfaces once), returns entries sorted by ts. watchIndex(onEntries, opts?) is the emulated watcher (polls readIndex).

This replaces the cross-account fan-out (store-registry.ts listEntityDocs('public') / resolveReadGraphs) as the app-facing discovery path: the consumer application submits public entities to the index and reads the index, instead of fanning out over every account's public documents. The fan-out survives only as an internal lib fallback — kept for the per-scope listing it also powers (e.g. resolveReadGraphs), never the app's discovery route.

discovery.ts knows no application domain — the consumer application defines the ref shape and its meaning. At migration the special account disappears: ownership moves to the decided global-index owner, submitToIndex becomes the native inbox_post_link (proposed/future) on the index's inbox, and readIndex queries the real index document. The consumer surface (submitToIndex / readIndex) is designed to survive that swap unchanged.

Emulated write guard (ng-proxy.ts)

The public ng proxy overrides sparql_update to enforce an emulated write cap: a write is refused unless the current user holds the target document's write cap. It passes through (no regression) unless a write policy exists and that specific document (the anchor arg) is governed by it — ungoverned docs (the mono-store default, no cap declared) flow through unchanged. This mirrors the target broker/verifier, which refuses a write without the document's write cap.

Identity store (accounts.ts)

The real NextGraph login (redirect to the broker, opening the single shared wallet) is perceived as a technical access barrier (see the login flow in decisions/shared-wallet-login-flow.md). This layer is not a login: it is an IdentityStore that holds the current identity id the consumer application relays to it:

• The identity id is set at wallet-import time by the consumer application and relayed to the lib via its current-identity call. It is persisted in localStorage so the id survives reloads and lands on the same account when the shared wallet re-opens. In practice the id is often a human-friendly handle the consumer application chose, but the lib's surface speaks only of an id.
• set(id) / clear() / get() only read/write the id in storage. They never call NextGraph (no session_stop / wallet_close) — the shared wallet stays open underneath. The real logout lives elsewhere (hidden in the consumer application's settings/debug), because it forces a new redirect.
• Framework-agnostic: no React, no DOM beyond an optional injected AccountStorage (a window.localStorage, a test fake, or null for SSR). The React Context/Provider stays in the consumer application. normalizeId (case-insensitive, optional leading @ stripped, trimmed) is the pure normalizer, reusable as the shim key normalizer.

SPARQL injection hardening (sparql.ts)

Every module that builds SPARQL by interpolation (inbox, store-registry) routes untrusted values through sparql.ts first, because a " closes a literal and a > closes an IRI, letting an injected value wreck the shim graph (the account → document trust root):

• escapeLiteral — for LITERAL position ("..."): escapes backslash, double-quote, C0 whitespace. Lossless (literals legitimately carry arbitrary text — JSON payloads, display names).
• escapeIri — for untrusted values embedded into an IRI (<PREFIX:${…}>, e.g. an identity id minted into an account-subject IRI): percent-encodes every IRI-hostile character so any id (spaces, unicode, punctuation) stays usable while breakout is impossible.
• assertNuri — for trusted-shaped NURIs coming back from ng (did:ng:...): validates and throws on IRI-breaking chars rather than emitting a malformed/injected query.

These are re-exported from @ng-eventually/client so the consumer application reuses the same escaping when it builds SPARQL.