ng-eventually/docs/simulation.md

How this library emulates mature NextGraph on ONE shared wallet

The consumer writes against @ng-eventually/client as if NextGraph already shipped per-entity documents in public/protected/private stores, capabilities and inboxes. It hasn't (see nextgraph-current-state.md). This file is the lib's own engineering doctrine on how it fabricates that mature face on top of one single shared wallet / broker. Everything here is polyfill-era and disappears at migration (migration-guide.md).

The premise: one shared wallet, everything readable

Current NextGraph has no cross-wallet read (OpenRepo is a TODO at engine/verifier/src/verifier.rs:1423; a foreign NURI raises RepoNotFound; a session only holds its own 3 stores in self.repos). So "each user their own wallet" is blocked at the root — no data ever crosses the boundary between two wallets.

The lib's answer: everyone opens the same wallet. NextGraph sees a single identity → everything is physically readable. "Multi-user" becomes an application fiction the lib maintains. On top of that one wallet the lib rebuilds, by emulation, the per-user stores + capabilities + inbox the consumer codes against.

Physical wallet vs virtual wallet — never enumerate the physical one

Because the emulation runs on ONE shared wallet, distinguish two levels:

• Physical wallet — the real NextGraph wallet everyone opens. Its local store holds every account's documents PLUS the lib's own internals (the shim index, the inbox docs, the discovery index) as named graphs. It accumulates without bound across sessions/runs. Listing / scanning "all documents" of the physical wallet is meaningless AND O(size) — it mixes every user's data with lib internals, and it is exactly what a sparql_query with no anchor (GRAPH ?g { … }) does (it spans every synced graph). Never do it. The physical wallet is a substrate, not something to enumerate.

• Virtual wallet — the lib's emulation of one user's wallet: the set of documents the shim attributes to that account (its per-scope index in store-registry.ts). This is what "the user owns". Over a virtual wallet, "list my documents" is meaningful and bounded (only that account's docs).

Consequence for reads (see read-model.md): to list a user's entities you enumerate the virtual wallet — the account's scope index (bounded, O(my docs)), NOT the physical union — then read those specific documents with a per-doc anchored sparql_query. A non-empty / bloated physical wallet then costs nothing, because the physical union is never scanned. Discovery (all public events) is the one bounded enumeration hack and goes through the discovery index, not a physical scan.

At migration each virtual wallet becomes a real per-user wallet; the physical/virtual distinction — and the "never enumerate the physical wallet" rule — dissolves into native per-wallet reads.

Two axes, never conflate them (store ≠ document)

The single most load-bearing distinction. Two orthogonal axes the terminology historically fused:

• Axis A — which native STORE? A wallet has 3: private_store_id, protected_store_id, public_store_id. Historic origin of "mono-store / multi-store" (use 1 store vs the 3).
• Axis B — how many DOCUMENTS in a store? A store contains documents; the document (= repo = @graph) is the sharing + rights boundary. The ReadCap — hence isolation — is PER-DOCUMENT.

docCreate(sessionId, "Graph", "data:graph", "store", undefined) → the shared wallet's PRIVATE store. The trailing store arg left undefined targets the private store (this is what store-registry.ts's createDoc() does). So every document the shim creates physically lives in ONE store (private), and the public|protected|private scope is a LOGICAL LABEL tracked in RDF by the shim — not a NextGraph store. Therefore what a consumer's "multi-store" flag switches on is really multi-DOCUMENT with logical scope labels, never multi-store. Do not read Scope (types.ts) as a physical store — it is the logical label the registry attaches.

Why undefined and not a real store? Because doc_create cannot target a non-private native store today: StoreRepo is not JS-constructible (verified — see the parked getNativeStore note in migration-guide.md). The private store is reachable because it opens without RepoNotFound.

The shared-wallet shim (store-registry.ts)

Emulates the target infrastructure — where each user owns their own public/protected/private stores — on top of one shared wallet.

• One document per (account × scope) inside the shared wallet, created via the docs.docCreate primitive. The scope (public|protected|private) is a logical attribute tracked here, not a physical store.
• The sharedWalletShim is the mapping account → its 3 scope-document NURIs, persisted as RDF in the shared wallet's private store (the anchor, always known from the session: RegistrySession.privateStoreId). That makes login cross-device: another device opening the same wallet reads the same shim and finds the same accounts. It is the account→document trust root — which is why every untrusted value that reaches its SPARQL is escaped (see SPARQL hardening below).
• Per-entity documents + per-scope index. createEntityDoc(username, scope) makes a dedicated document for ONE entity (mirrors the target, where each entity is its own document/repo with a future inbox) and appends its NURI to the account's scope index document — the index doc plays the role of the future store-container (it lists the entity-document NURIs "in" that scope). listEntityDocs(scope) unions the contained NURIs across all accounts. This is a fallback / test-only path, NOT the read path: enumerating every account and handing the NURIs to useShape({ graphs }) opens/syncs other accounts' possibly- unsynced docs and HANGS (the ORM fan-out, ~75s — see read-model.md). The real READ path is readModel.readUnion(docs) (open/sync by-need + ONE anchorless union sparql_query); the app resolves the by-need doc set from the discovery index (public events) and listMyEntityDocs(username, scope) (my own account, bounded — no cross-account fan-out).
• GENERIC by construction. The registry knows only the three native scopes, zero application entity kind. The consumer maps its entities to a scope and injects the session + username normalization via configureStoreRegistry({ getSession, normalizeUser }) (polyfill.ts).

The store≠document two axes materialize here directly: the registry moves along axis B (more documents = more isolation), never axis A (it always writes into the one private store via docCreate(..., undefined)).

A virtual wallet's structure — the three emulated stores

A virtual wallet = one account in the shim, keyed by its virtual-wallet id (the technical identifier the consumer sets when the physical wallet is opened; it identifies which virtual wallet, and is NOT the consumer's application username). Its structure mirrors the target "1 user = 1 wallet with 3 native stores":

Virtual wallet (id)
├── public store    = docPublic  index → [ event doc NURI, PdR doc NURI, … ]
├── protected store = docProtected index → [ profile doc NURI, participation doc NURI, … ]
└── private store   = docPrivate index → [ settings doc NURI, … ]

So yes, the 3 native stores (public/protected/private) are present — but emulated: each "store" is an index document (AccountRecord.{docPublic,docProtected,docPrivate}) that LISTS the NURIs of the per-entity documents in that scope. It is not a physical native store.

Everything is physical in ONE place: the 3 index documents, every per-entity document, and the shim anchor itself all live in the shared physical wallet's private store (docCreate(..., undefined)). The 3-store structure is the per-account logical layer the lib maintains on top.

Physical wallet (shared, ONE) → private_store (physical) holds EVERYTHING:
   • the shim anchor:  virtual-wallet-id → { docPublic, docProtected, docPrivate }
   • every account's 3 scope-index docs + all per-entity docs + inbox + discovery index

At migration each virtual wallet's 3 index documents become the user's 3 real native stores, the entity documents move into them physically, and the virtual/physical distinction dissolves (see migration-guide.md).

SDK-shaped scope resolvers — the consumer holds NO store-id

The consumer must never construct a did:ng:${store_id} NURI itself: physical placement is the lib's job (the whole point of the SDK boundary). Two resolvers turn a logical scope into an opaque graph NURI without exposing any store-id:

• resolveScopeGraph(scope) — the graph where the current session writes entities of scope, and whose repo useShape subscribes to read them back. Use the returned value as BOTH the read scope (useShape(shape, nuri)) and the @graph write target. Placement lives HERE (Axis A): private → the private native store; public + protected → the protected native store, because doc_create/ORM cannot target a non-private/protected native store today (SDK blocker, migration-guide.md). At migration each scope resolves to the user's REAL per-scope store — the change is in this function, the consumer is unchanged.
• resolveInboxAnchor() — the anchor where emulated inbox deposits land: a DEDICATED inbox document (a reserved account's public scope document, from docCreate — a real repo NURI, stable across clients), not the shared wallet's private-store root. Why dedicated: the shim (the account→document trust root) lives in the private-store graph and is scanned on every loadShim; routing every inbox deposit into that SAME graph bloats it without bound (thousands of deposit triples across sessions), turning loadShim into a multi-second full-graph scan. A separate inbox document keeps the shim graph small and the deposits isolated. At migration it becomes the host's native inbox NURI.

Both resolve the native store ids from the injected session (RegistrySession.protectedStoreId / publicStoreId, alongside the existing privateStoreId anchor). The consumer hands the whole session to the lib at the ONE injection point (configureStoreRegistry({ getSession })) — that is wiring, not placement logic; everything else in the consumer speaks only in scopes. If the session omits protectedStoreId, the non-private scopes fall back to the private store rather than emit a broken NURI.

RepoNotFound and the orm_start_graph scope rule

A hard constraint inherited from the SDK: to read and write entities through the ORM, the store's repo must be explicitly opened in the verifier's self.repos HashMap. orm_start_graph with a store's NURI opens that repo; without it, orm_frontend_update fails with RepoNotFound.

• Scope for useShape: the store NURI, e.g. did:ng:${privateStoreId} (or, in the consumer, a per-user store once that migration happens).
• @graph (write target): the same store NURI.
• Never use did:ng:i as a scope. It subscribes to the user's whole site via a special code path (NuriTargetV0::UserSite) that does not open individual repos → breaks every write with RepoNotFound.

Both the private and the protected native stores were verified to open the same way for ORM+SPARQL (round-trip probe, no RepoNotFound). The original arbitration is preserved in decisions/private-store-nuri-scope.md.

The @ng-org double-proxy DataCloneError constraint

Validated hard constraint, not a style choice. docs.ts calls the real injected ng (getConfig().ng) DIRECTLY — never the public ng proxy (makeNg in ng-proxy.ts).

@ng-org/web's ng is already an iframe-RPC proxy (postMessage marshaling, see nextgraph-current-state.md § integration). Wrapping it in the lib's own JS Proxy (double proxy) breaks doc_create's postMessage marshaling → DataCloneError: function ... could not be cloned. Reaching the real ng held in the config avoids the double-proxy. This was verified: routing the shim's doc_create/SPARQL through the public proxy turned 4 multistore scenarios red; it was reverted. The integration boundary is:

• Through the lib's public proxy (validated): useShape (ORM + ReadCap filter), init/initNg, login.
• Through the real injected ng (docs.ts primitives): doc_create + all shim/inbox SPARQL.

docs.ts therefore imports no @ng-org package and must not import from ./ng-proxy.

Emulated ReadCap — per document (caps.ts + read-filter.ts)

In the target the broker only delivers documents the wallet holds a ReadCap for, so useShape already returns an authorized subset. Here (single shared wallet, everything readable) the lib reproduces that with a read-filtered VIEW:

• CapRegistry (caps.ts) models ReadCaps as faithfully as a data layer can. The access UNIT is the document = repo NURI (an item's @graph), never the item — because in nextgraph-rs a store is just a container repo and holding its cap does NOT grant the repos it references (no store-level read inheritance; verified). So the registry is purely per-document: grantRead / grantWrite / makePublic / open(doc, scope, owner) / canRead / canWrite / governsRead / hasReadPolicy. The consumer performs the acts of granting (create-public, grant-to-a-connection…) exactly as it will in the target; the lib injects no policy.
• read-filter.ts — makeReadFilteredView wraps the reactive set in a Proxy: iteration / size / forEach are filtered by caps.canRead(item['@graph'], user); everything else (add, delete, has, getById…) forwards to the target, preserving writes and reactivity. An item with no @graph, or in a document under no cap policy, is KEPT (the filter only restricts documents that declare a cap — no regression on ungoverned data). filterReadable is the pure variant.
• useShape (use-shape.ts) applies the view only if caps.hasReadPolicy() — otherwise it passes the real set through unchanged (no regression when the consumer declares no caps).

In a mono-store layout (every item in one repo) this is all-or-nothing on that document — exactly the native behaviour, and why fine-grained isolation requires one document per entity (axis B).

Making the ReadCap ACTIVE — current user + connection-driven grants

The filter only discriminates once the consumer (a) tells the SDK who is reading and (b) declares the access policy on the documents. Both are plain SDK calls; the consumer never touches the registry internals:

• setCurrentUser(id) (polyfill.ts) — the SDK's "current identity" call. useShape's filtered view reads it lazily, so the delivered subset always reflects the identity in effect at read time. Until it is set, the filter has no principal and (per canRead(doc, null)) only public documents pass — which is why isolation stayed dormant while the consumer never made this call.

• getCaps().open(doc, scope, owner) — declares a document's policy when the consumer creates it: public → world-readable; protected/private → owner reads, owner holds the write cap. open now also remembers (scope, owner) per document so a later connection-driven grant can find the protected ones.

• declareConnections(peers, as?) (polyfill.ts) — the SDK-shaped protected sharing act, now AUTHENTICATED / BILATERAL (connections.ts). Each call declares the CURRENT identity's OWN peers (as defaults to getCurrentUser()); the lib records that as a directed assertion authored by the current identity — a session can only ever assert its own side. A protected read cap is issued between two principals only when both have asserted the other (a materialized two-sided link, ConnectionRegistry.neighbors → CapRegistry.grantReadToConnections). Public docs stay world-readable; private docs stay owner-only. Re-callable; additive + idempotent. The consumer passes only principals — no document NURI, no store id.

  Why bilateral (adversarial finding). If a single directed assertion granted access, any reader could read any owner's protected documents by unilaterally self-declaring a connection. The two-sided requirement is the emulation of the target's mutual capability exchange: only a reciprocated link grants the cap. A unilateral / self-declared connection grants nothing (proven in test/connections.test.ts and test/isolation-active.test.ts case (b)).

The result is the target's discrimination reproduced end-to-end: private → owner; protected → owner + BILATERAL connections; public → all. Proven in test/isolation-active.test.ts: (a) an unconnected principal is denied a protected document, granted it after a two-sided declareConnections, and reads the public document throughout; (b) a unilateral/self-declared connection is denied.

This discrimination is only observable because each entity is its own document (the consumer creates per-entity docs via createEntityDoc and opens each) — in a mono-store layout the per-document ReadCap is all-or-nothing.

Write-guard coverage (honest scope)

The emulated write guard (ng-proxy.ts, sparql_update override) enforces the per-document write cap on the public ng proxy only. In practice the consumer's write paths (docs.sparqlUpdate, ORM ngSet) call the real injected ng directly — never the public proxy — for the validated DataCloneError reason above. So the guard is best-effort: it fires for any write routed through the public proxy, but the consumer's real write paths bypass it and are not guarded today. This is a deliberate, recorded limitation of the emulation (the write guard becomes effective only when the broker/verifier enforces caps natively at migration); the READ side is what makes isolation observably active.

The per-document ReadCap is now THE isolation path (item-level filter retired)

Isolation is enforced by the per-document ReadCap (caps.ts + read-filter.ts) alone: the access unit is the DOCUMENT (@graph = repo), grants are explicit (open / grantRead / makePublic) and, for protected, driven by the bilateral connection registry (connections.ts). Because the consumer now writes one document per entity (createEntityDoc + open per entity), the per-document cap discriminates at entity granularity — the target's behaviour.

The old item-level application-visibility filter (isolation.ts applyIsolation, a Set-of-records filter keyed on owner+scope) is retired from the consumer path: the app carries no access logic — it declares its identity and its bilateral connections and trusts the SDK. isolation.ts survives only as the home of the generic Connections interface (consumed by connections.ts / caps.grantReadToConnections) plus its own unit tests; its matrix functions are dead scaffolding kept for reference and removed at migration. There is no longer a second, coexisting app-layer filter to reconcile — the single axis is the per-document cap, exactly as in the target.

Emulated inbox + curator (inbox.ts)

Current NextGraph does not expose the inbox to the JS SDK (verifier has no InboxPost arm; no wasm sealing helper — see nextgraph-current-state.md § Inbox). Rather than fork the broker (fork-inbox-fallback.md), the lib emulates the inbox on the shared wallet:

• Target vs polyfill. Target: post seals a reference into the owner's native inbox (ng.inbox_post_link(...)) and a separate curator materializes deposits into the owned document. Here, everything is readable, so both sides are emulated in-lib.
• post(targetInbox, opts) appends a deposit { from, payload, ts } as RDF into the inbox DOCUMENT (in the shared wallet) via docs.sparqlUpdate. Each deposit is a unique RDF subject → concurrent deposits don't collide. from is BOUND to the current identity (getCurrentUser) — it is authenticated, not caller-supplied: omit it to stamp the current user, pass null to deposit ANONYMOUSLY, and a from naming ANOTHER principal is rejected as a spoof. This reproduces the protocol's "identified if known, anonymous otherwise" AND the target's guarantee that a client cannot forge another's sender identity (in the target the broker seals from from the wallet's own key; here the check closes the spoof the shared wallet would otherwise allow). The emulation stores from = null as absence of a triple, so it does not provide the target's crypto anonymity (from = None sealed), which only a native inbox would. Proven in test/inbox.test.ts case (c).
• read / materialize (alias) play the emulated CURATOR: they read the deposits back via docs.sparqlQuery, JSON-parse each payload, sort by ts.
• watch(targetInbox, onDeposits, { intervalMs }) is the emulated watcher: it polls read and fires when the deposit count changes (the polyfill has no reactive inbox subscription). Fires once immediately; returns an unsubscribe.

GENERIC: the module knows no domain — the consumer supplies the inbox document NURI and interprets payload. At migration post becomes the native inbox_post_link and the read side moves to a separate curator package (see the deferred global-index note in the top-level README and decisions/discovery-model.md). The inbox + watcher is the ONE deposit/materialization mechanism reused for BOTH meeting-point registration AND submission to a discovery index — same post API, same watcher.

Emulated discovery index + special account (discovery.ts)

Discovery is a surface on top of the inbox, not a new primitive. Access ≠ discovery: a public entity is world-readable with its NURI; the discovery index is how a client learns that NURI exists without holding a connection to its creator (see decisions/discovery-model.md). The model is: ONE global index = an owned document (public read), fed via ITS inbox, materialized by a curator. Nobody writes the index directly — a creator DEPOSITS a reference into the index's inbox; the curator ingests deposits into entries. Materialization is the natural dedup / moderation point.

• The special account (polyfill owner). "Who owns the global index" is undecided in the target (NextGraph is mono-user with no global data — a singleton app is the only glimpsed path). So the polyfill parks ownership on a RESERVED SPECIAL ACCOUNT in the shim — INDEX_ACCOUNT = "@index". It is a normal shim account (so its 3 scope documents are created on first sight like any other), but never a real user; it only HOSTS the index document. Its public scope document IS the index document, and its inbox receives the deposits — a stable NURI: every client opening the same shared wallet resolves the same account → same document, so all clients read/write ONE shared index.
• submitToIndex(ref, opts?) — the SDK act "make this discoverable". Deposits ref into the index document's inbox via inbox.post. from follows the inbox convention (bound to the current identity; anonymous when null). ref is opaque here — the consumer serializes whatever locates the entity (e.g. an entity document NURI + discovery metadata). PUBLIC-ONLY guard: when opts.doc names the document being surfaced, a document under a non-public (protected/private) read policy is REFUSED (caps.governsRead(doc) && !caps.canRead(doc, null)) — the global index is world-readable, so admitting a governed doc's NURI would leak it past its scope. Proven in test/discovery.test.ts case (d).
• readIndex() — the EMULATED CURATOR. Reads every submission, dedups by serialized ref (the moderation point: a duplicate submission surfaces once), returns entries sorted by ts. watchIndex(onEntries, opts?) is the emulated watcher (polls readIndex).

This REPLACES the cross-account fan-out (store-registry.ts listEntityDocs('public') / resolveReadGraphs) as the app-facing discovery path: the app submits public entities to the index and reads the index, instead of fanning out over every account's public documents. The fan-out survives only as an internal lib fallback — kept for the per-scope listing it also powers (e.g. resolveReadGraphs), never the app's discovery route.

GENERIC: discovery.ts knows no application domain — the consumer defines the ref shape and its meaning. At migration the special account disappears: ownership moves to the decided global-index owner, submitToIndex becomes the native inbox_post_link on the index's inbox, and readIndex queries the real materialized index document. The consumer surface (submitToIndex / readIndex) is designed to survive that swap unchanged.

Emulated write guard (ng-proxy.ts)

The public ng proxy overrides sparql_update to enforce an emulated write cap: a write is refused unless the current user holds the target document's WRITE cap. Passthrough (no regression) unless a WRITE policy exists AND that specific document (the anchor arg) is governed by it — ungoverned docs (the mono-store default, no cap declared) flow through unchanged. Mirrors the target broker/verifier, which refuses a write without the document's write cap.

Faux login (accounts.ts)

The real NextGraph login (redirect to the broker, opening the single SHARED wallet) is perceived as a technical access barrier, not a login (see login flow in decisions/shared-wallet-login-flow.md). THIS layer is the perceived login:

• The user picks a username (no password — declarative), persisted in localStorage so the "session" survives reloads and lands on the same account when the shared wallet re-opens.
• login() / logout() are FAUX: they only read/write the username in storage. They must NEVER call NextGraph (no session_stop / wallet_close) — the shared wallet stays open underneath. The real logout lives elsewhere (hidden in the consumer's settings/debug), because it forces a new redirect.
• Framework-agnostic: no React, no DOM beyond an optional injected AccountStorage (a window.localStorage, a test fake, or null for SSR). The React Context/Provider stays in the consumer. normalizeUsername (case-insensitive, optional leading @ stripped, trimmed) is the pure normalizer, reusable as the shim key normalizer.

SPARQL injection hardening (sparql.ts)

Every module that builds SPARQL by interpolation (inbox, store-registry) routes untrusted values through sparql.ts first, because a " closes a literal and a > closes an IRI, letting an injected value wreck the shim graph (the account → document trust root):

• escapeLiteral — for LITERAL position ("..."): escapes backslash, double-quote, C0 whitespace. Lossless (literals legitimately carry arbitrary text — JSON payloads, display names).
• escapeIri — for UNTRUSTED values embedded into an IRI (<PREFIX:${…}>, e.g. a username minted into an account-subject IRI): percent-encodes every IRI-hostile character so any username (spaces, unicode, punctuation) stays usable while breakout is impossible.
• assertNuri — for trusted-SHAPED NURIs coming back from ng (did:ng:...): validates and throws on IRI-breaking chars rather than emitting a malformed/injected query.

These are re-exported from @ng-eventually/client so the consumer reuses the same escaping when it builds SPARQL.