diff --git a/docs/briefs/2026-07-27-p1a-cap-surface.md b/docs/briefs/2026-07-27-p1a-cap-surface.md index ba8b974..382ad69 100644 --- a/docs/briefs/2026-07-27-p1a-cap-surface.md +++ b/docs/briefs/2026-07-27-p1a-cap-surface.md @@ -14,7 +14,8 @@ The spec below is unchanged — read it first. Everything from here to *Why this > - *"Processing inboxes … Not started"* — **done** (`src/connect.ts`), at both levels, including per-document inboxes. > - *"`Nuri` and `ReadCap` are plain strings"* — **superseded the same week**: they are template literal types, so the confusion the runtime guard catches is now also a compile error. The *Typing* section below records the change; the earlier sentences were not rewritten. > - The `:k:` segment throughout — **a ReadCap is `r:`** (`BlockRef::readcap_nuri`, `engine/repo/src/types.rs:518-521`), reported by NextGraph's developer and verified. `:k:` belongs to objects, files and commits. -> - *"That branch lists the store's documents… It is the owner's keyring. Upstream, the keyring is the wallet"* — **wrong when written**, and it is the sentence that produced a global in-memory "keyring". There is no keyring object; the wallet holds one root key per user. See [`../readcap-and-nuri-model.md`](../readcap-and-nuri-model.md) §4quater. +> - *"That branch lists the store's documents… It is the owner's keyring. Upstream, the keyring is the wallet"* — **wrong when written**, and it is the sentence that produced a global in-memory "keyring". There is no keyring object; the wallet holds one root key per user. See [`../readcap-and-nuri-model.md`](../readcap-and-nuri-model.md) §4quater. The word *keyring* is left standing everywhere below because this report is kept as written; read it as *"what the holder holds"*, which is what the code now calls it. +> - `fileOwnCaps` — **renamed and split.** Writing a created document's cap is `holdOwnCap`, reading them back is `readStoreCaps`, and a user's own structure (three stores + inbox) is `fileOwnStructure`. Searching the code for `fileOwnCaps` finds nothing. ## What landed @@ -171,7 +172,7 @@ Fixed in step 2 of [`2026-07-30-virtual-wallet-boundary.md`](2026-07-30-virtual- And it is inboxes, plural, at **two levels**: - the **user/wallet** inbox — where ReadCaps arrive; -- the inbox of **every document the user owns** — a document is addressable and has its own inbox upstream, so deposits land per document too. +- the inbox of **every document the user owns**, so deposits land per document too. *(The justification originally given here — "a document has its own inbox upstream" — is **false**; see the correction in [`2026-07-30-virtual-wallet-boundary.md`](2026-07-30-virtual-wallet-boundary.md) step 7. The design decision stands on the consumer's need and on the record's per-`repo_id` shape, not on an upstream document inbox that does not exist.)* What exists today and is reusable: `inbox.readSynced` (the cold, barrier-gated read meant exactly for "process the inbox at (re)connection"), the inline cap absorption in `inbox.read`, and `CapRegistry.onChange` to re-trigger the reads a late cap unblocks. What is missing is the **orchestration**: a connection hook that enumerates the identity's inboxes (wallet-level + one per owned document, via the scope indexes) and processes them, idempotently and without polling. @@ -188,10 +189,11 @@ Not started. It changes the consumer contract in the right direction (one less o ## Verification status -- **Unit suite green — 138 tests**, typecheck clean on `src`, `test` and the e2e harness. +- **Unit suite green — 146 tests**, typecheck clean on `src`, `test` and the e2e harness. - The typing was verified from a **consumer's** point of view, not just the library's: a synthetic app compiled against the entry points shows the two real mistakes (`shareCap(bareNuri, …)` and passing a raw `string` from storage) as compile errors, while every correct path — `capFor(doc)` → `shareCap(cap, inbox)`, and narrowing with the exported guards — needs no cast. - The acceptance test was **mutation-checked**: reverting both gardes (the discovery fold and the `readUnion` possession gate) makes `watch-shape.test.ts` (e) fail with the bare-referenced document reappearing. The test has teeth. -- **The e2e was updated but NOT run** — it needs a real broker. `capsReadFilter` was rewritten around possession and a new `capsShareCap` step exercises the full share→inbox→absorb path against the real broker; both await a run. +- **The e2e ran against the live broker (`nextgraph.eu`) on 2026-08-03 — 39 passed, 0 failed.** The first run was 22/8, and the eight refusals were not test noise: they exposed a **real hole in the surface**. `docs.docCreate` filed no cap for the creator, so a consumer could create a document through the public primitive and then be refused reading or writing it. Upstream that cannot happen — `doc_create` commits `AddRepo { read_cap }` to the store's Store branch, so the creator holds it from the first instant. Fixed at `packages/client/src/docs.ts:73`, and deliberately NOT replicated in `physical.ts`: the shim's own documents belong to no user, and `store-registry` files their caps where it knows whose they are. The remaining failures were the harness acting as a second identity without establishing it (`createEntityDoc(id, …)` with someone else connected) or reading an arbitrary document as an inbox; both are now `setCurrentUser` + `walletInbox`, which is what a consumer must do too. +- **An e2e run against a persistent wallet must use a FRESH identity per run.** The second run was green and the third was not, on unchanged code: moving the inbox tests onto `walletInbox(id)` made the inbox *stable for its owner* — which is the point of an inbox — so a fixed id accumulates every past run's deposits and `deposits.length === 2` drifts to 4. Green-then-red on identical code is the tell. The disposable thing is the **user**, not the inbox: `run.ts` now stamps `@inbox-user-`/`@watcher-`/`@friend-` with `Date.now()`, as it already did for `@alice-`. Any future step that resolves a durable per-user document (inbox, stores, Links) inherits this constraint. - **The cap registry is process-wide and `bun test` shares modules across files**, so suites that read without declaring caps now reset explicitly (`read-model.test.ts`, `watch-shape.test.ts`). Worth knowing before adding a suite. ## Documentation state diff --git a/docs/briefs/2026-07-30-virtual-wallet-boundary.md b/docs/briefs/2026-07-30-virtual-wallet-boundary.md index 39dc98b..db3272c 100644 --- a/docs/briefs/2026-07-30-virtual-wallet-boundary.md +++ b/docs/briefs/2026-07-30-virtual-wallet-boundary.md @@ -115,7 +115,7 @@ Applies to every exported surface, including ones added later: **if it is expose 1. ~~**Remove `discovery.***~~ — **DONE 2026-07-30.** `src/discovery.ts` and `test/discovery.test.ts` deleted; `INDEX_ACCOUNT`, `watchShape`'s public-scope fold and its discovery-index container subscription, `nurisFromRef`, the `submitToIndex` guard, and the e2e discovery block all removed. P1a's acceptance test did not need re-basing: `test/cross-user-access.test.ts` already proves the same property (a bare reference reads nothing, the link reads the document) on the model's own terms — following a link — so `watch-shape.test.ts` (e), which proved it on the discovery fold, was dropped. Docs realigned: the ADR is marked superseded, `read-model.md` now describes ONE regime (follow, never enumerate), and the root README's capability row records the removal. 2. ~~**"My inbox" + the inbox read guard**~~ — **DONE 2026-07-30.** `storeRegistry.walletInbox(id)` gives every virtual user its own inbox document, created on first sight and recorded in the doc-shim under its own predicate (`shim:docInbox`), read by its OWN query so an account record written before this existed still resolves — the fixed account SELECT did not grow a fourth required field. `isOwnInbox(nuri)` is the predicate; `inbox.read` / `readSynced` (hence `watch`, which reads through it) refuse an inbox that is not the connected wallet's, and refuse outright when no identity is set. **Depositing stays open** — `post` / `shareCap` are untouched, because that is the one legitimate cross-wallet act. The shared `resolveInboxAnchor` (a reserved account's document, an inbox COMMON to every wallet) was removed: it was unused by the library and violated *nothing common*. Locked by `test/isolation-active.test.ts` *an inbox may be DEPOSITED into by anyone, and READ only by its owner*, which walks the exact breach — Alice deposits, cannot read back; Mallory knowing the NURI absorbs nothing; anonymous is refused; Bob reads his own and only then does the cap land. - *Not done, and deliberately*: per-DOCUMENT inboxes. Upstream every document has one; here only the wallet does. **The PO has ruled they must come** (2026-07-30) — *"it can come in a second step, but it must come"* — so this is a commitment, not an option. The guard predicate (`isOwnInbox`) is where they plug in: it answers "is this inbox mine?", which extends to "…one of my documents' inboxes" without changing a single caller. + *Not done, and deliberately*: per-DOCUMENT inboxes. ~~Upstream every document has one~~ — **false, corrected 2026-08-03**: no document has an inbox upstream, and neither does the private store (see step 7's correction). Here only the wallet does. **The PO has ruled they must come** (2026-07-30) — *"it can come in a second step, but it must come"* — so this is a commitment, not an option. The guard predicate (`isOwnInbox`) is where they plug in: it answers "is this inbox mine?", which extends to "…one of my documents' inboxes" without changing a single caller. ### Two defects this step surfaced — the first still open, the second closed by steps 5–6 @@ -190,6 +190,8 @@ So the store-root pointer, the doc-shim and the account records go through the m The asymmetry holds at both levels, and a test walks it: **anyone deposits** into a document's inbox (that is how a third party reaches its owner at all), **only the owner reads** it. + **Correction 2026-08-03 — this step ANTICIPATES, it does not mirror.** The paragraph above overstated the upstream fact, and an adversarial doc review caught it. Verified: `new_store_default` attaches an inbox only `if !private` (`engine/verifier/src/verifier.rs:2994`), and `doc_create` goes through `new_repo_default`, which leaves `inbox: None` (`engine/repo/src/repo.rs:574`). The **only** two `AddInboxCap` commits in the engine are for the public and protected STORE repos (`engine/verifier/src/site.rs:128,149`). So today **no document has an inbox, and neither does the private store**. What is genuinely verified is the *shape*: `AddInboxCapV0` is keyed by `repo_id` (`engine/repo/src/types.rs:1973`), so the record accommodates an inbox on any repo. The step stands as delivered — a per-document inbox is what the consumer needs and what the record's shape supports — but it must be read as a bet on where upstream goes, not as an emulation of what it already does. + ## Relation to P1b P1b (per-document encryption, closing the read paths that bypass the guard) largely **becomes** this lot, better framed. Encrypting each document while leaving the wallet boundary open would be securing the windows with the door open — and conversely, once every access is confined to the connected wallet, "the emulated key is derivable" stops being the load-bearing weakness. This lot should therefore precede P1b, or absorb it. diff --git a/docs/decisions/discovery-model.md b/docs/decisions/discovery-model.md index d0bc977..7fc12e5 100644 --- a/docs/decisions/discovery-model.md +++ b/docs/decisions/discovery-model.md @@ -13,6 +13,8 @@ > > **`discovery.ts` and its tests were removed on 2026-07-30**, along with `watchShape`'s public-scope fold and `INDEX_ACCOUNT`. See [`../briefs/2026-07-30-virtual-wallet-boundary.md`](../briefs/2026-07-30-virtual-wallet-boundary.md). > +> One factual error below is worth naming so it is not carried into a future design: *"a native inbox (a primitive present on every document)"* is **false**. No document has an inbox upstream — only the public and protected STORE repos do (`engine/verifier/src/site.rs:128,149`; `doc_create` leaves `inbox: None`, `engine/repo/src/repo.rs:574`). See [`../nextgraph-current-state.md`](../nextgraph-current-state.md) § Inbox. +> > What survives, and is worth keeping from the text below: the **3-stage frame** (`discovery → synchronization → query`) is still exactly right, with stage 1 re-read as *"a link reached you"* rather than *"you consulted an index"*. You still cannot query what you have not synchronized, and you still do not synchronize what nobody gave you. The **inbox** is what feeds stage 1 — which makes it the bootstrap of the whole reachability graph, not a side feature. > > Kept in full below as a record of what was built and why, and of the reasoning that has to be re-read through the correction above. diff --git a/docs/decisions/private-store-nuri-scope.md b/docs/decisions/private-store-nuri-scope.md index 1152e7e..0e9905a 100644 --- a/docs/decisions/private-store-nuri-scope.md +++ b/docs/decisions/private-store-nuri-scope.md @@ -9,7 +9,8 @@ the shim opens repos. Original context: the consuming app. > **protected** store (`did:ng:${protected_store_id}`), verified to open without > `RepoNotFound`. **The central insight of this ADR still holds** and now applies > to **both** stores: you must open the repo via the store's NURI -> (`orm_start_graph`) or you get `RepoNotFound`. +> or you get `RepoNotFound`. *(How it is opened has since changed — see the note +> under Decision.)* ## Context @@ -39,10 +40,17 @@ entity to reuse) → falls back to `doc_create` and the same `RepoNotFound`. ## Decision **Option B**: use the store NURI as both the `useShape` scope AND the write -`@graph`, exactly like `expense-tracker-rdf`. This is why this lib's shim opens a -store repo via `orm_start_graph` before writing, and why **`did:ng:i` must never -be used as a scope** (it breaks writes with `RepoNotFound`). See the -`orm_start_graph` scope rule in [`../simulation.md`](../simulation.md). +`@graph`, exactly like `expense-tracker-rdf`. This is why this lib's shim opens the +store repo before writing, and why **`did:ng:i` must never be used as a scope** (it +breaks writes with `RepoNotFound`). See the scope rule in +[`../simulation.md`](../simulation.md). + +*The decision stands; the mechanism named in it has been replaced.* Opening was +`orm_start_graph` when this was written. It is now `ensureRepoOpen` — `doc_subscribe` +plus a wait for the first `State` (`packages/client/src/open-repo.ts:167`) — after +`orm_start_graph` was found to hang on a fan-out (`subscribe.ts:28,181`). What must be +read here is the invariant *"open the repo, by its store NURI, before writing"*, not the +call that used to implement it. ## Consequences diff --git a/docs/incidents/2026-07-14-write-loss-on-disconnect.md b/docs/incidents/2026-07-14-write-loss-on-disconnect.md index 0c65105..9e0d4e9 100644 --- a/docs/incidents/2026-07-14-write-loss-on-disconnect.md +++ b/docs/incidents/2026-07-14-write-loss-on-disconnect.md @@ -25,11 +25,11 @@ An entity written just before a period of inactivity can be **silently lost**: i Interpretation (**plausible mechanism, not settled**): the write was pushed into the local **outbox**, but the socket died before it was **durably flushed** into the broker topic; on reconnection, the outbox replay fails (`Err(TopicNotFound)`) because the topic was **never created on the broker side** → the event is abandoned. The account, for its part, had already been durably resolved (`resolveAccount → 1 record`): it is neither lost nor forked. -> **Epistemic caveat.** The evidence establishes the *symptom* (loss + `Err(TopicNotFound)` + `readScopeIndex → 0`). The exact *mechanism* is not settled between **(i) loss at write time** (the write never durably reaches the broker) and **(ii) cold-rehydration failure** (the write *is* on the broker but a fresh session does not reopen its own scope). The `Err(TopicNotFound)` on the outbox replay leans toward **(i) in this Firefox case**. See the @data repro below, which exhibits a neighboring symptom but **does not settle** (i) vs (ii). +> **Epistemic caveat.** The evidence establishes the *symptom* (loss + `Err(TopicNotFound)` + the scope read returning 0 — logged above as `readScopeIndex`, since renamed `readUserStore`). The exact *mechanism* is not settled between **(i) loss at write time** (the write never durably reaches the broker) and **(ii) cold-rehydration failure** (the write *is* on the broker but a fresh session does not reopen its own scope). The `Err(TopicNotFound)` on the outbox replay leans toward **(i) in this Firefox case**. See the @data repro below, which exhibits a neighboring symptom but **does not settle** (i) vs (ii). ## Causal chain (TRACED — reading of the NextGraph core, to be re-verified) -- The `SerializationError` closes the socket. The core emits the disconnection: `broker.rs` → `LocalBrokerMessage::Disconnected` → `disconnections_sender.send(...)` (≈ `broker.rs:1051`, to be re-verified — volatile number, navigate by symbol). +- The `SerializationError` closes the socket. The core emits the disconnection in **two** places, and they are not the same file: `engine/net/src/broker.rs:1074` sends `LocalBrokerMessage::Disconnected`, which the SDK turns into `disconnections_sender.send(...)` at `sdk/rust/src/local_broker.rs:648`. Navigate by symbol — the line numbers are volatile, and the earlier note in this file put the `send` in `broker.rs`, which it never was. - This disconnection is **pushed** to subscribers via `disconnections_subscribe(cb)` (PUSH stream). - **NextGraph reconnection is an unimplemented `// TODO`** (≈ `broker.rs:1051-1076`): nothing re-establishes the socket nor re-flushes the outbox. - `user_connect` returns a **snapshot** `{ server_id, server_ip, error, since }` at call time — not a stream, unusable for detecting a later drop. diff --git a/docs/nextgraph-current-state.md b/docs/nextgraph-current-state.md index 5ab71cc..86a3e1b 100644 --- a/docs/nextgraph-current-state.md +++ b/docs/nextgraph-current-state.md @@ -22,14 +22,24 @@ Where the ground truth lives, so future re-verification is cheap: dispatch (the truth on what is actually *processed*). - `engine/net/src/types.rs` — inbox types (`InboxPost`, `InboxMsg`, `InboxMsgContent`). - `engine/verifier/src/inbox_processor.rs` — inbox message handling. -- `engine/verifier/src/verifier.rs:1423` — the `OpenRepo` TODO (cross-wallet read). +- `engine/verifier/src/verifier.rs:2237` — `load_repo_from_read_cap`, the one path that + brings a repo in FROM a cap (`pub(crate)`, see § *Capability / ReadCap granularity*). +- `engine/verifier/src/verifier.rs:1423` — the `OpenRepo` TODO. It is **not** about + loading an unheld repo: it sits inside `open_branch_`, past + `self.repos.get_mut(repo_id).ok_or(RepoNotFound)?` (`:1331`), so the repo is already + held by the time that line runs. What is missing is the broker-side `OpenRepo` + request, worked around with a pin. - `engine/repo/src/types.rs` — `RootBranchV0.store: StoreOverlay` (repo → its store). ## The 5 store types -Every wallet has the **3 default stores** out of the box (session fields -`private_store_id`, `protected_store_id`, `public_store_id`). Group and Dialog -are created on demand. +The **3 default stores** belong to a **user**, not to the wallet. A wallet holds +`sites: HashMap` (`engine/wallet/src/types.rs:456`), and it is `SiteV0` +that carries `public` / `protected` / `private` (`engine/verifier/src/site.rs:31-37`) — +one wallet can hold several, which is exactly why "wallet" is the wrong unit to reason +in (see `docs/readcap-and-nuri-model.md` §4quinquies, *Nomenclature first*). A session exposes the three as +`private_store_id`, `protected_store_id`, `public_store_id` — those are the connected +USER's. Group and Dialog are created on demand. | Store | Read | Write | Creation | |---|---|---|---| @@ -104,11 +114,27 @@ offline"*; *"removing permissions … requires a SyncSignature"* (synchronous). ## Inbox -Every document has a native inbox. A non-editor can deposit a link (DID -cap) into it without being invited as an editor; the owner moderates. NURI: -`did:ng:d:`. Content: the `InboxMsgContent` enum (`ContactDetails`, +**Only two repos have an inbox today: a user's public and protected STORES.** Not +documents, and not the private store. `new_store_default` attaches one solely `if +!private` (`engine/verifier/src/verifier.rs:2994`), and `doc_create` goes through +`new_repo_default`, which leaves `inbox: None` (`engine/repo/src/repo.rs:574`). The only +`AddInboxCap` commits in the whole engine are the two in `engine/verifier/src/site.rs:128,149` +— one for the public store repo, one for the protected one. + +The *register* is nonetheless per-repo: `AddInboxCapV0 { repo_id, overlay, priv_key }` +(`engine/repo/src/types.rs:1973`) records **which repo** an inbox is opened for, so the +shape accommodates an inbox on any repo. Nothing creates one, which is a different +statement from the shape forbidding it — and it is why this lib's per-document inbox is +an ANTICIPATION of that shape, not an emulation of something upstream already does. + +A non-editor can deposit into an inbox without being invited as an editor; the owner +moderates. NURI: `did:ng:d:`. Content: the `InboxMsgContent` enum (`ContactDetails`, `DialogRequest`, `Link`, `Patch`, `ServiceRequest`, `ExtRequest`, -`RemoteQuery`, `SocialQuery`…). Messages are sealed (`crypto_box::seal`) to +`RemoteQuery`, `SocialQuery`…, `engine/net/src/types.rs:4249-4261`). Note what `Link` +is: a **unit variant, carrying nothing** — not a link, not a cap, just a discriminant. +Reading it as "the inbox can deliver a read capability" is the trap this file exists to +prevent; see the *Consequence for this lib* below, which says the same thing from the +other end. Messages are sealed (`crypto_box::seal`) to the inbox pubkey, so only the owner decrypts. The `from` field is optional, so an anonymous sender is possible. This is the "identified if known, anonymous otherwise" behaviour native to the protocol. @@ -241,7 +267,8 @@ with the whole wallet, which is why the read path is per-doc anchored: the ancho read makes a non-empty wallet irrelevant. At the real multi-store migration this is unchanged (the anchored read is native); only bringing a repo into the session changes: opening a real per-user store repo by cap becomes a native -broker sync (the `OpenRepo` TODO at `verifier.rs:1423`). Opening still requires the +broker sync, through `load_repo_from_read_cap` (`verifier.rs:2237`) — not through the +`OpenRepo` TODO at `:1423`, which concerns a repo already held. Opening still requires the repo's NURI + ReadCap — there is no store-level read inheritance (see § Capability / ReadCap granularity). @@ -355,10 +382,16 @@ Listing must go through a one-shot union `sparql_query` instead — see installed version) **does NOT expose**: Group/Dialog store creation; capability sharing (a NURI with rights); permission manipulation; inbox deposit/read. -Available JS methods: `doc_create`, `doc_subscribe`, `sparql_query`, +The JS methods this lib USES: `doc_create`, `doc_subscribe`, `sparql_query`, `sparql_update`, `orm_start_graph`, `orm_start_discrete`, `graph_orm_update`, -`discrete_orm_update`, `file_get`, `app_request_stream`. The docs announce *"An -API will be provided for permission manipulation"* (no date). +`discrete_orm_update`, `file_get`, `app_request_stream`. That is a working subset, +**not** the surface: `NGModule` exports **77** (`@ng-org/web@0.1.2-alpha.13`, +`dist/index.d.ts:140-268`), including `app_request`, `session_stop`, +`disconnections_subscribe`, `social_query_start`, `upload_start`/`upload_chunk`/ +`upload_done`, and the whole `wallet_*` family. Read "not in the list above" as "we do +not call it", never as "it does not exist" — several sections of this very file discuss +methods absent from that subset. The docs announce *"An API will be provided for +permission manipulation"* (no date). ## Integration & deployment model @@ -439,6 +472,15 @@ methods: `doc_subscribe`, `orm_start_graph`, `orm_start_discrete`, `file_get`, NextGraph's app/service execution model — important because it **invalidates** the idea of "a service with its own wallet sharing global data". +> **Provenance: NOT verified against `nextgraph-rs`.** Every other section of this file +> cites the engine; this one cites nothing, and nothing in the local clone corresponds to +> it — no app/service runtime, no settings-document type, no singleton notion. It arrived +> with `bea9f51`, moved wholesale from the consumer app's repo, and its own source (an +> exchange with the PO, the official docs, or an inference) was not recorded. The +> conclusion below carries real weight — it is what deferred a global index in this lib — +> so treat it as **a claim to re-confirm with the PO**, not as an engine fact. Do not +> extend it, and do not cite it as verified. + - **Apps AND services are mono-user.** They see only **what the user makes available** to them. There is **no global data** natively, and no central service holding shared data. @@ -734,6 +776,8 @@ semantics of the replay path itself (VERIFIED by reading `send_outbox`). **Consequence for this lib:** a queued write can be dropped without any observable error, and one unknown topic can take the rest of the queue with it. The polyfill's own -`outbox-log.ts` records write intents but cannot replay them into the core, and no +`outbox-log.ts` does not record anything: it exports a single `inspectOutbox()` that +READS the SDK's own `sessionStorage` outbox and logs how many peers still have queued +writes. It observes the symptom; it holds nothing it could replay, and no write-durability confirmation exists to await — so "the write returned" is not "the write is durable". diff --git a/packages/client/docs/sdk-reference.md b/packages/client/docs/sdk-reference.md index 706e999..4958807 100644 --- a/packages/client/docs/sdk-reference.md +++ b/packages/client/docs/sdk-reference.md @@ -234,9 +234,10 @@ Consequences a consumer must internalize: permission enum (`engine/repo/src/types.rs:1729`, `PermissionV0`) has `WriteAsync`/ `WriteSync` but **no** add-only/append permission and **no** public-writable grant. To surface data to others without a shared write, use the **inbox** (any identity — - even anonymous — can deposit into a document's native inbox; the owner materializes - deposits) or make the document **public-readable** and let each identity own its own - document. + even anonymous — can deposit; only the owner reads back) or make the document + **public-readable** and let each identity own its own document. *Per-document inboxes + are this library's, not the engine's: upstream only the public and protected store + repos carry one (`engine/verifier/src/site.rs:128,149`).* The consumer asks the SDK for what it needs and trusts the result; it does not construct NURIs, pick union-vs-anchor, or reason about caps. The domain-shaped list diff --git a/packages/client/e2e/run.ts b/packages/client/e2e/run.ts index f3084d5..f645ac7 100644 --- a/packages/client/e2e/run.ts +++ b/packages/client/e2e/run.ts @@ -224,7 +224,9 @@ async function main(): Promise { // ── inbox ─────────────────────────────────────────────────────────────── console.log("\n── inbox ──"); await step("inbox post → read round-trip", async () => { - const r = await sdk(frame, "inboxPostRead", { k: "a" }, { k: "b" }); + // Fresh user per run: an inbox is stable for its owner, so a reused id would + // read back the previous runs' deposits too (the wallet persists). + const r = await sdk(frame, "inboxPostRead", "@inbox-user-" + Date.now(), { k: "a" }, { k: "b" }); const payloads = (r.deposits || []).map((d: any) => JSON.stringify(d.payload)); check( "post then read returns both deposits (sorted)", @@ -233,7 +235,7 @@ async function main(): Promise { ); }); await step("inbox watch fires on deposit", async () => { - await sdk(frame, "inboxWatchStart"); + await sdk(frame, "inboxWatchStart", "@watcher-" + Date.now()); await frame.waitForFunction(() => (window as any).__sdk.inboxWatchState().fires >= 1, { timeout: 20000 }); const base = await sdkGet(frame, "inboxWatchState"); await sdk(frame, "inboxWatchDeposit", { landed: true }); @@ -329,7 +331,7 @@ async function main(): Promise { ); }); await step("shareCap: a cap delivered to an inbox reveals the doc", async () => { - const r = await sdk(frame, "capsShareCap"); + const r = await sdk(frame, "capsShareCap", "@friend-" + Date.now()); check( "shareCap → inbox processed → the shared doc becomes readable, and the delivery is not surfaced", r.before === 0 && r.after === 1 && r.surfacedDeposits === 0, diff --git a/packages/client/e2e/sdk-entry.ts b/packages/client/e2e/sdk-entry.ts index 213a383..0e3dde0 100644 --- a/packages/client/e2e/sdk-entry.ts +++ b/packages/client/e2e/sdk-entry.ts @@ -23,6 +23,7 @@ import { getCaps, resetCaps, shareCap, + connectedUser, } from "@ng-eventually/client/polyfill"; import { docs, @@ -368,10 +369,18 @@ const identity = new IdentityStore( }, // ── inbox ──────────────────────────────────────────────────────────────── - async inboxPostRead(payloadA: unknown, payloadB: unknown) { - const s = await sessionReady; - const target = await docs.docCreate(s.session_id, "Graph", "data:graph", "store", undefined); - setCurrentUser("inbox-user"); + /** + * `id` must be FRESH per run (run.ts stamps it). A user's inbox is stable over time — + * that is the point of it — so re-running against a reused id accumulates the previous + * runs' deposits on a persistent wallet, and the exact-count assertion drifts. The + * thing to make disposable is the user, not the inbox. + */ + async inboxPostRead(id: string, payloadA: unknown, payloadB: unknown) { + // The target must be that user's OWN inbox, not an arbitrary document: you may + // deposit into anyone's, you may only read your own. Establishing the identity + // FIRST is what makes `walletInbox` resolve (and file) that user's inbox. + setCurrentUser(id); + const target = await storeRegistry.walletInbox(id); await inbox.post(target, { payload: payloadA, from: null, ts: 1000 }); await inbox.post(target, { payload: payloadB, from: null, ts: 2000 }); const deposits = await inbox.read(target); @@ -380,9 +389,12 @@ const identity = new IdentityStore( }, // watch (doc_subscribe-based) fires when a deposit lands. _inboxWatch: { fires: 0, lastLen: -1, unsub: () => {}, target: "" }, - async inboxWatchStart() { - const s = await sessionReady; - const target = await docs.docCreate(s.session_id, "Graph", "data:graph", "store", undefined); + /** `id` fresh per run, for the same reason as {@link inboxPostRead}. */ + async inboxWatchStart(id: string) { + // Watching an inbox is READING it continuously, so the watcher stays connected + // for the whole probe — including across `inboxWatchDeposit`. + setCurrentUser(id); + const target = await storeRegistry.walletInbox(id); const rec = { fires: 0, lastLen: -1, unsub: () => {}, target }; (window as any).__sdk._inboxWatch = rec; rec.unsub = inbox.watch(target, (deposits) => { @@ -393,9 +405,7 @@ const identity = new IdentityStore( }, async inboxWatchDeposit(payload: unknown) { const rec = (window as any).__sdk._inboxWatch; - setCurrentUser("watcher"); await inbox.post(rec.target, { payload, from: null }); - setCurrentUser(null); }, inboxWatchState() { const r = (window as any).__sdk._inboxWatch; @@ -403,6 +413,7 @@ const identity = new IdentityStore( }, inboxWatchStop() { (window as any).__sdk._inboxWatch.unsub(); + setCurrentUser(null); }, // spoof guard: depositing as another principal throws. async inboxSpoofGuard() { @@ -440,10 +451,16 @@ const identity = new IdentityStore( }, async entityDocsBounded(idA: string, idB: string) { storeRegistry.resetRegistryCache(); + // Each user creates its OWN documents: you act as one virtual user at a time, + // and the caps of what you create are filed under the identity you were acting + // as. Creating B's document while connected as A is not a thing the model has. + setCurrentUser(idA); const dA1 = await storeRegistry.createEntityDoc(idA, "public"); const dA2 = await storeRegistry.createEntityDoc(idA, "public"); + setCurrentUser(idB); const dB1 = await storeRegistry.createEntityDoc(idB, "public"); // listMyEntityDocs(A) → only A's docs (poll: the index append can lag). + setCurrentUser(idA); let listA: string[] = []; for (let i = 0; i < 12; i++) { storeRegistry.resetRegistryCache(); @@ -451,6 +468,7 @@ const identity = new IdentityStore( if (listA.includes(dA1) && listA.includes(dA2)) break; await new Promise((r) => setTimeout(r, 1000)); } + setCurrentUser(null); return { dA1, dA2, dB1, listA, @@ -474,6 +492,10 @@ const identity = new IdentityStore( async reconnectSeed(id: string, scope: "public" | "protected" | "private") { storeRegistry.resetRegistryCache(); const s = await sessionReady; + // Seed AS the user whose document this is — otherwise the cap of the created + // document is filed under nobody and the very session that created it is + // refused the write below. + setCurrentUser(id); const entityNuri = await storeRegistry.createEntityDoc(id, scope); const marker = "recon-" + Date.now(); await docs.sparqlUpdate( @@ -505,11 +527,24 @@ const identity = new IdentityStore( * fail-without-the-fix proof (see run.ts's reconnection step comment). */ async reconnectRead(id: string, scope: "public" | "protected" | "private", entityNuri: string, marker: string) { - // DIAGNOSTIC: a RAW anchored read of the entity doc with NO open at all, first - // thing in the fresh session — reports how many rows the bare anchored query - // resolves for a not-yet-opened repo (the premise: 0 until opened). Uses the - // low-level docs primitive directly, bypassing readUnion's open step. const s = session ?? (await sessionReady); + // A fresh session holds nothing in memory: connect AS the user so the caps are + // restored from the durable registers (own documents from the Store branches, + // received ones from the Links) before anything is read back. + setCurrentUser(id); + await connectedUser(); + + storeRegistry.resetRegistryCache(); + const listed = await storeRegistry.listMyEntityDocs(id, scope); + // DIAGNOSTIC: a RAW anchored read of the entity doc with NO open — reports how + // many rows the bare anchored query resolves for a not-yet-opened repo (the + // premise: 0 until opened). Uses the low-level docs primitive directly, bypassing + // readUnion's open step. + // + // Placed AFTER `listMyEntityDocs`, which is what restores the caps of the user's + // own documents from the Store branch. Before it, the boundary refuses the read + // and the probe would measure the guard rather than the open — a number that + // looks like the premise holding while proving nothing about it. let rawRowCount = -1; try { const raw: any = await docs.sparqlQuery(s.session_id, "SELECT ?s ?p ?o WHERE { ?s ?p ?o }", undefined, asNuri(entityNuri)); @@ -517,9 +552,6 @@ const identity = new IdentityStore( } catch (e: any) { rawRowCount = -2; // threw (e.g. RepoNotFound / InvalidNuri) } - - storeRegistry.resetRegistryCache(); - const listed = await storeRegistry.listMyEntityDocs(id, scope); const subjects = await readModel.readUnion(listed.length ? listed : [asNuri(entityNuri)]); const markers: string[] = []; for (const subj of subjects) { @@ -770,24 +802,30 @@ const identity = new IdentityStore( * "receive" operation exists, and no principal is ever named to the registry. * Runs against the REAL broker inbox document, so it exercises the whole path. */ - async capsShareCap() { + async capsShareCap(friendId: string) { const s = await sessionReady; resetCaps(); - const doc = await docs.docCreate(s.session_id, "Graph", "data:graph", "store", undefined); - const friendInbox = await docs.docCreate(s.session_id, "Graph", "data:graph", "store", undefined); - injectedSetItems = [{ "@graph": doc, "@id": "1", v: "shared-item" }]; + // The recipient's OWN inbox — the address a cap is delivered to. Resolved while + // connected as them, since that is who owns it and who may later read it. + // `friendId` is fresh per run: this test's assertions survive accumulated caps, but + // the recipient's durable Links would grow run after run on a persistent wallet, + // making every later `connectedUser()` re-apply a longer and longer history. + setCurrentUser(friendId); + const friendInbox = await storeRegistry.walletInbox(friendId); setCurrentUser("owner-O"); + const doc = await docs.docCreate(s.session_id, "Graph", "data:graph", "store", undefined); + injectedSetItems = [{ "@graph": doc, "@id": "1", v: "shared-item" }]; getCaps().open(doc, "protected"); const cap = capFor(doc)!; - setCurrentUser("friend"); + setCurrentUser(friendId); const before = [...(libUseShape(null, null) as Iterable)].length; setCurrentUser("owner-O"); await shareCap(cap, friendInbox); - setCurrentUser("friend"); + setCurrentUser(friendId); const absorbed = await inbox.read(friendInbox); // processing it applies the cap const after = [...(libUseShape(null, null) as Iterable)].length; diff --git a/packages/client/src/caps.ts b/packages/client/src/caps.ts index 5a0296a..1db92e7 100644 --- a/packages/client/src/caps.ts +++ b/packages/client/src/caps.ts @@ -26,8 +26,10 @@ * * This class is the in-memory record of what the connected holder currently holds: * upstream's local user storage, not a durable register. The durable ones are - * emulated in `store-registry.ts` (`fileOwnCaps` for created documents, `addLink` / - * `readLinks` for received ones), and `connect.ts` restores from them. + * emulated in `store-registry.ts` — for created documents, `holdOwnCap` writes and + * `readStoreCaps` reads the Store branch back; for received ones, `addLink` / + * `readLinks` on the User branch. `connect.ts` restores the Links at connection; + * the own-document caps come back through `listMyEntityDocs`. * * One record PER holder, since one shared wallet hosts every identity. Switching * identity therefore SWITCHES records; it never wipes one (a wipe would make diff --git a/packages/client/src/docs.ts b/packages/client/src/docs.ts index bac45eb..d3c9d57 100644 --- a/packages/client/src/docs.ts +++ b/packages/client/src/docs.ts @@ -13,7 +13,7 @@ * app's storeRegistry usage), so this is a drop-in for those raw calls. */ -import { getConfig } from "./polyfill"; +import { getCaps, getConfig } from "./polyfill"; import { logAccess, enabled as accessLogEnabled } from "./access-log"; import { isNuri } from "./nuri"; import { assertMayReach } from "./reach"; @@ -61,6 +61,16 @@ export async function docCreate( `[ng-eventually] docCreate: the broker returned something that is not a NextGraph reference: ${JSON.stringify(nuri)}`, ); } + // **Creating a document gives you its cap.** Upstream that is not a courtesy but + // the mechanism: `doc_create` commits `AddRepo { read_cap }` to the store's Store + // branch, so the creator holds it from the first instant. Without this, a caller + // could create a document through this primitive and then be refused reading or + // writing it — which is what the e2e run against the live broker exposed. + // + // `physical.ts`'s counterpart deliberately does NOT do this: the shim's own + // documents belong to no virtual user, and `store-registry` files their caps + // itself, where it knows whose they are. + getCaps().mint(nuri); // A container creation is a WRITE; the NURI only exists after the call. logAccess("WRITE", nuri, "docCreate"); return nuri; diff --git a/packages/client/src/inbox.ts b/packages/client/src/inbox.ts index e51ab1c..a8fe16c 100644 --- a/packages/client/src/inbox.ts +++ b/packages/client/src/inbox.ts @@ -229,10 +229,19 @@ function capOfPayload(payload: unknown): ReadCap | null { * Reaching several recipients means calling this once per inbox, which is what the * real model does too: each delivery is sealed to one recipient. * - * Upstream this path is a GAP, not a disagreement: the field exists - * (`ContactDetails.read_cap`) but its message construction is `unimplemented!()` - * and the receiver discards the cap. The shape is right; the implementation is - * absent, so we emulate it meanwhile. + * Upstream this path is a GAP, not a disagreement — verified at both ends: + * - the field exists, `ContactDetails.read_cap: Option` + * (`engine/net/src/types.rs:4233`), but building a message that carries one is + * `read_cap: if with_readcap { unimplemented!() }` (`types.rs:3786`); + * - and the receiver ignores it: `InboxMsgContent::ContactDetails` writes only + * `ng:site`/`ng:protected` + `ng:*_inbox` into a fresh contact document + * (`engine/verifier/src/inbox_processor.rs:778-830`), never `details.read_cap`. + * + * Do NOT read `InboxMsgContent::Link` as the intended channel either: it is a **unit + * variant carrying nothing** (`engine/net/src/types.rs:4251`). + * + * The shape is right; the implementation is absent at both ends, so we emulate it + * meanwhile. */ export async function shareCap(cap: ReadCap, toInbox: Nuri): Promise { if (!hasReadCap(cap)) { diff --git a/packages/client/src/store-registry.ts b/packages/client/src/store-registry.ts index 316bf10..66b6dd6 100644 --- a/packages/client/src/store-registry.ts +++ b/packages/client/src/store-registry.ts @@ -4,7 +4,9 @@ * Stopgap / polyfill-era. Emulates the target infrastructure — where each * user owns their own public/protected/private stores — on top of one shared * wallet. It creates one document per (account × scope) inside that shared - * wallet (via the `docs.docCreate` primitive), so the `scope` + * wallet (via `physical.physicalCreate` — the UNGUARDED primitive, since a store is + * machinery and its cap is filed here, where it is known whose it is; the public + * `docs.docCreate` files the creator's cap itself), so the `scope` * (`public|protected|private`) is a logical attribute tracked here, not a * physical NextGraph store. Isolation is enforced by the app layer + the * emulated cap registry, not by crypto. @@ -1015,12 +1017,21 @@ export async function userStoreDoc(id: string, scope: Scope): Promise { * The inbox of a document this user owns — resolved, and created on first ask. * * Upstream a repo carries `inbox: Option` (`engine/repo/src/repo.rs:126`): - * an inbox is a keypair on the document, whose PRIVATE half its owner holds. That - * half is recorded with `AddInboxCap { repo_id, overlay, priv_key }` — *"into the - * user branch, so that a user can share with all its device"* - * (`engine/repo/src/types.rs:1969-1981`), the same branch that carries `AddLink`. - * So "which inboxes may I read" is answered by the User branch, and that is what - * this emulates. + * an inbox is a keypair on the repo, whose PRIVATE half its owner holds. That half is + * recorded with `AddInboxCap { repo_id, overlay, priv_key }` — *"into the user branch, + * so that a user can share with all its device"* (`engine/repo/src/types.rs:1973`), the + * same branch that carries `AddLink`. So "which inboxes may I read" is answered by the + * User branch, and that is what this emulates. + * + * **This ANTICIPATES: no document has an inbox upstream today.** `new_store_default` + * attaches one only `if !private` (`engine/verifier/src/verifier.rs:2994`), and + * `doc_create` goes through `new_repo_default`, which leaves `inbox: None` + * (`engine/repo/src/repo.rs:574`) — the only two `AddInboxCap` commits in the engine + * are for the public and protected STORE repos (`engine/verifier/src/site.rs:128,149`). + * What is verified is the SHAPE: the record is keyed by `repo_id`, so it accommodates + * an inbox on any repo. What is not verified is that anything upstream will create one + * per document. At migration this either becomes native or stays emulated here; either + * way the consumer-facing act is unchanged. * * Lazy on purpose: creating an inbox document for every entity up front would * double every `createEntityDoc` for inboxes most documents never receive anything