Compare commits
13 Commits
1f0bae461e
...
main
| Author | SHA1 | Date | |
|---|---|---|---|
| 6f0d0586e2 | |||
| 0d52c82ba9 | |||
| 518292498a | |||
| b2cb774124 | |||
| 8764daff4f | |||
| 60a9fd3ede | |||
| d7e0ee6a4b | |||
| ead5aececf | |||
| f2c5b30527 | |||
| 1791c31f42 | |||
| 127ca3159e | |||
| 138d37c02f | |||
| cf9500f0cf |
@@ -0,0 +1,145 @@
|
|||||||
|
# Brief — align the caps emulation with the real NextGraph model
|
||||||
|
|
||||||
|
**Brief (incubation) — 2026-07-20.** See the reference `docs/readcap-and-nuri-model.md`.
|
||||||
|
|
||||||
|
## Problem
|
||||||
|
|
||||||
|
`caps.ts` emulates read rights as an **ACL** (`Map<Nuri, Set<PrincipalId>>`, `grantRead(doc, grantee)`) — **the inversion** of the real NextGraph model (key possession). Consequences: no notion of a **cap-less reference**, grant/revocation **instantaneous and total** (instead of durable sealing + re-key), and an API (`declareConnections`) that consumers have to **re-declare every session**. This divergence makes it impossible to properly build models that rest on the real semantics — in particular **anonymous presence** (naming/counting without reading).
|
||||||
|
|
||||||
|
## Objective: shape-fidelity, NOT security
|
||||||
|
|
||||||
|
The polyfill does **NOT match** the security of finished NextGraph, and does not try to. The shared wallet plus the absence of crypto make the emulation **deliberately insecure** (everything is in plaintext, any marker is forgeable) — a dev/staging vehicle, not a goal. **Sole objective**: expose the **RIGHT SHAPE** of the future primitives so that consumers (Festipod) are coded against the **correct mental model** and **do not have to be rewritten** when NextGraph is finished.
|
||||||
|
|
||||||
|
Corollary: **"no crypto" is not a problem**; what matters is being **in the same logic, with RIGOR**. A criticism of the form "an attacker reads the plaintext / forges a marker" is **correct but out of scope**. What is **unacceptable** = exposing the **wrong shape** (e.g. an ACL where the real thing is key possession) → the consumer codes against a model that will not exist. **The ACL inversion of ReadCaps IS that lack of rigor** — the central defect to fix.
|
||||||
|
|
||||||
|
## Enforcement mechanism: LIGHTWEIGHT crypto simulation (anti-ACL, anti-shortcut)
|
||||||
|
|
||||||
|
For the shape to be **really** key-possession (and not an ACL in disguise), a doc's data is **stored encrypted** (per-doc symmetric encryption, however lightweight) and the **ReadCap = the key**. Invariant (cf. `docs/vision.md`):
|
||||||
|
|
||||||
|
> a **bare `did` (without a ReadCap)** does **NOT** allow reading; a **NURI with a ReadCap** is **sufficient and required**.
|
||||||
|
|
||||||
|
This **prevents the shortcuts** the adversary pointed out (#4/#6: reading the plaintext, `sparqlQuery`/`inbox.read` bypassing the filter) and **forbids** falling back on an ACL — that is the heart of "same logic, with rigor".
|
||||||
|
|
||||||
|
**Target — NOT the current state**: every surface that returns data will have to go through decryption-with-key. **Today this is FALSE, and far more broadly than this brief first stated** — mapping of 2026-07-27, VERIFIED: **only 4 sites consult the caps** (`use-shape`, `read-filter`, `read-model.readUnion`, `discovery.submitToIndex`). Everything else returns data with no guard:
|
||||||
|
|
||||||
|
| Surface | State |
|
||||||
|
|---|---|
|
||||||
|
| `docs.sparqlQuery` / `sparqlUpdate` | **bypasses** — they call the injected `ng` **directly** (an accepted constraint, to avoid a double-Proxy `DataCloneError`). **The widest breach**: a session id + a NURI are enough to read everything. |
|
||||||
|
| `inbox` (`read` / `readSynced` / `materialize` / `watch`) | **bypasses** — no cap consulted; the drops go to whoever asks for them |
|
||||||
|
| `store-registry` (**zero** reference to caps in the whole file) | **bypasses** — the account→NURI root of trust is universally readable |
|
||||||
|
| `discovery.readIndex` | **bypasses** on read (caps checked on write only) |
|
||||||
|
| `subscribe`, `open-repo` | **bypass** — the subscription push carries the doc state with no check |
|
||||||
|
| `watch-shape` | deliberately delegates to `readUnion` (does not re-filter) |
|
||||||
|
|
||||||
|
**And the WRITE guard is already stillborn**: `ng-proxy` guards `sparql_update`, but `docs` bypasses the proxy **by design**, and **all** internal writers go through `docs`. So the guard only fires for an app calling `ng.sparql_update` on the exported `ng` — which Festipod does not do. `grantWrite` / `canWrite` are **decorative**. *(This finding reinforces §1 of the adversarial review: writing is not an axis "to be added", it is an axis we believed was covered and is not.)*
|
||||||
|
|
||||||
|
**This inventory IS the scope of P1b.** The only existing guard (`caps.canRead`) is moreover a **set-membership ACL** — the very inversion the vision forbids.
|
||||||
|
|
||||||
|
1. **Two distinct reference shapes**: cap-less (names/locates without reading — aligned with the NURI without `:k:`) vs cap-bearing (id + key/token). Absent today.
|
||||||
|
2. **Grant = delivering a cap-token to a recipient** (emulating the sealing: the recipient *receives* the token in their inbox; it is **possession** of the token that authorizes reading — not an ACL row checked per principal).
|
||||||
|
3. **Enforcement by possession**: readers (`read-filter`, `use-shape`) only see what they **hold the token for**, not "what they are in the readers set for".
|
||||||
|
4. **Resolving a cap-less** = naming / proving existence / counting, **without** exposing the content (support for anonymous presence).
|
||||||
|
5. **Revocation = re-key** emulated: invalidate the old token, re-deliver a new one to the remaining authorized holders; **non-retroactive**.
|
||||||
|
|
||||||
|
## ~~Widened scope: the WriteCap (= membership)~~ — DROPPED (2026-07-21)
|
||||||
|
|
||||||
|
**This section was wrong and is kept struck through as a guardrail.** It imported a notion of *membership* read from the **current state** of `nextgraph-rs` (`AddMember`, `PermissionV0`, `member_pubkey`) and promoted it into a **target shape**. But (a) those types are **inert scaffolding** at runtime — `verify_sig` / `verify_perm` are only called in unit tests, and `Repo`s are built with `members: HashMap::new()`; and (b) the target model **has no notion of membership at all**: only **keys and URLs**, symmetric and asymmetric. A shape in terms of `member`/`role`/`permission` is therefore exactly the **wrong shape** that this brief exists to prevent.
|
||||||
|
|
||||||
|
**The methodological lesson, which is worth more than the dropped section**: reading NextGraph's current state in order to **deduce** the target shape is a mistake — the current state contains unfinished work that must not be frozen into the polyfill. The source serves to verify an existing **mechanism**, never to infer an **intent**.
|
||||||
|
|
||||||
|
Erroneous content kept below as a record:
|
||||||
|
|
||||||
|
<details>
|
||||||
|
<summary>Dropped section</summary>
|
||||||
|
|
||||||
|
**Why this is here and not elsewhere.** The brief was at first ReadCap-centric; an adversarial finding showed that it **does not compose** with its consumer: the Festipod brief on "Set-based sign-ups" needs to **deduplicate** participations (one user = one participation per event), and the only non-application-level basis available is the commit's **author signature** — hence a **write** primitive, not a read one. A polyfill that only exposes the ReadCap shape leaves the consumer to invent its own application-level dedup → exactly the wrong shape.
|
||||||
|
|
||||||
|
**The real shape (VERIFIED, cf. `readcap-and-nuri-model.md` §1)** — and it is **asymmetric** with reading, which is the easiest point to miss:
|
||||||
|
|
||||||
|
- **Reading = possession of a key.** No ACL. Whoever holds, reads.
|
||||||
|
- **Writing = membership + permissions** (`AddMember`, `AddPermission` on the `RootBranch`). It really **is an authorization list** — not possession. Emulating writing "by token possession" would be just as wrong as the current read ACL, in mirror image.
|
||||||
|
- **Commits ARE signed** by a `UserId` (a **technical** key, distinct from the profile) — so a dedup identifier **exists** natively, with no application-level pseudonym.
|
||||||
|
- **But verifying a signature requires being a member of the repo** (access to the `member_pubkey`). A non-member third party sees a signed commit without being able to attribute it.
|
||||||
|
- **The inbox drop is NOT authenticated** (anonymous sealed box): a declared `from` is content, not proof.
|
||||||
|
|
||||||
|
**What that imposes on the polyfill.** Expose `membership` as a primitive **distinct** from cap possession, with at minimum: adding/removing a member of a repo, reading the members map **when one is a member**, and **verifying the author of a commit** (→ an author digest, **per-overlay hence per-store**). It is this last point that unblocks the dedup on the Festipod side.
|
||||||
|
|
||||||
|
**The shape consequence, to be documented explicitly** (otherwise the consumer picks the wrong model): the author digest being **per-store**, the choice of how stores are carved up **is** the choice of the correlation level. A **per-user stable** store gives an identifier traceable **across events**; a **per-event** store gives a pseudonym **local to the event** — dedup possible, correlation impossible. Festipod needs the second. So the polyfill must make this carving **expressible**, not freeze it.
|
||||||
|
|
||||||
|
**Still open**: "can the creator of an event be a member of the store that contains the participations, without holding its read key?" — that is, membership (writing/verification) and possession (reading) genuinely **orthogonal**. If NextGraph couples them, verified dedup and anonymity are mutually exclusive, and it is the Festipod brief that must settle what it sacrifices. **To be verified before shaping the API.**
|
||||||
|
|
||||||
|
</details>
|
||||||
|
|
||||||
|
*(Question now moot: there is no membership. The dedup does not go through signature verification — see the Festipod brief on "sign-ups".)*
|
||||||
|
|
||||||
|
## Open questions
|
||||||
|
|
||||||
|
- **SETTLED (PO directive, 2026-07-21)**: we **simulate the crypto** (per-doc encrypted data, cap = key). "Semantics only" (a token registry) is **discarded** — it turns back into an ACL and lets the plaintext be read. What remains to settle is the **level** of simulation (real lightweight encryption vs masked read-model projection), **before P1**.
|
||||||
|
- NURI representation, cap-less vs cap-bearing, in the emulation (mirror `:k:`).
|
||||||
|
- ~~Should **keyless fetch** be allowed (resolving a cap-less into existence/count without the content)~~ — **SETTLED, and negatively (2026-07-27)**: not constructible. Addressing itself presupposes the cap, so there is nothing to expose. See the corrected Q1 verdict below. Kept struck through rather than deleted: the hypothesis is intuitive and will otherwise be re-formed.
|
||||||
|
- API migration: `declareConnections`/`grantRead` → `seal(cap, recipient)` + `inbox → received caps`. Breaks consumers (the app-side `declareConnections` disappears).
|
||||||
|
|
||||||
|
## P0 — "keyless-resolve" spike (the blocker, BEFORE any P1)
|
||||||
|
|
||||||
|
**Load-bearing question**: can a holder of a **cap-less reference** (`did:ng:o:{id}:v:{overlay}`, without `:k:`), **without ever reading the content**:
|
||||||
|
- **Q1 — Existence / fetch**: prove/retrieve the presence of the (encrypted) blocks from the broker? Or does the broker require a ReadCap/membership in order to serve the blocks?
|
||||||
|
- **Q2 — Deletion**: distinguish "exists" from "deleted"? *(The FRAGILE point: NextGraph is an append-only CRDT — a withdrawal = a **tombstone commit** that one would have to **read** in order to know about → potentially **the key is required**. And the **decrement on leave** depends on it.)*
|
||||||
|
- **Q3 — Confidentiality**: does the key (`:k:`) remain **required** in order to decrypt (keyless never gives the content)?
|
||||||
|
|
||||||
|
**Why this is the blocker**: the whole **anonymous counter** (counting/validating cap-less refs without reading) AND the **decrement on leave** depend on it. **If NO** → "anonymous counter via cap-less ref" is **not constructible in the target** → Festipod must **not** code that shape (guaranteed rewrite). **If YES** → P1 exposes `resolveCapLess(nuri) → {exists|deleted}` (never any content), and the emulation simulates it faithfully.
|
||||||
|
|
||||||
|
**Method** (cheap, decisive):
|
||||||
|
1. **Trace** in `nextgraph-rs` the broker/verifier **fetch authorization** path: who serves the blocks (`BlocksGet`/`TopicSync`/`OverlaySync`)? is a cap/membership checked, or is `id+overlay` enough? is the *outer* overlay public? is a deletion observable without the key?
|
||||||
|
2. *(Optional)* **decisive e2e test** (in the style of `e2e/reactivity-doc-subscribe.ts`): B holds the cap-less ref, attempts fetch/existence **without** the key, verifies that it **does not reach** the content. Empirical proof > source.
|
||||||
|
3. *(Or)* confirm with the NextGraph dev — the fastest.
|
||||||
|
|
||||||
|
**Deliverable**: YES/NO/PARTIAL per Q1/Q2/Q3 + the exact primitive (file:line) + the API shape to expose (if YES), or the finding that the counter changes (if NO).
|
||||||
|
**Gated decision**: YES → P1; NO → the sign-ups brief revisits the counter (not anonymous, or another primitive).
|
||||||
|
|
||||||
|
### Spike verdict (2026-07-21) — VERIFIED in `nextgraph-rs`
|
||||||
|
|
||||||
|
| | Answer | Evidence |
|
||||||
|
|---|---|---|
|
||||||
|
| **Q1 — existence/fetch without a cap** | **NO** *(corrected on 2026-07-27 — the initial "partial YES" verdict over-read the evidence)* | Read access control does indeed let you through (reads are not cap-gated) — **but addressing presupposes the cap**: no existence command at the SDK level; the only probe is internal to the crate, requires `BlockId`s **and** a loaded repo, and targets the **inner** overlay derived from the read secret. A cap-less reference has neither `BlockId` nor the required overlay. See `readcap-and-nuri-model.md`. |
|
||||||
|
| **Q2 — detect a deletion without the key** | **NO** | Append-only broker; a deletion is an **encrypted tombstone commit** (`RemoveRepo`), a no-op on the verifier side. Without the key one observes "some activity", never "a deletion". |
|
||||||
|
| **Q3 — confidentiality** | **YES** | Blocks stored as ciphertext; the key is `#[serde(skip)]` (`types.rs`), derived from the `ReadCapSecret`. Keyless **never** gives the content. |
|
||||||
|
|
||||||
|
**What that decides.**
|
||||||
|
- **P1 is unblocked**: `resolveCapLess(nuri) → { exists }` is the right shape — but **`{ exists | deleted }` is NOT**. Do not expose a `deleted` state; that would be inventing a capability the target will never have (precisely the failure mode this brief fights).
|
||||||
|
- **Withdrawal has to be a message, not an observation.** On the consumer side: an explicit *nudge*. The polyfill has **nothing** to emulate for that — it just must not pretend otherwise.
|
||||||
|
- **Settled by the Q1 correction**: the anonymous counter can**not** rest on an existence validation — that is not constructible. So it rests on something **declarative**, which is acceptable (outside the security scope) as long as the **exposed shape does not lie**: do not expose an existence primitive that the target will not offer.
|
||||||
|
|
||||||
|
## P1a — the surface
|
||||||
|
|
||||||
|
**Extracted into its own brief: [`2026-07-27-p1a-cap-surface.md`](2026-07-27-p1a-cap-surface.md).**
|
||||||
|
|
||||||
|
This batch is **specified and ready to implement**; it has its own note so that one can code from it without wading through the retracted material of this document.
|
||||||
|
|
||||||
|
In two lines: a single new type (`ReadCap`), a keyring (`capFor`), a per-document share to an inbox (`shareCap`) — and nothing else. The branded types, `resolveCapLess`, `receivedCaps`, `refOf`, `parseNuri` and `PrincipalId` were **discarded** after a double adversarial review; the reasons are in that note.
|
||||||
|
|
||||||
|
This brief remains the **overall effort**: P0 verdicts, P1b scope, P2–P4 batches, and the adversarial reviews.
|
||||||
|
|
||||||
|
## Phase sketch
|
||||||
|
|
||||||
|
- **P1a** — **the surface**: one new type (`ReadCap`), a keyring (`capFor`), per-document sharing to an inbox (`shareCap`). **The only batch that blocks Festipod.** Specified in its own note: [`2026-07-27-p1a-cap-surface.md`](2026-07-27-p1a-cap-surface.md). *(An earlier draft listed `DocRef`/`DocCap` branded types, `resolveCapLess` and a durable `sealCapTo` here — all three were **dropped** after adversarial review; the note says why.)*
|
||||||
|
- **P1b** — **the enforcement**: per-doc encryption (cap = key) and closing out the inventory of bypasses. Without it the shape is right but the isolation remains false — so nothing "anonymous" can be claimed.
|
||||||
|
- **P2** — replace the ACL with a **token possession** model (grant = deliver to a recipient; enforcement = possession). *Requalified by the adversarial review: the real content of P2 is **durability + cap-less + re-sharing by the holder**, not "inverting the ACL" — without crypto, inverting produces no observable delta.*
|
||||||
|
- **P3** — revocation by re-key (invalidation + re-delivery, non-retroactive).
|
||||||
|
- ~~**PW** — WriteCap = membership~~ **DROPPED (2026-07-27)**. This batch rested on a notion of membership that **does not exist** in the model (everything is keys and URLs); see the struck-through section above. It was moreover justified by a need for **dedup via signature verification** that the consumer turns out not to have: its dedup rests on the overlay, which is store-scoped. *For the record, two facts verified along the way, not to be re-discovered*: author signature verification **is not called at runtime**; and the author digest is **not** keyed under the read secret — it is keyed by the **outer** overlay, which is public *(it is the commit's **content** that is encrypted, hence the fact that verifying still presupposes being able to read)*. Detail in `nextgraph-current-state.md`.
|
||||||
|
- **P4** — adapt the consumer API + `migration-guide.md`. *The adversarial review requalifies this batch: it is not an API swap but a **consumer re-architecture** (the grant moves to connection acceptance and becomes persistent; `declareConnections` disappears).*
|
||||||
|
|
||||||
|
## Adversarial review (2026-07-20) — to be integrated
|
||||||
|
|
||||||
|
An adversary refuted the brief (7 findings — the 7th marked *(Plausible)*). **To be read through the filter of the Objective above** (shape, not security). The purely **security** criticisms — readable plaintext content (#4), forgeable markers — are **ACCEPTED / out of scope**: the polyfill does not seek to prevent them. What remains are the real **SHAPE / rigor** defects (to be fixed), and a question of **future model** (#5):
|
||||||
|
|
||||||
|
1. **WriteCap forgotten, and "possession" is FALSE there.** Writing is **membership/permissions** (`AddMember`) — an **authorization list**, not key possession (ref. §1); `ng-proxy.ts:28-48` guards every `sparql_update`. → keep a **WriteCap = membership track**; **possession concerns ONLY reading**.
|
||||||
|
2. **P2 "possession without crypto" = the ACL renamed.** Without crypto, "who holds which token" = `Map<doc, Set<holder>>` = the current `readers`: **no observable delta**. The real deltas are **durability + cap-less + re-sharing by the holder** — THAT is the content of P2, not "inverting the ACL".
|
||||||
|
3. **Non-retroactive revocation NOT EMULABLE** without versioning: `read-model.ts:112-118` only reads the current state → "invalidate the old token" = total removal = the inverse of the real thing (the former holder decrypts the **prior** versions). → emulate only "no new reads after re-key" + **document non-retroactivity as non-emulable**.
|
||||||
|
4. **cap-less "without exposing the content" ILLUSORY in the emulation**: content in **plaintext** in the shared wallet; `sparqlQuery`/`inbox.read` **bypass** the filter; `read-filter.ts:30-35` is all-or-nothing. → cap-less anonymity requires either **real crypto** or a **masked read-model projection** (counting without reading). "Replacement not overhaul" is **overstated**.
|
||||||
|
5. **Keyless-fetch = INFERRED and load-bearing**: add a **P0 spike** that verifies it **before** P1 (otherwise the model — polyfill AND Festipod — is not constructible).
|
||||||
|
6. **Migration ≠ API swap.** `declareConnections` is replayed every session because the map is ephemeral; durable seals move the grant to **connection acceptance** + persist "already sealed" — no analogue of `protectedDocsOf` + the re-derivation loop. **Consumer re-architecture.**
|
||||||
|
7. *(Plausible)* delivering a cap through the async inbox **does not re-trigger** `watchShape` (which subscribes to data docs, not to caps) → unreadable views left **stale** until another change. → plan for a cap-mutation signal.
|
||||||
|
|
||||||
|
**Consequence**: add **P0 (keyless-fetch spike)** up front and a **distinct WriteCap track**; requalify P2 (the real content = durability + cap-less + re-sharing, not "inverting the ACL"); record that **without crypto, read privacy is not applicable** (choose: real crypto vs masked projection).
|
||||||
|
|
||||||
|
Links: `readcap-and-nuri-model.md`, `packages/client/src/caps.ts`. On the consumer side, the Festipod brief "realign the sign-ups" depends on this effort.
|
||||||
@@ -0,0 +1,129 @@
|
|||||||
|
# Brief — P1a: the capability surface
|
||||||
|
|
||||||
|
**Status: specified, ready to implement.** Extracted from `2026-07-20-caps-emulation-alignment.md` (which remains the wider chantier: P0 findings, P1b enforcement, P2–P4, and the adversarial reviews). This file is the actionable lot; read it alone to implement.
|
||||||
|
|
||||||
|
Written 2026-07-27, after two adversarial reviews and three corrections from the PO. Background: `../vision.md` (why this library exists), `../readcap-and-nuri-model.md` (the target model, verified against `nextgraph-rs`).
|
||||||
|
|
||||||
|
## Why this lot exists
|
||||||
|
|
||||||
|
`caps.ts` currently models read rights as an **ACL** — a `Map<doc, Set<principal>>` plus `grantRead(doc, grantee)`. That is the **exact inversion** of the real model, where reading is **key possession**: whoever holds the key reads, and there is no authorization list anywhere.
|
||||||
|
|
||||||
|
This is not a security problem — the library is deliberately insecure and that is accepted (see `../vision.md`). It is a **shape** problem, and shape is the only thing this library exists to get right. A consumer coded against an ACL is coded against a model that will never exist, and will have to be rewritten.
|
||||||
|
|
||||||
|
## Scope: shape only, not enforcement
|
||||||
|
|
||||||
|
- **P1a (this brief)** — the surface consumers see.
|
||||||
|
- **P1b (separate)** — per-doc encryption and closing the read paths that bypass the guard.
|
||||||
|
|
||||||
|
Only P1a blocks the consumer, because the consumer must be written as if NextGraph were finished. P1b can follow.
|
||||||
|
|
||||||
|
> **After P1a the shape is right and the isolation is still fake.** Nothing may be claimed as "anonymous" or "private" until P1b lands. Say so in the README if it helps.
|
||||||
|
|
||||||
|
## Guiding constraint: stay close to NextGraph's concepts
|
||||||
|
|
||||||
|
Stated by the PO, and it is the acceptance criterion for the design as much as for the code:
|
||||||
|
|
||||||
|
> Stay as close as possible to NextGraph's concepts — and to its SDK's — to keep development simple and to keep the number of notions someone must discover small when they already know NextGraph and open this library.
|
||||||
|
|
||||||
|
Every invented name is **vocabulary debt**: the reader has to carry a translation table in their head. The first draft of this spec introduced eight new notions; adversarial review reduced it to two. Hold that line.
|
||||||
|
|
||||||
|
## The design
|
||||||
|
|
||||||
|
### 1. Types — one new name
|
||||||
|
|
||||||
|
A NURI is **one object**, with or without the key inside — upstream, `NuriV0 { target, access }`, where a cap-less NURI simply has an empty `access`. `did:ng:` is the **URI scheme prefix**, present on inboxes, branches and overlays alike; it does not mean "without cap". The discriminant is the **`:k:` segment**.
|
||||||
|
|
||||||
|
```ts
|
||||||
|
type Nuri = string // did:ng:o:{doc}:v:{overlay} — names, does not read
|
||||||
|
type ReadCap = string // …:k:{key} — names AND reads
|
||||||
|
```
|
||||||
|
|
||||||
|
`Nuri` **keeps its current meaning** in this package (~90 call sites, untouched): the cap-less form. `ReadCap` is the upstream name — do not invent another.
|
||||||
|
|
||||||
|
A parsed form `{ target, readCap? }` — a 1:1 mirror of `NuriV0 { target, access }` — may be used **inside** the library. It must not surface in the SDK-identical entry's signatures.
|
||||||
|
|
||||||
|
**Do not use branded types.** They were in the first draft and were dropped deliberately: the real SDK takes `nuri: String` and enforces at **runtime, through cryptography**. A compile-time guarantee is a concept NextGraph does not have, and a consumer who typed everything would have to *un-type* it when the real SDK arrives — the opposite of the goal. The cost was also measured: branded types force a cast at every ORM and SPARQL boundary.
|
||||||
|
|
||||||
|
### 2. The keyring — where caps come from
|
||||||
|
|
||||||
|
`doc_create` returns a **cap-less** NURI. So a rule like "no function ever goes from a bare reference to a cap" is wrong: it would leave a document's own creator unable to obtain that document's cap.
|
||||||
|
|
||||||
|
The real mechanism: on every document creation, an `AddRepo { read_cap }` is committed to a **branch of the store** (the store is itself a repo, with typed branches — "branch" here has nothing to do with git). That branch lists the store's documents, each with its read key. **It is the owner's keyring.** Upstream, the keyring is the **wallet**.
|
||||||
|
|
||||||
|
```ts
|
||||||
|
capFor(nuri: Nuri): ReadCap | undefined
|
||||||
|
```
|
||||||
|
|
||||||
|
The invariant, correctly stated:
|
||||||
|
|
||||||
|
> **You do not derive a cap from a bare reference. You look it up in your keyring — or you were given it.**
|
||||||
|
|
||||||
|
`capFor` absorbs `canRead(doc)` (`capFor(n) !== undefined`) and drops its ACL verb.
|
||||||
|
|
||||||
|
**The keyring is not the sharing mechanism.** Handing over a store cap would give away everything the store contains, present and future. That is not the gesture (see §3). This confusion is easy and expensive — it was made once already during design.
|
||||||
|
|
||||||
|
### 3. Sharing — one document, to one or more recipients
|
||||||
|
|
||||||
|
**The unit of sharing is the document**, consistent with the consumer's own doctrine ("the document is the unit of sharing and of rights").
|
||||||
|
|
||||||
|
```ts
|
||||||
|
shareCap(cap: ReadCap, toInbox: Nuri): Promise<void>
|
||||||
|
```
|
||||||
|
|
||||||
|
Recipients are addressed as **inboxes** — which `inbox.post(targetInbox: Nuri)` already does in this package. There is no `PrincipalId` here: that notion exists nowhere upstream, and the first draft removed `principal` from `canRead` (calling it the ACL inversion) only to reintroduce it here.
|
||||||
|
|
||||||
|
**Caps received need no dedicated operation.** They arrive as inbox deposits of kind `cap`, consumed by the **existing** `inbox.watch`. This also fixes a known gap: a cap delivered asynchronously now triggers a re-read naturally, instead of leaving stale views.
|
||||||
|
|
||||||
|
> **Upstream status: this is a GAP, not a disagreement.** The field exists (`ContactDetails.read_cap`, commented "*if user wants to share the content of profile*") but the message construction is `unimplemented!()`, its only caller passes "without read_cap", and the receiver **discards** the cap. The shape is right; the implementation is absent. We emulate it meanwhile — filed as `orm-tests/INBOX/2026-07-27-inbox-cap-delivery-not-implemented.md`, including what to remove from this library once upstream lands it.
|
||||||
|
|
||||||
|
### 4. Key rotation — automatic redelivery, not loss of access
|
||||||
|
|
||||||
|
When a key is rotated, the new one is **sent to the inbox** of users who keep access, and that inbox is **processed automatically** as soon as one of the user's clients connects.
|
||||||
|
|
||||||
|
So access is not lost, it is **deferred** until the next connection — consistent with local-first. Consequences for the surface:
|
||||||
|
|
||||||
|
- **No subscription obligation to expose.** The consumer implements nothing to "keep" an access.
|
||||||
|
- Redelivery uses **the same channel** as the initial delivery, so §3 covers both with no special case.
|
||||||
|
- **Revocation** stays what it is: stop redelivering, non-retroactive.
|
||||||
|
|
||||||
|
> An earlier draft said the opposite ("whoever does not stay subscribed loses access"). That came from an upstream comment describing the **current state**, read as if it gave the **intention**. It does not. Source verifies a mechanism; it never states a direction.
|
||||||
|
|
||||||
|
### 5. Public content — readable by URL, and NOT recursive
|
||||||
|
|
||||||
|
> **An item in the public store is public: whoever has the URL reads the content.** But **not recursively** — public content may *reference* private content, and the reference does not grant access to what it references.
|
||||||
|
|
||||||
|
This is a **second mechanism** alongside key possession, not an exception to it. The non-recursiveness is what carries the value: it allows a public object that **points at** private identity without disclosing it — exactly the pattern the consumer needs.
|
||||||
|
|
||||||
|
*Implementation detail the shape must not depend on*: NextGraph is moving toward **not encrypting** public store content (data still signed). And if the public store does not behave as this principle describes, **this library adapts** — not the consumer.
|
||||||
|
|
||||||
|
### 6. What disappears or is renamed
|
||||||
|
|
||||||
|
| Today | Becomes |
|
||||||
|
|---|---|
|
||||||
|
| `grantRead(doc, grantee)` | `shareCap(cap, toInbox)` |
|
||||||
|
| `canRead(doc, principal)` | absorbed by `capFor(nuri)` — the `principal` parameter **was** the ACL inversion |
|
||||||
|
| `protectedDocsOf(owner)` | **removed** — the re-derivation loop disappears |
|
||||||
|
| `makePublic(doc)` | `publishRepoLink` — the shareable link has an upstream name (`RepoLinkV0`) |
|
||||||
|
| `grantWrite` / `canWrite` | deferred to P1b — currently **decorative** (the guard never fires) |
|
||||||
|
| `resetCaps()` on identity change | **switch** keyrings, do **not** wipe |
|
||||||
|
| `PrincipalId` in the cap surface | **removed** — recipients are inboxes |
|
||||||
|
|
||||||
|
`resetCaps()` is the trap that can make this lot look finished while it is not: if switching identity still wipes, durability is a lie and the per-session re-declaration comes back under another name.
|
||||||
|
|
||||||
|
### 7. Boundary: SDK-identical entry vs `/polyfill`
|
||||||
|
|
||||||
|
Caps live under `/polyfill` today; `index.ts` is the SDK-identical entry. Keep it that way, and keep `index.ts` signatures on plain strings — that **is** what the real SDK does. The discrimination lives in what you can **obtain** (the keyring), not in what the compiler permits.
|
||||||
|
|
||||||
|
### 8. Acceptance test — no cryptography required
|
||||||
|
|
||||||
|
`watch-shape` currently harvests **every** `did:ng:` string it finds in a discovery reference and folds those documents into the **read** set. A bare reference therefore grants **full read** today — the semantics exactly inverted.
|
||||||
|
|
||||||
|
After P1a: a harvested bare reference yields **nothing**, for want of a cap in the keyring — which is what real NextGraph does. The test holds without a line of encryption, which is what makes the P1a/P1b split honest rather than cosmetic.
|
||||||
|
|
||||||
|
## Consumer impact
|
||||||
|
|
||||||
|
`declareConnections` **disappears**. This is not an API swap: today it re-declares every grant on every session because the ACL is in-memory. With delivered caps, the grant moves to the moment a connection is **accepted**, and persists. Plan for consumer re-architecture, and update `../migration-guide.md`.
|
||||||
|
|
||||||
|
## What this lot does NOT do
|
||||||
|
|
||||||
|
Closing the read paths that bypass the guard — `docs.sparqlQuery`/`sparqlUpdate`, the whole inbox, `store-registry`, `discovery.readIndex`, `subscribe`, `open-repo`. Only four sites consult caps today. That inventory is P1b's scope and is listed in `2026-07-20-caps-emulation-alignment.md`.
|
||||||
@@ -0,0 +1,57 @@
|
|||||||
|
# Write loss on socket death (`SerializationError`)
|
||||||
|
|
||||||
|
**Post-mortem — 2026-07-14 · Status: OPEN (not addressed).**
|
||||||
|
|
||||||
|
An entity written just before a period of inactivity can be **silently lost**: it is absent on reconnection. *(Whether the write never durably reached the broker, or reached it and is not read back on a cold reconnection, is **not settled** — see Epistemic caveat below. The wording here deliberately states only the observed symptom.)* The **account / identity survives** (no fork). Observed in real conditions (Festipod, Firefox) during a pause after login/creation.
|
||||||
|
|
||||||
|
## Symptom
|
||||||
|
|
||||||
|
1. The user logs in, the app creates an entity (a Festipod event).
|
||||||
|
2. A period of inactivity follows (idle, tab in the background…).
|
||||||
|
3. The broker socket dies spontaneously with `SOCKET IS CLOSED Some(Left(SerializationError))`.
|
||||||
|
4. On reconnection, the created entity has disappeared; the app reads back its own scope **empty**.
|
||||||
|
|
||||||
|
## Evidence (VERIFIED — live Firefox logs, verbatim)
|
||||||
|
|
||||||
|
```
|
||||||
|
… REPLAY TOPIC NOT FOUND <topic> IN OVERLAY <overlay>
|
||||||
|
… NEED REPLAY true
|
||||||
|
… SENDING EVENTS FROM OUTBOX RETURNED: Err(TopicNotFound)
|
||||||
|
[user1][polyfill] resolveAccount(user1) → 1 record ← the account SURVIVES (no fork)
|
||||||
|
[user1][polyfill] readScopeIndex(…) → 0 entities ← but the scope is EMPTY
|
||||||
|
… set reçu: 0 objets Event (public)
|
||||||
|
… SOCKET IS CLOSED Some(Left(SerializationError)) [51, 3, 223, …]
|
||||||
|
```
|
||||||
|
|
||||||
|
Interpretation (**plausible mechanism, not settled**): the write was pushed into the local **outbox**, but the socket died before it was **durably flushed** into the broker topic; on reconnection, the outbox replay fails (`Err(TopicNotFound)`) because the topic was **never created on the broker side** → the event is abandoned. The account, for its part, had already been durably resolved (`resolveAccount → 1 record`): it is neither lost nor forked.
|
||||||
|
|
||||||
|
> **Epistemic caveat.** The evidence establishes the *symptom* (loss + `Err(TopicNotFound)` + `readScopeIndex → 0`). The exact *mechanism* is not settled between **(i) loss at write time** (the write never durably reaches the broker) and **(ii) cold-rehydration failure** (the write *is* on the broker but a fresh session does not reopen its own scope). The `Err(TopicNotFound)` on the outbox replay leans toward **(i) in this Firefox case**. See the @data repro below, which exhibits a neighboring symptom but **does not settle** (i) vs (ii).
|
||||||
|
|
||||||
|
## Causal chain (TRACED — reading of the NextGraph core, to be re-verified)
|
||||||
|
|
||||||
|
- The `SerializationError` closes the socket. The core emits the disconnection: `broker.rs` → `LocalBrokerMessage::Disconnected` → `disconnections_sender.send(...)` (≈ `broker.rs:1051`, to be re-verified — volatile number, navigate by symbol).
|
||||||
|
- This disconnection is **pushed** to subscribers via `disconnections_subscribe(cb)` (PUSH stream).
|
||||||
|
- **NextGraph reconnection is an unimplemented `// TODO`** (≈ `broker.rs:1051-1076`): nothing re-establishes the socket nor re-flushes the outbox.
|
||||||
|
- `user_connect` returns a **snapshot** `{ server_id, server_ip, error, since }` at call time — not a stream, unusable for detecting a later drop.
|
||||||
|
- **No write-durability confirmation API**: a caller cannot `await` the guarantee that a write has reached the broker.
|
||||||
|
|
||||||
|
## What the SDK exposes but does not consume
|
||||||
|
|
||||||
|
`disconnections_subscribe` **does fire** on this failure — but neither the polyfill (`@ng-eventually/client`) nor the consumer app subscribes to it. The signal exists, nobody listens to it; on the app side, no mechanism retries or warns the user.
|
||||||
|
|
||||||
|
## Scope & not reproduced
|
||||||
|
|
||||||
|
- **Observed on Firefox only** to date. A manual test on another browser did not trigger the `SerializationError` nor its consequences.
|
||||||
|
- **@data reproduction (Chromium, real broker) — 2026-07-14, decisive.** The existing @data reconnection test (`reconnexion-meme-identite`) was a **false green**: it read A's repos back from the persistent profile's **local IndexedDB**, never from the broker. A **genuinely cold** reader (non-persistent `freshBrowser` context, the **same** wallet/account A, no local state — seeded from the wallet captured before the event) reads **0** events from A (`BARRIER timed-out (8000ms)`, `CONNECTION ESTABLISHED`). A **different** signature from the Firefox case (no socket death; the `OUTBOX empty` is the reader's, trivially empty) and it **does not settle** (i) vs (ii) — an empty barrier is compatible with both. Established on the other hand: **@data has never verified the broker durability of A's own reads**, and cold rehydration from the broker fails. Repro: `src/modules/event/features/reconnexion-froide-sans-local.feature` (Festipod).
|
||||||
|
- **To settle (i) vs (ii)**: independently verify that A's write reaches the broker — e.g. a *warm* reader / a second identity reads the event's public doc (the two-identity isolation scenario). If it sees it → the write is durable → the cold reader's 0 is a **(ii)** (rehydration). Otherwise → **(i)**.
|
||||||
|
|
||||||
|
## Fix leads (not arbitrated)
|
||||||
|
|
||||||
|
1. **Core** — fix the `SerializationError` **and** implement the reconnection TODO (re-establish the socket + re-flush the outbox).
|
||||||
|
2. **SDK / polyfill** — consume `disconnections_subscribe` → reconnection + outbox re-flush as a mitigation, independently of the core.
|
||||||
|
3. **Durability API** — expose a confirmation that a write has reached the broker, so that the caller can `await` it.
|
||||||
|
|
||||||
|
## Links
|
||||||
|
|
||||||
|
- `docs/nextgraph-current-state.md` — current state of the core (disconnection / reconnection to be cross-referenced here).
|
||||||
|
- Product impact + consumer-side caveat: Festipod concept `data-layer` → `caveat_write-durability-across-disconnect`.
|
||||||
@@ -137,6 +137,32 @@ A related exposed primitive: `social_query_start` (a federated query via inbox u
|
|||||||
`degree` hops) exists but is limited to contacts — it does not cover an anonymous
|
`degree` hops) exists but is limited to contacts — it does not cover an anonymous
|
||||||
notification to a non-connected host.
|
notification to a non-connected host.
|
||||||
|
|
||||||
|
### Delivering a ReadCap through the inbox — the field exists, the path does NOT — VERIFIED
|
||||||
|
|
||||||
|
The `ContactDetails` inbox message carries `read_cap: Option<ReadCap>`, commented
|
||||||
|
*"optional readcap on the profile, if user wants to share the content of profile"*
|
||||||
|
(`engine/net/src/types.rs`). Nothing behind that field is implemented:
|
||||||
|
|
||||||
|
- **Building it panics.** `InboxPost::new_contact_details(…, with_readcap: bool, …)`
|
||||||
|
(`engine/net/src/types.rs`) fills `read_cap` with `unimplemented!()` when
|
||||||
|
`with_readcap` is true, and `None` otherwise. Asking for a cap in the message is a
|
||||||
|
panic, not a feature.
|
||||||
|
- **Nobody asks for one.** Its ONLY caller is the `QrCodeProfileImport` path in
|
||||||
|
`engine/verifier/src/request_processor.rs`
|
||||||
|
(`post_to_inbox(InboxPost::new_contact_details(…))`), which passes `with_readcap =
|
||||||
|
false`. No message ever carries a cap.
|
||||||
|
- **The receiver discards it.** The `InboxMsgContent::ContactDetails(details)` arm of
|
||||||
|
`engine/verifier/src/inbox_processor.rs` reads `details.profile`, `details.name` and
|
||||||
|
`details.email` to build a `social:contact` document — it **never reads
|
||||||
|
`details.read_cap`**. Even a hand-crafted message carrying a cap would be dropped.
|
||||||
|
|
||||||
|
**Consequence for this lib:** there is no native channel to HAND a key to somebody. The
|
||||||
|
inbox transports an identity/profile pointer, not a read capability. Combined with
|
||||||
|
§ *The inbox is not usable from the JS SDK* (no `InboxPost` arm in the request processor
|
||||||
|
at all), cap delivery must be emulated end to end: the polyfill's emulated inbox and its
|
||||||
|
`CapRegistry` are not a shortcut around an existing mechanism, they stand in for a
|
||||||
|
mechanism that does not exist.
|
||||||
|
|
||||||
## The query capability — ONE local store, named graphs, union queries
|
## The query capability — ONE local store, named graphs, union queries
|
||||||
|
|
||||||
The single fact that makes read-time *listing* possible on the shared wallet, and
|
The single fact that makes read-time *listing* possible on the shared wallet, and
|
||||||
@@ -470,3 +496,242 @@ logout is exposed (`ng.session_stop()`, `ng.user_disconnect()`,
|
|||||||
redirect afterwards. This lib's identity store sidesteps all of it — the identity
|
redirect afterwards. This lib's identity store sidesteps all of it — the identity
|
||||||
id is set at wallet-import time and relayed to the lib, without a separate login;
|
id is set at wallet-import time and relayed to the lib, without a separate login;
|
||||||
see the identity store in [`simulation.md`](./simulation.md).
|
see the identity store in [`simulation.md`](./simulation.md).
|
||||||
|
|
||||||
|
## Authorship, existence, outer overlay, `Ext` (section added 2026-07-27)
|
||||||
|
|
||||||
|
Four capability facts about the current core, verified in `nextgraph-rs`. They bear on
|
||||||
|
what can be BUILT on top (can we deliver a key? can we tell whether a document exists?
|
||||||
|
can we attribute a write?) — they are not a security assessment. Each carries its
|
||||||
|
epistemic status; do not upgrade an INFERRED item without new evidence.
|
||||||
|
|
||||||
|
### Author-signature verification is never called at runtime — VERIFIED
|
||||||
|
|
||||||
|
`Commit::verify` (`engine/repo/src/commit.rs`) chains `verify_sig` → `verify_perm` →
|
||||||
|
`verify_full_object_refs_of_branch_at_commit`. Its only callers in the whole tree are
|
||||||
|
inside `#[cfg(test)] mod test` blocks (`engine/repo/src/commit.rs`,
|
||||||
|
`engine/repo/src/branch.rs`); `verify_sig` and `verify_perm` have no other caller. The
|
||||||
|
verifier's commit path calls a **different** `verify`:
|
||||||
|
`CommitBodyV0::<Body>::verify(commit, self, branch_id, repo_id, store)` in
|
||||||
|
`engine/verifier/src/verifier.rs` — the `CommitVerifier` trait, which APPLIES a body
|
||||||
|
(mutating verifier state); it is not a signature check.
|
||||||
|
|
||||||
|
Even if it were called it could not succeed. `verify_sig` resolves the author through
|
||||||
|
`Repo::member_pubkey` → `Repo.members`, and every `Repo` the verifier builds at runtime
|
||||||
|
sets `members: HashMap::new()` — `engine/verifier/src/user_storage/repo.rs` (with a
|
||||||
|
literal `//TODO: members`) and `engine/verifier/src/commits/mod.rs`. Only
|
||||||
|
`Repo::new_with_member` ever populates a member, and it is called only from tests. An
|
||||||
|
empty table makes `member_pubkey` return `NotFound` →
|
||||||
|
`CommitVerifyError::PermissionDenied`.
|
||||||
|
|
||||||
|
Reading authorship at all presupposes the read cap (VERIFIED): the author field is not a
|
||||||
|
UserId but `CommitContent::author_digest(user, overlay)`, a BLAKE3 keyed hash, and the
|
||||||
|
commit content sits in blocks ChaCha20-encrypted under `Object::convergence_key(store)`
|
||||||
|
(`engine/repo/src/object.rs`), whose key material is the store id **plus the
|
||||||
|
store-overlay-branch ReadCapSecret**. No read cap → the author field is not even
|
||||||
|
visible. *Nuance, VERIFIED:* the digest's own hashing key derives from
|
||||||
|
`overlay_id_for_read_purpose`, which for Public/Protected/Private/Group stores is
|
||||||
|
`OverlayId::outer(store_id)` — public. What is secret is the commit content, not the
|
||||||
|
hash key.
|
||||||
|
|
||||||
|
**Consequence for this lib:** "who wrote this triple" is unanswerable today — neither
|
||||||
|
cryptographically (nothing verifies) nor by identity (the digest is opaque without a
|
||||||
|
member table). Any authorship or provenance the polyfill needs must be carried in the
|
||||||
|
DATA it writes and re-read from there; an "authored by X" claim in the emulation has no
|
||||||
|
core check behind it.
|
||||||
|
|
||||||
|
### No existence probe at SDK level — addressing presupposes the cap — VERIFIED
|
||||||
|
|
||||||
|
`AppRequestCommandV0` (`engine/net/src/app_protocol.rs`) contains no existence command:
|
||||||
|
`Fetch`, `Pin`, `UnPin`, `Delete`, `Create`, `FileGet`, `FilePut`, `Header`, `InboxPost`,
|
||||||
|
`SocialQueryStart`, `SocialQueryCancel`, `QrCodeProfile`, `QrCodeProfileImport`,
|
||||||
|
`OrmStartGraph`, `OrmStartDiscrete`, `OrmGraphUpdate`, `OrmDiscreteUpdate`, `OrmStop`.
|
||||||
|
Nothing answers *"does document D exist?"*.
|
||||||
|
|
||||||
|
The single probe in the tree is internal and cannot answer it either:
|
||||||
|
`Verifier::has_blocks` (`engine/verifier/src/verifier.rs`) sends
|
||||||
|
`BlocksExist { blocks, overlay }`. It is `pub(crate)` (never reaches JS); it takes
|
||||||
|
**`BlockId`s** — content addresses you only hold if you already read the object; it takes
|
||||||
|
a **`&Repo` already loaded**; and it targets
|
||||||
|
`repo.store.overlay_for_read_on_client_protocol()` = the **inner** overlay
|
||||||
|
(`Store::inner_overlay` → `overlay_id_for_write_purpose(store_overlay_branch_readcap.key)`,
|
||||||
|
`engine/repo/src/store.rs`), derived from the read-cap secret.
|
||||||
|
|
||||||
|
**Consequence for this lib:** you cannot prove — nor disprove — the existence of a
|
||||||
|
document whose key you do not hold. **Addressing presupposes the cap.** Every "is it
|
||||||
|
there?" question therefore collapses into "can I read it?", which is why absence is only
|
||||||
|
ever established behind a sync barrier (see § *Findable-without-lookup vs subscribable*)
|
||||||
|
and never by probing.
|
||||||
|
|
||||||
|
### `expose_outer` is hard-coded to `false` — VERIFIED
|
||||||
|
|
||||||
|
Both constructors of `PinRepo` — `PinRepo::for_branch` and `PinRepo::from_repo`
|
||||||
|
(`engine/net/src/actors/client/pin_repo.rs`) — set `expose_outer: false`, and they are
|
||||||
|
the only two `PinRepoV0` constructions in the tree. No parameter carries the flag up:
|
||||||
|
`expose_outer` appears nowhere under `sdk/`. The broker side is fully wired
|
||||||
|
(`RepoInfo.expose_outer: HashSet<UserId>` in `engine/broker/src/server_broker.rs`, the
|
||||||
|
`if expose_outer` branch in `rocksdb_server_storage.rs`, the outer-overlay registration
|
||||||
|
in `server_storage/core/overlay.rs`), and the `PinRepo` responder even validates the flag
|
||||||
|
(refusing `expose_outer` from a peer that publishes no topic) — but no client ever sets
|
||||||
|
it.
|
||||||
|
|
||||||
|
**Consequence for this lib:** a store's **outer** overlay is never registered broker-side,
|
||||||
|
so there is no anonymous / capability-free read surface to build on. Everything is reached
|
||||||
|
through the inner overlay, i.e. through a read cap — the same cap-first addressing as
|
||||||
|
above. The "public store readable by everyone without permission" promise in the official
|
||||||
|
docs has no client-side switch today.
|
||||||
|
|
||||||
|
### The `Ext` protocol serves blocks with no control — VERIFIED
|
||||||
|
|
||||||
|
The `ExtObjectGetV0` responder (`engine/net/src/actors/ext/get.rs`) builds
|
||||||
|
`Store::new_from_overlay_id(&req.overlay, …)` from the OverlayId the **requester
|
||||||
|
declares**, then returns `Object::load_without_header(obj_id, None, &store)` blocks for
|
||||||
|
each requested id. No authentication, no verification that the requester belongs to that
|
||||||
|
overlay. The guards that were planned exist but are dead:
|
||||||
|
|
||||||
|
- `Authorization::ExtMessage` is matched in `Broker::authorize`
|
||||||
|
(`engine/net/src/broker.rs`) and returns `AccessDenied` — but **no caller ever passes
|
||||||
|
it**; the only `authorize` call sites pass `Discover`, `Admin` or `Client`. The
|
||||||
|
server-side `StartProtocol::Ext` arm in `engine/net/src/connection.rs` goes straight to
|
||||||
|
`StepReply::Responder`, never through `authorize`.
|
||||||
|
- the config flag whose comment reads *"are ExtRequest allowed on the server? this
|
||||||
|
requires the core to be ON."* — `allow_read` in `engine/net/src/types.rs` — is declared
|
||||||
|
and defaulted to `false`, and **read nowhere**.
|
||||||
|
- `ExtRequestContentV0::get_actor` handles `WalletGetExport` and `ExtObjectGet` and falls
|
||||||
|
through to `_ => unimplemented!()` for `ExtTopicSyncReq` — a **panic reachable from an
|
||||||
|
anonymous peer**. (The commented-out `// Self::ExtTopicSyncReq(a) => a.get_actor(),` on
|
||||||
|
that arm and the `// TODO inbox requests` in the enum are *direction hints*, labelled as
|
||||||
|
such — not current behaviour.)
|
||||||
|
|
||||||
|
**Consequence for this lib:** `Ext` is not a usable read path in either direction. Blocks
|
||||||
|
come back **encrypted**, and naming them requires ObjectIds you only have once you can
|
||||||
|
already read — so it grants no capability we could build on, and confirms the shape of
|
||||||
|
everything above: confidentiality lives entirely in the keys, and holding no key means
|
||||||
|
holding no partial access, just none.
|
||||||
|
|
||||||
|
## Known open issues (section added 2026-07-18)
|
||||||
|
|
||||||
|
Live limitations observed against the current core/SDK, each with its epistemic
|
||||||
|
status. **None is treated.** The status labels below are load-bearing — do not
|
||||||
|
upgrade an OPEN / UNDETERMINED / HYPOTHESIS item to "confirmed" or "fixed"
|
||||||
|
without new evidence.
|
||||||
|
|
||||||
|
### Write loss on socket death (`SerializationError`) — symptom VERIFIED, mechanism UNSETTLED, OPEN / untreated
|
||||||
|
|
||||||
|
A write made just before an idle period / spontaneous socket death
|
||||||
|
(`SOCKET IS CLOSED Some(Left(SerializationError))`) can be **silently lost**:
|
||||||
|
the entity is absent on reconnection while the account survives. Reconnection is
|
||||||
|
an unimplemented `// TODO` stub in the core (`broker.rs`, ≈ `1051-1076`);
|
||||||
|
`disconnections_subscribe` DOES fire on the failure but nothing — neither this
|
||||||
|
polyfill nor the consumer app — consumes it; and there is **no
|
||||||
|
write-durability-confirmation API** a caller could `await`. Full post-mortem
|
||||||
|
(logs, causal chain, correction leads, none arbitrated):
|
||||||
|
[`incidents/2026-07-14-write-loss-on-disconnect.md`](./incidents/2026-07-14-write-loss-on-disconnect.md).
|
||||||
|
|
||||||
|
### Cold-start read does not rehydrate the owner's own scope from the broker — symptom VERIFIED, root cause UNDETERMINED, OPEN / untreated
|
||||||
|
|
||||||
|
Decisive test (2026-07-14): a genuinely no-local cold reader — fresh
|
||||||
|
non-persistent browser context, SAME wallet + account — reads **0** of the
|
||||||
|
owner's own scope from the broker. The previously "passing" reconnect test was
|
||||||
|
FALSE-GREEN: it read the owner's repos from the persistent profile's LOCAL
|
||||||
|
IndexedDB, so it never proved broker durability. It is UNDETERMINED whether
|
||||||
|
**(i)** the write never durably reached the broker, or **(ii)** the write IS on
|
||||||
|
the broker but a fresh session cannot re-open the owner's own scope docs (a
|
||||||
|
cold-open / rehydration limitation) — both collapse to the same 0-read in this
|
||||||
|
setup. Next step (NOT done): disambiguate (i) vs (ii) with an independent warm /
|
||||||
|
second-identity read of the same doc. The same (i)/(ii) reserve is carried in
|
||||||
|
[`incidents/2026-07-14-write-loss-on-disconnect.md`](./incidents/2026-07-14-write-loss-on-disconnect.md)
|
||||||
|
(§ *Portée & non-reproduit*), whose Firefox case leans (i) — this cold-reader
|
||||||
|
signature is distinct (no socket death) and does not settle it.
|
||||||
|
|
||||||
|
### Reactive subscription may not echo the writer's OWN local commit — HYPOTHESIS (high-confidence), confirmation in progress (2026-07-18), NOT confirmed, NOT fixed
|
||||||
|
|
||||||
|
When a client does a local `sparqlUpdate` on a doc it is itself subscribed to
|
||||||
|
(`subscribeDoc`/`doc_subscribe`), the subscription callback appears NOT to fire
|
||||||
|
for its own local commit in the same session, so the polyfill's reactive re-read
|
||||||
|
chain never runs and consumers keep a stale value until the next connection
|
||||||
|
delivers a fresh initial `State`. REMOTE commits DO push correctly (verified:
|
||||||
|
cross-browser reactive update works). Verdict pending a live instrumented run.
|
||||||
|
Full write-up (suspect link, instrumentation, planned polyfill-side fix):
|
||||||
|
[`../packages/client/docs/sdk-reference.md`](../packages/client/docs/sdk-reference.md)
|
||||||
|
§ *Current emulation status*.
|
||||||
|
|
||||||
|
### Cold-start anchored read returns 0 rows instead of an error — symptom VERIFIED, mechanism INFERRED, healed polyfill-side
|
||||||
|
|
||||||
|
On a FRESH session over the SAME persistent wallet (reconnect, new page, re-login), an
|
||||||
|
anchored `sparql_query` against a document written in an earlier session comes back with
|
||||||
|
**0 rows and no error** — persisted documents read as empty. Observed on every anchored
|
||||||
|
reader of the polyfill and healed identically in each (`ensureRepoOpen` before the read,
|
||||||
|
`packages/client/src/open-repo.ts`): the discovery index (`discovery.ts` `readIndex`),
|
||||||
|
the per-scope index (`store-registry.ts` `readScopeIndex`), the by-need doc batch
|
||||||
|
(`read-model.ts` `readUnion`), and the store-root pointer read (`store-registry.ts`
|
||||||
|
`resolvePointer`). The heal is `doc_subscribe(nuri)` → await the first `State` (the sync
|
||||||
|
barrier) → THEN the anchored read, and it is verified to return the data.
|
||||||
|
|
||||||
|
The circularity that made it self-inflicted (VERIFIED by the fix working): `doc_subscribe`
|
||||||
|
WOULD open the repo, but the reactive layer only subscribes AFTER a listing produced
|
||||||
|
NURIs, and the listing is itself an anchored read of a not-yet-open index repo → 0 rows →
|
||||||
|
nothing to subscribe → nothing ever opens.
|
||||||
|
|
||||||
|
**Mechanism INFERRED, not established.** `resolve_target_for_sparql(Repo(id))`
|
||||||
|
(`engine/verifier/src/request_processor.rs`) does
|
||||||
|
`self.repos.get(repo_id).ok_or(RepoNotFound)`, so a repo genuinely absent from
|
||||||
|
`self.repos` should ERROR, not return 0 rows. The most plausible reading of the silent 0
|
||||||
|
is that the repo IS in `self.repos` (loaded from local user storage at bootstrap) while
|
||||||
|
its named graph in `graph_dataset` is not yet populated — commits not applied/synced yet
|
||||||
|
— so the query legitimately matches nothing. Not traced end to end; the tension with the
|
||||||
|
`RepoNotFound` path described in § *A repo is only queryable once OPENED/synced into the
|
||||||
|
store* is unresolved.
|
||||||
|
|
||||||
|
**Consequence for this lib:** a cold anchored read is NOT authoritative on its own — 0
|
||||||
|
rows does not mean absent. This is what imposes the open-then-read discipline on every
|
||||||
|
cold reader, and it is why the account trust root had to move behind a first-`State`
|
||||||
|
barrier (see § *The pointer → doc-shim indirection*).
|
||||||
|
|
||||||
|
### Account fork on concurrent provision — symptom VERIFIED, guarded polyfill-side, residue persists in wallets
|
||||||
|
|
||||||
|
On a fresh page, several independent callers hit `ensureAccount(A)` near-simultaneously
|
||||||
|
(the public and protected `watchShape`, container subscriptions, the app's owned-events
|
||||||
|
effect). When the account is genuinely new, each caller sees 0 and each provisions its
|
||||||
|
own set of three scope documents — an **in-session account fork**. The persisted residue
|
||||||
|
is a single account subject carrying MULTIPLE values for one scope predicate (observed:
|
||||||
|
five `shim:docPublic`), after which a writer and a later reader can resolve DIFFERENT
|
||||||
|
scope docs and the reader's anchored read returns 0.
|
||||||
|
|
||||||
|
Two polyfill-side guards, both in `packages/client/src/store-registry.ts`: `ensureInFlight`
|
||||||
|
(a bounded promise map keyed by account, so concurrent `ensureAccount` calls share ONE
|
||||||
|
resolve-or-provision) prevents new forks; `canonicalDoc` (pick the lexicographically
|
||||||
|
smallest NURI among all distinct values for a scope predicate — NURIs are
|
||||||
|
content-addressed, so the order is total and session-independent) makes resolution
|
||||||
|
deterministic on wallets that already carry fork residue. The earlier account-level
|
||||||
|
`provisionRetry` / `resolveAccountReliably` loop is gone, replaced by the doc-shim
|
||||||
|
barrier.
|
||||||
|
|
||||||
|
**Consequence for this lib:** the underlying enabler is core-side — there is no atomic
|
||||||
|
create-if-absent, and no existence probe to settle "does this account already exist?"
|
||||||
|
(see § *No existence probe at SDK level*), so provisioning is a read-then-create race the
|
||||||
|
polyfill has to serialize itself. The guards are mitigation, not a fix: a wallet already
|
||||||
|
corrupted stays corrupted, and only `canonicalDoc` keeps it readable.
|
||||||
|
|
||||||
|
### Outbox replay aborts on an unknown topic (`REPLAY TOPIC NOT FOUND`) — VERIFIED in core, already documented as an incident
|
||||||
|
|
||||||
|
`Verifier::send_outbox` (`engine/verifier/src/verifier.rs`) walks the queued events and,
|
||||||
|
for each, looks up `self.topics.get(&(overlay, topic_id))`. On a miss it logs
|
||||||
|
`REPLAY TOPIC NOT FOUND <topic> IN OVERLAY <overlay>` and sets `need_replay`, calls
|
||||||
|
`load_from_credentials_and_outbox(&events_to_replay)`, then in the send loop does
|
||||||
|
`self.topics.get(…).ok_or(NgError::TopicNotFound)?` — the `?` **aborts the whole outbox
|
||||||
|
flush**, so the remaining queued events are not sent. There is no per-event isolation and
|
||||||
|
no signal to the caller.
|
||||||
|
|
||||||
|
Already covered — **not duplicated here**: this is the core-side mechanism behind the
|
||||||
|
symptom described in § *Write loss on socket death (`SerializationError`)* above, whose
|
||||||
|
full post-mortem (logs, causal chain, the unarbitrated (i)/(ii) reserve) is
|
||||||
|
[`incidents/2026-07-14-write-loss-on-disconnect.md`](./incidents/2026-07-14-write-loss-on-disconnect.md).
|
||||||
|
The spontaneous socket death (`SOCKET IS CLOSED Some(Left(SerializationError))`) is
|
||||||
|
likewise covered there and in that section — the only fact added here is the abort
|
||||||
|
semantics of the replay path itself (VERIFIED by reading `send_outbox`).
|
||||||
|
|
||||||
|
**Consequence for this lib:** a queued write can be dropped without any observable error,
|
||||||
|
and one unknown topic can take the rest of the queue with it. The polyfill's own
|
||||||
|
`outbox-log.ts` records write intents but cannot replay them into the core, and no
|
||||||
|
write-durability confirmation exists to await — so "the write returned" is not "the write
|
||||||
|
is durable".
|
||||||
|
|||||||
@@ -0,0 +1,147 @@
|
|||||||
|
# NextGraph's ReadCap & NURI model — and the polyfill's caps emulation
|
||||||
|
|
||||||
|
**Established 2026-07-20**, VERIFIED by direct reading of the `nextgraph-rs` Rust core (except for points marked INFERRED). The `file:line` references are dated — line numbers are volatile, navigate by symbol/regex.
|
||||||
|
|
||||||
|
Purpose: to give the ground truth of NextGraph's access-rights model, in order to align the polyfill's `caps.ts` emulation (today an ACL — the inverse of the real model). This is the basis for the item "align ReadCap/WriteCap with NextGraph".
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. A ReadCap = possession of a key, NOT a per-identity ACL
|
||||||
|
|
||||||
|
A **ReadCap is fundamentally a cryptographic key that one holds**, not an ACL entry tied to a wallet. "Whoever holds the key can read."
|
||||||
|
|
||||||
|
- Structure: `ReadCap = ObjectRef = BlockRef { id: BlockId, key: SymKey }` (`engine/repo/src/types.rs:461, 463-471, 557, 565`).
|
||||||
|
- `id: BlockId` = **BLAKE3** digest (address of the encrypted object).
|
||||||
|
- `key: SymKey = ChaCha20Key([u8;32])` = the **decryption key**.
|
||||||
|
Holding the pair → the broker serves the encrypted blocks by `id`, and one decrypts **locally** with `key`.
|
||||||
|
- Granularity: per commit/object the `ObjectRef` **is** the cap; for a branch → its defining commit; for a repo → RootBranch; for a store → the root repo's cap (`types.rs:559-565`). `ReadCapSecret` = the key half (`:567-570`).
|
||||||
|
- **There is NO read-ACL.** A repo's membership/permissions (`RootBranch`, `AddMember`, `AddPermission`) govern **writing/admin**, not reading. Reading is guarded only by key possession.
|
||||||
|
|
||||||
|
## 2. Granting read access = sealing the key to the recipient
|
||||||
|
|
||||||
|
"Grant" = delivering the cap **sealed** (`crypto_box seal`, anonymous public-key encryption) to the recipient's **inbox pubkey** — only they can open it with their private key.
|
||||||
|
|
||||||
|
- Sealed inbox message: `InboxMsgBody.msg` = `crypto_box::seal(... to_inbox ...)`, opened with the inbox secret key (`engine/net/src/types.rs:4272, 4299, 4319`).
|
||||||
|
- The payload can carry a cap: `ContactDetails.read_cap: Option<ReadCap>` ("if user wants to share the content of profile") (`net/types.rs:4232-4233`) → **directed grant** (sealed to one recipient).
|
||||||
|
- **Undirected** variant: `RepoLinkV0.read_cap` = a shareable link that **whoever receives it** can open (`net/types.rs:5061-5078`).
|
||||||
|
|
||||||
|
So "wallet targeting" lives in the **sealing envelope**, not in the cap: the cap remains `{id, key}`, possession-based.
|
||||||
|
|
||||||
|
> **Current state (2026-07-27) — the path is a GAP, not a disagreement.** The `ContactDetails.read_cap` field exists, but the construction of the message is `unimplemented!()` (its only caller passes "without read_cap") and the receiver **discards** the cap it would receive. The *shape* is therefore the right one; the implementation is not there. The polyfill emulates it in the meantime — filed in the bug-inbox.
|
||||||
|
|
||||||
|
## 3. Revocation = re-key (coarse, non-retroactive)
|
||||||
|
|
||||||
|
A delivered key is not "taken back". To revoke = **re-encrypt** with a new key and re-seal it only to the remaining authorized holders.
|
||||||
|
|
||||||
|
- "Capabilities are not durable: they can be refreshed by members and previously shared Caps become obsolete/revoked… if [a member] doesn't subscribe, they lose access after the refresh" (`net/types.rs:5055-5058`).
|
||||||
|
- Mechanism: `RootCapRefresh` / `BranchCapRefresh` (`repo/src/commit.rs:616,630`; perms `types.rs:1748-1749`).
|
||||||
|
- Consequences: **coarse** (repo/branch scale), **non-retroactive** (what was read before remains known to the former holder; they only decrypt the versions **prior to** the refresh).
|
||||||
|
- **Durable** delivery of a cap = `PermaCap` — still **TODO** (`repo/types.rs:578`).
|
||||||
|
|
||||||
|
### DIRECTION — rotation does NOT cause access to be lost (confirmed by the PO, 2026-07-27)
|
||||||
|
|
||||||
|
**Do not read the comment above as the intent.** "*if they don't subscribe, they lose access after the refresh*" describes **the current state**, not the target. What NextGraph is aiming for:
|
||||||
|
|
||||||
|
> When a key is rotated, the new one is **sent to the inbox** of the users who retain the access right. That inbox is **processed automatically** as soon as one of the user's clients connects.
|
||||||
|
|
||||||
|
So access is **not lost**, it is **deferred** until the next connection — consistent with local-first. Shape consequences: **no subscription obligation** to expose to the consumer; a re-delivery takes **the same channel** as the initial delivery, so the sharing mechanism covers both with no special case. **Revocation** remains "stop re-delivering", non-retroactive.
|
||||||
|
|
||||||
|
## 4. NURI grammar: cap-less vs cap-bearing (the `:k:` segment)
|
||||||
|
|
||||||
|
**Clearing up the confusion first**: `did:ng:` is **not** a "cap-less" marker, it is the **URI scheme prefix** — present everywhere (inbox `did:ng:d:…`, branch `did:ng:b:…`, overlay `did:ng:v:…`, document `did:ng:o:…`). A NURI **is** a `did:ng:…`. So there is no "the did" on one side and "the NURI" on the other: it is **a single object**, with or without the key inside it — a single type upstream, `NuriV0 { target, access }`, where a cap-less NURI simply has an empty `access`.
|
||||||
|
|
||||||
|
The discriminant is the **`:k:{key}`** segment: present = cap-bearing; **absent = cap-less** (names/locates **without** granting the right to read). This is **first-class** in the type: `NuriV0.target` (ids) and `access`/`objects` (the cap) are **separate fields** — an id-only NURI parses with `access: vec![]` (`engine/net/src/app_protocol.rs:53-62, 99-118, 181-195, 659-677`).
|
||||||
|
|
||||||
|
**Cap-less** (id + optional overlay, no key) — formatters in `app_protocol.rs`, regexes in `net/types.rs`:
|
||||||
|
- `did:ng:o:{repo_id}` (`:315`, `RE_REPO_O` types.rs:52)
|
||||||
|
- `did:ng:o:{repo_id}:v:{overlay_id}` (`:263`, `RE_REPO` types.rs:55)
|
||||||
|
- `did:ng:o:{repo_id}:v:{overlay_id}:b:{branch_id}` (`RE_BRANCH` types.rs:58)
|
||||||
|
- `did:ng:o:{repo_id}:c:{commit_id}` (`:355`)
|
||||||
|
- `did:ng:b:{branch}` / `h:{topic}` / `v:{overlay}` / `d:{inbox}` (`:327,323,319,359`)
|
||||||
|
|
||||||
|
**Cap-bearing** (embeds the key):
|
||||||
|
- `did:ng:j:{id}:k:{key}` — object/file read cap (`repo/types.rs:511`, `RE_FILE_READ_CAP` types.rs:49)
|
||||||
|
- `did:ng:o:{repo}:c:{commit}:k:{key}` (`RE_COMMIT` types.rs:73)
|
||||||
|
- list `RE_OBJECTS` `…:[cj]:{id}:k:{key}…:l:{locator}` (types.rs:64)
|
||||||
|
|
||||||
|
The `:v:` segment is the **overlay**, which has its own section below — it is the point with the heaviest consequences for anonymous-presence models.
|
||||||
|
|
||||||
|
## 4bis. The overlay is the network space of a STORE — never of a document
|
||||||
|
|
||||||
|
**The overlay is a store's unit of network addressing.** At the broker, blocks are filed under a `(overlay, block_id)` key, and peers synchronize *within* an overlay. Two forms per store:
|
||||||
|
|
||||||
|
| | Derivation | Who can compute it |
|
||||||
|
|---|---|---|
|
||||||
|
| **outer** | `OverlayId::outer(store_id)` = **public** BLAKE3 | everyone (the store_id is enough) |
|
||||||
|
| **inner** | `OverlayId::inner(store_id, readcap_secret)` = **keyed** BLAKE3 | only whoever holds the store's read key |
|
||||||
|
|
||||||
|
Consistent with the rest of the model: no role and no list, only "do you hold the key that lets you derive this identifier". `outer` = the store's public name, `inner` = its private name.
|
||||||
|
|
||||||
|
**The `:v:` of a DOCUMENT NURI carries the overlay of its STORE** (VERIFIED, chain read end to end): `NuriV0::repo_graph_name(repo_id, overlay_id)` formats `o:{repo_id}:v:{overlay_id}`; in `doc_create` the value injected is `store.outer_overlay()` — the **containing** store, never the `repo_id`. A `Repo` carries **no** overlay field (only `store: Arc<Store>`); it is `Store` that carries `overlay_id`. **Mechanical counter-proof**: in `Store`, `get`/`put`/`del`/`has` all pass `&self.overlay_id` to the block storage — every document of a store shares the same block namespace, so a per-document overlay is structurally impossible.
|
||||||
|
|
||||||
|
### The consequence to know about: the `:v:` is a stable pseudonym
|
||||||
|
|
||||||
|
**All of one person's documents in their protected store carry the SAME `:v:`** = `outer(protected_store_id)`. So a cap-less reference — precisely the one used to "name without granting read" — **exposes store membership**, that is to say a **stable and permanent pseudonymous identifier of the person**. The store_id itself does not leak (BLAKE3 is not invertible), so it does not say *who*; but it is a **constant handle**, the same everywhere and forever, correlatable by anyone who collects cap-less references.
|
||||||
|
|
||||||
|
**The coupling that results, and that constrains any anonymous-presence model**: that same `:v:` is *simultaneously* (a) what makes it possible to **deduplicate** references without reading them — two references with the same `:v:` come from the same person — and (b) what makes it possible to **track** that person from one context to another. **It is the same bit of information.** You cannot get the dedup without conceding the tracking, nor remove the tracking without losing the dedup — short of changing how the stores are carved up, which moves the cursor but does not remove the trade-off.
|
||||||
|
|
||||||
|
*Nuances.* The NURI's `:v:` is the **outer** overlay, whereas client↔broker traffic and local storage use the **inner** one — a different value, but derived from the store as well, so the property holds in both cases. A `Dialog` store returns an `Inner`, still store-scoped.
|
||||||
|
|
||||||
|
**CORRECTED on 2026-07-27 — this hypothesis was FALSE.** We had inferred, then believed we had verified, that a holder **without a key** could fetch the encrypted blocks and therefore prove a document's **existence**. An adversarial review showed that the reasoning stopped at *access control* without looking at **addressing**:
|
||||||
|
|
||||||
|
- There is **no existence command at the SDK level**.
|
||||||
|
- The only probe (`BlocksExist`) is **internal to the crate**, requires `BlockId`s **and** an already **loaded** repo, and addresses the **inner** overlay — which is derived from the **read secret**.
|
||||||
|
- A cap-less reference carries a RepoId and the **outer** overlay: no `BlockId` to probe. And the outer is never registered anyway (`expose_outer` hard-coded to `false`, with no SDK parameter).
|
||||||
|
- The only primitive accessible to a non-member (`ExtObjectGet`) requires the ObjectIds **and their keys**.
|
||||||
|
|
||||||
|
> **Addressing itself presupposes the cap.** Proving a document's existence without holding its key is not constructible today, and nothing indicates that it is planned.
|
||||||
|
|
||||||
|
Transferable lesson: verifying that an access guard **lets you through** does not prove that an operation is reachable — you still have to be able to **name** what you are asking for.
|
||||||
|
|
||||||
|
## 4ter. The public store: readable by URL, and NOT recursive
|
||||||
|
|
||||||
|
Target principle (confirmed by the PO, 2026-07-27):
|
||||||
|
|
||||||
|
> **An element of the public store is public: whoever has the URL reads the content.**
|
||||||
|
> But **not recursively** — public content can *reference* private content, and the reference does **not** give access to the referenced.
|
||||||
|
|
||||||
|
This is a **second mechanism**, alongside key possession (§1) — not a breach of it. And it is the **non-recursiveness** that carries the value: it allows a public object that **points** to private identity, without divulging it. That is exactly the pattern an anonymous-presence model needs.
|
||||||
|
|
||||||
|
*Implementation detail, NOT to be carried by the shape*: NextGraph is moving toward **not encrypting** the content of the public store (the data remaining **signed**). A surface must not depend on it. And if the public store does not behave the way this principle describes, it is **the polyfill** that adapts, not the consumer.
|
||||||
|
|
||||||
|
## 4quater. The keyring: where the owner gets the caps for THEIR OWN documents
|
||||||
|
|
||||||
|
On every document creation, an `AddRepo { read_cap }` is committed to a **store branch** — the store being itself a repo, endowed with **typed** branches (the word "branch" has nothing to do with git: it is a compartment with a defined role). That branch lists **the store's documents, each with its read key**.
|
||||||
|
|
||||||
|
So it **is** the **owner's keyring**: the mechanism by which they find the caps of their own documents. Upstream of that, the keyring is the **wallet**.
|
||||||
|
|
||||||
|
**This is NOT the sharing mechanism.** An easy and costly confusion: concluding "we share at the store level" is wrong — delivering a store cap would give access to **all** of its content, present and future. **The unit of sharing is the document** (§2). The keyring is a private index, not an act of sharing.
|
||||||
|
|
||||||
|
*(VERIFIED for the `AddRepo { read_cap }` mechanism; the **exact name** of the branches and the enumeration of their types have not been re-traced — to be confirmed if this point becomes load-bearing.)*
|
||||||
|
|
||||||
|
## 5. What the polyfill emulates (caps.ts) — and where it diverges
|
||||||
|
|
||||||
|
`packages/client/src/caps.ts` models `readers: Map<Nuri, Set<PrincipalId>>` + `grantRead(doc, grantee)` (`:29-30, 41-42`) — **a per-document ACL of principals, that is the exact INVERSION of the real model** (key). Divergences:
|
||||||
|
|
||||||
|
| | Real NextGraph | caps.ts emulation |
|
||||||
|
|---|---|---|
|
||||||
|
| Nature | possession of a **key** | **ACL** (set of principals) |
|
||||||
|
| Grant | seal the key (crypto_box) to the inbox | add a principal to the set |
|
||||||
|
| Durability | **durable** (key delivered once) | **ephemeral** (Map empty every session → re-declared) |
|
||||||
|
| Revocation | coarse **re-key**, non-retroactive | removal from the set: **instantaneous and total** |
|
||||||
|
| Granularity | repo / branch / commit / object | **one cap per doc-NURI** |
|
||||||
|
| Ref. without rights | **cap-less NURI** (no `:k:`) | no such notion (the ACL says who may) |
|
||||||
|
|
||||||
|
**App-facing**: `declareConnections` (on the consumer side), which re-declares "my connections read my protected entities" **every session**, is an **artifact of this ephemeral ACL** — moot in the real model (there the seals are durable; one seals per-doc at share time, not per-session).
|
||||||
|
|
||||||
|
## 6. Implications for consumers (e.g. Festipod)
|
||||||
|
|
||||||
|
- "**protected scope = my network can read**" is **not** an ACL checked by the broker: it is "I have **sealed my read key** to each of my connections". The "scope = ACL" mental model is wrong at the NextGraph level.
|
||||||
|
- **Anonymous references are possible**: putting a **cap-less NURI** in a third party's collection lets that third party **name/count** without **reading the identity**; the cap-bearing one is sealed separately to the authorized parties only. (Basis for a presence model of the form "self-owned participation + curated cap-less Set + cap sealed to the connections".)
|
||||||
|
- **Alignment to do**: when the real cap operations become available, replace the emulated ACL with durable per-doc key sealing, and `declareConnections`-as-a-re-declared-ACL disappears.
|
||||||
|
|
||||||
|
## Caveats / gaps
|
||||||
|
|
||||||
|
- `file:line` references are dated (2026-07) — re-verify by symbol; the core moves.
|
||||||
|
- ~~INFERRED: keyless broker fetch (existence without a key)~~ — **RESOLVED and REFUTED, 2026-07-27**: not constructible. See the CORRECTED block in §4bis. Kept struck through because the hypothesis is intuitive and will otherwise be re-formed.
|
||||||
|
- Not traced: the full execution of `RootCapRefresh` on the verifier side (`verifier/src/commits/mod.rs:616`), wallet storage of `private_store_read_cap` (`repo/types.rs:945,976`).
|
||||||
@@ -0,0 +1,32 @@
|
|||||||
|
# Vision & principles of the `@ng-eventually/client` polyfill
|
||||||
|
|
||||||
|
## Purpose
|
||||||
|
|
||||||
|
A **stand-in faithful in SHAPE** to NextGraph's future primitives. **Single** objective: that consumers (Festipod) be **coded against the CORRECT mental model** — the one of finished NextGraph — and have **NOTHING to rewrite** when NextGraph provides the real primitives.
|
||||||
|
|
||||||
|
## What the polyfill is NOT
|
||||||
|
|
||||||
|
A **security** layer. The **shared wallet** (everyone shares the same keys) plus the absence of real crypto make the emulation **infinitely less secure** than a wallet-per-user — it is a **dev/staging vehicle**, not a goal. **Insecurity is ACCEPTED.** An attacker who bypasses the emulation is not our problem.
|
||||||
|
|
||||||
|
## The only criterion: shape-fidelity, with RIGOR
|
||||||
|
|
||||||
|
The **exposed surfaces** must match the **exact SHAPE** of the future primitives, **even where enforcement is simulated**. The **failure mode to avoid**: exposing the **wrong shape** → the consumer codes against a model that will not exist → rewrite. The **ACL** inversion of ReadCaps was exactly that defect (an ACL where the real thing is **key possession**) — a lack of rigor.
|
||||||
|
|
||||||
|
## Simulating crypto to PREVENT shortcuts
|
||||||
|
|
||||||
|
Without a minimum of crypto simulation, damaging shortcuts get taken (reading the plaintext, falling back on ACLs). The polyfill therefore **simulates** the final mechanism, enough to hold this **invariant**:
|
||||||
|
|
||||||
|
> **A `did` (bare id, WITHOUT a ReadCap) and a NURI (WITH a ReadCap) are treated GENUINELY differently: the former does NOT allow reading the data; the latter is SUFFICIENT and REQUIRED.**
|
||||||
|
|
||||||
|
Concretely: a document's data is **stored encrypted** (per-doc symmetric encryption, however lightweight); the **ReadCap = the key**; without it, **decrypting/reading is impossible**. No ACL, no plaintext accessible "on the side". Obtaining read access = **holding the key**, exactly as in the target model.
|
||||||
|
|
||||||
|
## Shape consequences (to respect everywhere)
|
||||||
|
|
||||||
|
- **Everything is keys and URLs.** There is **no** notion of membership, role, or authorization list in the model: only symmetric and asymmetric cryptography, URIs, and who holds which key. Any exposed shape that looks like an ACL, a `member`, a `role`, or a `permission` is a **wrong shape**, whatever scaffolding one may otherwise read in the current state of NextGraph.
|
||||||
|
- **Reading = possession of the read key** (ReadCap = `{id, key}`). A bare id (a `did` without a ReadCap) does not read.
|
||||||
|
- **Writing = possession of the write key** — a key **distinct** from the read key, hence a distinct axis, but **possession too**.
|
||||||
|
- **Sharing a cap = sealing it to a recipient** (**durable** delivery, at share time — NOT an ACL re-declared every session).
|
||||||
|
- **Revocation = re-key** (new key; former holders keep the old state). Non-retroactive.
|
||||||
|
- **Cap-less reference** (naming/pointing without reading) **distinct** from the cap-bearing reference.
|
||||||
|
|
||||||
|
See `readcap-and-nuri-model.md` (the real model, verified in `nextgraph-rs`) and `briefs/2026-07-20-caps-emulation-alignment.md` (the alignment effort).
|
||||||
@@ -244,7 +244,7 @@ helpers live in the consumer app; the SDK exposes the generic reactive/by-need r
|
|||||||
> [`read-model.md`](../../../docs/read-model.md),
|
> [`read-model.md`](../../../docs/read-model.md),
|
||||||
> [`simulation.md`](../../../docs/simulation.md).
|
> [`simulation.md`](../../../docs/simulation.md).
|
||||||
|
|
||||||
Today, on a single shared wallet emulating the mature platform, three gaps diverge
|
Today, on a single shared wallet emulating the mature platform, four gaps diverge
|
||||||
from the reactive contract:
|
from the reactive contract:
|
||||||
|
|
||||||
1. **Entity-list reads are one-shot, not reactive.** The reactive ORM cannot be used
|
1. **Entity-list reads are one-shot, not reactive.** The reactive ORM cannot be used
|
||||||
@@ -279,5 +279,38 @@ from the reactive contract:
|
|||||||
queryable. At the multi-store migration, opening a repo by cap becomes a native
|
queryable. At the multi-store migration, opening a repo by cap becomes a native
|
||||||
broker sync and the anchored read is unchanged.
|
broker sync and the anchored read is unchanged.
|
||||||
|
|
||||||
|
4. **The subscription may not echo the writer's OWN local commit — HYPOTHESIS
|
||||||
|
(high-confidence), confirmation in progress (2026-07-18); NOT confirmed, NOT
|
||||||
|
fixed.** Unlike gaps 1–3 (designed emulation stopgaps), this is a suspected
|
||||||
|
defect in the polyfill's own reactive assembly. When a client does a local
|
||||||
|
`sparqlUpdate` on a doc it is itself subscribed to (`subscribeDoc` /
|
||||||
|
`ng.doc_subscribe`), the subscription callback appears NOT to fire for its OWN
|
||||||
|
local commit in the same session — so the reactive re-read chain
|
||||||
|
([`../src/watch-shape.ts`](../src/watch-shape.ts) `watchShape` → `reread` →
|
||||||
|
[`../src/read-model.ts`](../src/read-model.ts) `readUnion`) never runs, and
|
||||||
|
consumers keep the STALE value until the next connection delivers a fresh
|
||||||
|
initial `State`. **Remote** commits DO push correctly (verified: cross-browser
|
||||||
|
reactive update works). A code review verified the consumer wiring is correct,
|
||||||
|
the doc IS in the subscribed set, and a triggered re-read WOULD return the new
|
||||||
|
value — leaving the self-commit echo as the only suspect link. That link is
|
||||||
|
**INFERRED**, not observed: the real `ng.doc_subscribe` runtime is not readable
|
||||||
|
from source, and [`../src/subscribe.ts`](../src/subscribe.ts)'s own doc-comment
|
||||||
|
CLAIMS local writes push a `Patch` — contradicted by the observation. (This
|
||||||
|
also sits in tension with § *The reactivity model* above, which documents the
|
||||||
|
target contract — one commit, every subscriber pushed, local or remote.) The
|
||||||
|
requirement at stake is multi-user: a value change (e.g. a participant count)
|
||||||
|
must propagate reactively to ALL viewers — other viewers (remote push, which
|
||||||
|
works) AND the writer's own view (this suspect link). **Treatment (PLANNED,
|
||||||
|
not done):** confirm first via the temporary instrumentation just added
|
||||||
|
([`../src/subscribe.ts`](../src/subscribe.ts) ≈`:119` logs
|
||||||
|
`doc_subscribe FIRE <nuri> (State|Patch)`;
|
||||||
|
[`../src/watch-shape.ts`](../src/watch-shape.ts) ≈`:341` logs
|
||||||
|
`reread TRIGGER by <nuri>` — line numbers volatile, grep the log strings);
|
||||||
|
then, IF confirmed, fix **polyfill-side** — a
|
||||||
|
local commit should notify the doc's active `subscribeDoc` callbacks.
|
||||||
|
Consumers must not compensate. Short entry:
|
||||||
|
[`nextgraph-current-state.md`](../../../docs/nextgraph-current-state.md) §
|
||||||
|
*Known open issues*.
|
||||||
|
|
||||||
When these gaps close, the read path collapses to the reference above: `useShape`
|
When these gaps close, the read path collapses to the reference above: `useShape`
|
||||||
everywhere, push everywhere, no polling and no re-query-on-signal assembly.
|
everywhere, push everywhere, no polling and no re-query-on-signal assembly.
|
||||||
|
|||||||
@@ -0,0 +1,280 @@
|
|||||||
|
/**
|
||||||
|
* DECISIVE real-broker determination: does `doc_subscribe` actually PUSH when a
|
||||||
|
* subscribed document is written?
|
||||||
|
*
|
||||||
|
* This is the reactive-layer coverage whose ABSENCE let a reactivity bug ship: the
|
||||||
|
* app's whole read-model reactivity rests on `subscribeDoc(nuri, cb)` (the polyfill
|
||||||
|
* wrapper over `ng.doc_subscribe`, `src/subscribe.ts`) firing `cb` again on every
|
||||||
|
* commit to the doc. Two pushes are load-bearing in production and were reported as
|
||||||
|
* NOT firing:
|
||||||
|
* (SELF) a session's own `sparqlUpdate` to a doc it subscribes to.
|
||||||
|
* (CROSS) another session writes to a doc the first session subscribes to.
|
||||||
|
*
|
||||||
|
* This runner exercises BOTH against the REAL broker, through the SAME public
|
||||||
|
* surface the app uses — `subscribeDoc` (via the harness's `stateProbe*` bridge,
|
||||||
|
* which passes the raw `AppResponse` straight through the polyfill wrapper),
|
||||||
|
* `docs.docCreate`, and `docs.sparqlUpdate` (`writeTo`). It records EVERY push as a
|
||||||
|
* typed event (`{ typeKey: "State" | "Patch" | "TabInfo" | …, elapsedMs }`) so the
|
||||||
|
* verdict is the ground truth "did the subscription callback fire again", not a
|
||||||
|
* re-read of the document. Each wait is a single event-driven promise+timeout on the
|
||||||
|
* push (NO re-read loop) — a timeout is a DEFINITE "did-not-fire", not a flaky miss.
|
||||||
|
*
|
||||||
|
* Standalone (NOT `bun test`). Run:
|
||||||
|
* bun run e2e/reactivity-doc-subscribe.ts
|
||||||
|
* (or `bun run test:e2e:reactivity` from packages/client)
|
||||||
|
*
|
||||||
|
* It reuses the exact real-broker plumbing of run.ts / broker.ts: the dedicated lib
|
||||||
|
* wallet, the broker iframe, `window.__sdk`. The CROSS case opens a SECOND page on
|
||||||
|
* the SAME persistent wallet context — a second concurrent verifier session on one
|
||||||
|
* shared wallet (as faithfulReconnect does) — and writes from it.
|
||||||
|
*/
|
||||||
|
|
||||||
|
import type { Frame, Page, BrowserContext } from "playwright";
|
||||||
|
import {
|
||||||
|
buildBundle,
|
||||||
|
serveHarness,
|
||||||
|
ensureWallet,
|
||||||
|
launchWalletContext,
|
||||||
|
setupBrokerPage,
|
||||||
|
} from "./broker";
|
||||||
|
|
||||||
|
type Check = { name: string; ok: boolean; detail?: string };
|
||||||
|
const results: Check[] = [];
|
||||||
|
function record(name: string, ok: boolean, detail?: string): void {
|
||||||
|
results.push({ name, ok, detail });
|
||||||
|
console.log(` [${ok ? "PASS" : "FAIL"}] ${name}${detail ? " — " + detail : ""}`);
|
||||||
|
}
|
||||||
|
|
||||||
|
type Event = { typeKey: string; elapsedMs: number };
|
||||||
|
|
||||||
|
// Call a bridge method inside a given iframe.
|
||||||
|
function sdk<T>(frame: Frame, method: string, ...args: unknown[]): Promise<T> {
|
||||||
|
return frame.evaluate(
|
||||||
|
([m, a]) => (window as any).__sdk[m as string](...(a as unknown[])),
|
||||||
|
[method, args] as const,
|
||||||
|
) as Promise<T>;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The decisive wait: resolve TRUE as soon as the probe's recorded push count grows
|
||||||
|
* past `base` (the subscription callback fired again), or FALSE on timeout. This is
|
||||||
|
* a promise+timeout on the PUSH itself — it polls only the in-memory event counter
|
||||||
|
* the `subscribeDoc` callback writes, NEVER re-reads the document. A FALSE here is a
|
||||||
|
* definite non-delivery within the window, not a missed re-read.
|
||||||
|
*/
|
||||||
|
async function waitForPush(frame: Frame, base: number, timeoutMs: number): Promise<boolean> {
|
||||||
|
try {
|
||||||
|
await frame.waitForFunction(
|
||||||
|
(b) => (window as any).__sdk.stateProbeEvents().length > (b as number),
|
||||||
|
base,
|
||||||
|
{ timeout: timeoutMs },
|
||||||
|
);
|
||||||
|
return true;
|
||||||
|
} catch {
|
||||||
|
return false; // timed out → the callback did NOT fire again within the window
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
const seq = (events: Event[]): string =>
|
||||||
|
events.length ? events.map((e) => `${e.typeKey}@${e.elapsedMs}ms`).join(" → ") : "(none)";
|
||||||
|
|
||||||
|
async function openSession(
|
||||||
|
ctx: BrowserContext,
|
||||||
|
url: string,
|
||||||
|
tag: string,
|
||||||
|
): Promise<{ page: Page; frame: Frame; sessionId: string }> {
|
||||||
|
const page = await ctx.newPage();
|
||||||
|
page.on("pageerror", (e) => console.error(`[iframe error:${tag}]`, e.message));
|
||||||
|
page.on("console", (m) => {
|
||||||
|
const t = m.text();
|
||||||
|
// Surface the polyfill's own "doc_subscribe FIRE" diagnostic (subscribe.ts) if
|
||||||
|
// access logging happens to be on — an independent confirmation of a push.
|
||||||
|
if (m.type() === "error") console.error(`[iframe console:${tag}]`, t);
|
||||||
|
else if (t.includes("doc_subscribe FIRE")) console.log(`[${tag}] ${t}`);
|
||||||
|
});
|
||||||
|
const frame = await setupBrokerPage(page, url);
|
||||||
|
await frame.waitForFunction(() => (window as any).__sdk !== undefined, { timeout: 30000 });
|
||||||
|
await frame.waitForFunction(() => (window as any).__sdk.status() === "connected", {
|
||||||
|
timeout: 60000,
|
||||||
|
});
|
||||||
|
const info = await sdk<{ session_id: string } | null>(frame, "sessionInfo");
|
||||||
|
const sessionId = info?.session_id ?? "(none)";
|
||||||
|
console.log(`[session:${tag}] connected — session_id=${sessionId}`);
|
||||||
|
return { page, frame, sessionId };
|
||||||
|
}
|
||||||
|
|
||||||
|
const SELF_TIMEOUT_MS = 10000;
|
||||||
|
const CROSS_TIMEOUT_MS = 15000;
|
||||||
|
const STATE_TIMEOUT_MS = 20000;
|
||||||
|
|
||||||
|
async function main(): Promise<void> {
|
||||||
|
console.log("[reactivity] building SDK page bundle...");
|
||||||
|
buildBundle();
|
||||||
|
console.log("[reactivity] ensuring dedicated lib wallet...");
|
||||||
|
await ensureWallet();
|
||||||
|
const { url, close: closeServer } = await serveHarness();
|
||||||
|
console.log(`[reactivity] harness served at ${url}`);
|
||||||
|
|
||||||
|
let ctx: BrowserContext | null = null;
|
||||||
|
try {
|
||||||
|
ctx = await launchWalletContext();
|
||||||
|
|
||||||
|
// ── Session A (the subscriber for both cases) ────────────────────────────
|
||||||
|
const A = await openSession(ctx, url, "A");
|
||||||
|
|
||||||
|
// ════════════════════════════════════════════════════════════════════════
|
||||||
|
// CASE 1 — SELF: A subscribes to D, then A itself writes to D.
|
||||||
|
// ════════════════════════════════════════════════════════════════════════
|
||||||
|
console.log("\n── CASE 1: SELF (single session — own write to own subscribed doc) ──");
|
||||||
|
{
|
||||||
|
const doc = await sdk<string>(A.frame, "docCreate");
|
||||||
|
console.log(` [SELF] created doc D = ${doc}`);
|
||||||
|
await sdk(A.frame, "stateProbeSubscribe", doc);
|
||||||
|
|
||||||
|
// Wait for the initial State (the sync barrier). TabInfo may precede it.
|
||||||
|
const gotState = await (async () => {
|
||||||
|
try {
|
||||||
|
await A.frame.waitForFunction(
|
||||||
|
() => (window as any).__sdk.stateProbeStateCount() >= 1,
|
||||||
|
{ timeout: STATE_TIMEOUT_MS },
|
||||||
|
);
|
||||||
|
return true;
|
||||||
|
} catch {
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
})();
|
||||||
|
const afterSubscribe = await sdk<Event[]>(A.frame, "stateProbeEvents");
|
||||||
|
console.log(` [SELF] pushes after subscribe: ${seq(afterSubscribe)}`);
|
||||||
|
record(
|
||||||
|
"SELF: initial State push arrives on subscribe (baseline sanity)",
|
||||||
|
gotState && afterSubscribe.some((e) => e.typeKey === "State"),
|
||||||
|
`sequence=${seq(afterSubscribe)}`,
|
||||||
|
);
|
||||||
|
|
||||||
|
// Now the decisive write: A's OWN sparqlUpdate to D.
|
||||||
|
const preWrite = afterSubscribe.length;
|
||||||
|
console.log(` [SELF] A writes to D (own sparqlUpdate); waiting ≤${SELF_TIMEOUT_MS}ms for a push…`);
|
||||||
|
await sdk(A.frame, "writeTo", doc, "self-1");
|
||||||
|
const fired = await waitForPush(A.frame, preWrite, SELF_TIMEOUT_MS);
|
||||||
|
|
||||||
|
const afterWrite = await sdk<Event[]>(A.frame, "stateProbeEvents");
|
||||||
|
const newEvents = afterWrite.slice(preWrite);
|
||||||
|
console.log(` [SELF] pushes AFTER own write: ${seq(newEvents)}`);
|
||||||
|
console.log(` [SELF] VERDICT: callback ${fired ? "FIRED" : "did NOT fire"} within ${SELF_TIMEOUT_MS}ms`);
|
||||||
|
record(
|
||||||
|
`SELF: subscription callback fires on the session's OWN write (≤${SELF_TIMEOUT_MS}ms)`,
|
||||||
|
fired,
|
||||||
|
`newPushes=${seq(newEvents)}`,
|
||||||
|
);
|
||||||
|
await sdk(A.frame, "stateProbeStop");
|
||||||
|
}
|
||||||
|
|
||||||
|
// ════════════════════════════════════════════════════════════════════════
|
||||||
|
// CASE 2 — CROSS-SESSION: A subscribes to D2; a SECOND session B (same shared
|
||||||
|
// wallet, own concurrent verifier session) writes to D2.
|
||||||
|
// ════════════════════════════════════════════════════════════════════════
|
||||||
|
console.log("\n── CASE 2: CROSS-SESSION (session B writes to a doc session A subscribes to) ──");
|
||||||
|
let B: { page: Page; frame: Frame; sessionId: string } | null = null;
|
||||||
|
try {
|
||||||
|
B = await openSession(ctx, url, "B");
|
||||||
|
} catch (e: any) {
|
||||||
|
console.log(` [CROSS] COULD-NOT-TEST: second concurrent session on the shared wallet failed to open: ${String(e?.message ?? e)}`);
|
||||||
|
record(
|
||||||
|
"CROSS: second concurrent session opened on the shared wallet",
|
||||||
|
false,
|
||||||
|
`open failed: ${String(e?.message ?? e)} — see Festipod multibrowser harness as the alternative venue`,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (B) {
|
||||||
|
// NB: `session_id` is a PER-PAGE local verifier counter (each fresh iframe
|
||||||
|
// numbers its first session "1"), so it is NOT a global identifier and cannot
|
||||||
|
// be used to prove distinctness. The REAL proof that A and B are two separate
|
||||||
|
// verifier sessions is behavioural: B's write reaches A only after a broker
|
||||||
|
// round-trip (a delayed Patch), not as an instant same-session echo.
|
||||||
|
console.log(
|
||||||
|
` [CROSS] both pages connected — A.session=${A.sessionId} B.session=${B.sessionId} (per-page local counter; distinctness shown by the cross-broker propagation below)`,
|
||||||
|
);
|
||||||
|
record(
|
||||||
|
"CROSS: a second concurrent page/session is open on the same shared wallet",
|
||||||
|
true,
|
||||||
|
`A=${A.sessionId} B=${B.sessionId} (session_id is a per-page counter, not a global id)`,
|
||||||
|
);
|
||||||
|
|
||||||
|
// A creates D2 and subscribes.
|
||||||
|
const doc2 = await sdk<string>(A.frame, "docCreate");
|
||||||
|
console.log(` [CROSS] A created doc D2 = ${doc2}`);
|
||||||
|
await sdk(A.frame, "stateProbeSubscribe", doc2);
|
||||||
|
const gotState2 = await (async () => {
|
||||||
|
try {
|
||||||
|
await A.frame.waitForFunction(
|
||||||
|
() => (window as any).__sdk.stateProbeStateCount() >= 1,
|
||||||
|
{ timeout: STATE_TIMEOUT_MS },
|
||||||
|
);
|
||||||
|
return true;
|
||||||
|
} catch {
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
})();
|
||||||
|
const afterSub2 = await sdk<Event[]>(A.frame, "stateProbeEvents");
|
||||||
|
console.log(` [CROSS] A pushes after subscribe: ${seq(afterSub2)}`);
|
||||||
|
record(
|
||||||
|
"CROSS: A receives its initial State on D2 (baseline sanity)",
|
||||||
|
gotState2 && afterSub2.some((e) => e.typeKey === "State"),
|
||||||
|
`sequence=${seq(afterSub2)}`,
|
||||||
|
);
|
||||||
|
|
||||||
|
// B writes to D2. Capture a write failure (e.g. RepoNotFound) explicitly —
|
||||||
|
// it would mean B cannot reach A's doc, which is itself a determination.
|
||||||
|
const preCross = afterSub2.length;
|
||||||
|
let writeThrew: string | null = null;
|
||||||
|
// Cross-session writes to a doc created by ANOTHER session can be slow: B must
|
||||||
|
// sync/open D2's repo before it can commit. Time it separately so the push
|
||||||
|
// latency is reported relative to when B's write actually LANDED, not to
|
||||||
|
// subscribe time.
|
||||||
|
console.log(` [CROSS] B writes to D2 from its own session…`);
|
||||||
|
const tWriteStart = Date.now();
|
||||||
|
try {
|
||||||
|
await sdk(B.frame, "writeTo", doc2, "cross-1");
|
||||||
|
} catch (e: any) {
|
||||||
|
writeThrew = String(e?.message ?? e);
|
||||||
|
console.log(` [CROSS] B's write THREW: ${writeThrew}`);
|
||||||
|
}
|
||||||
|
const writeMs = Date.now() - tWriteStart;
|
||||||
|
record("CROSS: session B's write to D2 did not throw", writeThrew === null, writeThrew ? writeThrew : `landed in ${writeMs}ms`);
|
||||||
|
|
||||||
|
console.log(` [CROSS] B's write returned in ${writeMs}ms; now waiting ≤${CROSS_TIMEOUT_MS}ms for A's push…`);
|
||||||
|
const tWaitStart = Date.now();
|
||||||
|
const crossFired = writeThrew ? false : await waitForPush(A.frame, preCross, CROSS_TIMEOUT_MS);
|
||||||
|
const pushAfterWriteMs = Date.now() - tWaitStart;
|
||||||
|
const afterCross = await sdk<Event[]>(A.frame, "stateProbeEvents");
|
||||||
|
const crossNew = afterCross.slice(preCross);
|
||||||
|
console.log(` [CROSS] A pushes AFTER B's write: ${seq(crossNew)}`);
|
||||||
|
console.log(
|
||||||
|
` [CROSS] VERDICT: A's callback ${crossFired ? `FIRED (${pushAfterWriteMs}ms after B's write landed)` : "did NOT fire"} within ${CROSS_TIMEOUT_MS}ms${writeThrew ? " (B's write threw first)" : ""}`,
|
||||||
|
);
|
||||||
|
record(
|
||||||
|
`CROSS: A's subscription callback fires on B's write (≤${CROSS_TIMEOUT_MS}ms after B's write landed)`,
|
||||||
|
crossFired,
|
||||||
|
`newPushes=${seq(crossNew)} (B write took ${writeMs}ms; push ${crossFired ? pushAfterWriteMs + "ms after" : "not seen"})${writeThrew ? ` — B write threw: ${writeThrew}` : ""}`,
|
||||||
|
);
|
||||||
|
await sdk(A.frame, "stateProbeStop");
|
||||||
|
}
|
||||||
|
} finally {
|
||||||
|
try { if (ctx) await ctx.close(); } catch { /* ignore */ }
|
||||||
|
closeServer();
|
||||||
|
}
|
||||||
|
|
||||||
|
// ── Determination summary (not a pass/fail gate — this is a probe) ──────────
|
||||||
|
console.log("\n══ doc_subscribe delivery determination ══");
|
||||||
|
for (const r of results) console.log(` [${r.ok ? "PASS" : "FAIL"}] ${r.name}${r.detail ? " — " + r.detail : ""}`);
|
||||||
|
const self = results.find((r) => r.name.startsWith("SELF: subscription callback fires"));
|
||||||
|
const cross = results.find((r) => r.name.startsWith("CROSS: A's subscription callback fires"));
|
||||||
|
console.log("\n SELF →", self ? (self.ok ? "FIRES" : "DOES-NOT-FIRE") : "could-not-test");
|
||||||
|
console.log(" CROSS →", cross ? (cross.ok ? "FIRES" : "DOES-NOT-FIRE") : "could-not-test");
|
||||||
|
}
|
||||||
|
|
||||||
|
main().catch((e) => {
|
||||||
|
console.error("[reactivity] fatal:", e);
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
@@ -28,6 +28,7 @@
|
|||||||
},
|
},
|
||||||
"scripts": {
|
"scripts": {
|
||||||
"test": "bun test",
|
"test": "bun test",
|
||||||
"test:e2e": "bun run e2e/run.ts"
|
"test:e2e": "bun run e2e/run.ts",
|
||||||
|
"test:e2e:reactivity": "bun run e2e/reactivity-doc-subscribe.ts"
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -29,7 +29,13 @@ import { subscribeDoc } from "./subscribe";
|
|||||||
import { ensureRepoOpen } from "./open-repo";
|
import { ensureRepoOpen } from "./open-repo";
|
||||||
import { getCurrentUser, getStoreRegistryDeps } from "./polyfill";
|
import { getCurrentUser, getStoreRegistryDeps } from "./polyfill";
|
||||||
import { escapeLiteral } from "./sparql";
|
import { escapeLiteral } from "./sparql";
|
||||||
import { accessLogPrefix } from "./access-log";
|
import {
|
||||||
|
accessLogPrefix,
|
||||||
|
enabled as accessLogEnabled,
|
||||||
|
logAccess,
|
||||||
|
logStage,
|
||||||
|
shortNuri,
|
||||||
|
} from "./access-log";
|
||||||
import type { Nuri, PrincipalId } from "./types";
|
import type { Nuri, PrincipalId } from "./types";
|
||||||
|
|
||||||
// --- deposit model --------------------------------------------------------
|
// --- deposit model --------------------------------------------------------
|
||||||
@@ -76,6 +82,28 @@ async function sessionId(): Promise<string> {
|
|||||||
return (await getStoreRegistryDeps().getSession()).sessionId;
|
return (await getStoreRegistryDeps().getSession()).sessionId;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// --- diagnostic logging helper ---------------------------------------------
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Best-effort, length-capped JSON rendering of a deposit payload for the
|
||||||
|
* inbox diagnostic log (see {@link enabled}/{@link logAccess}). This module
|
||||||
|
* stays domain-agnostic (see module header) — it never interprets payload
|
||||||
|
* fields, it only dumps them verbatim so the consumer's own shape (e.g. a
|
||||||
|
* Festipod participation: `{ participantId, eventId, … }`) is visible in the
|
||||||
|
* log without this module knowing that shape. Capped so one oversized payload
|
||||||
|
* can't blow up a log line; a payload that fails to stringify (e.g. a
|
||||||
|
* circular structure a caller mistakenly passed) falls back to `String()`.
|
||||||
|
*/
|
||||||
|
function summarizePayload(payload: unknown): string {
|
||||||
|
let s: string;
|
||||||
|
try {
|
||||||
|
s = JSON.stringify(payload) ?? String(payload);
|
||||||
|
} catch {
|
||||||
|
s = String(payload);
|
||||||
|
}
|
||||||
|
return s.length > 200 ? s.slice(0, 200) + "…" : s;
|
||||||
|
}
|
||||||
|
|
||||||
// --- SPARQL result helpers ------------------------------------------------
|
// --- SPARQL result helpers ------------------------------------------------
|
||||||
|
|
||||||
/** Tolerant extraction of SPARQL SELECT bindings across possible shapes. */
|
/** Tolerant extraction of SPARQL SELECT bindings across possible shapes. */
|
||||||
@@ -145,6 +173,17 @@ export async function post(targetInbox: Nuri, opts: PostOptions): Promise<void>
|
|||||||
<${P.ts}> "${ts}"${fromTriple} .
|
<${P.ts}> "${ts}"${fromTriple} .
|
||||||
}`;
|
}`;
|
||||||
await sparqlUpdate(sid, update, targetInbox, "deposit");
|
await sparqlUpdate(sid, update, targetInbox, "deposit");
|
||||||
|
// Domain-level diagnostic (on top of docs.ts's generic access-path WRITE log):
|
||||||
|
// who deposited WHAT into which inbox — the decoded payload, not just the
|
||||||
|
// triple-write. Gated by the same access-log flag; skip the JSON work when off.
|
||||||
|
if (accessLogEnabled()) {
|
||||||
|
logAccess(
|
||||||
|
"WRITE",
|
||||||
|
targetInbox,
|
||||||
|
"inbox deposit",
|
||||||
|
" from=" + (from ?? "anonymous") + " payload=" + summarizePayload(opts.payload ?? null),
|
||||||
|
);
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// --- read --------------------------------------------------------------
|
// --- read --------------------------------------------------------------
|
||||||
@@ -191,6 +230,26 @@ export async function read(targetInbox: Nuri): Promise<Deposit[]> {
|
|||||||
deposits.push({ from: fromValue ? fromValue : null, payload, ts });
|
deposits.push({ from: fromValue ? fromValue : null, payload, ts });
|
||||||
}
|
}
|
||||||
deposits.sort((a, b) => a.ts - b.ts);
|
deposits.sort((a, b) => a.ts - b.ts);
|
||||||
|
// Domain-level diagnostic (on top of docs.ts's generic access-path READ log
|
||||||
|
// of raw triple-rows): how many DEPOSITS were found, and the decoded data of
|
||||||
|
// each — the exact visibility needed to trace materialization at the owner
|
||||||
|
// side. Gated by the same access-log flag; skip the JSON work when off.
|
||||||
|
if (accessLogEnabled()) {
|
||||||
|
logAccess(
|
||||||
|
"READ",
|
||||||
|
targetInbox,
|
||||||
|
"inbox materialize",
|
||||||
|
" → " + deposits.length + " message(s)",
|
||||||
|
);
|
||||||
|
for (const d of deposits) {
|
||||||
|
logAccess(
|
||||||
|
"READ",
|
||||||
|
targetInbox,
|
||||||
|
"inbox message",
|
||||||
|
" ts=" + d.ts + " from=" + (d.from ?? "anonymous") + " payload=" + summarizePayload(d.payload),
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
return deposits;
|
return deposits;
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -218,6 +277,11 @@ export const materialize = read;
|
|||||||
* the unit fake-ng path (no `doc_subscribe`) so `bun test` is unaffected.
|
* the unit fake-ng path (no `doc_subscribe`) so `bun test` is unaffected.
|
||||||
*/
|
*/
|
||||||
export async function readSynced(targetInbox: Nuri): Promise<Deposit[]> {
|
export async function readSynced(targetInbox: Nuri): Promise<Deposit[]> {
|
||||||
|
// Marks the cold, connection-triggered entry point in the trace — the BARRIER
|
||||||
|
// line (open-repo.ts) and the "inbox materialize"/"inbox message" lines below
|
||||||
|
// (from the read() this wraps) follow right after, so a live session shows
|
||||||
|
// the whole owner-reconnect sequence together.
|
||||||
|
logStage("READSYNCED " + shortNuri(targetInbox) + " (cold, barrier-gated)");
|
||||||
await ensureRepoOpen(targetInbox);
|
await ensureRepoOpen(targetInbox);
|
||||||
return read(targetInbox);
|
return read(targetInbox);
|
||||||
}
|
}
|
||||||
@@ -251,7 +315,22 @@ export function watch(
|
|||||||
if (stopped) return;
|
if (stopped) return;
|
||||||
try {
|
try {
|
||||||
const deposits = await read(targetInbox);
|
const deposits = await read(targetInbox);
|
||||||
if (!stopped && deposits.length !== lastCount) {
|
const changed = deposits.length !== lastCount;
|
||||||
|
// Owner-side processing decision: did this push actually grow the
|
||||||
|
// deposit set (→ onDeposits fires, the polyfill's stand-in for
|
||||||
|
// materialization) or was it a no-op push (→ skipped)? This is the
|
||||||
|
// exact line to check for the "must reconnect an extra time" symptom:
|
||||||
|
// a push whose read still sees the OLD count means the barrier/read
|
||||||
|
// raced the write, not that watch itself failed to fire.
|
||||||
|
if (accessLogEnabled()) {
|
||||||
|
logAccess(
|
||||||
|
"READ",
|
||||||
|
targetInbox,
|
||||||
|
"inbox watch",
|
||||||
|
" → " + deposits.length + " message(s)" + (changed ? " (materializing)" : " (unchanged, skip)"),
|
||||||
|
);
|
||||||
|
}
|
||||||
|
if (!stopped && changed) {
|
||||||
lastCount = deposits.length;
|
lastCount = deposits.length;
|
||||||
onDeposits(deposits);
|
onDeposits(deposits);
|
||||||
}
|
}
|
||||||
|
|||||||
Reference in New Issue
Block a user