0eb25286c8
Deux mouvements de surface, aucun changement de comportement. **`packages/client` → `packages/sdk`, `@ng-eventually/client` → `@ng-eventually/sdk`.** « client » ne disait rien : ce paquet EST le SDK que l'application appelle, et c'est tout ce qu'elle appelle. L'ancien nom reste comme mot-clé de recherche dans `docs/source-layout-by-fate.md` et le tableau des paquets du README. **Une seule entrée.** L'entrée `./polyfill` disparaît ; ses symboles applicatifs — `configure`, `configureStoreRegistry`, `setCurrentUser`, `connectedUser` et leurs types — vivent dans un bloc `POLYFILL-ERA` de `src/index.ts`. Ce que la seconde porte portait mérite d'être nommé avant d'être retiré : *ce qu'on importe de ce chemin est exactement ce qu'on supprimera à la migration*. Une seule porte perd ce signal — rien à la ligne d'import ne distingue `configure`, qui part, de `docs`, que le vrai SDK remplace sur place. Trois choses le portent désormais : le bloc lui-même, l'inventaire d'exports de `docs/api-contract.md` (épinglé par `test/vocabulary.test.ts`, donc il ne peut pas rancir en silence), et le contrôle de vocabulaire sur les noms publiés. **Six symboles quittent la surface au passage**, et la fusion est ce qui a rendu le choix visible plutôt qu'hérité : - `getConfig` / `getStoreRegistryDeps` — câblage interne, atteint par `shared-wallet/bootstrap` ; - `resetConfig` / `resetStoreRegistry` / `resetCaps` — remises à zéro de test, atteintes par leur chemin interne, ce qui est leur raison d'être ; - le `share` direct — `inbox.share` a toujours été la même fonction, et la publier deux fois brouillait la frontière qu'elle servait à marquer. Corrections d'affirmations fausses trouvées en chemin : le contrat annonçait `isNuri` / `hasReadCap` sur la porte SDK alors qu'ils ne sont plus exportés depuis le passage au permissif en entrée (`NuriLike` validé à la porte) ; le README du paquet documentait `capFor`, `shareCap`, `getCaps` et `publishRepoLink`, dont aucun n'existe ; et le README de l'app d'exemple affirmait que la suite e2e la pilote, ce qui reste à faire. 179 tests unitaires, typecheck bibliothèque / exemple / harnais, e2e 42/42 contre le broker en ligne — mesuré une fois après le renommage, une fois après la fusion.
835 lines
53 KiB
Markdown
835 lines
53 KiB
Markdown
# Current-state NextGraph — what the SDK/broker do and do NOT expose
|
|
|
|
**Owner:** this library. `@ng-eventually/sdk` exists because the *current*
|
|
NextGraph JS SDK is immature. This file is the authoritative reference on what
|
|
today's SDK/broker actually give us — the ground truth every polyfill in this
|
|
lib compensates for. Read [`simulation.md`](./simulation.md) for how we emulate
|
|
the mature behaviour on top of these limits, and
|
|
[`migration-guide.md`](./migration-guide.md) for what changes when they lift.
|
|
|
|
Verified against `nextgraph-rs` (local clone at `../nextgraph-rs`, sibling of
|
|
this repo) and the installed npm packages. Store/permission facts cross-checked
|
|
with the official docs ([Documents & Stores](https://docs.nextgraph.org/en/documents/),
|
|
[Getting started](https://docs.nextgraph.org/en/getting-started/)).
|
|
|
|
## Source pointers (`nextgraph-rs`)
|
|
|
|
Where the ground truth lives, so future re-verification is cheap:
|
|
|
|
- `sdk/js/lib-wasm/src/lib.rs` — the wasm API actually exposed to JS.
|
|
- `engine/net/src/app_protocol.rs` — `AppRequestCommandV0` enum, `NuriV0` formats.
|
|
- `engine/verifier/src/request_processor.rs` — the effective `app_request`
|
|
dispatch (the truth on what is actually *processed*).
|
|
- `engine/net/src/types.rs` — inbox types (`InboxPost`, `InboxMsg`, `InboxMsgContent`).
|
|
- `engine/verifier/src/inbox_processor.rs` — inbox message handling.
|
|
- `engine/verifier/src/verifier.rs:2237` — `load_repo_from_read_cap`, the one path that
|
|
brings a repo in FROM a cap (`pub(crate)`, see § *Capability / ReadCap granularity*).
|
|
- `engine/verifier/src/verifier.rs:1423` — the `OpenRepo` TODO. It is **not** about
|
|
loading an unheld repo: it sits inside `open_branch_`, past
|
|
`self.repos.get_mut(repo_id).ok_or(RepoNotFound)?` (`:1331`), so the repo is already
|
|
held by the time that line runs. What is missing is the broker-side `OpenRepo`
|
|
request, worked around with a pin.
|
|
- `engine/repo/src/types.rs` — `RootBranchV0.store: StoreOverlay` (repo → its store).
|
|
|
|
## The 5 store types
|
|
|
|
The **3 default stores** belong to a **user**, not to the wallet. A wallet holds
|
|
`sites: HashMap<String, SiteV0>` (`engine/wallet/src/types.rs:456`), and it is `SiteV0`
|
|
that carries `public` / `protected` / `private` (`engine/verifier/src/site.rs:31-37`) —
|
|
one wallet can hold several, which is exactly why "wallet" is the wrong unit to reason
|
|
in (see `docs/readcap-and-nuri-model.md` §4quinquies, *Nomenclature first*). A session exposes the three as
|
|
`private_store_id`, `protected_store_id`, `public_store_id` — those are the connected
|
|
USER's. Group and Dialog are created on demand.
|
|
|
|
| Store | Read | Write | Creation |
|
|
|---|---|---|---|
|
|
| **Private** | Owner only | Owner only | Default |
|
|
| **Protected** | Owner + link+permission holders | Owner + permissioned collaborators | Default |
|
|
| **Public** | Everyone, no capability | Owner only | Default |
|
|
| **Group** | Group members | Group members (collaborative) | On demand |
|
|
| **Dialog** | The two users only | The two users only | On demand |
|
|
|
|
Doc citations (verbatim): Private — *"only you have access to … not possible to
|
|
share"*; Protected — *"share … but they will need a special link and permission"*;
|
|
Public — *"equivalent to your website … without the need for special permissions"*;
|
|
Group — *"each Group is a separate Store … documents inherit the permissions of
|
|
the store"*; Dialog — *"hold all the data you exchange with another user (and only
|
|
with that other user) … You cannot add more users"*.
|
|
|
|
## Document = repo (there is no `Document` type)
|
|
|
|
*"A Repo is the equivalent of an E2EE group for one and only one Document."*
|
|
**1 document = 1 repo** (commits + permissions). Identifier: `did:ng:o:<RepoID>`.
|
|
|
|
There is **no `Document` type** in `nextgraph-rs` (verified 2026-06-29): a
|
|
"document" is simply *any repo*. A **store is a special repo** (`is_store=true`,
|
|
with `Store`/`Overlay`/`User` branches) — so *a store is a document, but a
|
|
document is not necessarily a store*.
|
|
|
|
**Containment (store → repos) is by REFERENCE, not by a list.** A store does not
|
|
hold a `Vec<RepoId>`: it references its repos through an **RDF graph** in its
|
|
Overlay/User branch. Conversely each repo names its parent store via
|
|
`RootBranchV0.store: StoreOverlay` — **a repo belongs to exactly one store**.
|
|
|
|
## Capability / ReadCap granularity — the load-bearing fact for this lib
|
|
|
|
`ReadCap = ObjectRef`. Granularity is at the **repo AND branch** level (each
|
|
branch has its own `read_cap`), down to the **block** (`ObjectKey`/ChaCha20 key).
|
|
Write is managed at the **document (repo)** level.
|
|
|
|
**No automatic read inheritance.** Holding a **store's** ReadCap does **not**
|
|
grant the repos it contains — **you need each repo's own ReadCap**. The optional
|
|
`inherit_perms_users_and_quorum_from_store: Option<ReadCap>` shares only
|
|
users/quorum (write/permissions), **not** read-cap possession. (Repos of a
|
|
`private_store` inherit implicitly.)
|
|
|
|
> Consequence for this lib's emulation (see [`simulation.md`](./simulation.md)):
|
|
> the read access unit is the repo = each item's `@graph` — a per-document
|
|
> filter, never per-store and never per-item. This is exactly what
|
|
> `emulated-verifier/caps.ts` (`CapRegistry`) and `emulated-verifier/read-filter.ts` model: no store-level
|
|
> inheritance, purely per-document caps. In a mono-store layout (all items in one
|
|
> repo) the filter is therefore all-or-nothing on that document — which *is* the
|
|
> native behaviour, and why fine-grained isolation requires one document per
|
|
> entity. Read isolation is cryptographic in the target: with no cap for a repo, a
|
|
> union / reactive read returns empty (the repo is never decrypted), while a
|
|
> targeted read of an unheld repo returns `RepoNotFound`. There is no
|
|
> cap-introspection API, and there is nothing to introspect: reading is key
|
|
> possession, so the polyfill asks the only question the model admits —
|
|
> `capFor(doc)`, "do I hold it?". Its *implementation* is emulation-only; its shape
|
|
> is the target's.
|
|
|
|
### Store ↔ document confusion (recurring)
|
|
|
|
The isolation axis is the **document (repo/`@graph`)**, never the **store**: a
|
|
store *contains* several documents and does not share their read caps. See the
|
|
two-axes warning in [`simulation.md`](./simulation.md): "multi-store" in this
|
|
lib's emulation means **multiple DOCUMENTS in one shared store**, not multiple
|
|
stores.
|
|
|
|
## Capability sharing / NURI
|
|
|
|
**What a NURI transports, and the two acts that cover every use of it, are in
|
|
[`readcap-and-nuri-model.md`](./readcap-and-nuri-model.md) §4sexies** — including why a
|
|
key-less reference is the ordinary case rather than a degenerate one, and why nothing is
|
|
checked at access time. The analysis of what exists vs what is merely declared is in
|
|
[`document-links.md`](./document-links.md).
|
|
|
|
Sharing transmits a **NURI** embedding the crypto capability (read and/or write).
|
|
No central ACL: holding the NURI *is* the right. *"adding permissions can be done
|
|
offline"*; *"removing permissions … requires a SyncSignature"* (synchronous).
|
|
|
|
## Inbox
|
|
|
|
**Only two repos have an inbox today: a user's public and protected STORES.** Not
|
|
documents, and not the private store. `new_store_default` attaches one solely `if
|
|
!private` (`engine/verifier/src/verifier.rs:2994`), and `doc_create` goes through
|
|
`new_repo_default`, which leaves `inbox: None` (`engine/repo/src/repo.rs:574`). The only
|
|
`AddInboxCap` commits in the whole engine are the two in `engine/verifier/src/site.rs:128,149`
|
|
— one for the public store repo, one for the protected one.
|
|
|
|
**But the engine SUPPORTS an inbox on any repo — "does not" and "cannot" are different
|
|
statements.** `inbox: Option<PrivKey>` is a field of EVERY `Repo`
|
|
(`engine/repo/src/repo.rs:126`), not of a store structure. `AddInboxCapV0` is keyed by
|
|
`repo_id` (`engine/repo/src/types.rs:1973`) — *"Repo the Inbox is opened for"*. And
|
|
`update_inbox_cap_v0` applies it via `self.repos.get_mut(repo_id)` with **no `is_store`
|
|
check of any kind** (`engine/verifier/src/verifier.rs:1920`). It is generic by
|
|
construction, and available at any time: `AddInboxCap` is a User-branch commit
|
|
(`engine/repo/src/commit.rs:1043-1050`) whose type documents the late case — *"DEPS to
|
|
the previous AddInboxCap commit(s) if it is an update"*.
|
|
|
|
So a per-document inbox is **not an anticipation**: it is an engine capability that no
|
|
code path exercises automatically and that no level-2 or level-3 API exposes. This lib
|
|
implements it aligned on the engine's model.
|
|
|
|
**An inbox address is TRANSMITTED, never published — and nothing in the engine says who
|
|
may open one.** Two facts that decide more than they look:
|
|
|
|
- `inboxes: HashMap<PubKey, RepoId>` is a field of the **Verifier**
|
|
(`engine/verifier/src/verifier.rs:105`), rebuilt empty on each construction (`:520`,
|
|
`:2820`). The inbox → repo association is **local to a session**, not a published
|
|
fact. A depositor learns a pubkey because it was **sent** to them — in a
|
|
`ContactDetails` message (`contact.inbox`) or through a profile QR code; the reply
|
|
path reads its own `repo.inbox` to include it (`request_processor.rs:736-750`).
|
|
- There is therefore **no engine guard on who opens an inbox for a repo**.
|
|
`AddInboxCap` lands on the committer's OWN User branch, so anyone may write one naming
|
|
anyone's repo. It simply reaches nobody: no one was told that pubkey means that
|
|
document.
|
|
|
|
*Consequence for this lib, and it is a real divergence:* we **publish** the address on
|
|
the document (its Header branch) because that is the only way a third party can find it
|
|
in an emulation with no message channel. That creates a vector the engine does not have
|
|
— whoever can write the document can redirect its deposits — so `openDocumentInbox`
|
|
guards on ownership. That guard compensates OUR design; it does not mirror an upstream
|
|
rule. Do not cite it as one.
|
|
|
|
A non-editor can deposit into an inbox without being invited as an editor; the owner
|
|
moderates. NURI: `did:ng:d:<inbox_id>`. Content: the `InboxMsgContent` enum (`ContactDetails`,
|
|
`DialogRequest`, `Link`, `Patch`, `ServiceRequest`, `ExtRequest`,
|
|
`RemoteQuery`, `SocialQuery`…, `engine/net/src/types.rs:4249-4261`). Note what `Link`
|
|
is: a **unit variant, carrying nothing** — not a link, not a cap, just a discriminant.
|
|
Reading it as "the inbox can deliver a read capability" is the trap this file exists to
|
|
prevent; see the *Consequence for this lib* below, which says the same thing from the
|
|
other end. Messages are sealed (`crypto_box::seal`) to
|
|
the inbox pubkey, so only the owner decrypts. The `from` field is optional, so an
|
|
anonymous sender is possible. This is the "identified if known, anonymous
|
|
otherwise" behaviour native to the protocol.
|
|
|
|
The recipient's own verifier unseals each queued message and applies it inline when
|
|
it processes its inbox — there is no separate curator or materialization service.
|
|
|
|
### The inbox is not usable from the JS SDK
|
|
|
|
- `app_request(request)` is exposed, and `AppRequestCommandV0::InboxPost` +
|
|
`AppRequest::inbox_post()` exist, but the verifier's `request_processor`
|
|
has no `InboxPost` arm (arms actually handled: `OrmStart(Discrete)`,
|
|
`Fetch`, `FileGet`, `OrmUpdate`, `OrmDiscreteUpdate`, `SocialQueryStart`,
|
|
`QrCodeProfile(Import)`, `Header`, `Create`, `FilePut`). Sending an `InboxPost`
|
|
triggers nothing.
|
|
- Building an `InboxPost` requires crypto sealing on the Rust side; no wasm
|
|
helper exposes it, and **no `inbox` method exists in `@ng-org/web` at all**.
|
|
*(`inbox_post_link`, named across this repo's docs, is OUR proposed name from
|
|
[`fork-inbox-fallback.md`](./fork-inbox-fallback.md) — grep `nextgraph-rs` and it
|
|
is nowhere. Nothing is announced about the eventual JS surface for inboxes: its
|
|
name and shape are unknown, not merely unimplemented.)*
|
|
- Inbox deposit is only triggered internally by `QrCodeProfileImport`
|
|
(`post_to_inbox(new_contact_details)`) and `social_query_start` (contact
|
|
propagation via inbox).
|
|
|
|
**Consequence:** there is no clean way to "drop a Link" into an arbitrary
|
|
document's inbox from the JS SDK today. This lib emulates the inbox instead of
|
|
patching the broker — see [`simulation.md`](./simulation.md) (emulated inbox) and
|
|
[`fork-inbox-fallback.md`](./fork-inbox-fallback.md) (the Rust-patch path not taken).
|
|
A related exposed primitive: `social_query_start` (a federated query via inbox up to
|
|
`degree` hops) exists but is limited to contacts — it does not cover an anonymous
|
|
notification to a non-connected host.
|
|
|
|
### Delivering a ReadCap through the inbox — the field exists, the path does NOT — VERIFIED
|
|
|
|
The `ContactDetails` inbox message carries `read_cap: Option<ReadCap>`, commented
|
|
*"optional readcap on the profile, if user wants to share the content of profile"*
|
|
(`engine/net/src/types.rs`). Nothing behind that field is implemented:
|
|
|
|
- **Building it panics.** `InboxPost::new_contact_details(…, with_readcap: bool, …)`
|
|
(`engine/net/src/types.rs`) fills `read_cap` with `unimplemented!()` when
|
|
`with_readcap` is true, and `None` otherwise. Asking for a cap in the message is a
|
|
panic, not a feature.
|
|
- **Nobody asks for one.** Its ONLY caller is the `QrCodeProfileImport` path in
|
|
`engine/verifier/src/request_processor.rs`
|
|
(`post_to_inbox(InboxPost::new_contact_details(…))`), which passes `with_readcap =
|
|
false`. No message ever carries a cap.
|
|
- **The receiver discards it.** The `InboxMsgContent::ContactDetails(details)` arm of
|
|
`engine/verifier/src/inbox_processor.rs` reads `details.profile`, `details.name` and
|
|
`details.email` to build a `social:contact` document — it **never reads
|
|
`details.read_cap`**. Even a hand-crafted message carrying a cap would be dropped.
|
|
|
|
**Consequence for this lib:** there is no native channel to HAND a key to somebody. The
|
|
inbox transports an identity/profile pointer, not a read capability. Combined with
|
|
§ *The inbox is not usable from the JS SDK* (no `InboxPost` arm in the request processor
|
|
at all), cap delivery must be emulated end to end: the polyfill's emulated inbox and its
|
|
`CapRegistry` are not a shortcut around an existing mechanism, they stand in for a
|
|
mechanism that does not exist.
|
|
|
|
## The query capability — ONE local store, named graphs, union queries
|
|
|
|
The single fact that makes read-time *listing* possible on the shared wallet, and
|
|
the reason the reactive ORM must **not** be used as the listing primitive. Verified
|
|
directly in `nextgraph-rs`.
|
|
|
|
### One oxigraph Store per session; each repo is a NAMED GRAPH
|
|
|
|
- The verifier keeps **ONE** local oxigraph `Store` per session:
|
|
`graph_dataset: Option<Store>` (`engine/verifier/src/verifier.rs:94`).
|
|
- Every synced repo's data is inserted into that one store as a **distinct NAMED
|
|
GRAPH keyed by the repo** — `update_graph` writes each repo's main-branch triples
|
|
under `NuriV0::repo_graph_name(&repo_id, &overlay_id)`
|
|
(`engine/verifier/src/commits/transaction.rs:646-701`, the `ov_graphname` /
|
|
`repo_graph_name` around line 669). So all opened repos coexist as named graphs in
|
|
one dataset — a `GRAPH ?g { ... }` body can span them.
|
|
|
|
### The NURI target decides scope: one repo vs the LOCAL UNION
|
|
|
|
`sparql_query`'s scope is resolved from the NURI target by
|
|
`resolve_target_for_sparql` (`engine/verifier/src/request_processor.rs:256-285`):
|
|
|
|
- `Repo(repo_id)` / `PrivateStore` → `Some(repo_graph_name)` = **ONE repo's graph**.
|
|
- `UserSite` / `None` → `Ok(None)` = the **UNION of all named graphs**. That `None`
|
|
is passed to oxigraph's `store.query(parsed, default_graph)`
|
|
(`engine/oxigraph/src/oxigraph/store.rs:200`) as the default graph, and
|
|
`sparql_query` first calls `dataset.set_default_graph_as_union()` when the query
|
|
has no explicit dataset (`request_processor.rs:656-675`, the
|
|
`if dataset.has_no_default_dataset()` block). Union = "every named graph currently
|
|
in the store".
|
|
|
|
### The wasm binding defaults the target to UserSite when no `nuri` is passed
|
|
|
|
The binding (`sdk/js/lib-wasm/src/lib.rs` — both the nodejs variant at ~`350-405`
|
|
and the web variant at ~`553-610`) reads the target from the `nuri` arg: a string
|
|
`nuri` is parsed; an **absent** `nuri` falls back to
|
|
`NuriV0::new_entire_user_site()` = `UserSite`
|
|
(`engine/net/src/app_protocol.rs:488-490`). Therefore:
|
|
|
|
> **`sparql_query(sid, query, base, /*anchor*/ undefined)` queries the LOCAL UNION
|
|
> across all synced/opened graphs; with a string anchor it is restricted to that one
|
|
> repo.** A `GRAPH ?g { ... }` body then spans/attributes across the local graphs.
|
|
|
|
### A repo is only queryable once OPENED/synced into the store
|
|
|
|
A repo's triples enter `graph_dataset` (hence the union) only after the repo is
|
|
opened/synced into `self.repos` **and** its commits applied via `update_graph`.
|
|
|
|
**VERIFIED (T03.k) — there is NO JS primitive to sync an *unknown* repo.** Every
|
|
JS entry point that could "open" a repo — `sparql_query` anchored,
|
|
`doc_subscribe` (`Fetch::Subscribe`), `orm_start_graph` — resolves its target via
|
|
`resolve_target`/`resolve_target_for_sparql`, which does
|
|
`self.repos.get(repo_id).ok_or(RepoNotFound)` (`request_processor.rs:155/163/264/269`).
|
|
None of them PULLS a repo that is absent from `self.repos`; they only touch a repo
|
|
already there. The primitive that actually loads a repo from its ReadCap,
|
|
`Verifier::load_repo_from_read_cap` (`verifier.rs:2237`), is **`pub(crate)` —
|
|
unexposed to JS**; it is only reached internally (bootstrap, inbox processing). So
|
|
from JS today a repo becomes queryable ONLY by being `doc_create`d in this session
|
|
(own docs) or synced by an internal path — never on demand by NURI+ReadCap.
|
|
|
|
**Consequence for this lib's mono-wallet polyfill:** every account's documents are
|
|
`doc_create`d in the one shared wallet within the same session, so they are all
|
|
already in `self.repos`. `surface/read-model.ts` reads the bounded, by-need set of docs
|
|
with one anchored `sparql_query` per doc (`SELECT ?s ?p ?o WHERE { ?s ?p ?o }`,
|
|
anchor = the doc NURI): the anchor resolves that same-session repo directly (no
|
|
separate open needed) and restricts the query to its graph, so it is O(1) per doc,
|
|
independent of the store's size. An absent repo throws `RepoNotFound` on its own
|
|
read and is skipped, never aborting the batch.
|
|
|
|
The read path avoids an anchorless union-scan. An anchorless
|
|
`SELECT … WHERE { GRAPH ?g { ?s ?p ?o } }` spans every named graph in the store —
|
|
O(wallet size). On a shared wallet that accumulates docs across runs that cost grows
|
|
with the whole wallet, which is why the read path is per-doc anchored: the anchored
|
|
read makes a non-empty wallet irrelevant. At the real multi-store
|
|
migration this is unchanged (the anchored read is native); only bringing a repo into
|
|
the session changes: opening a real per-user store repo by cap becomes a native
|
|
broker sync, through `load_repo_from_read_cap` (`verifier.rs:2237`) — not through the
|
|
`OpenRepo` TODO at `:1423`, which concerns a repo already held. Opening still requires the
|
|
repo's NURI + ReadCap — there is no store-level read inheritance (see
|
|
§ Capability / ReadCap granularity).
|
|
|
|
### Findable-without-lookup vs subscribable (first-`State` barrier) — DISJOINT
|
|
|
|
Two properties a fresh session might want from a document, and **no single document
|
|
has both**:
|
|
|
|
- **Findable without a lookup.** The ONLY NURI a fresh session can NAME with nothing
|
|
but the session in hand is the store-root, `did:ng:${privateStoreId}` (from
|
|
`session.private_store_id`). Any per-document repo is `did:ng:o:<RepoID>` with a
|
|
**random** RepoID minted by `doc_create` — **not derivable**, so it must be looked
|
|
up somewhere first.
|
|
- **Subscribable with a sync BARRIER.** `doc_subscribe(nuri)` delivers `TabInfo` then
|
|
an initial **`State`** (`verifier.rs:470`/`:476`); that first `State` is the sync
|
|
barrier — **after it, presence is guaranteed and absence is definitive** (pinned
|
|
empirically by CONTRACT 3 in `packages/sdk/e2e/`). But this barrier exists only
|
|
for a repo `doc_subscribe` can open, i.e. a `did:ng:o:<RepoID>` repo. A **store-root
|
|
has no first-`State` barrier**: an anchored read on it can return 0 rows during
|
|
sync-lag with no signal distinguishing "still syncing" from "genuinely empty".
|
|
|
|
> **`doc_fetch_repo_subscribe` / `doc_fetch_private_subscribe` are NOT alternatives to
|
|
> `doc_subscribe`** — checked 2026-08-03, because they look like ready-made
|
|
> "open a repo" calls and they are not. Neither performs any I/O: each **builds an
|
|
> `AppRequest` and returns it serialized** (`sdk/js/lib-wasm/src/lib.rs:1890`, `:1900`),
|
|
> with no `session_id` and no callback. `doc_subscribe` builds the *same* request
|
|
> (`AppRequest::doc_fetch_repo_subscribe`, `engine/net/src/app_protocol.rs:930` →
|
|
> `Fetch(Subscribe)`), then adds the session id and runs it through
|
|
> `app_request_stream_` (`lib.rs:1921-1923`). They exist for a caller that wants to
|
|
> construct the request and dispatch it itself. So `ensureRepoOpen`'s
|
|
> `doc_subscribe` + wait-for-first-`State` is not duplicating an available call — using
|
|
> them instead would mean re-implementing what `doc_subscribe` already does. The
|
|
> difference in `doc_fetch_private_subscribe` is only its target
|
|
> (`NuriV0::new_private_store_target()`, the private store-root), which changes nothing
|
|
> about the barrier: a store-root still has none.
|
|
|
|
These two are **mutually exclusive**: the guessable target (store-root) is not
|
|
barrier-authoritative, and the barrier-authoritative target (`o:` repo) is not
|
|
guessable. **Consequence:** you cannot build a lookup table that is BOTH reachable
|
|
cold (findable) AND authoritative on a cold read (barrier). A cold "0 rows" read of a
|
|
store-root graph is therefore fundamentally ambiguous — which is the trap the shim's
|
|
account map fell into (see next section).
|
|
|
|
### The pointer → doc-shim indirection (how the polyfill shim resolves accounts)
|
|
|
|
`shared-wallet/account-registry.ts` keeps a map `identifier → {docPublic, docProtected, docPrivate}`
|
|
(the "shim", the account→document trust root). It must be reachable by a fresh
|
|
reconnecting session (findable) AND authoritative on a cold read (so a fresh page
|
|
does not mistake sync-lag for "account absent" and PROVISION a fork). Since no single
|
|
document is both (previous section), the shim uses an **indirection**:
|
|
|
|
1. **doc-shim** — a `doc_create`d graph document (`did:ng:o:...`, hence a first-`State`
|
|
barrier). **All `VirtualUserRecord`s live inside it.** Because it is subscribable, an
|
|
anchored read behind its `ensureRepoOpen` barrier is **authoritative**: a cold 0
|
|
means the account is genuinely absent.
|
|
2. **pointer** — a single well-known, **write-once** triple in the store-root graph,
|
|
`<urn:ng-eventually:shim:root> <urn:ng-eventually:shim:shimDoc> <docShimNuri>`.
|
|
The store-root is findable-without-lookup, so a fresh session can always read it;
|
|
the pointer being the OLDEST, write-once triple in that graph, it is near-always
|
|
already synced on a cold read.
|
|
|
|
**Resolution** (`resolveShimDoc`): read the pointer from the store-root → open the
|
|
named doc-shim through its barrier (`ensureRepoOpen`) → read the account
|
|
AUTHORITATIVELY. First login (no pointer): `doc_create` the doc-shim, publish the
|
|
pointer, done. A **pointer fork** (two devices each writing a pointer before either
|
|
synced) is reconciled to the lexicographically-smallest doc-shim NURI
|
|
(content-addressed, so every device converges on the same doc-shim).
|
|
|
|
**The account-level retry is GONE.** Before this indirection the shim lived directly
|
|
in the store-root graph, so an account read had no barrier and a cold 0 was ambiguous;
|
|
the lib compensated with a **bounded account-level retry** (`provisionRetry` /
|
|
`resolveAccountReliably`) that re-read the account several times before concluding
|
|
"new". Moving the records behind the doc-shim barrier makes the account read
|
|
authoritative on the FIRST read, so **that retry was removed** — a barrier is
|
|
deterministic where a retry only guessed. The only residual bounded guard is a small
|
|
re-read of the **pointer** itself (`pointerGuard`, one write-once triple): it can
|
|
NEVER re-provision or fork an account — at worst it takes a couple extra reads to see
|
|
a pointer that is still landing.
|
|
|
|
**No legacy migration.** A wallet written under the OLD scheme (accounts directly in
|
|
the store-root graph, no pointer) is NOT recovered: opening it under the current scheme
|
|
simply provisions a fresh doc-shim, and the pre-indirection store-root records are
|
|
ignored. This is deliberate — the only such wallets are dev data — so the resolution
|
|
path carries no legacy-migration step; it always reads the account authoritatively from
|
|
the doc-shim.
|
|
|
|
### The union is read-only — writes must target one document
|
|
|
|
`resolve_target_for_sparql(update=true)` returns `InvalidTarget` for `UserSite` /
|
|
`None` (`request_processor.rs:275-282`). So `sparql_update` cannot write "to the
|
|
union": every write must name one document's `@graph` — exactly what the
|
|
polyfill's `docs.sparqlUpdate` already does.
|
|
|
|
### No reactive SPARQL — `sparql_query` is one-shot
|
|
|
|
`sparql_query` is non-streamed: it computes a `QueryResults` and returns once
|
|
(`lib-wasm/src/lib.rs:352-405` / `553-610`). There is no "subscribe to a union
|
|
query". The only reactive primitives are the streamed ones: `orm_start_graph`,
|
|
`orm_start_discrete`, `doc_subscribe`, `app_request_stream`.
|
|
|
|
### The ORM fan-out hang — verified root cause
|
|
|
|
The reactive ORM is structurally unfit for a fan-out of per-entity / not-yet-synced
|
|
graphs, and this is *why* subscribing such a fan-out hangs:
|
|
|
|
- `OrmStartGraph` first loops over every graph in the requested scope and calls
|
|
`open_for_target(&nuri.target, /*publisher*/ true)` on each
|
|
(`request_processor.rs:53-66`), and `orm/graph/initialize.rs` does the same
|
|
fan-out again for the graphs the ORM discovers (~`125-128`).
|
|
- `open_for_target` → `resolve_target` → `self.repos.get(repo_id).ok_or(RepoNotFound)`
|
|
(`request_processor.rs:286-294` calling `resolve_target` at `:147`, the
|
|
`RepoNotFound` at `:155/:163`).
|
|
- A freshly-created per-entity doc, or any not-yet-synced other-account doc,
|
|
is absent from `self.repos`, so `RepoNotFound` propagates through the `?` and
|
|
aborts the whole `orm_start_graph`. The subscription never emits its initial, so
|
|
the ORM `readyPromise` never resolves and the subscription hangs when a fan-out of
|
|
per-entity graphs is passed in.
|
|
|
|
**Consequence:** passing per-entity / unsynced graphs to the reactive ORM is broken.
|
|
Listing must go through a one-shot union `sparql_query` instead — see
|
|
[`read-model.md`](./read-model.md).
|
|
|
|
## JS SDK limits (`@ng-org/web`)
|
|
|
|
`@ng-org/web` (verified `0.1.2-alpha.13` = `upstream/main` at 2026-05-21, the
|
|
installed version) **does NOT expose**: Group/Dialog store creation; capability
|
|
sharing (a NURI with rights); permission manipulation; inbox deposit/read.
|
|
|
|
The JS methods this lib USES: `doc_create`, `doc_subscribe`, `sparql_query`,
|
|
`sparql_update`, `orm_start_graph`, `orm_start_discrete`, `graph_orm_update`,
|
|
`discrete_orm_update`, `file_get`, `app_request_stream`. That is a working subset,
|
|
**not** the surface: `NGModule` exports **77** (`@ng-org/web@0.1.2-alpha.13`,
|
|
`dist/index.d.ts:140-268`), including `app_request`, `session_stop`,
|
|
`disconnections_subscribe`, `social_query_start`, `upload_start`/`upload_chunk`/
|
|
`upload_done`, and the whole `wallet_*` family. Read "not in the list above" as "we do
|
|
not call it", never as "it does not exist" — several sections of this very file discuss
|
|
methods absent from that subset. The docs announce *"An API will be provided for
|
|
permission manipulation"* (no date).
|
|
|
|
## Integration & deployment model
|
|
|
|
NextGraph is consumed via an **iframe proxy** (`@ng-org/web`): the third-party
|
|
app contains no engine, it delegates to a hosted ng-app (default `nextgraph.net`)
|
|
that runs the engine in an iframe.
|
|
|
|
### The JS packages
|
|
|
|
- **`@ng-org/web`** — **published**. Lightweight postMessage proxy (no wasm
|
|
embedded). **The** third-party integration path; `@ng-org/orm` and every
|
|
example depend on it. This lib wraps it.
|
|
- **`@ng-org/api-web`** — **private** (unpublished). Full in-browser engine
|
|
(loads `@ng-org/lib-wasm` in a Web Worker). Consumed only by `app/nextgraph`
|
|
(the ng-app frontend) — **not** a third-party integration target.
|
|
- **`@ng-org/lib-wasm`** — the compiled wasm engine (contains the verifier).
|
|
Source `sdk/js/lib-wasm/`.
|
|
- **`nextgraph`** (npm) — the NodeJS API (`pkg-node` build).
|
|
- **`@ng-org/orm`** — reactive ORM (`useShape`…), built on `@ng-org/web`.
|
|
|
|
### Where the verifier runs
|
|
|
|
In the standard web model, the verifier runs **in the iframe**: `app/nextgraph`
|
|
loads `api-web` → `lib-wasm` in a Web Worker, browser-side. The broker (`ngd`)
|
|
only does **transport and storage**.
|
|
|
|
**Consequence:** changing verifier logic (`request_processor`,
|
|
`inbox_processor`) means rebuilding the **ng-app**, not the broker.
|
|
|
|
### iframe model & build-time retargeting
|
|
|
|
`@ng-org/web` redirects to the hosted ng-app, which reloads the third-party app
|
|
in an iframe after auth, then relays over `postMessage`. **Retargetable at build
|
|
time** (`sdk/js/web/src/index.ts`, `import.meta.env`):
|
|
|
|
| Variable | Target |
|
|
|---|---|
|
|
| `NG_REDIR_SERVER` | default `nextgraph.net` |
|
|
| `NG_DEV3` | `127.0.0.1:3033` |
|
|
| `NG_DEV` | `localhost:14402`/`14404` |
|
|
| `NG_DEV_LOCAL_BROKER` | `localhost:1421` |
|
|
|
|
**No runtime override** — `init()` takes no broker URL. To point at a
|
|
self-hosted ng-app: **rebuild `@ng-org/web`** (pure TS, no wasm → trivial build).
|
|
|
|
### Proxy ↔ iframe ↔ worker plumbing (generic)
|
|
|
|
The call path is **entirely generic** (no allowlist): `@ng-org/web` is a JS
|
|
`Proxy` relaying *any* method name over `postMessage`; `app/nextgraph` dispatches
|
|
via `Reflect.apply(ng[method], …)`. So a new wasm function in simple
|
|
request/response form is *reachable* without touching the JS — but that's an
|
|
untyped **hack** (quick test, not a plan). The **streamed** case needs an entry
|
|
on both sides (`E` in `@ng-org/web` + `streamed_api` in api-web; current streamed
|
|
methods: `doc_subscribe`, `orm_start_graph`, `orm_start_discrete`, `file_get`,
|
|
`app_request_stream`).
|
|
|
|
> This is exactly why `docs.ts` in this lib calls the **real injected `ng`**
|
|
> directly and never layers our own `Proxy` on top of `@ng-org/web`'s
|
|
> iframe-RPC proxy — see the `DataCloneError` double-proxy constraint in
|
|
> [`simulation.md`](./simulation.md).
|
|
|
|
### The broker (ngd)
|
|
|
|
- Already supports the inbox natively (`inbox_post`, `inbox_register`,
|
|
`inbox_pop_for_user` in `engine/net/src/server_broker.rs`) — a standard `ngd`
|
|
would route the inbox, **no broker patch needed**. The gap is in the
|
|
verifier/SDK layer, not the broker.
|
|
- **WebSocket** daemon (`async-tungstenite`), **stateful**: RocksDB under
|
|
`--base-path`, persisted PeerId (critical volume).
|
|
- CLI: `--local PORT`, `--domain DOMAIN:PORT,LOCAL_PORT` (behind a TLS-terminated
|
|
reverse proxy — Traefik/Coolify).
|
|
- **Serves no static assets**: the ng-app frontend is a separate static deploy
|
|
(`pnpm webfilebuild`). First boot is **interactive** (admin-wallet invitation
|
|
link). Official Dockerfiles are **broken**.
|
|
|
|
## Apps & services: shared app data goes through a hardcoded app store (section rewritten 2026-08-03)
|
|
|
|
NextGraph's app/service execution model, and the answer to "can an application hold data common to all its users".
|
|
|
|
> **Provenance — two layers, do not mix them.** What the **engine** contains is verified below and is vocabulary only. What the **model will be** was stated by the NextGraph developer on 2026-08-03 and is **not implemented**: treat it as the target's declared direction, not as something the clone can confirm. Per [`../README.md`](../README.md)'s design principle, never infer the target's shape from the source's current state — an absent implementation says nothing about the intended one. The previous version of this section drew exactly that inference and concluded the opposite of what the developer states; it has been replaced.
|
|
|
|
**What the engine actually has — types, no behaviour (verified 2026-08-03):**
|
|
|
|
- `AppManifestV0` (`engine/wallet/src/permissions.rs:113`) carries `nuri`, `origin` (webapps), `singleton: bool`, `access_requests`, `installs` (Viewer / Editor / ReadService / WriteService / Model, keyed by PrimaryClass), `dependencies`, and presentation fields.
|
|
- The JS surface exists: `init(callback, singleton, access_requests)` (`sdk/js/web/src/index.ts:51`) relays `singleton` to the wallet origin by `postMessage`. Every example passes `true`.
|
|
- **Nothing consumes it.** The `permissions` module is declared by `engine/wallet/src/lib.rs:19` and imported by no other crate; `AppManifest` is constructed nowhere; no code reads `singleton`; the surrounding `AccessRequest` / `AccessGrant` machinery is in the same state. There is no app runtime, no app store, no global-document type.
|
|
- The field's doc comment reads `/// cannot create Documents?` — with the question mark, in the source. It is an open design note, and the developer's statement below settles it the other way. Do not treat it as the semantics.
|
|
- **A second, published gloss of the same flag disagrees with that doc comment, and it is the one that matches the developer's statement.** `sdk/js/web/README.md:90,108` annotates the argument as *"will your app create many docs in the system, or should it be launched as a unique instance"* — i.e. `singleton` is about **instance multiplicity**, not about being forbidden to create documents. That is consistent with "a singleton app can also manage ordinary per-user documents" below, and it is the reading to carry. Both README examples pass `true`.
|
|
- Unrelated homonyms, so a grep does not mislead: singleton *commits* (`engine/repo/`), the broker singleton (`engine/net/src/broker.rs`), Oxigraph's `empty_singleton` (SPARQL optimiser).
|
|
|
|
**The declared model (NextGraph developer, 2026-08-03 — not implemented):**
|
|
|
|
- A singleton app can **also** manage ordinary per-user documents, i.e. cover what a non-singleton app does. When both modes are needed, declaring one app as singleton is enough — there is no need for two apps.
|
|
- Centralized data for an application takes the form of a **document, or a store, shared by all its users and hardcoded in the app**.
|
|
- The **app's developer holds the write rights** on the app's documents and stores, and can **delegate** them.
|
|
- Delegation is **never to all users**. User contributions reach app-owned data **through an inbox** — this is NextGraph's general model, in which nothing is freely writable by everyone.
|
|
|
|
**Reading of the last point, since it decides the shape of any index:** an app-owned index is not a place users write to. It is a document they can read (its NURI being hardcoded) and **deposit into**, with an authority derived from the developer materializing the deposits — the same deposit-then-materialize shape the inbox already imposes elsewhere, moved up to the app level.
|
|
|
|
**Open questions to put to the developer before designing on this:**
|
|
|
|
- What exactly is hardcoded — the store's or document's NURI, and does that carry the read capability?
|
|
- How is write delegation transmitted, and is it revocable?
|
|
- Who processes the app store's inbox: an instance running with the developer's rights, a service, something else? A deposit nobody materializes is not an index.
|
|
|
|
**Bearing on [`decisions/discovery-model.md`](decisions/discovery-model.md):** that ADR's superseding block leans in part on the singleton-app path being "not implemented, uncertain". The path is still not implemented, but it is no longer uncertain in direction — the developer names it as *the* way to hold data common to all users. That does not reinstate discovery: the "you cannot discover, you can only follow links" verdict rests on its own footing (the PO, 2026-07-30). It does mean the *singleton-app* half of that reasoning must be re-put rather than cited as closed.
|
|
|
|
## Third-party wallet auto-import constraint
|
|
|
|
Verified empirically (2026-06-17): with the **hosted** broker (`nextgraph.net`),
|
|
a third-party web app **cannot** provision/import a wallet programmatically. A
|
|
wallet must **pre-exist** in the browser before the auth redirect can succeed.
|
|
|
|
Mechanism (from `@ng-org/web`'s `ngweb.js` dist):
|
|
|
|
- **`init()` top-level REDIRECTS**: when `window.self === window.top` it does
|
|
`window.location.href = https://nextgraph.net/redir/#/?o=<url>`. The app's code
|
|
stops running.
|
|
- **Every `ng.*` method is relayed** by `parent.postMessage` to `nextgraph.net`,
|
|
and the handler **throws `"you must call init() first"` until a session is
|
|
established** (internal `d !== false` guard). This includes
|
|
`wallet_import_from_code`, `add_in_memory_wallet`, `session_in_memory_start`.
|
|
- The third-party app runs **inside the iframe only AFTER** the broker has opened
|
|
a wallet and established the session. There is **no window** where our code runs
|
|
*before* the broker's wallet gate → **nothing to hook an auto-import onto**.
|
|
|
|
Of the wallet-import methods offered on `nextgraph.eu`, only the **wallet FILE**
|
|
(`.ngw`) is a static, reusable export; TextCode/QR are temporary device↔device
|
|
transfers (5 min, both devices online, single use) — unusable to embed. The only
|
|
real way to eliminate the cross-origin round-trip is to self-host/fork the ng-app
|
|
(see [`fork-inbox-fallback.md`](./fork-inbox-fallback.md)).
|
|
|
|
## Login is not programmable
|
|
|
|
NextGraph login is a web redirect to the broker page (`nextgraph.net`). There
|
|
is no way to open a wallet silently — at least one broker-redirect pass per device
|
|
is required. Session persistence: the wallet is remembered iframe-side
|
|
(`localStorage` long-term + `sessionStorage` for the active session); on reload,
|
|
`init()` recovers the session without re-triggering the redirect while the
|
|
broker session exists (`sdk/js/web/src/index.ts`, `sdk/js/api-web/main.ts`). A
|
|
full browser restart (losing `sessionStorage`) can re-trigger the gate. A real
|
|
logout is exposed (`ng.session_stop()`, `ng.user_disconnect()`,
|
|
`ng.wallet_close()` in `sdk/js/lib-wasm/src/lib.rs`) but forces a new
|
|
redirect afterwards. This lib's identity store sidesteps all of it — the identity
|
|
id is set at wallet-import time and relayed to the lib, without a separate login;
|
|
see the identity store in [`simulation.md`](./simulation.md).
|
|
|
|
## Authorship, existence, outer overlay, `Ext` (section added 2026-07-27)
|
|
|
|
Four capability facts about the current core, verified in `nextgraph-rs`. They bear on
|
|
what can be BUILT on top (can we deliver a key? can we tell whether a document exists?
|
|
can we attribute a write?) — they are not a security assessment. Each carries its
|
|
epistemic status; do not upgrade an INFERRED item without new evidence.
|
|
|
|
### Author-signature verification is never called at runtime — VERIFIED
|
|
|
|
`Commit::verify` (`engine/repo/src/commit.rs`) chains `verify_sig` → `verify_perm` →
|
|
`verify_full_object_refs_of_branch_at_commit`. Its only callers in the whole tree are
|
|
inside `#[cfg(test)] mod test` blocks (`engine/repo/src/commit.rs`,
|
|
`engine/repo/src/branch.rs`); `verify_sig` and `verify_perm` have no other caller. The
|
|
verifier's commit path calls a **different** `verify`:
|
|
`CommitBodyV0::<Body>::verify(commit, self, branch_id, repo_id, store)` in
|
|
`engine/verifier/src/verifier.rs` — the `CommitVerifier` trait, which APPLIES a body
|
|
(mutating verifier state); it is not a signature check.
|
|
|
|
Even if it were called it could not succeed. `verify_sig` resolves the author through
|
|
`Repo::member_pubkey` → `Repo.members`, and every `Repo` the verifier builds at runtime
|
|
sets `members: HashMap::new()` — `engine/verifier/src/user_storage/repo.rs` (with a
|
|
literal `//TODO: members`) and `engine/verifier/src/commits/mod.rs`. Only
|
|
`Repo::new_with_member` ever populates a member, and it is called only from tests. An
|
|
empty table makes `member_pubkey` return `NotFound` →
|
|
`CommitVerifyError::PermissionDenied`.
|
|
|
|
Reading authorship at all presupposes the read cap (VERIFIED): the author field is not a
|
|
UserId but `CommitContent::author_digest(user, overlay)`, a BLAKE3 keyed hash, and the
|
|
commit content sits in blocks ChaCha20-encrypted under `Object::convergence_key(store)`
|
|
(`engine/repo/src/object.rs`), whose key material is the store id **plus the
|
|
store-overlay-branch ReadCapSecret**. No read cap → the author field is not even
|
|
visible. *Nuance, VERIFIED:* the digest's own hashing key derives from
|
|
`overlay_id_for_read_purpose`, which for Public/Protected/Private/Group stores is
|
|
`OverlayId::outer(store_id)` — public. What is secret is the commit content, not the
|
|
hash key.
|
|
|
|
**Consequence for this lib:** "who wrote this triple" is unanswerable today — neither
|
|
cryptographically (nothing verifies) nor by identity (the digest is opaque without a
|
|
member table). Any authorship or provenance the polyfill needs must be carried in the
|
|
DATA it writes and re-read from there; an "authored by X" claim in the emulation has no
|
|
core check behind it.
|
|
|
|
### No existence probe at SDK level — addressing presupposes the cap — VERIFIED
|
|
|
|
`AppRequestCommandV0` (`engine/net/src/app_protocol.rs`) contains no existence command:
|
|
`Fetch`, `Pin`, `UnPin`, `Delete`, `Create`, `FileGet`, `FilePut`, `Header`, `InboxPost`,
|
|
`SocialQueryStart`, `SocialQueryCancel`, `QrCodeProfile`, `QrCodeProfileImport`,
|
|
`OrmStartGraph`, `OrmStartDiscrete`, `OrmGraphUpdate`, `OrmDiscreteUpdate`, `OrmStop`.
|
|
Nothing answers *"does document D exist?"*.
|
|
|
|
The single probe in the tree is internal and cannot answer it either:
|
|
`Verifier::has_blocks` (`engine/verifier/src/verifier.rs`) sends
|
|
`BlocksExist { blocks, overlay }`. It is `pub(crate)` (never reaches JS); it takes
|
|
**`BlockId`s** — content addresses you only hold if you already read the object; it takes
|
|
a **`&Repo` already loaded**; and it targets
|
|
`repo.store.overlay_for_read_on_client_protocol()` = the **inner** overlay
|
|
(`Store::inner_overlay` → `overlay_id_for_write_purpose(store_overlay_branch_readcap.key)`,
|
|
`engine/repo/src/store.rs`), derived from the read-cap secret.
|
|
|
|
**Consequence for this lib:** you cannot prove — nor disprove — the existence of a
|
|
document whose key you do not hold. **Addressing presupposes the cap.** Every "is it
|
|
there?" question therefore collapses into "can I read it?", which is why absence is only
|
|
ever established behind a sync barrier (see § *Findable-without-lookup vs subscribable*)
|
|
and never by probing.
|
|
|
|
### `expose_outer` is hard-coded to `false` — VERIFIED
|
|
|
|
Both constructors of `PinRepo` — `PinRepo::for_branch` and `PinRepo::from_repo`
|
|
(`engine/net/src/actors/client/pin_repo.rs`) — set `expose_outer: false`, and they are
|
|
the only two `PinRepoV0` constructions in the tree. No parameter carries the flag up:
|
|
`expose_outer` appears nowhere under `sdk/`. The broker side is fully wired
|
|
(`RepoInfo.expose_outer: HashSet<UserId>` in `engine/broker/src/server_broker.rs`, the
|
|
`if expose_outer` branch in `rocksdb_server_storage.rs`, the outer-overlay registration
|
|
in `server_storage/core/overlay.rs`), and the `PinRepo` responder even validates the flag
|
|
(refusing `expose_outer` from a peer that publishes no topic) — but no client ever sets
|
|
it.
|
|
|
|
**Consequence for this lib:** a store's **outer** overlay is never registered broker-side,
|
|
so there is no anonymous / capability-free read surface to build on. Everything is reached
|
|
through the inner overlay, i.e. through a read cap — the same cap-first addressing as
|
|
above. The "public store readable by everyone without permission" promise in the official
|
|
docs has no client-side switch today.
|
|
|
|
### The `Ext` protocol serves blocks with no control — VERIFIED
|
|
|
|
The `ExtObjectGetV0` responder (`engine/net/src/actors/ext/get.rs`) builds
|
|
`Store::new_from_overlay_id(&req.overlay, …)` from the OverlayId the **requester
|
|
declares**, then returns `Object::load_without_header(obj_id, None, &store)` blocks for
|
|
each requested id. No authentication, no verification that the requester belongs to that
|
|
overlay. The guards that were planned exist but are dead:
|
|
|
|
- `Authorization::ExtMessage` is matched in `Broker::authorize`
|
|
(`engine/net/src/broker.rs`) and returns `AccessDenied` — but **no caller ever passes
|
|
it**; the only `authorize` call sites pass `Discover`, `Admin` or `Client`. The
|
|
server-side `StartProtocol::Ext` arm in `engine/net/src/connection.rs` goes straight to
|
|
`StepReply::Responder`, never through `authorize`.
|
|
- the config flag whose comment reads *"are ExtRequest allowed on the server? this
|
|
requires the core to be ON."* — `allow_read` in `engine/net/src/types.rs` — is declared
|
|
and defaulted to `false`, and **read nowhere**.
|
|
- `ExtRequestContentV0::get_actor` handles `WalletGetExport` and `ExtObjectGet` and falls
|
|
through to `_ => unimplemented!()` for `ExtTopicSyncReq` — a **panic reachable from an
|
|
anonymous peer**. (The commented-out `// Self::ExtTopicSyncReq(a) => a.get_actor(),` on
|
|
that arm and the `// TODO inbox requests` in the enum are *direction hints*, labelled as
|
|
such — not current behaviour.)
|
|
|
|
**Consequence for this lib:** `Ext` is not a usable read path in either direction. Blocks
|
|
come back **encrypted**, and naming them requires ObjectIds you only have once you can
|
|
already read — so it grants no capability we could build on, and confirms the shape of
|
|
everything above: confidentiality lives entirely in the keys, and holding no key means
|
|
holding no partial access, just none.
|
|
|
|
## Known open issues (section added 2026-07-18)
|
|
|
|
Live limitations observed against the current core/SDK, each with its epistemic
|
|
status. **None is treated.** The status labels below are load-bearing — do not
|
|
upgrade an OPEN / UNDETERMINED / HYPOTHESIS item to "confirmed" or "fixed"
|
|
without new evidence.
|
|
|
|
### Write loss on socket death (`SerializationError`) — symptom VERIFIED, mechanism UNSETTLED, OPEN / untreated
|
|
|
|
A write made just before an idle period / spontaneous socket death
|
|
(`SOCKET IS CLOSED Some(Left(SerializationError))`) can be **silently lost**:
|
|
the entity is absent on reconnection while the account survives. Reconnection is
|
|
an unimplemented `// TODO` stub in the core (`broker.rs`, ≈ `1051-1076`);
|
|
`disconnections_subscribe` DOES fire on the failure but nothing — neither this
|
|
polyfill nor the consumer app — consumes it; and there is **no
|
|
write-durability-confirmation API** a caller could `await`. Full post-mortem
|
|
(logs, causal chain, correction leads, none arbitrated):
|
|
[`incidents/2026-07-14-write-loss-on-disconnect.md`](./incidents/2026-07-14-write-loss-on-disconnect.md).
|
|
|
|
### Cold-start read does not rehydrate the owner's own scope from the broker — symptom VERIFIED, root cause UNDETERMINED, OPEN / untreated
|
|
|
|
Decisive test (2026-07-14): a genuinely no-local cold reader — fresh
|
|
non-persistent browser context, SAME wallet + account — reads **0** of the
|
|
owner's own scope from the broker. The previously "passing" reconnect test was
|
|
FALSE-GREEN: it read the owner's repos from the persistent profile's LOCAL
|
|
IndexedDB, so it never proved broker durability. It is UNDETERMINED whether
|
|
**(i)** the write never durably reached the broker, or **(ii)** the write IS on
|
|
the broker but a fresh session cannot re-open the owner's own scope docs (a
|
|
cold-open / rehydration limitation) — both collapse to the same 0-read in this
|
|
setup. Next step (NOT done): disambiguate (i) vs (ii) with an independent warm /
|
|
second-identity read of the same doc. The same (i)/(ii) reserve is carried in
|
|
[`incidents/2026-07-14-write-loss-on-disconnect.md`](./incidents/2026-07-14-write-loss-on-disconnect.md)
|
|
(§ *Portée & non-reproduit*), whose Firefox case leans (i) — this cold-reader
|
|
signature is distinct (no socket death) and does not settle it.
|
|
|
|
### Reactive subscription may not echo the writer's OWN local commit — HYPOTHESIS (high-confidence), confirmation in progress (2026-07-18), NOT confirmed, NOT fixed
|
|
|
|
When a client does a local `sparqlUpdate` on a doc it is itself subscribed to
|
|
(`subscribeDoc`/`doc_subscribe`), the subscription callback appears NOT to fire
|
|
for its own local commit in the same session, so the polyfill's reactive re-read
|
|
chain never runs and consumers keep a stale value until the next connection
|
|
delivers a fresh initial `State`. REMOTE commits DO push correctly (verified:
|
|
cross-browser reactive update works). Verdict pending a live instrumented run.
|
|
Full write-up (suspect link, instrumentation, planned polyfill-side fix):
|
|
[`../packages/sdk/docs/sdk-reference.md`](../packages/sdk/docs/sdk-reference.md)
|
|
§ *Current emulation status*.
|
|
|
|
### Cold-start anchored read returns 0 rows instead of an error — symptom VERIFIED, mechanism INFERRED, healed polyfill-side
|
|
|
|
On a FRESH session over the SAME persistent wallet (reconnect, new page, re-login), an
|
|
anchored `sparql_query` against a document written in an earlier session comes back with
|
|
**0 rows and no error** — persisted documents read as empty. Observed on every anchored
|
|
reader of the polyfill and healed identically in each (`ensureRepoOpen` before the read,
|
|
`packages/sdk/src/emulated-verifier/open-repo.ts`): the user's own documents,
|
|
the user's store (`shared-wallet/account-registry.ts` `readUserStore`), the by-need doc batch
|
|
(`surface/read-model.ts` `readUnion`), and the store-root pointer read (`shared-wallet/account-registry.ts`
|
|
`resolvePointer`). The heal is `doc_subscribe(nuri)` → await the first `State` (the sync
|
|
barrier) → THEN the anchored read, and it is verified to return the data.
|
|
|
|
The circularity that made it self-inflicted (VERIFIED by the fix working): `doc_subscribe`
|
|
WOULD open the repo, but the reactive layer only subscribes AFTER a listing produced
|
|
NURIs, and the listing is itself an anchored read of a not-yet-open index repo → 0 rows →
|
|
nothing to subscribe → nothing ever opens.
|
|
|
|
**Mechanism INFERRED, not established.** `resolve_target_for_sparql(Repo(id))`
|
|
(`engine/verifier/src/request_processor.rs`) does
|
|
`self.repos.get(repo_id).ok_or(RepoNotFound)`, so a repo genuinely absent from
|
|
`self.repos` should ERROR, not return 0 rows. The most plausible reading of the silent 0
|
|
is that the repo IS in `self.repos` (loaded from local user storage at bootstrap) while
|
|
its named graph in `graph_dataset` is not yet populated — commits not applied/synced yet
|
|
— so the query legitimately matches nothing. Not traced end to end; the tension with the
|
|
`RepoNotFound` path described in § *A repo is only queryable once OPENED/synced into the
|
|
store* is unresolved.
|
|
|
|
**Consequence for this lib:** a cold anchored read is NOT authoritative on its own — 0
|
|
rows does not mean absent. This is what imposes the open-then-read discipline on every
|
|
cold reader, and it is why the account trust root had to move behind a first-`State`
|
|
barrier (see § *The pointer → doc-shim indirection*).
|
|
|
|
### Account fork on concurrent provision — symptom VERIFIED, guarded polyfill-side, residue persists in wallets
|
|
|
|
On a fresh page, several independent callers hit `ensureAccount(A)` near-simultaneously
|
|
(the public and protected `watchShape`, container subscriptions, the app's owned-events
|
|
effect). When the account is genuinely new, each caller sees 0 and each provisions its
|
|
own set of three scope documents — an **in-session account fork**. The persisted residue
|
|
is a single account subject carrying MULTIPLE values for one scope predicate (observed:
|
|
five `shim:docPublic`), after which a writer and a later reader can resolve DIFFERENT
|
|
scope docs and the reader's anchored read returns 0.
|
|
|
|
Two polyfill-side guards, both in `packages/sdk/src/shared-wallet/account-registry.ts`: `ensureInFlight`
|
|
(a bounded promise map keyed by account, so concurrent `ensureAccount` calls share ONE
|
|
resolve-or-provision) prevents new forks; `canonicalDoc` (pick the lexicographically
|
|
smallest NURI among all distinct values for a scope predicate — NURIs are
|
|
content-addressed, so the order is total and session-independent) makes resolution
|
|
deterministic on wallets that already carry fork residue. The earlier account-level
|
|
`provisionRetry` / `resolveAccountReliably` loop is gone, replaced by the doc-shim
|
|
barrier.
|
|
|
|
**Consequence for this lib:** the underlying enabler is core-side — there is no atomic
|
|
create-if-absent, and no existence probe to settle "does this account already exist?"
|
|
(see § *No existence probe at SDK level*), so provisioning is a read-then-create race the
|
|
polyfill has to serialize itself. The guards are mitigation, not a fix: a wallet already
|
|
corrupted stays corrupted, and only `canonicalDoc` keeps it readable.
|
|
|
|
### Outbox replay aborts on an unknown topic (`REPLAY TOPIC NOT FOUND`) — VERIFIED in core, already documented as an incident
|
|
|
|
`Verifier::send_outbox` (`engine/verifier/src/verifier.rs`) walks the queued events and,
|
|
for each, looks up `self.topics.get(&(overlay, topic_id))`. On a miss it logs
|
|
`REPLAY TOPIC NOT FOUND <topic> IN OVERLAY <overlay>` and sets `need_replay`, calls
|
|
`load_from_credentials_and_outbox(&events_to_replay)`, then in the send loop does
|
|
`self.topics.get(…).ok_or(NgError::TopicNotFound)?` — the `?` **aborts the whole outbox
|
|
flush**, so the remaining queued events are not sent. There is no per-event isolation and
|
|
no signal to the caller.
|
|
|
|
Already covered — **not duplicated here**: this is the core-side mechanism behind the
|
|
symptom described in § *Write loss on socket death (`SerializationError`)* above, whose
|
|
full post-mortem (logs, causal chain, the unarbitrated (i)/(ii) reserve) is
|
|
[`incidents/2026-07-14-write-loss-on-disconnect.md`](./incidents/2026-07-14-write-loss-on-disconnect.md).
|
|
The spontaneous socket death (`SOCKET IS CLOSED Some(Left(SerializationError))`) is
|
|
likewise covered there and in that section — the only fact added here is the abort
|
|
semantics of the replay path itself (VERIFIED by reading `send_outbox`).
|
|
|
|
**Consequence for this lib:** a queued write can be dropped without any observable error,
|
|
and one unknown topic can take the rest of the queue with it. The polyfill's own
|
|
`shared-wallet/outbox-log.ts` does not record anything: it exports a single `inspectOutbox()` that
|
|
READS the SDK's own `sessionStorage` outbox and logs how many peers still have queued
|
|
writes. It observes the symptom; it holds nothing it could replay, and no
|
|
write-durability confirmation exists to await — so "the write returned" is not "the write
|
|
is durable".
|