Reconnexion/ng-eventually
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat!: retirer readUnion de la surface publiee, et lire un document par requete ancree

Browse Source 
readUnion etait N requetes ancrees plus une politique de lot : une lecture
qui echoue avalee en reponse vide, un document dont on ne detient pas la cle
ecarte au lieu d'etre refuse, un regroupement en sacs de proprietes, et le
filtrage du prefixe reserve. Rien en amont ne repond a cette composition, et
son annotation citait sparql_query -- une citation exacte attachee a une
affirmation fausse. Une composition ne s'aligne pas sur une fonction.

Ce qui reste est ce qu'il y avait dessous : docs.sparqlQuery ancre sur un
document, qui va deja chercher la cle servie par le magasin public -- donc
une reference nue suffit toujours pour lire un document public, sans rien
changer a useShapeQuery. Lire plusieurs documents, c'est cet appel une fois
par document, et la tolerance a une lecture qui echoue revient a l'appelant,
la ou l'amont la laisse : une requete ancree sur un depot que le verifier ne
detient pas rejette.

Cote src, l'unique lecture interne restante devient read-document.ts : un
document, une requete ancree, precedee de l'ouverture du depot -- barriere
que index-deposit consulte pour distinguer une absence d'un depot non
synchronise. Le harnais e2e et l'exemple composent la leur, avec leur catch
visible au point d'appel.

Version majeure : 1.0.0-dev.8.
This commit is contained in:
Sylvain Duchesne
2026-09-21 14:08:05 +02:00
parent 05a0f7b148
commit 0c21adebf2
18 changed files with 659 additions and 516 deletions
Download Patch File Download Diff File Expand all files Collapse all files
packages/polyfill/src/surface/read-document.ts
+141
View File
@@ -0,0 +1,141 @@
/**
* read-document - read ONE document with an anchored SPARQL query, and group the rows
* it returns per subject. Library-internal: nothing here is published.
*
* -- What this replaced, and why the batch went ----------------------------
* Until `1.0.0-dev.8` this library published `readUnion(docs)`: the same anchored read,
* issued once per document, in parallel, with a batch policy of its own (a failing
* document was swallowed into an empty result unless the caller passed a callback to be
* told). Nothing upstream has a counterpart for that - it was N calls plus a policy -
* and its provenance annotation cited `sparql_query`, a citation that is exact attached
* to a claim that is false. **A composition cannot be aligned on a single function.**
*
* Every caller in production passed ONE document. So the batch is gone, and what is left
* is the thing upstream really does have: one anchored `sparql_query`. There is no
* swallow left to explain either - this throws, and the caller decides, which is the
* only honest answer once "one document failed" and "the read failed" are the same
* event.
*
* -- Why ANCHORED, and never anchorless -----------------------------------
* An anchored `sparql_query(sid, "SELECT ?s ?p ?o WHERE { ?s ?p ?o }", base, doc)` is
* restricted to the anchor repo's graph: `resolve_target_for_sparql(Repo)` ->
* `Some(repo_graph_name)`, which becomes the query's DEFAULT graph. A body with no
* `GRAPH` wrapper reads only that default graph, so only that document's triples - O(1)
* in the document's own size, independent of how many other graphs the local store holds.
*
* The footgun this avoids: an anchorless query (`anchor` undefined -> `UserSite` ->
* `set_default_graph_as_union`) spans EVERY named graph in the session store. On a shared
* wallet that accumulates across runs, that is O(wallet size) - the observed ~90s
* timeouts. Replacing several anchored reads with one anchorless query would also be a
* silent WIDENING of what a read can reach, so it is not an optimisation available here.
*
* NB (verified, `docs/read-model.md` probe step 4): an explicit `GRAPH ?g { ... }` body
* iterates the named graphs regardless of the default graph, so an anchor does not bound
* such a body. This read therefore uses a default-graph body - no `GRAPH` wrapper - so
* the anchor's one-repo restriction actually applies.
*
* -- The open in front of it is the EMULATION, not the query --------------
* On a fresh session over the same persistent wallet the target repo is present but
* UNSYNCED, and an anchored query against it legitimately matches nothing - no error,
* just zero rows. So {@link ensureRepoOpen} runs first and awaits the sync barrier, which
* is also where a document in a PUBLIC store hands over its `ReadCap`. That step is
* `polyfill-era` machinery: it is what one shared wallet costs, it is annotated where it
* lives (`emulated-verifier/open-repo.ts`), and it goes at migration with the shim.
*
* Calling it here rather than leaving it to the caller is deliberate and load-bearing:
* `emulated-verifier/index-deposit.ts` decides "absent" versus "unreadable" from
* {@link getSyncState}, and a read that never opened the repo would leave that state
* `unknown` - which that arm reads as "a zero-row answer MEANS it". Skipping the open
* would therefore turn a cold start into a destroyed deposit queue, silently. It is the
* failure that path has paid for twice; the barrier is not an optimisation to move.
*/
import { sparqlQuery } from "./docs";
import { ensureRepoOpen } from "../emulated-verifier/open-repo";
import { isMachinerySubject } from "../emulated-verifier/machinery";
import { session } from "../shared-wallet/account-registry";
import type { Nuri } from "../model/types";
/**
* One subject read from a document, with its properties (predicate -> values).
*
* `subject` is typed `string`, not `Nuri`: a subject is an ordinary RDF subject and may
* be ANY IRI. An application that writes its entity under the document's own reference
* gets a reference back here, but that is its convention, not this type's promise - a
* document may hold several subjects under IRIs of its writer's choosing, and each comes
* back as written.
*/
export interface DocumentSubject {
/** The subject IRI (`?s`), exactly as the document carries it. */
subject: string;
/** predicate IRI -> the list of object values (literals or IRIs) for it. */
props: Record<string, string[]>;
}
/** Tolerant extraction of SPARQL SELECT bindings across the shapes `ng` can return. */
function bindings(
result: unknown,
): Array<Record<string, { value: string } | undefined>> {
if (!result) return [];
if (Array.isArray(result))
return result as Array<Record<string, { value: string }>>;
const anyRes = result as {
results?: { bindings?: Array<Record<string, { value: string }>> };
};
return anyRes.results?.bindings ?? [];
}
/**
* Read `doc`'s triples with one anchored query and return them grouped per subject.
*
* **Throws** on anything that stops the read: a reference this holder may not reach
* (`docs.sparqlQuery` refuses it), a broker that does not answer, a query that rejects.
* That is the whole point of having no batch any more - there is no other document for a
* failure to be tolerated in favour of, so "it failed" reaches the caller as a failure
* rather than as an empty answer. A document that is genuinely empty, or whose repo has
* not synced, returns `[]` WITHOUT throwing - and telling those two apart is the
* caller's job ({@link getSyncState} is what answers it).
*
* A document holding SEVERAL subjects yields several entries - one per distinct subject,
* carrying that subject as written. Properties of different subjects are never merged.
* Placing one business entity per document stays the recommended practice (a key is per
* repo, so isolation needs a repo per entity), but that is a recommendation about
* WRITING: the read reports what is there, and does not get to enforce it by making the
* other arrangement invisible.
*
* The library's own compartments live as reserved SUBJECTS inside the very documents a
* consumer reads (the Header branch carrying a document's inbox address is the first).
* They are machinery, not the entity's properties, and are dropped here - once, for
* every compartment present and future.
*/
// @provenance readDocumentSubjects kind=invention level=none ref=none - the anchored query it issues is `docs.sparqlQuery`, aligned and annotated THERE; what is added here has no upstream counterpart at any level: the cold-start open a shared wallet forces, and the grouping of a SELECT's rows into per-subject property bags. Internal, and named as ours so a later reader knows it is ours to change
export async function readDocumentSubjects(
doc: Nuri,
label = "readDocument",
): Promise<DocumentSubject[]> {
// The cold-start open, and the public store's chance to hand over its key, BEFORE the
// anchored read - see the module header for why this cannot move to the caller.
await ensureRepoOpen(doc);
const { sessionId } = await session();
// Anchored to `doc` -> default graph = this repo. No `GRAPH ?g` wrapper, so the
// anchor's one-repo restriction applies.
const rows = bindings(
await sparqlQuery(sessionId, "SELECT ?s ?p ?o WHERE { ?s ?p ?o }", undefined, doc, label),
);
const bySubject = new Map<string, DocumentSubject>();
for (const row of rows) {
const s = row.s?.value;
if (isMachinerySubject(s)) continue;
const p = row.p?.value;
const o = row.o?.value;
if (s === undefined || !p || o === undefined) continue;
let entry = bySubject.get(s);
if (!entry) {
entry = { subject: s, props: {} };
bySubject.set(s, entry);
}
(entry.props[p] ??= []).push(o);
}
return [...bySubject.values()];
}
packages/polyfill/src/surface/read-model.ts
-282
View File
@@ -1,282 +0,0 @@
/**
* read-model — the listing primitive of the polyfill: read a bounded, by-need set
* of documents, each with its own anchored `sparql_query`, and return the triples
* grouped per subject. This is the mechanism documented in docs/read-model.md.
*
* ── Why per-doc anchored, rather than an anchorless union-scan ─────────────
* An anchored `sparql_query(sid, "SELECT ?s ?p ?o WHERE { ?s ?p ?o }", base, doc)`
* is restricted to the anchor repo's graph: `resolve_target_for_sparql(Repo)` →
* `Some(repo_graph_name)`, which becomes the query's default graph. A body with no
* `GRAPH` wrapper reads only that default graph → only that doc's triples, O(1) per
* doc, independent of how many other graphs the local store holds.
*
* The footgun this avoids: an anchorless query (`anchor` undefined → `UserSite` →
* `set_default_graph_as_union`) spans EVERY named graph currently in the session
* store. On a shared / bloated wallet that accumulates across runs, that is
* O(wallet size) → the observed ~90s timeouts. So the read path never union-scans
* all graphs — it reads exactly the bounded by-need set, one anchored query per doc.
*
* NB (verified, docs/read-model.md § probe step 4): an explicit `GRAPH ?g { … }`
* body iterates the named graphs regardless of the default graph, so an anchor does
* not bound such a body. The per-doc read therefore uses a default-graph body (no
* `GRAPH` wrapper) so the anchor's one-repo restriction actually applies.
*
* ── Why not the reactive ORM fan-out ──────────────────────────────────────
* `useShape({ graphs: […manyDocs] })` drives `orm_start_graph` over a fan-out of
* per-entity graphs; a freshly-created / not-yet-synced doc in that fan-out makes
* `RepoNotFound` abort the whole subscription → the readyPromise never resolves →
* the ~75s hang (docs/nextgraph-current-state.md § The ORM fan-out hang). Listing
* is instead a set of one-shot anchored `sparql_query`s. There is no reactive
* union query, so reactivity is assembled by re-querying on a change signal.
*
* ── Generic by construction ───────────────────────────────────────────────
* No application domain here: the consumer passes the doc NURIs to read (from
* the discovery index for public events, or its own scope docs for my-entities)
* and interprets the returned per-subject property bags. All NextGraph I/O routes
* through the T01.a `docs` primitives (the real injected `ng`), so this module
* imports no `@ng-org` package.
*
* At the real multi-store migration the per-doc anchored read is unchanged (native
* SPARQL, anchored to one repo); only bringing a repo into the session (open by cap)
* changes — the anchored query already resolves a same-session repo directly.
*/
import { docCreate, sparqlUpdate, sparqlQuery } from "./docs";
import { getCaps, getStoreRegistryDeps } from "../shared-wallet/bootstrap";
import { mustNotAttempt } from "../emulated-verifier/reach";
import { ensureReposOpen } from "../emulated-verifier/open-repo";
import { assertNuri } from "./sparql";
import { toNuri } from "../model/nuri";
import { isMachinerySubject } from "../emulated-verifier/machinery";
import type { Nuri, NuriLike } from "../model/types";
// Keep the primitives referenced so tree-shaking never drops the import used by
// the (side-effecting) open step below; `docCreate`/`sparqlUpdate` are not used
// here but the module intentionally depends only on the docs primitive surface.
void docCreate;
void sparqlUpdate;
/** One subject read from a doc, with its properties (predicate → values). */
// @provenance UnionSubject kind=invention level=none ref=none — no upstream type groups triples per (document, subject); a polyfill property bag, to be mapped into app types at the boundary
export interface UnionSubject {
/**
* The subject IRI (`?s`), exactly as the document carries it.
*
* Typed `string`, not `Nuri`: a subject is an ordinary RDF subject and may be ANY
* IRI. An application that writes its entity under the document's own NURI gets a
* NURI back here, but that is its convention, not this type's promise — a document
* may hold several subjects under IRIs of the consumer's choosing, and each comes
* back as written.
*
* The need that once made this `Nuri` is real and is met by {@link graph}: a
* consumer must be able to pass back what it just read without a cast —
* `shareNote(note.doc)`, `leaveMessage(note.doc)` — and a cast at that boundary
* would re-open exactly the confusion the template literal types exist to close.
* `graph` is the document reference, so that is the field to carry around.
*/
subject: string;
/**
* The document this subject was read from — the reference the caller passed to
* {@link readUnion}, unchanged. This is the anchor, it identifies the document
* stably, and it is what goes back into any call of this surface.
*/
graph: Nuri;
/** predicate IRI → the list of object values (literals or IRIs) for it. */
props: Record<string, string[]>;
}
/** Tolerant extraction of SPARQL SELECT bindings across possible shapes. */
function bindings(
result: unknown,
): Array<Record<string, { value: string } | undefined>> {
if (!result) return [];
if (Array.isArray(result))
return result as Array<Record<string, { value: string }>>;
const anyRes = result as {
results?: { bindings?: Array<Record<string, { value: string }>> };
};
return anyRes.results?.bindings ?? [];
}
async function sessionId(): Promise<string | number> {
return (await getStoreRegistryDeps().getSession()).sessionId;
}
/**
* Read one doc with an anchored default-graph query, tolerant per-doc.
*
* The anchor (`doc` NURI) restricts the query to that repo's graph as the default
* graph (`resolve_target_for_sparql(Repo)` → `Some(repo_graph_name)`); a body with
* no `GRAPH` wrapper reads exactly that default graph → only this doc's triples.
* This is O(1) in the doc's own size and independent of the rest of the (possibly
* bloated / shared) session store — it never iterates other graphs.
*
* COLD-START (fresh session, same persistent wallet): the repo is NOT in
* `self.repos` until something opens it, and an anchored query against an unopened
* repo silently returns 0 rows (never `RepoNotFound`). {@link readUnion} therefore
* opens the batch's repos ({@link ensureReposOpen}) BEFORE this read runs, so the
* anchored query resolves a same-session repo directly. A genuinely-absent repo
* still yields `[]` (in isolation, never aborting the others). Returns the doc's
* rows, or `[]` on failure.
*
* At the real multi-store migration this becomes a real sync: opening a per-user
* store repo by cap is a native broker fetch (`verifier.rs:1423` `OpenRepo` TODO).
*/
async function readDoc(
sid: string | number,
doc: Nuri,
onUnreadable?: (doc: Nuri, error: unknown) => void,
): Promise<Array<Record<string, { value: string } | undefined>>> {
try {
const nuri = assertNuri(doc);
// Anchored to `nuri` → default graph = this repo. No `GRAPH ?g` wrapper, so
// the anchor's one-repo restriction applies (an explicit `GRAPH ?g` body would
// iterate all named graphs regardless of the anchor — see docs § probe step 4).
const res = await sparqlQuery(
sid,
"SELECT ?s ?p ?o WHERE { ?s ?p ?o }",
undefined,
nuri,
"readDoc",
);
return bindings(res);
} catch (error) {
console.error("[read-model] read failed for", doc, error);
// The swallow STAYS — a batch read has always tolerated one bad document, and a caller
// building a list from optional values must not lose the whole read to one hole. What is
// new (2026-08-26) is that the failure is no longer INDISTINGUISHABLE from an empty
// document: a caller that needs to tell them apart says so by passing `onUnreadable`.
//
// It needed telling apart because of inbox consumption. A deposit is now retired before
// its arm runs, so an arm that reads `[]` and concludes "absent, nothing to do" turns a
// broker hiccup into permanent loss — and one that concludes "must be a failure" lets a
// single unresolvable reference stall every deposit behind it. Only the truth serves.
onUnreadable?.(doc, error);
return [];
}
}
/**
* Read a BOUNDED, by-need set of docs — each with its OWN anchored query — and
* return the triples grouped per subject. `docs` are the NURIs to read (the
* consumer resolves them by need — index for public, own scope docs for mine).
* Docs that fail are skipped (see {@link readDoc}); a failing doc never aborts the
* batch.
*
* A document holding SEVERAL subjects yields several entries — one per distinct
* subject, carrying that subject as written, all sharing the document as their
* `graph`. Properties of different subjects are never merged. Placing one business
* entity per document stays the recommended practice (a key is per repo, so
* isolation needs a repo per entity), but it is a recommendation about writing: the
* read reports what is there.
*
* ── Why this enumerates, and why that is OURS and not the target's ────────
* Upstream DOES have a query that spans many documents: `resolve_target_for_sparql`
* (`engine/verifier/src/request_processor.rs`) accepts `UserSite` — or no target at
* all — and returns `Ok(None)`, which the verifier itself annotates "QUERYING UNION
* GRAPH". It is refused for updates (`InvalidTarget`) and allowed for reads. So
* "the target cannot read several documents at once" is FALSE, and this path must
* not be read as compensating for a missing upstream capability.
*
* What makes that union unusable HERE is our own emulation: one shared wallet holds
* EVERY virtual identity, so a scan over all graphs crosses identity boundaries that
* upstream does not have (there, a wallet is one person's). The cost argument —
* O(wallet size) — follows from the same artefact. Both reasons are `polyfill-era`.
*
* Each doc is therefore read with an anchored default-graph query, O(1) per doc,
* in parallel via `Promise.all`.
*
* OPEN, and deliberately not settled here: whether enumerating is what an
* application should be taught at all. Targeting `PrivateStore` is a first-class
* option upstream, and whether it reaches a document's CONTENTS — or only the store
* repo's own graph, documents being repos of their own — decides it. Until that is
* established, treat this shape as unconfirmed rather than as the aligned answer.
*/
// @provenance readUnion kind=aligned level=2 ref=sdk/js/lib-wasm/src/lib.rs:sparql_query — one anchored query per document — upstream's own anchoring (`resolve_target_for_sparql`), composed client-side
// @provenance readUnion.enumeration kind=invention level=none ref=none — upstream offers a union query over many documents (`UserSite`/no target → union graph); enumerating instead is OURS, forced by the shared wallet, and UNCONFIRMED as the shape to teach — see the docstring's open question
// @provenance readUnion.swallow kind=invention level=none ref=none — a failing document is skipped and reads as empty; upstream has no batch read at all, so it has no counterpart for what a partial failure should do
export async function readUnion(
docsLike: NuriLike[],
onUnreadable?: (doc: Nuri, error: unknown) => void,
): Promise<UnionSubject[]> {
const sid = await sessionId();
// Drop the empties BEFORE validating, not after: this call has always tolerated a
// list with holes in it — a scope index can carry a blank entry, and a caller
// building a list from optional values should not have to compact it. Validating
// first turned that tolerance into a throw, which took down a whole reconnect run.
// Empty is absence, and absence is not a malformed reference.
const unique = [...new Set(docsLike.filter(Boolean))].map((d) => toNuri(d, "readUnion"));
if (unique.length === 0) return [];
// COLD-START heal (polyfill-era): on a fresh session over a persistent wallet the
// target repos are not yet in `self.repos`, so an anchored read would return 0
// rows. Open/subscribe each repo ONCE (idempotent, per session) and await its
// initial-state push before the anchored reads. No-op once opened / when the
// injected `ng` has no `doc_subscribe` (unit fake). See open-repo.ts.
//
// Called on the WHOLE set, before the boundary is consulted, because opening is also
// where a document in a PUBLIC store hands over its cap (see public-store.ts): a
// document filtered out first would never get the chance to answer. `ensureRepoOpen`
// still refuses to open what this user may not touch — it asks, it does not enter.
await ensureReposOpen(unique);
// RULE 2 — do not even attempt. Drop the documents whose cap this user does not
// hold before reading anything: upstream you cannot address a repo you have no cap
// for, so asking about one is not "a read that will be refused", it is a read that
// has no meaning. (The passage points enforce rule 1 regardless — see reach.ts — so
// a lapse here is caught, not exploited.)
const reachable = unique.filter((d) => !mustNotAttempt(d));
// One anchored query per doc, in parallel, tolerant (a bad doc yields []).
const perDoc = await Promise.all(
reachable.map(async (d) => ({
doc: assertNuri(d),
rows: await readDoc(sid, d, onUnreadable),
})),
);
// Possession gate, kept as defence in depth behind rule 2 above: `reachable`
// already excluded these, so this loop should never drop anything. The read is
// anchored per document, so the unit of possession is the document: the cap key is
// the doc NURI, whatever subjects that document turns out to carry.
const caps = getCaps();
// Keyed by (document, subject) — an entry is one subject INSIDE one graph, which is
// the identity upstream gives an object too: the ORM carries `@id` and `@graph` as
// two distinct read-only properties, and fabricates an `@id` when the writer leaves
// it empty (`sdk/js/orm/src/connector/GraphOrmSubscription.ts`). Several objects per
// graph is therefore the PROVIDED case, and `@id` is what tells them apart within a
// `@graph`. Two documents carrying the same subject IRI stay two entries: they are
// two objects, distinguished by their graph.
//
// Placing one business entity per document remains the recommended practice — a key
// is per repo, so isolating an entity requires a repo of its own. That is a
// recommendation about WRITING, and the read does not get to enforce it by making
// the other arrangement invisible.
const bySubject = new Map<string, UnionSubject>();
for (const { doc, rows } of perDoc) {
if (caps.isEnforcing() && caps.capFor(doc) === undefined) continue;
// Anchored to `doc`, so every row belongs to `doc` — hence `graph` is the caller's
// reference to it. The subject comes back exactly as the document carries it.
for (const row of rows) {
// The polyfill's own compartments live as reserved SUBJECTS inside the very
// documents the consumer reads (the Header branch carrying a document's inbox
// address is the first). They are machinery, not this entity's properties —
// drop them here, once, for every compartment present and future.
const s = row.s?.value;
if (isMachinerySubject(s)) continue;
const p = row.p?.value;
const o = row.o?.value;
if (s === undefined || !p || o === undefined) continue;
// NUL cannot appear in an IRI, so the pair never collides with either half.
const key = `${doc}\u0000${s}`;
let entry = bySubject.get(key);
if (!entry) {
entry = { subject: s, graph: doc, props: {} };
bySubject.set(key, entry);
}
(entry.props[p] ??= []).push(o);
}
}
return [...bySubject.values()];
}