Reconnexion/ng-eventually
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix!: la garde circulaire disparait, et une nouvelle tentative se fait au nom de qui a ete refuse

Browse Source 
Troisieme tour de revue adversariale, trois constats, tous reproduits avant
correction.

1. La garde etait CIRCULAIRE, et c'etait structurel. `namedDocumentsReachable`
   demandait si la capacite est DETENUE. Pour un document en store public, le seul
   canal qui puisse jamais la deposer est `fetchReadCap` - l'appel meme que la
   garde retenait. Elle attendait une reponse que seule l'action qu'elle bloquait
   pouvait produire, et toute cette classe restait irrecuperable.

   La bonne question n'est pas une meilleure garde, c'est de RETENTER. Les deux
   facons d'etre refuse se reparent par des choses opposees, et le memo les
   distingue deja : un echec n'est deliberement pas memoise, donc une nouvelle
   tentative redemande vraiment au reseau - c'est la reparation de cette classe,
   pas un cout a eviter ; une absence est memoise, donc une nouvelle tentative ne
   demande a personne et coute un instantane, jusqu'au jour ou la capacite arrive
   d'ailleurs, seule chose qui repare celle-la. La garde est supprimee : le module
   est plus petit qu'au tour precedent.

2. La reouverture faisait un ACCES sous la mauvaise identite. Republier REFILTRE
   une requete existante ; rouvrir EMET un abonnement et une requete au store
   public pour un document que cet abonne a nomme. C'est un acte neuf, pas un
   ancien qui ressort, et l'etat final etait `isSuccess` avec `data: []` pour un
   document illisible - le mensonge que cette tache supprime, par la porte du
   correctif. `refusedFor` retient QUI a ete refuse ; on ne retente que tant que
   c'est encore lui le connecte.

   Etabli plutot qu'affirme : une application ne peut pas changer d'identite en
   cours de session. `ensureIdentity` est le seul appel publie qui en etablit une,
   et `resolveIdentity` commence par `getCurrentUser()`. Corrige quand meme.

3. `Promise.all` court-circuite a la premiere rejection ARRIVEE, et un fetch qui
   rejette n'attend pas la barriere de synchronisation : il double un refus portant
   sur un document nomme avant lui. Le comportement est corrige plutot que le
   commentaire - `allSettled` puis parcours dans l'ordre du scope.

Et le quatrieme controle e2e est retire, pas repare : il passait avec toute la
machinerie de reprise supprimee. Un controle qui ne peut echouer pour aucune
regression de ce qu'il garde se lit comme de la couverture et vaut moins que rien.
This commit is contained in:
Sylvain Duchesne
2026-09-21 16:28:24 +02:00
parent aecb7cf2e8
commit f875047099
5 changed files with 234 additions and 90 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.project/concepts/app-contract/polyfill-surface/contract_polyfill-surface.md
+2 -2
View File
@@ -195,7 +195,7 @@ Where a call must first find out whether something already exists — a document
**A document you NAME in `graphs` is reached before the read opens, since `1.0.0-dev.9`.** Naming a document in a scope is the same act as anchoring `docs.sparqlQuery` on it, and it is now answered the same way: the key is asked of a public store if you do not already hold it, the document is refused out loud if you hold no key for it, and its repo is synced before the question is put. A named document you cannot read publishes `isError`, where until this version it reached the engine, came back, and was emptied by the per-document filter into `isSuccess` with `data: []` - the one snapshot this surface reserves for "this scope holds nothing". A named document that has never synced published that same snapshot for a document that is not empty; it too is now `isError` or a longer `isPending`, never a false emptiness. **Naming SEVERAL documents fails as a whole** when one of them cannot be reached: a short list is the same lie as an empty one, so nothing is answered rather than half of it. What is NOT touched, and each stays a success: `graphs: "did:ng:i"` or `""` (you named no document - the per-document filter is the whole of the answer there, and an empty result means you hold nothing of that shape), `graphs: []` (no objects, by NextGraph's own definition), and a named document you can read that holds nothing of the shape.
**And it converges by itself when the key arrives.** A named scope refused this way is not a dead view: the keyring signal that already re-published a filtered read now also RE-OPENS a scope the barrier refused, as soon as the holder reaches every document it names. So the ordinary sequence - a page whose components subscribe while the connection work is still restoring keys, or a document shared with you while you are looking at the screen - shows `isError` and then the document, with no call from the application. Two things follow, and both are deliberate: an interface that renders `isError` will briefly show it where it used to show an empty success, so render the load state rather than only `data`; and the retry is TARGETED - a key landing for some other document leaves the scope refused rather than spending a round-trip on it, so a failure that no key can lift (the broker did not answer) stays until `refetch()`, which is what `isError` has always meant here.
**And it converges by itself.** A named scope refused this way is not a dead view: the keyring signal that already re-published a filtered read now also re-ATTEMPTS a scope the barrier refused. So the ordinary sequence - a page whose components subscribe while the connection work is still restoring keys, a document shared with you while you are looking at the screen, or a broker that hiccupped once and healed - shows `isError` and then the document, with no call from the application. Both ways of being refused are covered, and they are repaired by opposite things: one needs the key to arrive from elsewhere, the other needs the store to be asked again, and every keyring movement re-attempts both. One consequence to design for: an interface that renders `isError` will briefly show it where it used to show an empty success, so branch on the load state rather than on `data` alone. Nothing here polls, so a scope refused in a session whose keyring never moves again stays `isError` until your own `refetch()` - which is what `isError` has always meant here, and what it means for every other failure this call reports.
**What it costs.** One public-store ask and one sync barrier per document named, once per session. The opens for a scope naming several documents run together, so the wait is the slowest of them rather than their sum. A document that never answers the barrier is bounded by an internal fallback rather than waiting for ever, and until `1.0.0-dev.9` that same document left the read `isPending` with no bound at all.
@@ -277,7 +277,7 @@ What each level means here, in this package's own terms:
- **`useShapeQuery` publishes `isError` for a document you NAME and cannot read.** Until this version a document named in `scope.graphs` was handed straight to the engine; whatever came back was filtered per document by what you hold, and a document you hold no key for was emptied out of the result. The snapshot was `isSuccess` with `data: []` - indistinguishable, byte for byte, from the scope genuinely holding nothing. It now asks the public store for the key, refuses out loud if there is none to be had, and crosses the sync barrier first, so the same read answers `isError`. **Handle `isError` on a named scope**; an interface that only branched on `isPending` / `isSuccess` will show its success state with nothing in it. Naming several documents fails as a whole when one of them fails.
- **A named scope now costs round-trips it did not cost before**, once per named document per session: the key fetch, and the subscribe that syncs the repo. The opens run together, so several named documents cost the slowest wait rather than the sum. `graphs: "did:ng:i"` is unchanged and spends nothing.
- **A refused named scope RECOVERS on its own** when the key arrives, through the same keyring signal that already re-published a filtered read. Without that a refusal would be terminal, and trading a silent empty answer for a permanently stuck error would have been no improvement at all. Expect to see `isError` briefly on a page that subscribes before its keys are restored.
- **A refused named scope RECOVERS on its own**, through the same keyring signal that already re-published a filtered read - whether what it needed was a key arriving from elsewhere or the public store simply being asked again. Without that a refusal would be terminal, and trading a silent empty answer for a permanently stuck error would have been no improvement at all. Expect to see `isError` briefly on a page that subscribes before its keys are restored. There is no polling: in a session whose keyring never moves again, `refetch()` is the recovery, as it is for every other failure this call reports.
- **`inbox.postToDocument`, `inbox.readForDocument` and `inbox.readSyncedForDocument` reject where they used to report "no inbox".** Resolving a document's inbox address reads that document, and when the read could not happen - the broker could not answer for it, or its public store could not be asked for the key - all three used to render the breakdown as the document's owner having opened nothing: a refusal on the deposit side, an empty list on the read side. They now reject. A document that genuinely has no inbox is unchanged, and still answers exactly as before.
- **Nothing that was already an ANSWER moves.** An empty document is a success holding zero objects, `graphs: []` is a success holding none, a document in a public store that exposes no key is a plain "no", and a document with no inbox answers `[]`. The whole change is that a *failure* stopped borrowing those shapes.
- **Why this direction, and not a richer set of outcomes.** The step that collapsed the two was this package's own emulation of a public store serving its key; NextGraph has no counterpart for it, because upstream nothing consults a capability on the read path at all. So there is no signature to mirror, and the alignment falls to the engine's own observable: an anchored query on a repo the verifier does not hold answers `NgError::RepoNotFound`, a rejected promise - **a read rejects, or it does not**. Inventing a third outcome for a caller to branch on would have been a vocabulary the target will never grow.
.tasks/doing/T01.j-a-failure-must-not-read-as-an-absence.md
+24 -4
View File
@@ -129,7 +129,7 @@ Two findings, both reproduced here before anything was changed, both diagnoses v
**Finding 1, blocking - a refused named scope was TERMINAL.** Reproduced as `use-shape-query.test.ts` (r): a named document whose cap arrives AFTER the subscribe stayed `{isError: true, data: []}` for ever. Diagnosis confirmed by reading: the keyring handler is `if (query !== null) publish(query, generation)`, and the barrier aborts BEFORE any query is created, so the one channel that makes a late cap converge went straight past the state that needed it most. It broke a guarantee this package publishes and that `read-filter.ts`, `use-shape-query.ts` and `inbox.ts` all restate - a document shared with you while you watch becomes readable with no call from the application. **Trading a silent empty answer for a stuck error is worse than the defect**, and the ordinary sequence (a page subscribing while its keyring is still being restored) hits it.
Fix: the keyring signal now RE-OPENS a scope the barrier refused, gated on `namedDocumentsReachable()` - every named document reachable now. Blast radius traced before writing it, and the gate is the load-bearing half: re-opening on any keyring change would re-run the barrier for every cap a connection files, re-ask the network for any document whose fetch had FAILED (a failure is deliberately not memoised) and re-publish the same error to every subscriber each time. `barrierRefused` keeps a `configure` with no `ormSubscription` - which no cap will ever repair - out of the retry entirely. The indiscriminate variant was tried and breaks two tests, (s) and the pre-existing teardown test (k).
Fix, round 2: the keyring signal RE-OPENS a scope the barrier refused, gated on `namedDocumentsReachable()`. **That gate was itself wrong and round 3 refuted it - see below.**
**Finding 2 - the INERT claim was false.** Reproduced by reading and pinned as (u): `ensureRepoOpen` runs whatever the registry is enforcing, so the inert case gained a real `doc_subscribe` and a barrier wait. The open is right and stays; what was wrong was calling it "unchanged". The measured COST was real though: the barrier was SERIAL, so N documents that never push a `State` cost N bounded fallbacks - 8s each in production. Now parallel, so the wait is the slowest rather than the sum, with the refusals still read in scope order afterwards. Pinned by (t), which fails on the serial form.
@@ -137,13 +137,33 @@ Fix: the keyring signal now RE-OPENS a scope the barrier refused, gated on `name
**What the e2e still does NOT prove, stated plainly.** The POSITIVE recovery - the cap for the refused document itself arriving - is proven at unit level only (test r, non-vacuous). Driving it live needs a second actor delivering through `inbox.share`, and on one page that means switching the current identity while the observable is subscribed, which would re-open the refused scope under the OTHER identity. That is a pre-existing property of this observable (`publish` has always read the keyring of whoever is connected), it cannot arise in an application (one identity per page load, no call takes an identifier), and the repo has already ruled that such montages belong in the applicative suite - `capsShareCap` was moved there for exactly this reason. Left as a named follow-up rather than encoded here as if it were the library's behaviour.
### CORRECTED AGAIN - round 2's repair was REFUTED, three findings, all reproduced here first
**Finding 1, blocking and STRUCTURAL - the gate was circular.** `namedDocumentsReachable()` tested `mayReach`, i.e. the cap is HELD. For a named document in a PUBLIC store the only channel that can ever file that cap is `fetchReadCap` - the very call that had failed and set the flag. The gate waited on an answer that only the action it gated could produce, so that whole class was unrecoverable however many caps arrived. Reproduced as `use-shape-query.test.ts` (v): the broker hiccups on the `exposedReadCap` read, the fetch rejects, the broker heals, an ordinary cap arrival follows, and the view never re-asks.
**The readiness question, answered rather than patched.** There are exactly two ways this barrier can refuse, and the memo already tells them apart. The store could not be ASKED: a failure is deliberately not memoised (an earlier fix in this repo), so a re-attempt genuinely re-asks the network - which IS the repair for this class, not a cost to avoid. Or the store ANSWERED and there is no cap to be had: an absence is memoised, so a re-attempt asks nobody, re-runs the barrier against what is already known, and republishes the same refusal for the price of one snapshot - and it stops the day the cap arrives from elsewhere, which is the only thing that repairs THAT class. So the right signal is not a cleverer predicate, it is **attempting again**: cheap for the class it cannot help, correct for the class it can, self-limiting because the moment an attempt succeeds there is a query again. `namedDocumentsReachable()` is DELETED - the repair makes the module smaller than round 2 left it. Pinned by (v) and, for the cost, by (s), which now asserts both halves (the re-attempt ran, and it spent no round-trip) so it cannot pass vacuously.
**Finding 2 - the re-open performed an access under the wrong identity, and my "pre-existing" defence was wrong in kind.** The reviewer is right: re-publishing RE-FILTERS an existing query, while a re-open NEWLY ISSUES a subscribe and a public-store fetch for a document that subscriber named. That is a new action taken on behalf of whoever happens to be connected, not an old one resurfacing, and the end state was `{isSuccess: true, data: []}` for a document the subscriber cannot read - this task's own defect, through the repair's door. Fixed by binding the refusal to a holder: `refusedFor` records WHOSE barrier failed, and a re-attempt is made only while that holder is still the connected one. Pinned by (w), which fails without the binding.
**ESTABLISHED rather than asserted, as asked: an application cannot change identity mid-session.** `ensureIdentity()` is the only published call that settles one, and it delegates to `settleIdentity` -> `resolveIdentity`, whose first act is `const already = getCurrentUser()` - so a second call returns the identity already set. `setCurrentUser` is not exported from `src/index.ts`. So Finding 2's sequence is reachable only from a harness playing several identities on one page. It is fixed anyway, because a new access issued in someone else's name is a defect in kind whatever can reach it, and because the fix costs one comparison.
**Finding 3 - the determinism claim was false.** `Promise.all` short-circuits on the first rejection to ARRIVE, and a fetch that rejects does not wait for the sync barrier, so it overtakes a refusal on a document named before it. I chose to fix the BEHAVIOUR rather than the claim: `Promise.allSettled` plus a walk in scope order, so the document reported is the first one the caller named that failed, whatever it failed of, and the parallelism is kept. Pinned by (x), which fails on `Promise.all`.
**The fourth e2e check was removed, not repaired.** "A cap landing for ANOTHER document leaves a refused scope refused" passed with the whole re-attempt machinery deleted - the outcome is a standing refusal either way. A check that cannot fail for any regression of the thing it guards reads as coverage and is worse than none. Its properties are measured in the unit suite by (v), (s) and (w).
### The decision to KEEP the barrier rather than take the escape hatch
Taken deliberately, at the second offer. Removing it would restore, on a published read, a silent `isSuccess` with `data: []` for a document the caller cannot reach - the defect this task exists for, now proven live rather than argued. The escape hatch's own premise is "a stuck error is worse than a silent empty"; after these three repairs there is no stuck error left to weigh against it. Every class that can refuse is re-attempted on the only readiness signal this library has, and the one class that is not - the identity changed under the observable - cannot arise in an application and is answered honestly rather than by acting in someone else's name. The repairs are also subtractive: a circular predicate deleted, a short-circuit replaced by an ordered walk, a flag given the one extra fact it was missing. What remains is a scope refused in a session whose keyring never moves again, which stays `isError` until `refetch()` - the published meaning of `isError` on this surface, and what it already meant for every other failure it carries.
### Gate
- `bun test` -> `435 pass / 0 fail`, `Ran 435 tests across 40 files` (baseline `421 pass / 0 fail`, measured before any change; +14 new tests, no test deleted or skipped)
- `bun test` -> `438 pass / 0 fail`, `Ran 438 tests across 40 files` (baseline `421 pass / 0 fail`, measured before any change; +17 new tests, no test deleted or skipped - (s) was REWRITTEN, not weakened: it asserted the targeting predicate that round 3 proved circular, and now asserts the property that is actually true and worth guarding, on both halves so it cannot pass vacuously)
- `bunx tsc --noEmit -p packages/polyfill/tsconfig.json` -> exit 0
- `bunx tsc --noEmit -p packages/polyfill/e2e/tsconfig.json` -> exit 0
- `python3 ~/.claude/skills/concept/lint.py` -> 0 ERROR (warnings pre-existing; the new leaf carries none)
- polyfill e2e, run at HEAD: `Polyfill e2e summary: 47 passed, 0 failed, 47 total - batch 3.4 min, slowest cold sync 29s`. 43 was the count before this task; the four added are the named-scope checks described above. NOTE: the broker login is flaky and independent of this work - three of ten runs in this session died with `[e2e broker login] gave up after 45.0s - the wallet-list screen never changed`, at a different step each time, on the login UI before any library code runs. Every such run was re-run to a clean result.
- polyfill e2e, run at HEAD: `Polyfill e2e summary: 46 passed, 0 failed, 46 total - batch 3.5 min, slowest cold sync 29s`. 43 was the count before this task; three named-scope checks were added and a fourth removed as non-discriminating. NOTE: the broker login is flaky and independent of this work - three of eleven runs in this session died with `[e2e broker login] gave up after 45.0s - the wallet-list screen never changed`, at a different step each time, on the login UI before any library code runs. Every such run was re-run to a clean result.
**What the e2e PROVES**, and it is the whole of it: against the real broker, a named scope reads for the holder who owns the document; a holder of nothing is refused out loud with `dataLen: 0` rather than told the document is empty; and a public document opens for that same holder from nothing but its reference, through the barrier's own fetch. **What it does NOT prove**: anything about the re-attempt. Every recovery path - the public class repaired by re-asking, the cost of a doomed re-attempt, and the holder a re-attempt is made as - is measured only in the unit suite, against the `CapRegistry`, which is where that mechanism lives and which the e2e's own header says the broker does not implement. Driving the positive recovery live needs a second actor delivering through `inbox.share`, and belongs in the applicative suite.
### The contract
@@ -175,7 +195,7 @@ Fix: the keyring signal now RE-OPENS a scope the barrier refused, gated on `name
- Every e2e run logs, at connection: `connect failed: docs.sparqlQuery: refused - the connected user does not hold this document's cap ... at readStoreCaps <- restoreOwnCaps <- restoreAndDrain`. That is `assertMayReach`'s own pre-existing message (this work would have produced a different one), so `restoreOwnCaps` is being refused one of the connected user's OWN store documents at connect, and the failure is logged rather than raised.
- The e2e broker login fails intermittently at 45s on the wallet-list screen, before any library code runs - 2 of 8 runs in this session.
- A two-actor journey for "a document is shared with someone while they are watching, and their named `useShapeQuery` converges" belongs in the applicative suite (`e2e/notebook.ts`), beside the journey that already checks "after Alice shares it, the same reference opens it". It is the live form of unit test (r).
- Pre-existing and out of scope, surfaced while tracing the fix: this observable re-publishes, and now re-opens, under whoever is connected at that instant rather than under the identity that subscribed. Harmless for an application (one identity per page load, no call takes an identifier); it only shows on a harness playing several identities on one page.
- This observable re-PUBLISHES under whoever is connected at that instant rather than under the identity that subscribed. That part is genuinely pre-existing and out of scope - `publish` has always read the keyring of the current holder. The RE-ATTEMPT no longer does: it is bound to the holder that was refused (`refusedFor`), because issuing a subscribe and a fetch is a new access rather than a re-filtering, and the two must not be lumped together - round 2's Result did lump them, and that was wrong.
- `docs/internal-contract.md:7` still names `surface/watch-shape.ts`, `surface/read-model.ts` and a `storeRegistry` namespace listing three removed symbols. Out of this task's boundary.
- `knowledge_what-a-failed-read-answers-upstream` cites `213338f6`, the checkout T01.h found to be three ORM releases behind. The facts it cites are engine-side rather than ORM-side, but T01.k owns reconciling what a citation's version means.
packages/polyfill/e2e/run.ts
+8 -21
View File
@@ -517,27 +517,14 @@ async function main(): Promise<void> {
`snap=${JSON.stringify(publicSnap)}`,
);
// That fetch FILED a cap, which fires `CapRegistry.onChange` - the channel a
// refused scope is re-opened on since `1.0.0-dev.9`, because the barrier aborts
// before any query exists and `publish` has nothing to re-filter. This cap says
// nothing about the protected document, so the refusal must stand.
//
// What this measures, exactly, and it is less than it might look: the OUTCOME is
// unchanged. It does not separate "no re-open was attempted" from "one was
// attempted and refused again" - both end here - so the targeting itself is
// measured in the unit suite, by the notification count (`use-shape-query.test.ts`
// (s)). Saying which of the two a green check proves is the whole lesson of the
// round that shipped this barrier with no named-scope coverage at all.
//
// The positive direction - the cap for the refused document ITSELF arriving, and
// the view converging - needs a second actor delivering it, and belongs in the
// applicative suite; `use-shape-query.test.ts` (r) is its unit form.
const afterOtherCap = await sdkGet<any>(frame, "useShapeQuerySnapshot", refused);
check(
"a cap landing for ANOTHER document leaves a refused scope refused",
afterOtherCap.isError && !afterOtherCap.isSuccess && afterOtherCap.dataLen === 0,
`snap=${JSON.stringify(afterOtherCap)}`,
);
// A fourth check stood here until 2026-09-21: "a cap landing for ANOTHER document
// leaves a refused scope refused". It was REMOVED rather than repaired, because it
// passed with the whole re-attempt machinery deleted - the outcome is a standing
// refusal either way, so nothing it could observe told the two apart. A check that
// cannot fail for any regression of the thing it guards is worse than no check: it
// reads as coverage. The re-attempt is measured in `test/use-shape-query.test.ts`
// - (v) for the class repaired by re-asking the network, (s) for the one that is
// not and for what a doomed re-attempt costs, (w) for the holder it is made as.
await sdk(frame, "useShapeQueryStop", refused);
await sdk(frame, "useShapeQueryStop", publicRef);
});
packages/polyfill/src/surface/use-shape-query.ts
+72 -43
View File
@@ -118,11 +118,32 @@
* surface, it is a worse one - and it broke a guarantee this package publishes, which
* `emulated-verifier/read-filter.ts` and `surface/inbox.ts` both restate.
*
* So the signal re-OPENS a refused scope, and does it only when the refusal has actually
* been lifted ({@link namedDocumentsReachable}). Re-opening on any keyring change would
* re-run the barrier for every cap a connection files - re-asking the network for any
* document whose fetch had FAILED, since a failure is deliberately not memoised, and
* re-publishing the same error to every subscriber each time.
* So the signal re-ATTEMPTS a refused scope, for the holder that was refused.
*
* -- Why there is no cleverer gate than "attempt again" ----------------------------
* The obvious one is to re-open only once the refusal has been lifted - "does the holder
* hold every named document's cap now". It was written that way first, and it is CIRCULAR
* for a whole class: for a document in a public store the only channel that can ever file
* that cap is `fetchReadCap`, the very call the gate was there to withhold. A scope
* refused because the store could not be ASKED then waited on an answer that only the
* action it gated could produce, and no number of cap arrivals ever released it. A second
* adversarial pass proved it with the broker healing and ten caps landing afterwards.
*
* Attempting is the cheaper question anyway, because the two ways to be refused are
* repaired by opposite things and the memo already tells them apart:
*
* - **the store could not be asked.** A failure is deliberately NOT memoised
* (`public-store.ts`), so a re-attempt genuinely re-asks the network - which is the
* whole repair for this class, not a cost to avoid.
* - **the store answered and there is no cap to be had.** An absence IS memoised, so a
* re-attempt asks nobody: it re-runs the barrier against what is already known and
* republishes the same refusal. It costs one snapshot, and it stops the day the cap
* arrives from somewhere else, which is the only thing that can repair this class.
*
* It is self-limiting either way: the moment an attempt succeeds there is a query again,
* and this branch is never reached. What it is NOT is a timer - nothing here polls, and
* a scope refused in a session where the keyring never moves again stays `isError` until
* the caller's own `refetch()`, which is what `isError` has always meant on this surface.
*
* -- An observable cannot un-emit: `data` survives an error -----------------
* A one-shot call rejects and is done. An observable has already handed a list to a
@@ -136,7 +157,7 @@
import { getCaps, getConfig } from "../shared-wallet/bootstrap";
import { filterReadable } from "../emulated-verifier/read-filter";
import { ensureRepoOpen } from "../emulated-verifier/open-repo";
import { assertMayReach, mayReach } from "../emulated-verifier/reach";
import { assertMayReach } from "../emulated-verifier/reach";
import { isNuri } from "../model/nuri";
import type { Nuri, OrmSubscriptionHandle, OrmSubscriptionLike } from "../model/types";
// The SDK's own types, imported as TYPES (erased at build, so no runtime `@ng-org`
@@ -259,15 +280,24 @@ export function useShapeQuery<T extends BaseType = BaseType>(
* a change signal that beats `readyPromise` must not be read as an answer. */
let ready = false;
/**
* Why there is no query in force: the BARRIER refused this scope, rather than the
* shape query failing to open or nothing having been tried yet.
* WHOSE refusal is standing, or `null` when the barrier is not what failed.
*
* The distinction is what the keyring signal needs. A cap arriving can lift a refusal
* and can lift nothing else - a `configure` with no `ormSubscription` is not going to
* be fixed by one - so re-opening on any keyring change while any error stood would
* churn for a misconfiguration that no amount of retrying repairs.
* It carries two things the keyring signal needs, and the second is what a first
* attempt at this got wrong.
*
* - **That the BARRIER is what failed**, rather than the shape query failing to open
* or nothing having been tried yet. A `configure` with no `ormSubscription` is not
* going to be repaired by a cap arriving, so retrying it on every keyring change
* would churn for a misconfiguration no amount of retrying repairs.
* - **For WHICH holder.** A re-open is not a re-publication: it issues a subscribe
* and a public-store fetch for a document THIS subscriber named. Deciding to make
* that request from whatever identity happens to be connected when an unrelated cap
* lands would perform an access on behalf of somebody who never asked for it - and
* leave the original subscriber reading a query opened in another name, which comes
* back as `isSuccess` holding nothing once its own filter runs. That is the exact
* lie this barrier exists to remove, arriving through the repair's own door.
*/
let barrierRefused = false;
let refusedFor: string | null = null;
function emit(): void {
for (const l of listeners) {
@@ -391,36 +421,32 @@ export function useShapeQuery<T extends BaseType = BaseType>(
* `assertMayReach` says it out loud immediately after - without that, an unheld
* document would simply produce no objects and read as a document holding none.
*
* The opens run TOGETHER and the refusals are read afterwards, in scope order. A first
* pass ran them one at a time so that the first unreachable document would be the one
* reported; that determinism is preserved by the loop below, while the cost of the
* serial form was not acceptable - each open waits on the sync barrier, bounded by
* `OPEN_TIMEOUT_MS` (8s) for a document that never pushes a `State`, so a scope naming
* N of them paid N timeouts before answering anything. `Promise.all` attaches a handler
* to every one of them, so a rejection never escapes as an unhandled one, and the opens
* that did succeed are memoised per session exactly as they would have been.
* The opens run TOGETHER, and every outcome is then read in SCOPE ORDER, so the
* document reported is the first one the caller named that failed - whatever it failed
* of. Two earlier shapes were wrong about this and the difference is not cosmetic, it
* is which document the error names:
*
* Each open is memoised per session, so a re-subscribe costs a map hit.
* - one at a time: correct order, unacceptable cost. Each open waits on the sync
* barrier, bounded by `OPEN_TIMEOUT_MS` (8s) for a document that never pushes a
* `State`, so a scope naming N of them paid N timeouts before answering anything.
* - `Promise.all`: right cost, wrong order. It SHORT-CIRCUITS on the first rejection
* to arrive, which is a race - a fetch that rejects does not wait for the sync
* barrier, so it overtakes a refusal on a document named before it, and the error
* named the wrong document while a comment beside it claimed otherwise.
*
* `allSettled` has neither fault: nothing is discarded, nothing rejects early, and no
* rejection escapes as an unhandled one. The opens that succeeded are memoised per
* session exactly as they would have been, so a re-subscribe costs a map hit.
*/
async function reachNamedDocuments(): Promise<void> {
const docs = namedDocuments(scope);
if (docs.length === 0) return;
await Promise.all(docs.map((doc) => ensureRepoOpen(doc)));
for (const doc of docs) assertMayReach(doc, "useShapeQuery");
}
/**
* Does the holder now reach every document the scope NAMES? The precondition for
* re-opening a scope the barrier refused - and the whole of it.
*
* Synchronous and cap-only, so it asks nothing of the network: it answers "has the
* thing that blocked us been lifted", which is exactly what a keyring change can have
* done and the only question worth spending a re-open on. A scope that is still
* unreachable is left alone, so a connection filing a dozen caps does not buy a dozen
* round-trips for a view none of them was about.
*/
function namedDocumentsReachable(): boolean {
return namedDocuments(scope).every((doc) => mayReach(doc));
const outcomes = await Promise.allSettled(docs.map((doc) => ensureRepoOpen(doc)));
for (const [i, doc] of docs.entries()) {
const outcome = outcomes[i];
if (outcome && outcome.status === "rejected") throw outcome.reason;
assertMayReach(doc, "useShapeQuery");
}
}
/**
@@ -435,13 +461,13 @@ export function useShapeQuery<T extends BaseType = BaseType>(
function open(): void {
closeQuery();
const gen = ++generation;
barrierRefused = false;
refusedFor = null;
void (async () => {
try {
await reachNamedDocuments();
} catch (error) {
if (gen !== generation) return;
barrierRefused = true;
refusedFor = getCaps().holderKey();
setUnknown(error);
return;
}
@@ -494,9 +520,12 @@ export function useShapeQuery<T extends BaseType = BaseType>(
// is the one this signal exists for: the BARRIER refused the scope, so the query
// was never created and there is nothing for `publish` to re-filter. A view left
// like that would be TERMINAL - the very case this channel was added to prevent,
// arriving by a door that did not exist when the channel was written. So re-open,
// and only when the refusal has actually been lifted.
if (barrierRefused && namedDocumentsReachable()) open();
// arriving by a door that did not exist when the channel was written.
//
// So ATTEMPT AGAIN, for the holder that was refused. There is deliberately no
// further test of whether the refusal has been lifted: see the note below on why
// the obvious one is circular, and why an attempt is the cheaper question anyway.
if (refusedFor !== null && refusedFor === getCaps().holderKey()) open();
});
open();
}
packages/polyfill/test/use-shape-query.test.ts
+128 -20
View File
@@ -60,6 +60,9 @@ interface SubRec { nuri: string; cb: (r: unknown) => void }
* creation the page does around the read. It is NOT what answers the read any more.
*/
function makeFake(options: { pushState?: boolean } = {}) {
/** Anchors whose anchored read throws. Mutable after boot, so a suite builds a healthy
* world first and breaks only the one call it is about. */
const unreachable = new Set<string>();
// A broker that never pushes the first `State` is the pathological case the bootstrap
// open's bounded fallback exists for - not the ordinary one, which CONTRACT 3 in
// `e2e/` measures at about a millisecond. It is what makes the barrier's cost visible
@@ -103,6 +106,10 @@ function makeFake(options: { pushState?: boolean } = {}) {
});
const sparql_query = mock(async (...a: unknown[]) => {
// The broker cannot answer for this repo right now - an outage, a repo this session
// has not synced, a rejected read. It REJECTS, as `resolve_target_for_sparql` does
// for a repo the verifier does not hold, and never answers 0 rows.
if (unreachable.has(a[3] as string)) throw new Error("RepoNotFound (emulated outage)");
const query = a[1] as string;
const anchor = a[3] as string | undefined;
// Pointer SELECT (store-root -> doc-shim).
@@ -136,7 +143,7 @@ function makeFake(options: { pushState?: boolean } = {}) {
}));
return { results: { bindings } };
}
for (const marker of ["inboxCap", "readCap", "contains"] as const) {
for (const marker of ["inboxCap", "readCap", "contains", "exposedReadCap"] as const) {
if (query.includes(`<urn:ng-eventually:shim:${marker}>`)) {
const key = marker === "contains" ? "e" : "c";
const bindings = quads
@@ -158,7 +165,7 @@ function makeFake(options: { pushState?: boolean } = {}) {
return () => {};
});
return { doc_create, sparql_update, sparql_query, doc_subscribe, _quads: quads };
return { doc_create, sparql_update, sparql_query, doc_subscribe, _quads: quads, _unreachable: unreachable };
}
function inject(ng: ReturnType<typeof makeFake>, orm: FakeOrm) {
@@ -706,7 +713,7 @@ describe("useShapeQuery", () => {
unsub();
});
it("(s) a cap for some OTHER document does not re-open a scope that is still unreachable", async () => {
it("(s) a re-attempt that cannot succeed spends NO round-trip: an absence is memoised", async () => {
const ng = makeFake();
const orm = makeOrm();
inject(ng, orm);
@@ -720,32 +727,40 @@ describe("useShapeQuery", () => {
const unsub = obs.subscribe(() => {
notes += 1;
});
await tick();
await tick();
for (let i = 0; i < 10 && !obs.getSnapshot().isError; i++) await tick();
expect(obs.getSnapshot().isError).toBe(true);
/** How many times the network was asked ABOUT this document - anchored reads only. */
const asks = (): number =>
ng.sparql_query.mock.calls.filter((c) => c[3] === note).length;
const afterRefusal = asks();
const notesAfterRefusal = notes;
// A keyring change that says nothing about THIS scope - every connection files
// several, and an inbox delivery for another document files one more. Re-opening on
// each of them would re-run the barrier, re-ask the network for any document whose
// fetch had FAILED (a failure is deliberately not memoised) and re-publish the same
// error to every subscriber, for as long as the session lasts.
// What this replaced, and why. An earlier version gated the re-attempt on "does the
// holder hold every named document's cap yet", so that an unrelated cap would not
// buy a round-trip. That gate was CIRCULAR for a document in a public store - the
// only channel that can ever file its cap is the very fetch the gate was blocking -
// and an adversarial pass proved the class unrecoverable. The gate is gone: every
// keyring change re-attempts, for the holder that was refused.
//
// The count of NOTIFICATIONS is what separates the targeted retry from the
// indiscriminate one: re-opening republishes, and republishing wakes every view
// bound to this observable. Asserting on the ORM instead proves nothing here - the
// barrier refuses before the engine is ever asked, either way.
getCaps().learn(mintCap("did:ng:o:something-else"));
await tick();
await tick();
// What makes that affordable is the memo, and it is worth pinning rather than
// assuming: the network ANSWERED here (this document exposes no cap), and an answer
// is remembered. So the re-attempts below re-run the barrier and ask nobody. A
// failure is the other case - deliberately not memoised, so it IS re-asked, which is
// exactly the repair for it (see (v)).
for (let round = 0; round < 3; round++) {
getCaps().learn(mintCap(`did:ng:o:unrelated-${round}`));
for (let i = 0; i < 5; i++) await tick();
}
// BOTH halves, or this test is vacuous: a re-attempt that never ran would also leave
// the count untouched. The republication is what proves the barrier was re-run.
expect(notes).toBeGreaterThan(notesAfterRefusal);
expect(obs.getSnapshot().isError).toBe(true);
expect(notes).toBe(notesAfterRefusal);
expect(asks()).toBe(afterRefusal);
unsub();
});
// -- what the barrier COSTS, which is not nothing -------------------------
it("(t) named documents cross the barrier TOGETHER, so N that never sync cost one wait, not N", async () => {
const ng = makeFake({ pushState: false });
const orm = makeOrm();
@@ -808,6 +823,99 @@ describe("useShapeQuery", () => {
unsub();
});
// -- the three the second adversarial pass found -------------------------
it("(v) a PUBLIC document whose FETCH failed is asked again when the keyring moves", async () => {
const ng = makeFake();
const orm = makeOrm();
inject(ng, orm);
setCurrentUser("alice");
const pub = await createEntityDoc("alice", "public");
seedEvent(orm, pub, `${pub}:q:one`, "Public");
setCurrentUser("bob");
// Bob holds nothing, and the public store is the ONLY channel that can ever hand him
// this document's cap - which is exactly what the broker now cannot answer.
ng._unreachable.add(pub);
const obs = useShapeQuery(EventShape, { graphs: [pub] });
const unsub = obs.subscribe(() => {});
for (let i = 0; i < 10 && !obs.getSnapshot().isError; i++) await tick();
expect(obs.getSnapshot().isError).toBe(true);
// The broker heals, and an ordinary cap arrival follows - a connection restoring a
// key, an inbox delivery for some other document. The first gate written for this
// asked "does the holder hold every named document's cap yet", and for a PUBLIC
// document only the failed fetch can ever make that true: the gate waited on an
// answer that only the action it gated could produce.
ng._unreachable.delete(pub);
getCaps().learn(mintCap("did:ng:o:unrelated-arrival"));
for (let i = 0; i < 20 && !obs.getSnapshot().isSuccess; i++) await tick();
const snap = obs.getSnapshot();
expect(snap.isError).toBe(false);
expect(snap.isSuccess).toBe(true);
expect(titles(snap.data)).toEqual(["Public"]);
unsub();
});
it("(w) a refused scope never re-opens under a DIFFERENT identity", async () => {
const ng = makeFake();
const orm = makeOrm();
inject(ng, orm);
setCurrentUser("alice");
const hers = await createEntityDoc("alice", "protected");
seedEvent(orm, hers, `${hers}:q:one`, "Hers");
setCurrentUser("bob");
const obs = useShapeQuery(EventShape, { graphs: [hers] });
const unsub = obs.subscribe(() => {});
for (let i = 0; i < 10 && !obs.getSnapshot().isError; i++) await tick();
expect(obs.getSnapshot().isError).toBe(true);
// The page becomes Alice. No application can do this - `ensureIdentity` is the only
// published call that settles an identity and it returns the one already set
// (`access-gate.resolveIdentity` reads `getCurrentUser()` first) - but the re-open is
// a NEW access, not a re-filtering of an existing one, and a new access must never be
// issued on behalf of somebody who did not ask for it.
setCurrentUser("alice");
getCaps().learn(mintCap("did:ng:o:unrelated-arrival"));
for (let i = 0; i < 10; i++) await tick();
expect(orm.openCount()).toBe(0); // nothing was subscribed or queried as Alice
// ...and Bob's view must not come back as a SUCCESS holding nothing, which is the lie
// this whole task exists to remove, arriving by the repair's own door.
setCurrentUser("bob");
getCaps().learn(mintCap("did:ng:o:unrelated-arrival-2"));
for (let i = 0; i < 10; i++) await tick();
expect(obs.getSnapshot().isSuccess).toBe(false);
expect(obs.getSnapshot().isError).toBe(true);
unsub();
});
it("(x) the failure reported is the FIRST named document that failed, in scope order", async () => {
const ng = makeFake();
const orm = makeOrm();
inject(ng, orm);
setCurrentUser("alice");
const hers = await createEntityDoc("alice", "protected");
const alsoHers = await createEntityDoc("alice", "public");
setCurrentUser("bob");
// Two named documents failing for two DIFFERENT reasons. The first answers and holds
// no cap for Bob (a refusal); the second cannot be asked at all (a rejection, which
// arrives first because it does not wait for the sync barrier). Running the opens
// together and letting the first rejection win would report the SECOND document.
ng._unreachable.add(alsoHers);
const obs = useShapeQuery(EventShape, { graphs: [hers, alsoHers] });
const unsub = obs.subscribe(() => {});
for (let i = 0; i < 10 && !obs.getSnapshot().isError; i++) await tick();
const message = String((obs.getSnapshot().error as Error).message);
expect(message).toContain(hers);
expect(message).not.toContain(alsoHers);
unsub();
});
// -- teardown ------------------------------------------------------------
it("(k) the last unsubscribe closes the shape query", async () => {