Files
ng-eventually/.tasks/doing/T01.j-a-failure-must-not-read-as-an-absence.md
T
Sylvain Duchesne 4af0be433d fix!: joindre un document n-est pas en avoir des nouvelles, et une reinit invalide ce qu-elle abandonne
Deux defauts, aucun des deux sur la reprise - elle a passe cette revue intacte et
n'a pas ete rouverte.

1. La barriere verifiait qu'un document nomme peut etre JOINT, jamais qu'il a
   REPONDU. Elle attendait `ensureRepoOpen` puis demandait `assertMayReach`, qui
   est la question de la capacite, et ne consultait pas l'etat de synchro que
   l'ouverture venait de produire. Un document dont l'abonnement a reussi mais dont
   le premier `State` n'est jamais venu publiait donc `isSuccess` avec `data: []` :
   le defaut d'origine, dans le cas precis pour lequel la barriere a ete ajoutee.

   La verification rejoint le reste de la decision, dans `publish`, donc rejouee a
   chaque publication au lieu d'etre memorisee. Les trois valeurs, enumerees avant
   d'ecrire la condition : `"synced"` fait autorite - zero ligne VEUT DIRE zero
   objet, la ligne tenue depuis cinq revues -, `"timed-out"` et `"syncing"` non, et
   `"unknown"` PASSE deliberement, parce qu'il veut dire « pas de semantique de
   `State` sur ce `ng` » et que le traiter autrement est exactement la regression
   du cas inerte livree aux tours 1 et 3.

   `isPending` plutot que `isError`, des deux que le contrat autorise : rien n'a
   echoue, il manque l'autorite de dire que le document est vide.

   Le test (t) EPINGLAIT le defaut - il attendait `isSuccess` apres trois
   barrieres expirees. Corrige. Ce n'est pas un test affaibli : l'ancienne
   assertion etait fausse.

   Et un trou dans ma propre suite, trouve par mutation : inverser la condition
   pour refuser `"unknown"` passait tous les tests, faute d'un `ng` SANS
   `doc_subscribe` nulle part. (cc) le couvre.

2. Regression de mon correctif du tour 4. Terminer une ouverture abandonnee
   faisait tourner sa continuation juste apres que la reinit avait vide les
   tables, donc elle ecrivait une entree `opened` perimee dans le registre neuf -
   et toute ouverture ulterieure de ce nuri court-circuitait dessus. Le depot
   n'etait jamais abonne dans la nouvelle session. Une epoque de registre corrige
   cela : une reinit l'incremente, une continuation dont l'epoque ne correspond
   plus n'ecrit rien. Une reinit INVALIDE ce qu'elle abandonne, en plus de le
   terminer.

`getSyncState` reste interne, et la doctrine cesse de l'offrir : elle pointait sur
un symbole qu'aucun appelant ne peut importer.
2026-09-24 11:03:07 +02:00

255 lines
50 KiB
Markdown

---
id: T01.j
title: A read that cannot reach its document must fail, not answer empty
parent: T01
depends_on: [T01.i]
created: 2026-09-21
claimed_by: T01.j-worker
claimed_at: 2026-09-21T14:45:49+02:00
blocked: false
---
## Description
A read whose document is unreachable currently answers an EMPTY SUCCESS. It must answer a failure. This is the "failure disguised as an absence" family, which this repo has closed roughly eight times and still carries two open `bug_` leaves for. Study it properly before changing anything: the facts below are established, the DESIGN is not.
### The established facts - do NOT re-derive them, and do NOT read `nextgraph-rs` for them
**Upstream (VERIFIED).** An anchored SPARQL query on a repo the verifier does not know REJECTS. The deciding line is `engine/verifier/src/request_processor.rs:269`: `let repo = self.repos.get(repo_id).ok_or(NgError::RepoNotFound)?;` inside `resolve_target_for_sparql`. It crosses into JS as a REJECTED PROMISE carrying the string `"RepoNotFound"` (`sdk/js/lib-wasm/src/lib.rs:403` nodejs, `:606` browser). The observable partition upstream is "in `self.repos`" vs "not in `self.repos`": "does not exist" and "exists but this session does not know it" are INDISTINGUISHABLE, while "known but unsynced" collides with "genuinely empty". There is NO read-capability check anywhere on that path - `resolve_target_for_sparql` reads only `nuri.target`.
**Our chain, three layers (VERIFIED by reading this working copy).**
- `fetchReadCap` (`packages/polyfill/src/emulated-verifier/public-store.ts:190`) returns a `boolean` collapsing three unrelated situations: the emulation is not in force (`:196`), the network could not be asked (`:223`), the document exposes no cap (`:225`). Its own comment at `:220` states the collapse out loud. `downloadReadCap`, directly below, DELIBERATELY distinguishes resolving from rejecting and documents why - so the distinction is produced and then destroyed one line later. And NO caller reads the boolean: `ensureRepoOpen:208`, `docs.sparqlQuery:155`, `branch-registers.ts:316`, `fetchReadCaps:277` all call it as a bare statement.
- The read then proceeds on a document that was never opened.
- `useShapeQuery` answers `isSuccess` holding zero objects.
**What the target says the answer should be.** `useShapeQuery`'s return type is modelled on TanStack Query's `useQuery`, which distinguishes purely by whether the query function REJECTS: a rejection gives `isError` with the error; a resolution gives `isSuccess` with whatever data, empty included. That maps exactly onto upstream's `RepoNotFound`. So an unreachable document must surface as `isError`, and `isSuccess` with zero objects must mean "it is there and it holds nothing".
### The design is yours to establish, and the enumeration comes FIRST
Do NOT start from a preferred shape. `.project/rules/nextgraph/no-divergence-from-nextgraph.md` and `polyfill-must-match-the-target` both apply, and so does the discipline of listing the cases before concluding - this repo has a recorded incident of deciding from one convenient case and having to be dragged back row by row.
**List the callers and their states before choosing anything.** For each of the four call sites above: what is it doing, what does it already hold, and what should it do when the cap cannot be obtained? Include the INERT case - before the first cap exists the emulation deliberately does not enforce, and reading works anyway; whatever you build must stay inert there or it will break every early path.
**The constraint that shapes the answer.** `fetchReadCap` has NO upstream counterpart - upstream never consults a capability on the read path. So there is no SDK signature to mirror, and the cascade goes to the ENGINE'S MODEL: upstream's observable is that a read REJECTS or it does not. That is an argument about where the distinction should become visible, not a licence to invent a rich vocabulary of outcomes that the target does not have. Weigh it; do not treat this paragraph as the decision.
**Re-examine the reverted barrier on its own merits.** `useShapeQuery` opens nothing named in `scope.graphs` - the barrier was removed by T01.d. A fix was written during T01.f and REVERTED, because it had been justified as unblocking `readUnion`'s removal, which was FALSE (an index is read by anchor, not by shape). The revert killed a bad JUSTIFICATION, not a bad change. The defect is real and unowned: an application that names an unopened document gets an empty success. Decide it fresh, on the enumeration.
**Check the two open bug leaves** (`bug_a-lost-inbox-watch-is-never-retried`, `bug_an-empty-inbox-list-can-mean-a-failure`). They are the same family. State whether this work closes either. Do NOT claim it does without a test that proves it - a bug leaf deleted on a plausible argument is worse than one left open.
### Blast radius - this is a behaviour change applications see
Reads that silently answered empty will start failing. That is the POINT, and it is also how this breaks callers that were quietly relying on the silence. So: the published contract at `.project/concepts/app-contract/polyfill-surface/contract_polyfill-surface.md` must state the new behaviour, with a version bump per its own `## Change policy` (a call that narrows is a MAJOR). Follow the existing pre-release scheme, do not invent one. Expect internal callers and tests to surface real breakage - each one is a case to understand, never a reason to soften the change.
**Record the durable fact** in `.project/concepts/app-contract/` as a `knowledge_` leaf: what the target answers for an unreachable, an unsynced and an empty document, and which of those a caller can tell apart. That fact is worth keeping whatever the fix turns out to be. Do NOT open a `bug_` leaf for something this task repairs.
### One incidental repair, since you are in this territory anyway
`docs/internal-contract.md`'s `open-repo.ts` section is stale: wrong line numbers, it names a non-existent `ensurePhysicalRepoOpen` export, and since T01.i it also names the deleted `ensureReposOpen`. Correct what that section says about the functions you touch. Do NOT undertake a general sweep of that document - the drift is broad and pre-existing, and fixing all of it is not this task.
### Verification
**The e2e must run.** Unit tests cannot judge this: the whole subject is what happens when the network does not answer. Run the polyfill e2e suite and report its counts verbatim. If the broker is unreachable, report the suite as UNRUN - never substitute a typecheck for the measurement.
**Hard gate:** `bun test` -> 0 fail; `bunx tsc --noEmit -p packages/polyfill/tsconfig.json` -> exit 0; `bunx tsc --noEmit -p packages/polyfill/e2e/tsconfig.json` -> exit 0; `python3 ~/.claude/skills/concept/lint.py` -> 0 ERROR. Anti-cheat: no `any`, no `@ts-ignore`, no `as unknown as`, no deleted or skipped test, no weakened assertion. Specifically forbidden here: catching the new failure somewhere upstream to keep an old test green - if a test breaks, it is telling you something.
**An adversarial review is part of this task, not optional.** When the work is done, it goes to a fresh reviewer mandated to REFUTE it, on at least these angles: a path where the failure is still swallowed; a path where the change now fails something that legitimately has nothing (an empty document must still be a SUCCESS); and the inert case. A finding counts only with a concrete failure scenario. Re-run the adversary on whatever you fix in response.
## Progress
### The enumeration, written before any code
Every row was read in this working copy. `fetchReadCap` reaches the network only when `caps.isEnforcing()` AND `caps.capFor(doc) === undefined` - held or inert, it answers without asking, so most rows below are already unaffected by whatever the network does.
**Site 1 - `ensureRepoOpen(nuri, holderKey?)`, `open-repo.ts:209`.** Doing: bringing a repo into the session before an anchored read (the cold-start heal), asking the public store first so a bare reference to a public document suffices. Holds: the nuri. Asks only when `holderKey === undefined`. Today, when the cap cannot be obtained: `fetchReadCap` answers `false` exactly as if the document exposed none, `mustNotAttempt` is then true, and the function returns SILENTLY. Its eight live callers (`readDocumentSubjects`, `readUserStore`, `restoreOwnCaps`, `readInboxCapPairs`, `readLinks`, `inbox.readSynced`, `inbox.readSyncedForDocument`, `inbox.processInbox`) all read straight afterwards, so the failure does surface - as `assertMayReach`'s "the connected user does not hold this document's cap", which is the WRONG reason and calls for the wrong action (give up, rather than retry). Should: propagate. INERT: returns before any round-trip, `mustNotAttempt` false, the open proceeds - unchanged. Note the six callers reading their own store/inbox hold that cap already, so they never ask at all.
**Site 2 - `docs.sparqlQuery`, `docs.ts:155`** (published). Doing: the anchored read itself; the ask is what makes a bare reference to a public document readable. Today: `false`, then `assertMayReach` throws the same wrong reason. A failure, but mislabelled. Should: reject, with the true reason. INERT: no ask, `mayReach` true, the read proceeds - unchanged.
**Site 3 - `documentInboxAddress(doc)`, `branch-registers.ts:317`.** Doing: reading a document's Header branch for its inbox address - the question a third party arrives with holding nothing but the reference. Today: `false` -> `mustNotAttempt` -> returns `undefined`, which means "this document has no inbox". `inbox.postToDocument` renders that as "its owner never opened one"; `inbox.readForDocument` / `readSyncedForDocument` render it as zero deposits. **This is the pure failure-as-absence, on a published call, in both directions.** The same function carries a SECOND one: the `try/catch` around the address query returns `undefined` for a broker that did not answer - the comment beside it justifies the catch by a refusal that `mustNotAttempt` has already excluded one line above, so what it actually swallows is breakdowns. Both repaired. Should: propagate; `undefined` keeps meaning "no inbox", which is an answer. INERT: unchanged.
**Site 4 - `fetchReadCaps(docs)`, `public-store.ts:277`.** Doing: the batch form, `Promise.all` over the singular with a tolerance of its own. Holds: nothing. **Zero callers** - grep-verified across `src`, `test` and `e2e`, and it is not exported from `src/index.ts`. Should: deleted, on T01.i's own precedent for `ensureReposOpen`. Leaving it would leave an unowned batch policy over a call that now rejects.
**Site 5, the one that is not a `fetchReadCap` call site - `useShapeQuery`** (published). It asks nothing and opens nothing since T01.d. Doing: one shape in one SDK `Scope`, read through `RdfOrmSubscription`. Holds: `scope.graphs`, which NAMES documents. Today, for a named document whose cap is not held: the ORM answers, `filterReadable` drops every item, and the snapshot is `isSuccess` with `data: []` - **the empty success this task is named after**. For a named document whose cap IS held but which this session has never synced: the ORM answers nothing and the snapshot is again `isSuccess` with `data: []`, for a document that is not empty. Should: before the query opens, each NAMED document in `graphs` is asked of the public store, refused out loud if its cap is not held, and crossed over the sync barrier; any of those failing publishes `isError`.
**The cases that must NOT change, and why each is a success rather than a failure:**
- `graphs: "did:ng:i"` or `""` (the SDK's `normalizeScope` maps `""` to `did:ng:i`): the whole local dataset. The application named no document, so there is nothing to guard and the capability filter is the right and only answer - `isSuccess` with `data: []` means "you hold nothing of this shape". Untouched.
- `graphs: []` or `undefined`: no objects, by the SDK's own definition. `isSuccess` with `data: []`. Untouched.
- a `graphs` entry that is not a `did:ng:` reference: it names no document this library can hold a cap for, so there is no cap question; left to the engine.
- a named document, held and synced, holding nothing of the shape: the barrier passes, the ORM answers empty, `isSuccess` with `data: []`. **This is the line that must not move.**
- a held document exposing no read cap: `fetchReadCap` answers `false` and MEMOISES it. An absence was read; it is an answer.
- a held document with no inbox: `documentInboxAddress` answers `undefined`, `depositsForDocument` answers `[]`.
- the INERT case throughout: before the first cap exists, `isEnforcing()` is false, `fetchReadCap` returns without a round-trip, `mayReach` answers true for everything, and nothing new refuses. **CORRECTED 2026-09-21 after an adversarial review**: this row said "every row above is unchanged there", and that was FALSE. The REFUSAL is inert; the OPEN is not. `ensureRepoOpen` runs whatever the registry is enforcing, so a named scope now issues a real `doc_subscribe` and waits on the sync barrier where it used to open the query at once. That is deliberate - without it a named document this session has never synced answers `isSuccess` with `data: []`, the very lie the barrier removes - but it is a change, and calling it "unchanged" hid a cost. Pinned by `use-shape-query.test.ts` (u).
**Partial scopes.** `graphs: [docA, docB]` with only docB held refuses the WHOLE query rather than answering docB's objects. A short answer is the same lie as an empty one, and this repo has already ruled twice against skip-and-carry-on: `readUnion`'s per-document tolerance was removed as an invention in `1.0.0-dev.8`, and an ORM write refuses the whole batch when one document is not writable.
### Round 5 - the sync-state enumeration, written before the condition
`useShapeQuery` checked that a named document may be REACHED and never that the open it had just awaited ARRIVED. Every value `getSyncState` can answer with, in every state it can be in when `reachNamedDocuments` has resolved - because treating the wrong one as not-synced is how the inert case regressed in rounds 1 and 3:
| value | what it means (`open-repo.ts`) | how it is reached after the open resolved | verdict |
|---|---|---|---|
| `"synced"` | the first `State` arrived - the TRUTH signal | the ordinary path with a real `doc_subscribe` | **authoritative.** Zero rows MEANS zero objects; `isSuccess` with `data: []` is the answer, and this is the line four passes have held |
| `"timed-out"` | the bounded fallback fired with NO `State`; opened best-effort, sync UNCONFIRMED | a broker that accepts the subscription and never pushes | **not authoritative.** This is the whole defect: the document may hold anything, and the read has no standing to call it empty |
| `"syncing"` | subscribed, no `State` yet | unreachable once the open has resolved - the open resolves on `State` or on the fallback, and each sets its own value. Treated as not authoritative anyway, because if it ever IS reached the barrier demonstrably was not | **not authoritative** (defensively) |
| `"unknown"` | never requested, **or the fake-ng no-op path: no `State` semantics** | an injected `ng` with no `doc_subscribe` - the whole unit suite, and any consumer whose `ng` lacks it. Also, until round 5 fixed it, a stale `opened` entry left by a reset | **authoritative, deliberately.** There is no barrier to have missed where there are no barrier semantics. Treating it as not-synced is exactly the inert regression rounds 1 and 3 shipped, and it would leave every early path pending for ever |
So the condition is `"timed-out"` or `"syncing"`, and `"unknown"` passes. With a real broker the post-open state is only ever `"synced"` or `"timed-out"`, so the check bites precisely where `State` semantics exist.
**`isPending` rather than `isError`**, and the contract allows either (`:196`, `:280`). Chosen because nothing FAILED: the subscribe succeeded, the query opened, the read did not throw - what is missing is an authoritative answer, and `isPending` is this surface's word for a question still open, while `isError` has meant "something threw" through five passes. It also needs no recovery machinery of its own: `publish` re-reads the level on every publication, so if the `State` does arrive later and brings data, the ORM's own change signal republishes and the read succeeds. What remains, and it is stated rather than hidden: a document whose `State` never arrives leaves the read pending for that session. The contract already names that state and tells an application to handle a `isPending` that does not end.
### The design, and what was rejected
Chosen: **`fetchReadCap` stops converting a failed download into `false`.** `downloadReadCap` right below it already distinguishes resolving from rejecting and documents why; the collapse was one `catch` destroying that distinction one line later. Removing the `catch` leaves `boolean` honestly meaning "a cap was obtained", and gives all three live call sites the one thing each of them wanted. Plus the `useShapeQuery` barrier above, which is where the published empty success actually comes from.
Rejected, with the deciding reason for each:
- **A three-valued result (`held` / `absent` / `unknown`), each caller deciding.** Rejected on the enumeration: all three live callers want the same thing - fail - so the vocabulary would buy nothing, and `fetchReadCap` has no upstream counterpart, so a richer outcome set than the engine's ("a read rejects or it does not") would be invented vocabulary an application learns and unlearns.
- **Fixing only `fetchReadCap` and leaving `useShapeQuery` alone.** Rejected: `useShapeQuery` never calls `fetchReadCap`, so this would leave the published reactive read answering `isSuccess` with `data: []` for a document it cannot read - the exact headline defect.
- **Making the capability filter refuse instead of drop.** Rejected: under `did:ng:i` dropping is exactly right (the broker delivers only what your keys open), and by the time the filter runs the engine has already answered. The question belongs at the ASK, not at the result.
- **Refusing only the unreachable graphs and reading the rest.** Rejected: see partial scopes above.
- **Adding no open barrier to `useShapeQuery`, only the refusal.** Rejected: an unsynced named document then still reads `isSuccess` with `data: []`, which is the same lie for a document that is not empty. Every other read path in this library crosses that barrier; this one stopped because T01.d removed the scope TRANSLATION that produced the document set, and the caller now names documents directly, so the barrier has a subject again.
## Result
Status: done. `1.0.0-dev.9`, MAJOR. Three commits: `75313f7` (the behaviour change), `b0eec7b` (the contract, the knowledge leaf and the narrow `internal-contract` repair), `534d8f9` (two guard tests).
### What each call site does now when the cap cannot be obtained
- **`ensureRepoOpen`** (`open-repo.ts:209`) - propagates. The open has not happened, and the read that follows must not proceed as though it had. It used to decline silently and let the anchored read blame the reader for holding no cap.
- **`docs.sparqlQuery`** (`docs.ts:155`) - rejects with the true reason instead of `assertMayReach`'s "you do not hold this document's cap", which is a different event calling for a different action.
- **`documentInboxAddress`** (`branch-registers.ts:317`) - propagates, and so does its own address read, whose `catch` was swallowing breakdowns into `undefined`. `undefined` now means only "no inbox".
- **`fetchReadCaps`** (`public-store.ts:277`) - DELETED. Zero callers, grep-verified, not in `src/index.ts`; an unowned per-document tolerance over a call that now rejects.
- **`useShapeQuery`** - crosses the barrier on every document `scope.graphs` NAMES before opening the query: public-store fetch, refusal out loud, sync barrier. Any of them failing publishes `isError`.
### The design, and what was rejected
`fetchReadCap` stops converting a rejected download into `false`. `downloadReadCap` already distinguishes the two and documents why; one `catch` destroyed it a line later. `boolean` now honestly means "a cap was obtained". Rejected: a three-valued result (all three live callers want the same thing - fail - and the engine has no such vocabulary); fixing only `fetchReadCap` (`useShapeQuery` never calls it, so the headline defect would have survived); making the capability filter refuse instead of drop (under `did:ng:i` dropping is exactly right, and by then the engine has already answered - the question belongs at the ASK); partial answers for a multi-document scope (a short list is the same lie as an empty one); the refusal without the open (an unsynced named document would still read `isSuccess` with `data: []`, for a document that is not empty).
### How "genuinely empty" stayed a success
Every row of the enumeration's "must not change" list is covered by a test that passes both with and without the change: `use-shape-query.test.ts` (m) a named, reachable document holding nothing of the shape is `isSuccess` with `data: []`; (h) `graphs: []`; (p) `graphs: ""`; (j) the whole dataset filtered down to the holder's own; (o) the INERT case; `document-inbox-surfaces-failure.test.ts` third test, a document nobody opened an inbox on answering `[]` on both reads and refusing the deposit with "has no inbox"; `public-store.test.ts`, a document exposing no cap still answering `false` and still asked only once.
The four tests that DO discriminate were verified non-vacuous by temporarily removing the change: (l) and (n) on the barrier, and both inbox failure tests. (q) caught a real leak when the generation check was removed.
### What broke, and what it was really asserting
Four tests in `public-store.test.ts`, all of them `expect(await fetchReadCap(unreachable)).toBe(false)`. What they were asserting was the MEMO's arithmetic - "a failure is not remembered, an absence is" - which is why every `asksAbout` count in that file is unchanged. The value they asserted alongside it was the collapse itself. They now assert a rejection, which is strictly stronger, plus one new test that the rejection names the document and keeps the broker's own words on `cause`.
Nothing else broke. The reason is worth recording: the three live `fetchReadCap` callers that read a STORE or an INBOX already hold that cap (`fileOwnStructure` / `holdOwnCap` file it at connection), so `fetchReadCap` answers from the registry and never reaches the network. The blast radius is much narrower than four call sites suggest - it is a third party naming a document it does not hold.
### CORRECTED after the adversarial review (round 1 was REFUTED)
Two findings, both reproduced here before anything was changed, both diagnoses verified independently.
**Finding 1, blocking - a refused named scope was TERMINAL.** Reproduced as `use-shape-query.test.ts` (r): a named document whose cap arrives AFTER the subscribe stayed `{isError: true, data: []}` for ever. Diagnosis confirmed by reading: the keyring handler is `if (query !== null) publish(query, generation)`, and the barrier aborts BEFORE any query is created, so the one channel that makes a late cap converge went straight past the state that needed it most. It broke a guarantee this package publishes and that `read-filter.ts`, `use-shape-query.ts` and `inbox.ts` all restate - a document shared with you while you watch becomes readable with no call from the application. **Trading a silent empty answer for a stuck error is worse than the defect**, and the ordinary sequence (a page subscribing while its keyring is still being restored) hits it.
Fix, round 2: the keyring signal RE-OPENS a scope the barrier refused, gated on `namedDocumentsReachable()`. **That gate was itself wrong and round 3 refuted it - see below.**
**Finding 2 - the INERT claim was false.** Reproduced by reading and pinned as (u): `ensureRepoOpen` runs whatever the registry is enforcing, so the inert case gained a real `doc_subscribe` and a barrier wait. The open is right and stays; what was wrong was calling it "unchanged". The measured COST was real though: the barrier was SERIAL, so N documents that never push a `State` cost N bounded fallbacks - 8s each in production. Now parallel, so the wait is the slowest rather than the sum, with the refusals still read in scope order afterwards. Pinned by (t) - which, as round 5 found, was at the same time asserting the WRONG outcome for those timed-out barriers, and is corrected there.
**And the thing that let both through: the e2e never walked this path.** Confirmed - both `useShapeQuery` journeys read `{graphs: "did:ng:i"}`, so `43 passed / 43` was never evidence for the barrier, and reporting it as such was the real failure of round 1. Four live checks added over a NAMED scope: the owner reads her own document (the barrier does not break a live named read); a holder of nothing is `isError` with the refusal's own message and `dataLen: 0`, never an empty success; a PUBLIC document opens for that same holder from a bare reference, through the barrier's own fetch; and a cap landing for ANOTHER document leaves the refused scope refused. That fourth one measures the OUTCOME only - it does not separate "no re-open was attempted" from "one was attempted and refused again"; the targeting itself is measured by the notification count in `use-shape-query.test.ts` (s). Saying which of the two a green check proves is the whole lesson of the round that shipped this barrier with no named-scope coverage.
**What the e2e still does NOT prove, stated plainly.** The POSITIVE recovery - the cap for the refused document itself arriving - is proven at unit level only (test r, non-vacuous). Driving it live needs a second actor delivering through `inbox.share`, and on one page that means switching the current identity while the observable is subscribed, which would re-open the refused scope under the OTHER identity. That is a pre-existing property of this observable (`publish` has always read the keyring of whoever is connected), it cannot arise in an application (one identity per page load, no call takes an identifier), and the repo has already ruled that such montages belong in the applicative suite - `capsShareCap` was moved there for exactly this reason. Left as a named follow-up rather than encoded here as if it were the library's behaviour.
### CORRECTED AGAIN - round 2's repair was REFUTED, three findings, all reproduced here first
**Finding 1, blocking and STRUCTURAL - the gate was circular.** `namedDocumentsReachable()` tested `mayReach`, i.e. the cap is HELD. For a named document in a PUBLIC store the only channel that can ever file that cap is `fetchReadCap` - the very call that had failed and set the flag. The gate waited on an answer that only the action it gated could produce, so that whole class was unrecoverable however many caps arrived. Reproduced as `use-shape-query.test.ts` (v): the broker hiccups on the `exposedReadCap` read, the fetch rejects, the broker heals, an ordinary cap arrival follows, and the view never re-asks.
**The readiness question, answered rather than patched.** There are exactly two ways this barrier can refuse, and the memo already tells them apart. The store could not be ASKED: a failure is deliberately not memoised (an earlier fix in this repo), so a re-attempt genuinely re-asks the network - which IS the repair for this class, not a cost to avoid. Or the store ANSWERED and there is no cap to be had: an absence is memoised, so a re-attempt asks nobody, re-runs the barrier against what is already known, and republishes the same refusal for the price of one snapshot - and it stops the day the cap arrives from elsewhere, which is the only thing that repairs THAT class. So the right signal is not a cleverer predicate, it is **attempting again**: cheap for the class it cannot help, correct for the class it can, self-limiting because the moment an attempt succeeds there is a query again. `namedDocumentsReachable()` is DELETED. **CORRECTED: the claim that this made the module smaller than round 2 left it was FALSE** - measured, `use-shape-query.ts` went 534 -> 563 lines and 203 -> 204 non-comment across that round, and is 664 / 232 now. It has grown at every round. One predicate left; the reasoning that had to be written down beside the code grew more than it. Pinned by (v) and, for the cost, by (s), which now asserts both halves (the re-attempt ran, and it spent no round-trip) so it cannot pass vacuously.
**Finding 2 - the re-open performed an access under the wrong identity, and my "pre-existing" defence was wrong in kind.** The reviewer is right: re-publishing RE-FILTERS an existing query, while a re-open NEWLY ISSUES a subscribe and a public-store fetch for a document that subscriber named. That is a new action taken on behalf of whoever happens to be connected, not an old one resurfacing, and the end state was `{isSuccess: true, data: []}` for a document the subscriber cannot read - this task's own defect, through the repair's door. Fixed by binding the refusal to a holder: `refusedFor` records WHOSE barrier failed, and a re-attempt is made only while that holder is still the connected one. Pinned by (w), which fails without the binding.
**CORRECTED: "an application cannot change identity mid-session" was FALSE.** `ensureIdentity` does return the already-settled identity, which is what I checked - but that is not the only path. `shared-wallet/access-gate.ts:386` calls `adoptCurrentUser(normalized)` when a REVIVED barrier is confirmed, reached through the published `init()`, with a comment saying in as many words that the value may well not be the one that left. I checked one call site and generalised from it. The identity CAN change under a live page, which is the second reason the holder comparison had to go.
**Finding 3 - the determinism claim was false.** `Promise.all` short-circuits on the first rejection to ARRIVE, and a fetch that rejects does not wait for the sync barrier, so it overtakes a refusal on a document named before it. I chose to fix the BEHAVIOUR rather than the claim: `Promise.allSettled` plus a walk in scope order, so the document reported is the first one the caller named that failed, whatever it failed of, and the parallelism is kept. Pinned by (x), which fails on `Promise.all`.
**The fourth e2e check was removed, not repaired.** "A cap landing for ANOTHER document leaves a refused scope refused" passed with the whole re-attempt machinery deleted - the outcome is a standing refusal either way. A check that cannot fail for any regression of the thing it guards reads as coverage and is worse than none. Its properties are measured in the unit suite by (v), (s) and (w).
### The decision to KEEP the barrier rather than take the escape hatch
Taken deliberately, at the second offer. Removing it would restore, on a published read, a silent `isSuccess` with `data: []` for a document the caller cannot reach - the defect this task exists for, now proven live rather than argued. The escape hatch's own premise is "a stuck error is worse than a silent empty"; after these three repairs there is no stuck error left to weigh against it. Every class that can refuse is re-attempted on the only readiness signal this library has, and the one class that is not - the identity changed under the observable - cannot arise in an application and is answered honestly rather than by acting in someone else's name. The repairs are also subtractive: a circular predicate deleted, a short-circuit replaced by an ordered walk, a flag given the one extra fact it was missing. What remains is a scope refused in a session whose keyring never moves again, which stays `isError` until `refetch()` - the published meaning of `isError` on this surface, and what it already meant for every other failure it carries.
### ROUND 4 - refuted again, and the recovery changed SHAPE rather than gaining another flag
Three defects and two false claims, all reproduced here before anything was touched.
**Finding 1, blocking: the recovery was EDGE-TRIGGERED, so the edge that caused a refusal was the one it could not see.** `refusedFor` was only set after the async barrier, so an `onChange` firing WHILE the barrier crossed found no query and no record and did nothing - and that same change is what made the walk refuse. Reproduced as (y): inert at subscribe, so the barrier skips the public-store ask entirely; the connect burst files the holder's first cap mid-barrier; the walk then refuses; no keyring event is left to come. `refetch()` answers at once, so the error was for nothing.
**This is the shape that had to change, not the flag.** Rounds 2 and 3 both built recovery as "record that we refused, then react to a later event", and both records were the fault - one gated on a condition that could not become true, one written after the window in which the deciding event arrives. So the recovery stopped depending on edges:
- **the decision is a LEVEL.** Which named documents the holder may reach is re-read on every publication ({@link publish} calls `namedRefusal()`), never carried from the barrier. A cap arriving turns `isError` into the document through the channel that already existed, and nothing is remembered, so there is no event to miss. This is also what makes the holder question moot: a query opened while somebody else was connected cannot come back to this subscriber as a success it is not entitled to, so the comparison that Finding 2 breaks is not needed at all.
- **an attempt that RACED a change notices, instead of waiting.** `open()` samples `CapRegistry.version()` synchronously before the barrier and compares after it fails; a mismatch means it decided on stale ground, so it makes another attempt itself. It terminates: an attempt repeats only because the keyring moved, moves are finite, and the moves an attempt causes itself are the caps it files - at most one per named document, after which that document is held and asks nothing (`file()` does not notify when the value is unchanged, and `markInPublicStore` does not notify at all - both checked).
`CapRegistry.version()` is the one new primitive: a counter bumped where `notify` is, with a note saying why a listener alone is not enough.
**Finding 2, major: the holder comparison broke the ordinary sign-in.** `holderKey()` is `""` while nobody is connected, so a refusal taken before the identity settles - which is where components mount - never matched again once it did. Reproduced as (z). The comparison is GONE, replaced by the publish-time check above.
**Finding 3: "costs one snapshot" held only for the absence class.** Over a FAILED fetch, each unrelated arrival re-asks the broker - that is the repair for that class and it stays - but it also published a fresh snapshot carrying a fresh `Error` for an unchanged state, which contradicts the stability this surface documents. FIXED rather than documented away: `setUnknown` compares by message and keeps the standing snapshot, so subscribers see no update while nothing has changed. Reproduced and pinned by (aa), which asserts snapshot IDENTITY.
**A defect of my own, found by measuring rather than by review.** The round-4 change made the unit suite intermittently fail - two 5s timeouts in `cold-read-for-document`, about one run in four, where round 3 was stable 4/4. Traced by instrumenting the open registry rather than guessed: `resetOpenedRepos()` abandons in-flight opens WITHOUT settling them, so a caller awaiting one waits out the full 8s fallback on a subscription that has been torn down, then proceeds with its sync state cleared - and `"unknown"` is the LAX reading (`index-deposit.noRowsMeansAbsent` accepts it as a definitive absence). A latent failure-as-absence bug in this task's own family, which the re-attempts made fire. Fixed: a reset now ENDS the opens it abandons and marks them `"timed-out"`, the reading that refuses to call nothing an absence. Pinned by a new test in `open-repo.test.ts`; six full-suite runs are green at the usual 2.4s.
Two fixtures were also racing themselves and are corrected, with the reason written beside them: `use-shape-query.test.ts`'s fake minted into `did:ng:o:doc<n>`, the same namespace as `wallet-fake.ts`, so two fakes aliased each other's repos in the process-global open registry - the lesson `wallet-fake` already carries at its own scale, one scope wider; and the INERT test asserted `isEnforcing() === false` and then subscribed, while signing in arms the registry a tick later, so it was pinning the schedule. It now runs with nobody signed in, which is inert for the whole test.
### ROUND 5 - two defects, neither about recovery
The recovery design survived this pass and was not reopened. Both findings reproduced here before anything was changed.
**Finding 1, blocking: the barrier checked that a document may be REACHED and never that it ARRIVED.** `reachNamedDocuments` awaited `ensureRepoOpen` and then asked `assertMayReach`, which is the CAPABILITY question; it never consulted the sync state that open had just produced. So a document whose subscribe succeeded and whose first `State` never came published `isSuccess` with `data: []` - the original defect, in the exact case the barrier was added to close. Reproduced as (bb), and test (t) was PINNING it: it waited for `isSuccess` after three timed-out barriers and asserted the three titles. (t) is corrected, and that is not a weakened assertion - the old one was wrong, and the commit says so.
The fix sits where the rest of the decision already sits: `publish` calls `namedDocumentsArrived()` on every publication, so the question is re-asked at the level rather than remembered from the barrier. The enumeration of the three values, written before the condition, is above under "Round 5". The short form: `"synced"` is authority (zero rows MEANS zero objects - the line that has held through five passes), `"timed-out"` and `"syncing"` are not, and **`"unknown"` PASSES deliberately** because it means "no `State` semantics on this `ng`" and treating it as not-synced is exactly the inert regression rounds 1 and 3 shipped.
**`isPending`, not `isError`**, of the two the contract allows. Nothing failed: the subscribe succeeded, the query opened, no read threw - what is missing is the authority to call the document empty, and `isPending` is this surface's word for a question still open while `isError` has meant "something threw" for five passes. It also needs no machinery: `publish` re-reads the level, so a `State` that lands later and brings data republishes through the ORM's own change signal. Stated rather than hidden: a document whose `State` never arrives leaves that read pending for the session, which is the state the contract already tells applications to handle.
**A GAP IN MY OWN SUITE, found by mutation rather than by review.** Inverting the condition to treat `"unknown"` as not-arrived - the precise regression the blast-radius note warned about - passed every test. Nothing in the file exercised an `ng` WITHOUT `doc_subscribe`, which is the only way `"unknown"` is reached, and it is what `wallet-fake` does by default and what any consumer lacking the member does. Closed by (cc), which now fails on that inversion.
**Finding 2, major, and a regression from my own round-4 fix.** Settling an abandoned open made its continuation run just after `resetOpenedRepos()` had cleared the maps, so it wrote a stale `opened` entry into the fresh registry - and every later `ensureRepoOpen` for that nuri short-circuited on it. The repo was never subscribed in the new session and a read over it answered as though it had been. Before round 4 that line ran only when the 8s fallback fired, so a re-open inside the window did re-subscribe; ending the open promptly was right, but it moved the write into the window where it does damage. Fixed with a registry epoch: a reset bumps it, and a continuation whose epoch no longer matches writes nothing. A reset now INVALIDATES what it abandons as well as ending it. Pinned by a new `open-repo.test.ts` case that fails without the epoch check.
**`getSyncState`: decided, not left dangling.** It stays INTERNAL, and `knowledge_what-a-failed-read-answers-upstream` no longer offers it - that pointer was at a symbol no caller can import. The reason for not exporting: it has no counterpart upstream at any level, so publishing it would hand an application one more polyfill-era symbol to unlearn, on a surface that is busy removing exactly those; and the distinction it carries is now folded into `useShapeQuery`'s own states, which is where a caller needs it. The leaf now also states the consequence plainly instead of papering over it: a caller using the anchored `docs.sparqlQuery` directly gets zero rows for an unsynced document and has no published way to tell that from an empty one - the same collapse upstream has.
### Gate
- `bun test` -> `445 pass / 0 fail`, `Ran 445 tests across 40 files` (baseline `421 pass / 0 fail`, measured before any change; +24 new tests, none deleted or skipped). **Run six times: green every time, 2.78-2.98s.** Two tests were REWRITTEN rather than weakened, each because the design under them changed and the old assertion had become a statement about something that is no longer true: (s) asserted the targeting predicate round 3 proved circular, and now asserts that a re-ask is made for the class it can repair and not for the one it cannot, each half guarding the other against vacuity; (w) asserted that no query is opened under another identity, a rule that left two classes terminal, and now asserts the stronger invariant that replaced it - no snapshot ever claims success for a document the CURRENT holder cannot reach.
- **Mutation-tested, ten ways.** Round 5: removing the arrival check kills (bb) and (t); inverting it to refuse `"unknown"` kills (cc); removing the registry-epoch invalidation kills the new `open-repo` case. Earlier rounds: Removing the raced-edge re-attempt kills (y); removing the publish-time refusal kills (w); removing the snapshot-stability guard kills (aa) and (k); removing the re-attempt entirely kills (r)(s)(v)(w)(z); `Promise.all` in place of `allSettled` kills (x). Round 2's circular gate kills (s)(t)(v) and round 3's holder rule kills (w)(z), so the suite now discriminates against both previously-shipped designs. Removing the abandoned-open settle kills the new `open-repo` test.
- `bunx tsc --noEmit -p packages/polyfill/tsconfig.json` -> exit 0
- `bunx tsc --noEmit -p packages/polyfill/e2e/tsconfig.json` -> exit 0
- `python3 ~/.claude/skills/concept/lint.py` -> 0 ERROR (warnings pre-existing; the new leaf carries none)
- polyfill e2e, run at HEAD: `Polyfill e2e summary: 46 passed, 0 failed, 46 total - batch 3.5 min, slowest cold sync 30s`. **No reviewer has ever run this suite; every count reported for it in this task is mine alone.** 43 was the count before this task; three named-scope checks were added and a fourth removed as non-discriminating. NOTE: the broker login is flaky and independent of this work - three of thirteen runs in this session died with `[e2e broker login] gave up after 45.0s - the wallet-list screen never changed`, at a different step each time, on the login UI before any library code runs. Every such run was re-run to a clean result.
**What the e2e PROVES**, and it is the whole of it: against the real broker, a named scope reads for the holder who owns the document; a holder of nothing is refused out loud with `dataLen: 0` rather than told the document is empty; and a public document opens for that same holder from nothing but its reference, through the barrier's own fetch. **What it does NOT prove**: anything about the re-attempt. Every recovery path - the public class repaired by re-asking, the cost of a doomed re-attempt, and the holder a re-attempt is made as - is measured only in the unit suite, against the `CapRegistry`, which is where that mechanism lives and which the e2e's own header says the broker does not implement. Driving the positive recovery live needs a second actor delivering through `inbox.share`, and belongs in the applicative suite.
### The contract
`1.0.0-dev.9`, MAJOR, per the leaf's own `## Change policy` ("an existing call narrows: ... it reports a state you did not have to handle before"). `packages/polyfill/package.json` bumped. Two NON-guarantees corrected because the change makes them false: "Neither shape read fetches a public store's key for you" (now `useShape` only, plus `did:ng:i`), and "a shape query the engine refuses never finishes loading" (the commonest way in is closed for a NAMED scope). Two out-of-leaf documents carried the same now-false claim and were corrected in one sentence each: `docs/api-contract.md:262` and `docs/readcap-and-nuri-model.md:367`. `emulated-verifier/read-filter.ts`'s header said both shape reads ask nothing - corrected, with the note that refusing at the ASK and filtering the RESULT remain two different questions.
**A rights movement, stated as the grid requires.** `useShapeQuery` now fetches a public store's cap for a document named in `graphs`, which files that cap in the connected holder's ring. That is a GAIN on that call. It grants nothing new overall: it is exactly the reachability `docs.sparqlQuery` anchored on the same document already gave, and `exposeReadCap` is called only for `scope === "public"` (VERIFIED in T01.f). No document becomes readable that was not already readable by naming it. Said in `## Non-guarantees`.
`knowledge_what-a-failed-read-answers-upstream` records the durable fact: two observable outcomes only, the two collisions ("does not exist" vs "not known to me"; "unsynced" vs "genuinely empty"), no capability check on the read path, and the three rules that follow - including that the unsynced/empty collision is closed by the sync barrier and never by an error.
`docs/internal-contract.md` section 6 only: line numbers, the `ensurePhysicalRepoOpen` export this module has never had (it is `shared-wallet/physical.ts:134`), the deleted `ensureReposOpen`, the missing `openRepoUnguarded`, and `ensureRepoOpen`'s second parameter. A dated note says the rest of that document's drift was left alone deliberately.
### The two bug leaves: NEITHER is closed
- `bug_a-lost-inbox-watch-is-never-retried` - **not closed.** It asks for a recovery signal that does not depend on the identity performing a capability-bearing act. Nothing here adds one. This work arguably makes it more visible (a failed open reports a truer reason), which is not the same as retrying.
- `bug_an-empty-inbox-list-can-mean-a-failure` - **not closed.** Its mechanism is `resolveAccount` yielding nothing RATHER THAN raising, inside `enumerateMyInboxes`, which then reports a list that is complete and empty. That path was not touched: `readInboxCapPairs` already rethrows what it catches, and `record?.docPrivate` returning `undefined` still answers `[]` with no shortfall. No test proves otherwise, so the leaf stays.
### Surprises
- **VERIFIED: `fetchReadCaps` was dead.** Zero callers anywhere in `src`, `test` or `e2e`, and absent from `src/index.ts`. Its last caller went with `readUnion` in `1.0.0-dev.8`.
- **VERIFIED: `useShapeQuery`'s empty success does not come from `fetchReadCap` at all.** That call site does not exist on this path - T01.d removed it with the scope translation. It comes from the capability filter emptying a named document's objects, and from a named-but-unsynced document answering nothing. Tests (l) and (n) fail without the barrier; the task's three-layer chain describes the ANCHORED path, which is a different (and already loud) one.
- **VERIFIED: the blast radius is narrow.** Six of the eight `ensureRepoOpen` callers hold the cap they are about to use, so they never ask the network. That is why only four tests broke.
- **VERIFIED: `g === ""` in `namedDocuments` is unreachable** - `isNuri("")` is false. Kept as documentation of the SDK's own alias, said so in the comment and the commit. Test (p) is therefore a behaviour guard, not a discriminating test, and is labelled as one.
- **VERIFIED, and it was under-reported in round 1: the barrier's cost.** A named scope spends one public-store fetch and one sync barrier per document per session. The opens now run TOGETHER, so the wait is the slowest rather than the sum; the serial form cost N bounded fallbacks, 8s each in production, which an adversarial review measured. Stated in the contract.
- **The round-1 Result contained two claims this review falsified**, both corrected above: that the INERT case was unchanged (the open is new), and that the e2e was evidence for the barrier (it never walked a named scope). Both were overstatements of what had actually been checked, and the second is the one that mattered - a green suite reported as proof of a path it does not exercise.
### Follow-ups, reported and not fixed
- `packages/polyfill/src/shared-wallet/account-registry.ts:78` imports `documentInboxAddress` and never uses it - a dead import. `tsconfig.base.json` sets no `noUnusedLocals`, so nothing flags it.
- Every e2e run logs, at connection: `connect failed: docs.sparqlQuery: refused - the connected user does not hold this document's cap ... at readStoreCaps <- restoreOwnCaps <- restoreAndDrain`. That is `assertMayReach`'s own pre-existing message (this work would have produced a different one), so `restoreOwnCaps` is being refused one of the connected user's OWN store documents at connect, and the failure is logged rather than raised.
- The e2e broker login fails intermittently at 45s on the wallet-list screen, before any library code runs - 2 of 8 runs in this session.
- A two-actor journey for "a document is shared with someone while they are watching, and their named `useShapeQuery` converges" belongs in the applicative suite (`e2e/notebook.ts`), beside the journey that already checks "after Alice shares it, the same reference opens it". It is the live form of unit test (r).
- This observable re-PUBLISHES under whoever is connected at that instant rather than under the identity that subscribed. That part is genuinely pre-existing and out of scope - `publish` has always read the keyring of the current holder. The RE-ATTEMPT no longer does: it is bound to the holder that was refused (`refusedFor`), because issuing a subscribe and a fetch is a new access rather than a re-filtering, and the two must not be lumped together - round 2's Result did lump them, and that was wrong.
- `docs/internal-contract.md:7` still names `surface/watch-shape.ts`, `surface/read-model.ts` and a `storeRegistry` namespace listing three removed symbols. Out of this task's boundary.
- `knowledge_what-a-failed-read-answers-upstream` cites `213338f6`, the checkout T01.h found to be three ORM releases behind. The facts it cites are engine-side rather than ORM-side, but T01.k owns reconciling what a citation's version means.