fix: un dépôt est délivré une fois et retiré, comme en amont

This commit is contained in:
Sylvain Duchesne
2026-08-26 14:11:56 +02:00
parent abfb619c76
commit f6a48a4507
24 changed files with 1486 additions and 176 deletions
@@ -30,8 +30,8 @@ What it never does: **process an inbox so that a `ReadCap` sent to it arrives**.
The two arms this package has:
- **A `ReadCap` delivery** (`inbox-dispatch.LINK_KIND`, whose name is upstream's own word for the variant) — held for the owner's ring and filed durably with `branch-registers.addLink`, the emulated `AddLink { read_cap }`. A payload declaring itself one of these and carrying nothing that reads is REFUSED and reported, which is the envelope validation above; it used to be handed back as consumer data.
- **A bare `reference`** — a deposit whose payload IS a `did:ng:` reference and nothing else: no operation, no claim, no copy of any value. `emulated-verifier/index-deposit.ts`, `applyIndexDeposit`, resolves it and LOOKS: which document the inbox belongs to (the owner's own `AddInboxCap` pairing, `readInboxCapPairs`), what that document declares it indexes by (`urn:ng-helpers:index:field`, on the document's own NURI as subject), whether the reference is already in, what the referenced object says for that field — and only then writes ONE entry (`urn:ng-helpers:index:value`, subject = the object's NURI). Since anyone may deposit, a payload carrying an operation would be a licence to rewrite someone else's document, and a payload carrying a value would put in the index something the object does not say. **An index only ever grows**: nothing there removes an entry and nothing there can express a removal, which is what makes an empty read harmless — absent, unreadable and failed all mean *not added this time*, and a later deposit adds it. An unresolved reference is reported and costs nothing; an already-indexed object is passed over; a document declaring no field, or several, refuses rather than write something wrong — and that refusal leaves READING its entries untouched.
- **A `ReadCap` delivery** (`inbox-dispatch.LINK_KIND`, whose name is upstream's own word for the variant) — held for the owner's ring and filed durably with `branch-registers.addLink`, the emulated `AddLink { read_cap }`. A payload declaring itself one of these and carrying nothing that reads is REFUSED and reported, which is the envelope validation above; it used to be handed back as consumer data. Being refused does not save it: the arm ran, so the deposit was already consumed.
- **A bare `reference`** — a deposit whose payload IS a `did:ng:` reference and nothing else: no operation, no claim, no copy of any value. `emulated-verifier/index-deposit.ts`, `applyIndexDeposit`, resolves it and LOOKS: which document the inbox belongs to (the owner's own `AddInboxCap` pairing, `readInboxCapPairs`), what that document declares it indexes by (`urn:ng-helpers:index:field`, on the document's own NURI as subject), whether the reference is already in, what the referenced object says for that field — and only then writes ONE entry (`urn:ng-helpers:index:value`, subject = the object's NURI). Since anyone may deposit, a payload carrying an operation would be a licence to rewrite someone else's document, and a payload carrying a value would put in the index something the object does not say. **An index only ever grows**: nothing there removes an entry and nothing there can express a removal. That once made an empty read harmless — absent, unreadable and failed all meant *not added this time*, and a later drain settled it. Consumption ends that, and forces a distinction the read layer erases: `read-model.readDoc` answers `[]` whether the document is empty or the broker failed. Three sources of "no rows" are now told apart (`index-deposit.noRowsMeansAbsent`). A read that **threw** — reported through the new `onUnreadable` hook — is `unreadable`. A repo whose sync this session never confirmed is also `unreadable`, and this is the one that is easy to miss: on a fresh session over a persistent wallet the repo is present but its commits have not arrived, so the anchored query matches nothing **and raises nothing** — `open-repo` keeps `synced` and `timed-out` apart for exactly this, and says so in those words. A document this holder cannot even attempt — no cap, so the read layer drops it before issuing a query — is `unreadable` too, and that one is counter-intuitive enough to state plainly: "no cap" LOOKS deterministic and is not, because for a stranger's public document possession comes from a network fetch that swallows its errors and memoises the failure, so one outage is indistinguishable from a document that never existed. Only a document that was actually QUERIED, from a confirmed repo, and still read empty is genuinely absent — that is `refused`: reported, costs its own deposit, and the deposits behind it keep their turn. `unreadable` THROWS and stops the drain, so an outage cannot walk the queue. Getting the split wrong is costly in both directions and both were shipped before it settled: treating a failure as an absence destroyed whole queues on a cold start, and treating an absence as a failure let one dead reference — deposited by anyone — stall an index's queue one deposit per connection. An unresolved reference costs its own deposit and is reported; an already-indexed object is passed over; a document declaring no field, or several, refuses rather than write something wrong (both are clean reads, so both are `refused` and neither stops the drain) — and that refusal leaves READING its entries untouched.
*(Those two IRIs are `@ng-helpers/indexing`'s namespace, kept letter for letter because renaming them is a data-format change nobody decided. This package writing another package's namespace is odd, and it is recorded as odd rather than acted on.)*
@@ -43,9 +43,15 @@ The two arms this package has:
**After a deposit whose recipient is absent.** A deposit that landed arms a deferred stand-in for the session the owner does not have. What it achieves is the only part written down outside the package: **deposits converge** — a share becomes readable without the recipient re-opening the page. *How* is deliberately not documented here. The mechanism is unpublished by design: nothing lets a caller ask for it, name another user's inbox, or turn it off, and the confidentiality of the whole shared-wallet emulation rests on its being unreachable. Its limit is stated rather than hidden — it lives in a page, so if the page goes away the deposit waits for its owner's next connection, which is the real path and not a repair.
**And it applies the registers, not the documents.** Running for an owner who is not the one holding the page, it takes the arms that write the verifier's own bookkeeping and stops there: writing application data into an absent owner's documents from a stranger's session is a different act, and this package does not take it. So a `reference` deposit is left — traced, not reported, because waiting is not failing — and the owner's own next connection applies it. Nothing is lost: see below.
**And it applies the registers, not the documents.** Running for an owner who is not the one holding the page, it takes the arms that write the verifier's own bookkeeping and stops there: writing application data into an absent owner's documents from a stranger's session is a different act, and this package does not take it. So a `reference` deposit is left — traced, not reported, because waiting is not failing — and the owner's own next connection applies it. It is left in the QUEUE too, not merely unapplied: what a stranger's drain declines to apply it must not consume either, or the owner would lose it to a session that was never going to apply it. See below.
**Nothing is ever consumed.** No path retires an applied deposit: an inbox here is re-readable where upstream it is a queue. That is what makes every abandon, every reported refusal and every watch that could not be opened harmless — what was not applied is still there for the next push or the next connection. It is also what makes re-applying safe: every arm is idempotent, so seeing every deposit again on every run lands on the same result.
**What is applied is CONSUMED — one act, one place.** Since 2026-08-26 a deposit this package claims is retired from its inbox by `emulated-verifier/inbox-consume.ts`, which holds the only removal and the only loop that performs it; both triggers above call it and neither carries a copy. The order is upstream's, read in its source: the broker removes the message from the queue *before* delivering it (`take_first_msg` → `MSGS.take_first_value`, a read followed by a `del`), and the verifier then applies it and merely logs what it could not (`log_err!("Error during process_inbox {e}")`, never re-enqueued). So **remove, apply, log, move on** — a deposit is delivered once whether or not its arm succeeded, and a refusal is reported once instead of at every push and every connection for ever.
**What is NOT claimed is not consumed**, and the distinction carries the safety of the whole thing. Two things are left in the queue: a consumer's own payload, which declares no kind of this package's and is not a message at all; and an arm that belongs to the owner's own session while a stranger's is running — the `reference` case above. `inbox-dispatch.claimsDeposit` asks that question *before* the removal, precisely so that what a stranger's drain declines to apply it also declines to pop. Upstream the question does not arise, because only a registered reader is ever popped for (`inbox_pop_for_user`); here one inbox mixes both kinds, so it has to be read off the payload. Getting it backwards is the one way consuming could lose data outright — a stranger's timer retiring a reference the owner never indexed.
**A failure that happens before the pop still leaves the deposit.** A drain whose *read* fails takes nothing out, so what is queued waits for the next attempt; a drain whose *arm* fails has already consumed what it was given. The two are different sentences and the package's log says which is which.
**The idempotence that remains is not a replay compensation.** Filing the same Link twice is still a no-op, and a reference already in an index is still passed over — because the same cap or the same object can legitimately arrive as **two distinct deposits** (a re-share, a rotation, two depositors pointing at one thing), not because one deposit is seen twice. Upstream keeps the same check with a queue that pops (`ContactAlreadyExists`), which is the proof it never answered the replay.
## NextGraph itself
@@ -122,7 +122,7 @@ What was shared with you becomes readable after `ensureIdentity()`.
`urn:ng-eventually:` is reserved, and the reservation has two faces. In a document: triples whose **subject** falls under that prefix are dropped on read and never returned by `readUnion`; every other IRI is returned. In a deposit: two payload shapes are this package's own and it acts on them — a payload that IS a reference (a bare `did:ng:…` string and nothing else), and an object whose `kind` property is a string starting with `urn:ng-eventually:inbox:`. **Every other payload is yours, and is delivered to you untouched.**
**Do not use either shape to carry your own data.** Depositing one is still accepted and it is still returned to you by `inbox.read` and its variants — nothing is withheld — but this package now also acts on it, and what you observe is one of two things: it writes an entry into the document the inbox belongs to, if that document is one it can index; otherwise it reports a refusal on its own log stream (`console.error`, ungated, as below). That refusal does not clear on its own the way an unreachable inbox does: a deposit is not consumed, so the same refusal is reported again at every arrival on that inbox and at every connection, for as long as the deposit sits there. Carry your own data in an object with a `kind` of your own — `{ kind: "join", … }` is yours and stays yours — and never as a bare reference on its own.
**Do not use either shape to carry your own data.** Depositing one is still accepted and it is still returned to you by `inbox.read` and its variants — nothing is withheld — but this package now also acts on it, and what you observe is one of two things: it writes an entry into the document the inbox belongs to, if that document is one it can index; otherwise it reports a refusal on its own log stream (`console.error`, ungated, as below). That refusal is reported ONCE and then the deposit is gone: this package consumes what it claims, whether or not the arm succeeded. Until `1.0.0-dev.4` the same refusal came back at every arrival on that inbox and at every connection, for as long as the deposit sat there. The same now applies when the referenced document resolves but carries **nothing for the field its index declares** — a draft, say: the reference is discarded and reported, where before it sat in the queue and would have been indexed by a later drain once the document gained that field. Deposit the reference again after the document says what it needs to say. Carry your own data in an object with a `kind` of your own — `{ kind: "join", … }` is yours and stays yours — and never as a bare reference on its own.
Only a document's owner writes to it. Holding its read key never grants a write.
@@ -162,7 +162,15 @@ The same rule reaches what a call hands BACK, not only what it looked up first:
**The watching is in place by the time `ensureIdentity()` resolves**, whatever else that call made of its own work: a connection that could not restore something still rejects, and the identity it settled is watched all the same. It lasts exactly as long as that identity stays connected — changing identity or clearing it stops it, and whoever connects next is watched in their own right, so nothing of the previous one keeps applying.
**Failing to apply one inbox denies nothing.** It is reported on this package's own log stream (`console.error`, carrying this package's prefix) and never gated by `debugAccessLog` — a diagnostic may be opt-in, a failure may not. The deposit stays in its queue, so the next arrival on that inbox, or the next connection, applies it; the other inboxes were never involved, and nobody is refused anything.
**A deposit this package applies is delivered ONCE, and a failure is not retried.** It is taken out of its inbox before the arm that handles it runs, so it is gone whether that arm succeeded or not — the same order the target has, where the broker removes a message from the queue as it hands it over and the verifier merely logs what it could not apply. A failure is reported on this package's own log stream (`console.error`, carrying this package's prefix), never gated by `debugAccessLog` — a diagnostic may be opt-in, a failure may not — and then the next deposit is processed. **Do not count on a failed deposit being applied later: there is no retry, no dead-letter queue and no way to ask what was lost.** If an arrival matters to your application, the record you keep of it must be your own. What this buys is the end of the opposite behaviour: an inbox that grew for ever, and one malformed deposit re-reported at every arrival and every connection for the life of the session.
**An unapplicable item and a failed one are not the same event.** A deposit this package cannot make sense of is reported and the next one is processed — one malformed item never holds up the queue behind it. A deposit whose handling could not be COMPLETED is different: the drain STOPS, the inbox is reported, and everything not yet taken out is preserved. Only the deposit in flight is lost. So a failing broker costs one deposit per attempt, not the whole queue, and a drain that stopped is retried by your next `ensureIdentity()`.
**A reference this package cannot read counts as failed, not as absent** — including a document you hold no capability for. That is deliberate and it costs latency: such a deposit stops one drain, and the deposits behind it land on the next connection instead of this one. The alternative was worse and was measured: treating "cannot read it" as "it is not there" destroyed deposits on a transient outage, and destroyed the re-deposits sent to repair them, because a failed capability fetch is remembered for the session. Only a document actually queried, from a synced repo, that still reads empty is called absent. **Consequence to plan for:** a queue holding many references you cannot read drains one per connection.
**`inbox.processInbox` now REJECTS on a broker failure**, where it used to resolve — including when a document a deposit referenced could not be READ. A referenced document that is merely absent is a different case: that is reported and the queue carries on. Awaiting it, handle the rejection: the deposit it was applying at that moment is already consumed, the rest of that inbox is intact, and calling again resumes from the next one. Connection-time draining is unaffected — `ensureIdentity()` reports such a queue and still grants the session, exactly as before.
**Failing to apply one inbox still denies nothing else.** The other inboxes were never involved, the session is not refused, and deposits this package does not claim — your own payloads — are untouched and stay readable through `inbox.read` and `inbox.watch`.
**Any number of subscriptions on one document coexist.** Opening a document, watching an inbox and following a scope no longer silence one another. Until 2026-08-17 a second `subscribeDoc` on a document killed the first, silently — nothing rejected, the first caller's unsubscribe still appeared to work, and what an application saw was a view that stopped re-reading and an inbox that stopped notifying, with no trace anywhere near the cause. A subscriber that joins a document somebody else already opened is handed the initial `State` its own subscription would have pushed it, so joining late is not the same as never firing; and unsubscribing silences that caller and no other, including when it happens from inside a push.
@@ -202,10 +210,10 @@ What each level means here, in this package's own terms:
- **minor** — a symbol is added and nothing existing moves (`inbox.readSyncedForDocument` arrived this way).
- **patch** — a fix that changes neither the published surface nor anything above under `## Guarantees`.
**A tag says where it comes from.** A release cut on `main` carries a **full version** (`1.0.0`), and the three rules above govern what changes between two full versions. Work still on a branch carries a **pre-release** of the version it is heading for (`1.0.0-dev.3`), which sorts *below* that version by construction — so you can pin what exists today while the tag itself tells you the surface has not been released and may still move before it is. Between two pre-releases of the same version nothing is promised: re-pull and read this leaf again. When the branch lands, the full version appears alongside; the pre-release keeps resolving, so no reference you pinned is ever withdrawn from under you.
**A tag says where it comes from.** A release cut on `main` carries a **full version** (`1.0.0`), and the three rules above govern what changes between two full versions. Work still on a branch carries a **pre-release** of the version it is heading for (`1.0.0-dev.4`), which sorts *below* that version by construction — so you can pin what exists today while the tag itself tells you the surface has not been released and may still move before it is. Between two pre-releases of the same version nothing is promised: re-pull and read this leaf again. When the branch lands, the full version appears alongside; the pre-release keeps resolving, so no reference you pinned is ever withdrawn from under you.
**Tags carry the package name**, because this repository publishes more than one engagement and their versions move independently: `polyfill/v1.0.0-dev.3` is this package, `ng-e2e-helpers/v…` is the other one. A bare `v…` tag would say nothing about which surface it froze the day the two diverge — which is the day one of them takes a major and the other does not.
**Tags carry the package name**, because this repository publishes more than one engagement and their versions move independently: `polyfill/v1.0.0-dev.4` is this package, `ng-e2e-helpers/v…` is the other one. A bare `v…` tag would say nothing about which surface it froze the day the two diverge — which is the day one of them takes a major and the other does not.
`1.0.0` is a baseline, not a claim of maturity: it is the number that makes your pin mean something. Nothing was released before it, so none of the changes named above is a bump from anything — but the next release very likely is a major. What exists today is `1.0.0-dev.3`, on a branch: pin that string exactly, and anchor your `usage_` leaf's `against:` on it — `against: @ng-eventually/polyfill@1.0.0-dev.3`, the string you pinned, never the version it is heading for. `1.0.0-dev.3` narrows an existing call rather than moving a signature: two payload shapes an application could carry freely are now this package's own, and one it cannot apply is reported — a state you did not have to handle before, which is a **major** by the rules above. It lands inside the pre-release line rather than on a new major number because `1.0.0` has not been cut, so there is nothing yet to break away from.
`1.0.0` is a baseline, not a claim of maturity: it is the number that makes your pin mean something. Nothing was released before it, so none of the changes named above is a bump from anything — but the next release very likely is a major. What exists today is `1.0.0-dev.4`, on a branch: pin that string exactly, and anchor your `usage_` leaf's `against:` on it — `against: @ng-eventually/polyfill@1.0.0-dev.4`, the string you pinned, never the version it is heading for. `1.0.0-dev.4` narrows a **guarantee** and adds a rejection: a deposit this package applies is now consumed, so a failure that used to be retried at the next arrival or the next connection is now final and reported once; and `inbox.processInbox` rejects on a broker failure where it used to resolve. An application that leaned on the retry — however implicitly — has to keep its own record instead, and one that awaited `processInbox` without a `catch` now has a rejection to handle. Either is a **major** by the rules above. `1.0.0-dev.3` before it narrowed an existing call the same way: two payload shapes an application could carry freely became this package's own, and one it cannot apply is reported. It lands inside the pre-release line rather than on a new major number because `1.0.0` has not been cut, so there is nothing yet to break away from.
There is no changelog file and no deprecation window: **the sections above are the release note.** A removal or a narrowing lands in `## Surface` and `## Guarantees` in the same version that ships it, and a symbol is never left published-but-dead as a courtesy. Diff this leaf between two pulls — `## Guarantees` and `## Non-guarantees` before `## Surface`, because that is where a narrowing shows up first.
@@ -53,3 +53,7 @@ Everything else the package exports is offered and NOT consumed here — the scr
**`declareSuite` fits a journey suite and not a check batch.** Only the applicative suite uses it; the other three declare a local `record`/`check` pair and print their own summary, because their shape is a flat list of checks discovered as the run goes rather than named journeys declared up front. The up-front declaration is exactly what makes the report bounded, so this is not a request to relax it — but three of four suites falling outside it suggests the package covers one of two real shapes.
**The screen inventory is exported and unreachable in practice.** `BROKER_SCREENS` and the `WALLET_*` page descriptions are published, but a suite that crosses the broker with `setupBrokerPage` never needs them, and one that wanted to extend the crossing has no supported way to hand a modified inventory back in. As it stands they read as internals that happen to be exported.
## Needs
**A bounded wait for a condition, reporting what it last saw.** These suites hand-roll one in at least three places — polling until `listMyEntityDocs` shows a new document, until an append is visible in this session, until a triple lands — each with its own loop, its own bound and its own failure message. A second consumer, the indexing layer's suite in another repository, independently wrote the same thing to know when a deposit had become an entry. The shape both arrived at is the same: re-READ by ordinary means until a predicate holds, return the first reading that satisfies it, and on expiry name what never arrived **and the last reading seen** — the second half is what turns a timeout into a diagnosis. It is not `within`: that bounds one call, this bounds a convergence. Two independent rediscoveries suggest it belongs on the engagement rather than in each suite.
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@ng-eventually/polyfill",
"version": "1.0.0-dev.3",
"version": "1.0.0-dev.4",
"type": "module",
"description": "Polyfill of the NextGraph JS SDK over @ng-org/web + @ng-org/orm, with emulated capabilities and inbox. Drop-in; remove at migration.",
"main": "./src/index.ts",
@@ -579,8 +579,11 @@ export async function enumerateMyInboxes(): Promise<InboxEnumeration> {
* upstream an inbox is consumed, and processing a message *applies* it. Applying a
* Link means writing it here.
*
* Idempotent — re-applying the same Link is a no-op, so re-processing an inbox
* (a second tab, a reconnect) costs nothing.
* Idempotent — filing the same Link twice is a no-op. Kept after deposits became consumed
* (2026-08-26), because the case it answers was never the replay: the SAME cap legitimately
* arrives as two distinct deposits when a document is shared twice or a key is rotated, and
* both are consumed and both applied. Without it that user's User branch carries the same
* `ng:link` twice.
*
* `forHolder` names WHOSE User branch this lands on when it is not the connected
* identity's — the emulated inbox processor
@@ -598,7 +601,17 @@ export async function addLink(cap: ReadCap, forHolder?: PrincipalId): Promise<vo
if (!store) return;
if ((await readLinks(forHolder)).includes(cap)) return;
const s = await session();
try {
// PROPAGATES since 2026-08-26. It used to catch and log, which was survivable only while
// nothing was consumed: the deposit stayed, and the next connection filed the cap again.
// Now the deposit is popped BEFORE this runs, so a swallowed failure is a cap lost for
// good — and worse, `applyLink` went on returning `true`, so the drain's trace counted a
// Link it had not filed.
//
// Its ONE caller does NOT catch, deliberately: most of what happens above this line
// (`ensureAccount`, `readLinks`, `session`) is deposit-INDEPENDENT infrastructure that
// fails for the whole queue at once, so catching per deposit meant popping and destroying
// every remaining one while reporting each. The throw travels to the drain, which reports
// the deposit it lost, reports the inbox, and stops — preserving everything not yet taken.
await registerUpdate(
s.sessionId,
`INSERT DATA { <${USER_BRANCH_SUBJECT}> <${P.link}> "${escapeLiteral(cap)}" }`,
@@ -606,9 +619,6 @@ export async function addLink(cap: ReadCap, forHolder?: PrincipalId): Promise<vo
"addLink",
holderRing(forHolder),
);
} catch (error) {
console.error(accessLogPrefix() + " addLink failed:", error);
}
}
/**
@@ -82,8 +82,8 @@ import type { Nuri } from "../model/types";
function reportUndrained(inbox: Nuri, error: unknown): void {
console.error(
accessLogPrefix() +
" connected, but this queue could not be drained — a share deposited in it is not " +
"applied yet, and connecting again will try it again: " +
" connected, but this queue could not be drained — whatever it had already taken out " +
"is consumed and gone; what is still in it waits for the next attempt: " +
shortNuri(inbox) +
":",
error,
@@ -120,12 +120,19 @@ const inFlight = new Map<string, Promise<void>>();
* ── Why a queue is NOT the connection, corrected 2026-08-16 ───────────────
* This said "there is no third category here" and rejected on an undrained inbox too. The
* reasoning was that an unapplied share and a lost restore look the same to a person. What
* it missed is what happens NEXT: an inbox is not consumed by failing, so a queue that
* cannot be applied is still on the list at the following connection, and the one after
* it missed is what happens NEXT. At the time nothing was ever consumed, so a queue that
* could not be applied was still on the list at the following connection, and the one after
* that. One unapplicable item therefore did not delay a share — it **locked the person out
* of the application, permanently**, and got worse rather than better with time. Reported
* live by a consuming application, three sign-ins out of three.
*
* Consuming (2026-08-26) removes that particular ratchet — a deposit whose arm fails is now
* gone, so it cannot deny the same session twice — but it does NOT restore the old rule.
* Rejecting here would still be wrong, and now for a worse reason: the item is already
* consumed by the time its arm fails, so denying the session would trade a permanent
* lockout for a permanent lockout AND a lost deposit. Applying one item stays data, and
* data is reported, never a refusal to connect.
*
* That is strictly worse than the silence the 2026-08-13 shape replaced, and it collapsed
* a distinction that has to stay open: **failing to REACH the queues** (the session cannot
* proceed at all — reject, unchanged) against **failing to apply what is in one** (data,
@@ -135,8 +142,11 @@ const inFlight = new Map<string, Promise<void>>();
* restore step is a plain `await` and still rejects, so a broker that cannot answer fails
* the connection there, before any queue is reached. Only the per-inbox drain is caught.
*
* Not silence: {@link reportUndrained} is loud and ungated, and the caps a share carries
* are not lost — the deposit stays in the queue, so the next connection applies it.
* Not silence: {@link reportUndrained} is loud and ungated. What it no longer promises,
* since deposits became consumed on 2026-08-26, is that nothing was lost: the deposit the
* drain was applying when it fell over is GONE, and only what is still queued reaches the
* next connection. The message says exactly that, and so does the test
* `an I/O failure while filing a Link STOPS the drain, so the rest of the queue survives`.
*
* Not fixed with a retry, a timeout or a flag on purpose: deciding *what to do* about a
* broker that cannot answer belongs to the caller, and it can only decide if it is told.
@@ -223,6 +233,12 @@ export async function connectedUser(): Promise<void> {
// is abandoning, which this function has always called "not a failure", and
// reporting it would put a broker-looking error in the log every time a page
// switches user. The loop is about to return for the same reason.
//
// This guard cannot tell an abandonment from a genuine failure that happened to
// coincide with a user switch, and it does not have to: since 2026-08-26 a deposit
// that was consumed and not applied is reported by `inbox-consume.deliverDeposits`
// BEFORE the error is thrown, so what this drops is the duplicate, never the only
// notice of a loss. Dropping it here was silent for exactly one day.
if (!stillConnected()) return;
reportUndrained(inbox, error);
}
@@ -0,0 +1,212 @@
/**
* Consuming an inbox — the ONE act that retires a deposit, and the ONE loop that performs it.
*
* ── Upstream pops, and the order is not an accident ───────────────────────
* Read at all three of its sites, because the order is the whole contract:
*
* 1. the broker REMOVES the message from the queue — `ClientEvent::InboxPopRequest` →
* `inbox_pop_for_user` → `take_first_msg` → `MSGS.take_first_value`, which is
* `get_first_key_value` followed by `del` (`engine/repo/src/kcv_storage.rs:557`,
* `engine/storage-rocksdb/src/kcv_storage.rs:239-243`). The removal happens in the
* STORE, before anything has looked at the content;
* 2. only then is it delivered (`ClientMessageContentV0::InboxReceive { from_queue: true }`,
* `engine/net/src/actors/client/client_event.rs:68`);
* 3. the verifier applies it, and a failure is LOGGED AND SWALLOWED —
* `log_err!("Error during process_inbox {e}")` — never re-enqueued. It then asks for
* the next one (`engine/verifier/src/verifier.rs:1680-1715`).
*
* So: **remove, then apply, then log whatever went wrong, then move on.** A deposit is
* delivered exactly once whether or not its arm succeeded, and an arm that throws loses it
* — upstream that message left the store before the arm ever ran. Removing FIRST is
* therefore not a risk this module takes, it is the behaviour it reproduces; and it makes
* the reverse mistake impossible, since a removal that fails stops the apply instead of
* letting an applied deposit stay in the queue.
*
* ── Why a deliverability question exists here and not upstream ────────────
* Upstream an inbox holds nothing but typed messages, and WHO may pop one is settled by
* reader registration rather than by content: `inbox_pop_for_user(user)` pops only from
* inboxes that user is a registered reader of, so a message is never popped by anybody but
* its own recipient. Here an inbox holds a MIX — this package's deposits and a consumer's
* own payloads, which `surface/inbox.ts` hands back untouched — so "may this be popped
* now?" has to be asked of the content. That is `inbox-dispatch.claimsDeposit`, and it is
* asked BEFORE the removal for the same reason the broker checks registration before
* `take_first_msg`: what is not deliverable to this consumer was never popped for it.
*
* ── One removal, and it is here ───────────────────────────────────────────
* Both triggers this package drains from — the owner's own session (`inbox.processInbox`)
* and the deferred stand-in for an absent owner (`inbox.processForOwner`, scheduled by
* `emulated-verifier/inbox-processor.ts`) — call {@link deliverDeposits} and nothing else.
* Neither carries a copy of the removal: the triggers differ in WHO drains and WHEN, never
* in what consuming means. `grep -rn "DELETE" packages/polyfill/src/` shows the only
* deposit removal in the package is {@link removeDeposit} below.
*/
import { registerUpdate } from "./register-write";
import { applyDeposit, claims, claimsDeposit } from "./inbox-dispatch";
import type { Applying } from "./inbox-dispatch";
import { getStoreRegistryDeps } from "../shared-wallet/bootstrap";
import { accessLogPrefix, logStage, shortNuri } from "../shared-wallet/access-log";
/**
* One deposit as it is STORED: the RDF subject `inbox.post` minted for it, and what it
* carries. The subject is the removal key — `post` gives every deposit a fresh unique
* subject, so naming it retires that deposit and no other, where a `ts` could collide.
*
* Deliberately NOT the published {@link import("../surface/inbox").Deposit}: a subject is
* emulation detail (upstream a queued message has no address of its own, it has a
* position), so it stays inside the package and an application never learns it.
*/
export interface StoredDeposit {
/** The RDF subject the deposit lives under, in its inbox document's default graph. */
readonly subject: string;
/** What the deposit carries — handed to the dispatch as-is. */
readonly payload: unknown;
}
/**
* Retire ONE deposit from its inbox. **The only removal in this package.**
*
* `DELETE WHERE { <subject> ?p ?o }` — every triple of that one deposit, whichever
* predicates it happens to carry (`from` is optional). Two established forms are combined
* here, and neither half is invented:
*
* - `DELETE WHERE` with a VARIABLE predicate (`<s> ?p ?o`) is the shape verified against
* the real broker for removing a whole subject
* (`docs/decisions/sparql-delete-for-orm-objects.md`);
* - it is ANCHORED with no `GRAPH <…>` wrapper, like every other register write in this
* package (`public-store.exposeReadCap`, `branch-registers.publishInboxAddress`). That
* is not a shortcut, it is where the data IS: `inbox.post` writes the anchored DEFAULT
* graph and `DEPOSITS_QUERY` reads it, so a `GRAPH`-wrapped delete would name a graph
* the deposits were never written to. The decision above shows the wrapped form because
* its subject is an ORM object in a named graph; the anchor plays that role here.
*
* A `;`-joined compound update is exercised nowhere in this library, so this stays one
* statement.
*
* Through `registerUpdate`, and that is deliberate: retiring a deposit is the verifier's
* own bookkeeping, not an application write — upstream it happens inside the broker's
* store, and no application makes it. That door skips the OWNERSHIP guard, which a drain
* running for an absent owner could not pass, and keeps `assertMayReach` against the named
* ring: whoever consumes must hold the inbox's cap, and `processForOwner` names the
* owner's ring exactly as it does for every other step it takes.
*
* A failure PROPAGATES. Nothing is applied for a deposit that could not be retired — the
* caller's loop stops and its drain reports itself where drains report — because applying
* one that stayed in the queue is the replay this module exists to end.
*/
// @provenance removeDeposit kind=aligned level=1 ref=engine/broker/src/server_storage/core/inbox.rs:take_first_msg — upstream the store REMOVES the message as it hands it over (`MSGS.take_first_value` = `get_first_key_value` + `del`); here the queue is RDF in a document, so the same act is a DELETE of the deposit's own subject
async function removeDeposit(stored: StoredDeposit, ctx: Applying): Promise<void> {
const { sessionId } = await getStoreRegistryDeps().getSession();
await registerUpdate(
sessionId,
`DELETE WHERE { <${stored.subject}> ?p ?o }`,
ctx.inbox,
"inboxConsume",
ctx.ring,
);
}
/**
* Deliver every deposit in `stored` to the dispatch, consuming each as it goes. Answers
* how many produced an EFFECT — which is what a drain traces, and all any caller wants.
*
* The loop both triggers share, so that "what consuming means" cannot come to differ
* between the owner's own drain and the deferred one. Per deposit, in this order:
*
* 1. **claimed?** — a payload no arm answers for is not popped at all, and the loop
* moves on leaving it in the inbox. That covers a consumer's own data (not a message,
* so nothing is reported) and a deposit whose arm belongs to the owner's own session
* while a stranger's is running (see `inbox-dispatch.claimsDeposit`);
* 2. **remove** — the pop, before anything is applied, as upstream;
* 3. **apply** — the `match`, one arm per declared type. A refusal inside an arm is
* reported on the log stream and the deposit is already gone, which is upstream's
* `log_err!` + move on, and the end of the re-report-forever loop this package had.
*
* Sequential rather than parallel, for the reason every drain in this package is: an arm
* writes what it applies (`branch-registers.addLink`, on the owner's User branch), so two
* in flight over one queue interleave their reads and writes.
*/
// @provenance deliverDeposits kind=aligned level=1 ref=engine/verifier/src/verifier.rs:inbox — upstream's pop-apply-log-next loop, over a queue that is a document here rather than a column family
export async function deliverDeposits(
stored: readonly StoredDeposit[],
ctx: Applying,
): Promise<number> {
let applied = 0;
for (const deposit of stored) {
// The holder can move between two arms as easily as during the read that found these.
// Stopping leaves every deposit not yet reached in the queue for its owner — which is
// the whole reason this is asked BEFORE the pop rather than after it.
if (!holderStillStands(ctx)) {
logStage(
"ABANDONED " + shortNuri(ctx.inbox) + " — the identity changed while it was being " +
"processed; its remaining deposits stay for their owner",
);
return applied;
}
if (!claimsDeposit(deposit.payload, ctx)) continue;
// No subject, no pop. `?d` is bound by the deposits query, so this should be
// unreachable — but `depositsFrom` defaults it to "" rather than dropping the row, and
// an empty subject would issue `DELETE WHERE { <> ?p ?o }`: a malformed statement whose
// effect on a real broker nobody has established. Refusing to APPLY as well is the
// load-bearing half — applying without retiring is precisely the replay this module
// exists to end, and it would repeat at every drain for as long as the row sat there.
if (deposit.subject === "") {
console.error(
accessLogPrefix() + " a deposit in " + shortNuri(ctx.inbox) + " carries no subject, " +
"so it cannot be retired — refusing to apply it rather than applying it again at " +
"every drain. Nothing was consumed and nothing was written.",
);
continue;
}
await removeDeposit(deposit, ctx);
// Popping is an AWAIT, and the arm below reads AMBIENT state — so the identity can move
// across exactly this gap, between the decision to deliver and the delivery itself.
// Re-asked rather than assumed: applying now would file this deposit for whoever holds
// the page instead of for its owner, which is the mis-filing the whole `Applying`
// context exists to prevent.
//
// BOTH guards, and that matters: `processInbox` names its owner as a constant
// (`ownerHoldsTheSession: () => true`) because it only ever runs in that owner's own
// session, so the claim check alone cannot see the move — only `stillTheHolder` can.
// Checking one and not the other left this hole open, which is how it was missed.
//
// Reported ungated and then ABANDONED: the deposit is gone and cannot be got back, and
// whatever is behind it in the queue belongs to a session that is no longer here.
if (!holderStillStands(ctx) || !claims(deposit.payload, ctx)) {
console.error(
accessLogPrefix() + " a deposit in " + shortNuri(ctx.inbox) + " was consumed but not " +
"applied: the identity changed while it was being taken out of the queue, and " +
"applying it now would file it for the wrong holder. It cannot be recovered; what " +
"is still queued stays for its owner.",
);
return applied;
}
// The arm's own I/O failure is REPORTED HERE, at the source, and only then allowed to
// travel. It must stop the drain — everything not yet popped has to survive — but a
// throw is at the mercy of whoever catches it, and one caller legitimately swallows:
// `connect.connectedUser` drops the error when the identity has moved, because an
// ownership refusal there is abandoning and not a failure. That guard cannot tell the
// two apart, so an arm that failed WHILE the page switched user destroyed a deposit and
// printed nothing at all. Reporting before rethrowing closes that without touching a
// policy that is right for its own case: the loss is announced by the only code that
// knows a deposit was consumed.
try {
if (await applyDeposit(deposit.payload, ctx)) applied += 1;
} catch (error) {
console.error(
accessLogPrefix() + " a deposit in " + shortNuri(ctx.inbox) + " was consumed and " +
"could not be applied — it cannot be recovered; what is still queued stays for " +
"the next attempt:",
error,
);
throw error;
}
}
return applied;
}
/** Is the identity this drain was authorised for still the one holding the page? A drain
* that NAMED its holder rather than resolving it has nothing to lose and always stands. */
function holderStillStands(ctx: Applying): boolean {
return ctx.stillTheHolder === undefined || ctx.stillTheHolder();
}
@@ -85,15 +85,39 @@ export interface Applying {
/**
* Whether {@link owner} is ALSO the identity holding the session.
*
* It decides what an arm may do, and the line is not arbitrary. A drain running under
* It decides what is DELIVERED, and the line is not arbitrary. A drain running under
* SOMEONE ELSE's session (the deferred stand-in for an absent owner,
* `emulated-verifier/inbox-processor.ts`) usurps the owner's identity, and it does so
* for the verifier's own bookkeeping only — a register write on the owner's User
* branch. Writing APPLICATION data into the owner's documents from a stranger's page is
* a different act, and this package does not take it: the deposit is not consumed, so
* it stays for the owner's own next connection, which is the real path.
* a different act, and this package does not take it.
*
* Since 2026-08-26 it also decides what is CONSUMED, and the two must stay the same
* decision: what a stranger's drain declines to apply, it must not pop either, or the
* owner would lose a deposit to a session that was never going to apply it. So this is
* read by {@link claimsDeposit}, before the removal, and not by the arm afterwards.
*
* A PREDICATE rather than a value, because it is asked once per deposit: a drain runs
* across many awaits and the owner can arrive or leave inside it. A drain that captured
* it once would keep treating every later deposit as the owner's long after he stopped
* holding the page.
*/
readonly ownerHoldsTheSession: boolean;
readonly ownerHoldsTheSession: () => boolean;
/**
* Is {@link owner} STILL the identity this drain was authorised for? Consulted before
* each deposit, never once for the run.
*
* Only a drain that resolved its holder from the ambient session needs it —
* `inbox.processInbox`, whose guard ran at entry while its arms read and write many
* awaits later, so the identity can move mid-run and its remaining deposits must be left
* for whoever owns them. It matters more now than it did: a deposit reached after the
* switch would be POPPED under the wrong identity, not merely mis-filed.
*
* Omitted by a drain that NAMED its holder instead of resolving it
* (`inbox.processForOwner`, which runs under a stranger's session on purpose and would
* abandon every run if it asked). Absent means "nothing can invalidate this run".
*/
readonly stillTheHolder?: () => boolean;
}
/**
@@ -132,21 +156,84 @@ export function linkPayload(cap: ReadCap): { kind: string; cap: ReadCap } {
* item is reported and denies nobody their session, nor the OTHER deposits their turn. A
* genuine I/O failure is a different thing and still propagates — the caller reports the
* whole inbox and moves to the next one.
*
* **The deposit is already gone when this runs**, and the wording says so. It was popped
* before the arm was entered (`inbox-consume.deliverDeposits`), exactly as upstream logs
* `Error during process_inbox` for a message the broker took out of the queue before
* delivering it. So this is reported ONCE, not at every push and every connection for as
* long as the deposit sits there — which is what it used to promise, in these words.
*/
function report(ctx: Applying, why: string): void {
console.error(
accessLogPrefix() + " could not apply a deposit in " + shortNuri(ctx.inbox) +
" — it stays in it and the next attempt will try again: " + why,
" — it has been consumed and will not be retried: " + why,
);
}
/**
* May this deposit be POPPED for the consumer described by `ctx`? Asked before anything is
* removed, and the reason it has to exist is a divergence, not a preference.
*
* Upstream nothing asks it: an inbox holds only typed messages, and WHO may take one is
* settled by reader registration rather than content — `inbox_pop_for_user(user)` walks
* only the inboxes that user reads, so a message is never popped by anyone but its
* recipient. Here one inbox holds a MIX, so the same question has to be put to the payload.
*
* Two answers are "no", and neither is a failure:
*
* - **a consumer's own payload.** It declares no kind of this package's, so no arm claims
* it, and nothing is reported: it is not an unapplied message, it is not a message. It
* is also the thing `inbox.read` and `inbox.watch` hand back, so consuming it would
* take from an application the data it deposited for itself.
* - **an arm that belongs to the owner's OWN session, while a stranger's is running.**
* The deferred stand-in usurps an absent owner's identity for the verifier's own
* bookkeeping only; writing application data into that owner's documents is an act it
* does not take (see {@link Applying.ownerHoldsTheSession}). Upstream such a pop simply
* never happens — only the recipient pops. So the deposit is left in the queue for the
* owner's next connection, which is the real path, and is NOT consumed by a drain that
* was never going to apply it. Getting this wrong is the one way consuming could lose
* data outright: a stranger's timer would retire a reference the owner never indexed.
*
* The trace lives here rather than in the arm, because this is now where the decision is.
*/
// @provenance claimsDeposit kind=divergent level=1 ref=engine/broker/src/server_broker.rs:inbox_pop_for_user — upstream deliverability is reader REGISTRATION and the content is never consulted; here an inbox mixes this package's deposits with a consumer's own, so it is read off the payload
export function claimsDeposit(payload: unknown, ctx: Applying): boolean {
const claimed = claims(payload, ctx);
if (!claimed && kindOf(payload) === REFERENCE_KIND) {
logStage(
"WAITING " + shortNuri(ctx.inbox) + " — a reference deposit is applied by its own " +
"owner's session; it stays for " + ctx.owner,
);
}
return claimed;
}
/**
* The same question with NO trace — for the re-ask that follows the pop.
*
* Separate because the trace is only true the FIRST time. Asked again after the deposit has
* been removed, {@link claimsDeposit} announced "it stays for <owner>" about something that
* had just been destroyed, one line above the report saying so. A predicate that logs is
* fine where it is asked once; this one is asked twice, and only the first ask is a wait.
*/
export function claims(payload: unknown, ctx: Applying): boolean {
const kind = kindOf(payload);
if (kind === null) return false;
if (kind === REFERENCE_KIND && !ctx.ownerHoldsTheSession()) return false;
return true;
}
/**
* Apply ONE deposit for the owner of its inbox. Answers whether it produced an effect —
* which is what a drain counts, and all any caller has ever needed from it.
*
* Nothing is consumed here, by any arm: an inbox stays re-readable, so every abandon and
* every refusal above is harmless — what was not applied is still there for the next push
* or the next connection.
* **Called only for a deposit already consumed**, and only through
* `inbox-consume.deliverDeposits`, which asked {@link claimsDeposit} first and popped it
* second. Every arm below therefore runs on a deposit that has left its inbox: a refusal
* is final and is reported once, and there is no next attempt to leave anything for. The
* arm that must NOT run under a stranger's session is not guarded here — it is not
* delivered here at all, which is the only way a refusal and a non-delivery stay
* distinguishable now that one of them destroys.
*/
// @provenance applyDeposit kind=aligned level=1 ref=engine/verifier/src/inbox_processor.rs:process_inbox — the same `match` over the declared content, one arm per variant, `NotImplemented` for the rest
export async function applyDeposit(payload: unknown, ctx: Applying): Promise<boolean> {
@@ -190,6 +277,21 @@ async function applyLink(payload: unknown, ctx: Applying): Promise<boolean> {
return false;
}
getCaps().learnFor(ctx.ring, cap);
// PROPAGATES, deliberately, and this arm is where that was got wrong twice in one day.
// `addLink` used to swallow, so a cap that was never filed was reported as applied. The
// first correction caught the throw HERE and reported per deposit — which was worse: most
// of what `addLink` does before its write (`ensureAccount`, `readLinks`, `session`) is
// deposit-INDEPENDENT infrastructure that fails for the whole queue at once, so catching
// it meant popping and destroying every remaining deposit one by one while reporting each.
//
// A genuine I/O failure is not an unapplicable item: it stops the drain, the caller
// reports the whole inbox, and everything not yet popped is preserved for the next
// attempt. That is the rule stated beside {@link report}, and BOTH arms follow it — but
// the reference arm only since its `unreadable` outcome was split out of `refused` on the
// same day. It had been the opposite: `read-model.readDoc` swallows a failed read and
// answers `[]`, so an outage arrived as an ordinary refusal and the drain walked the
// whole queue destroying it. Correcting one arm and leaving the other is precisely how
// that survived a round of review.
await addLink(cap, ctx.owner);
return true;
}
@@ -197,20 +299,14 @@ async function applyLink(payload: unknown, ctx: Applying): Promise<boolean> {
/**
* The **reference** arm: resolve it and look — see `index-deposit.ts` for the whole of it.
*
* Only in the owner's OWN session, for the reason {@link Applying.ownerHoldsTheSession}
* gives: an entry is application data in the owner's document, and a drain running under a
* stranger's session writes the verifier's registers and nothing else. Left waiting is not
* left silent, and it is not a failure either — the deposit is not consumed, so the owner's
* next connection applies it, which is the path this package documents.
* Only ever entered in the owner's OWN session, and {@link claimsDeposit} is what makes
* that true: under a stranger's session this deposit is not delivered, so it is not popped
* either and it waits for its owner. The test used to stand HERE and return `false`, which
* was harmless while nothing was consumed and would now retire a reference nobody applied.
* That is why deliverability had to become a question asked before the pop rather than a
* branch inside the arm.
*/
async function applyReference(reference: Nuri, ctx: Applying): Promise<boolean> {
if (!ctx.ownerHoldsTheSession) {
logStage(
"WAITING " + shortNuri(ctx.inbox) + " — a reference deposit is applied by its own " +
"owner's session; it stays for " + ctx.owner,
);
return false;
}
const application = await applyIndexDeposit(reference, ctx.inbox);
switch (application.outcome) {
case "applied":
@@ -222,14 +318,33 @@ async function applyReference(reference: Nuri, ctx: Applying): Promise<boolean>
case "unchanged":
return false;
case "skipped":
// A property of the OBJECT, not a failure: it resolved, and there is nothing to
// index it by. Traced, not reported — see `index-deposit.ts`.
logStage(
"NOT INDEXED " + shortNuri(application.object) + " — " + application.reason,
// A property of the OBJECT rather than a failure — it resolved, and there is nothing
// to index it by. It was TRACED and not reported while deposits were re-read for
// ever: an object that carried nothing for the field yet would be indexed by a later
// drain once it did. Consumption ends that, so the deposit is discarded here and now,
// and a discard may not be gated behind `debugAccessLog` — a diagnostic may be
// opt-in, something the caller can no longer recover may not.
report(
ctx,
"a reference to " + shortNuri(application.object) + " was discarded — " +
application.reason,
);
return false;
case "refused":
// DETERMINISTIC: a misconfigured index, an inbox that is not a document's. Trying
// again changes nothing, so this is an unapplicable item — reported, and the deposits
// behind it keep their turn.
report(ctx, application.why);
return false;
case "unreadable":
// The read established NOTHING — it threw, or it answered nothing from a repo whose
// sync this session never confirmed (`index-deposit.noRowsMeansAbsent`). THROWS, so
// the drain stops and everything still queued survives — the same rule the Link arm
// follows. Treating this as an ordinary refusal let a single outage walk the whole
// queue, popping and destroying every reference while reporting each as absent.
// `inbox-consume.deliverDeposits` reports the deposit this one cost before letting
// the throw travel. An object that is genuinely absent is `refused`, not this: it
// costs its own deposit and the queue behind it carries on.
throw new Error(application.why);
}
}
@@ -37,13 +37,18 @@ const running = new Map<Nuri, Promise<void>>();
* `inbox.watch` use — and it is NOT gated by the access-log flag: a diagnostic may be
* opt-in, a failure may not.
*
* One failed item denies nothing, here as at connection: the inbox is not consumed by
* failing, the deposit stays in it, and the next push or the next connection applies it.
* One failed item denies nothing, here as at connection. What it no longer promises is a
* RETRY: since 2026-08-26 a deposit is popped before its arm runs
* (`emulated-verifier/inbox-consume.ts`), so whatever had already been delivered when this
* drain fell over is gone, exactly as upstream loses a message the broker took out of the
* queue. What is still QUEUED is untouched and the next push or connection reaches it —
* which is a different sentence, and the wording says so rather than promising the old one.
*/
function reportFailure(inbox: Nuri, error: unknown): void {
console.error(
accessLogPrefix() + " could not apply what is in this inbox — its deposits stay in it " +
"and the next attempt will try again: " + shortNuri(inbox) + ":",
accessLogPrefix() + " could not finish draining this inbox — whatever it had already " +
"taken is consumed; what is still queued waits for the next attempt: " +
shortNuri(inbox) + ":",
error,
);
}
@@ -187,9 +187,12 @@ async function applyWhatArrived(obs: Observation, inbox: Nuri): Promise<void> {
// is not theirs. (Landing after it, the run ABANDONS quietly instead and returns; both
// leave the deposit where it is.) Abandoning is what this package has always called
// "not a failure", and reporting it would put a broker-looking error in the log every
// time a page switches user. Nothing is lost: an inbox is not consumed by being
// abandoned, so the deposit is still there for its owner's next connection. Same rule,
// same words, as the drain loop in `connect.connectedUser`.
// time a page switches user. Nothing is lost: abandoning happens BEFORE the deposit it
// stopped at is taken out of the queue (`emulated-verifier/inbox-consume.ts` asks
// whether the holder still stands, then pops), so that deposit and every one behind it
// are still there for its owner's next connection. What had already been popped was
// applied for the holder that was authorised at the time. Same rule, same words, as the
// drain loop in `connect.connectedUser`.
if (!current(obs)) return;
throw error;
}
@@ -388,7 +391,9 @@ async function enumerate(obs: Observation): Promise<void> {
*
* It never rejects. Failing to WATCH is reported where failures go and denies nobody their
* session — the same rule the drain loop above it follows, and for the same reason: what
* has not been applied stays in its queue for the next attempt.
* has not been DELIVERED stays in its queue for the next attempt. (Delivered, not applied:
* since 2026-08-26 a deposit is popped before its arm runs, so one whose arm then failed is
* gone — see `emulated-verifier/inbox-consume.ts`. Failing to WATCH takes nothing out.)
*/
// @provenance startObservingInboxes kind=aligned level=1 ref=engine/verifier/src/verifier.rs:inbox — upstream a connected verifier is handed each inbox message as it arrives and applies it; here the arrival signal is a document push, and the set of inboxes is the User branch's `AddInboxCap` records
export async function startObservingInboxes(): Promise<void> {
@@ -19,8 +19,11 @@
* seconds later. Alice deposits into Bob's inbox; twenty seconds later this package drains
* Bob's inbox, while the connected session is still Alice's. When Bob IS connected in this
* page, the observation has already applied it long before the window closes and the timer
* finds nothing left to do — harmless (draining is idempotent) and not worth a special
* case: the two answer different questions, and only this one answers the absent owner.
* finds an EMPTY queue — since 2026-08-26 that is literally true rather than merely
* harmless: the observation CONSUMED the deposit, so there is nothing left to see, where
* before the timer re-read it and relied on every arm being idempotent. Not worth a special
* case either way: the two answer different questions, and only this one answers the absent
* owner.
*
* ── That is identity usurpation, and it is deliberate ─────────────────────
* It is only possible because one shared wallet holds every virtual identity, and it is
@@ -24,11 +24,22 @@
* fields and corrupt each other's entries.
*
* ── AN INDEX ONLY EVER GROWS ──────────────────────────────────────────────
* Nothing here removes an entry, and nothing here can express a removal. That is what
* makes the failure story trivial: since the only write is an addition, a read that comes
* back empty — object gone, unreadable, or the broker silent — can only ever mean "not
* added this time", and a later deposit adds it. Nothing has to tell an absence from a
* failure, so nothing can get that wrong. The price, stated plainly because someone will
* Nothing here removes an entry, and nothing here can express a removal, so the only write
* is an addition and curation stays convergent whatever order references arrive in.
*
* That used to make the failure story trivial too: a read coming back empty — object gone,
* unreadable, or the broker silent — meant "not added this time" whichever it was, so
* nothing had to tell an absence from a failure. **That is no longer true, and it is the
* sharpest consequence of consuming.** An absence is deterministic and costs its own deposit;
* a silent broker is transient and must stop the drain, or one outage walks the queue and
* destroys every reference in it. `read-model.readDoc` answers `[]` for both, so this module
* passes it an `onUnreadable` reporter and asks which one happened.
*
* What it no longer means, since deposits became consumed on 2026-08-26, is "and a later
* drain adds it". THIS deposit is gone once it has been looked at, so a broker that was
* silent for the one second the reference was examined costs that reference permanently.
* The recourse is a NEW deposit, which someone has to make; nothing retries on its own.
* Every such case is reported ungated, so it is at least never silent. The price, stated plainly because someone will
* need it: an entry cannot be taken out of an index, by anyone, including its owner.
*
* Harmless is not the same as invisible: every reference that could not be resolved is
@@ -46,6 +57,8 @@
import { readUnion } from "../surface/read-model";
import { sparqlUpdate } from "../surface/docs";
import { readInboxCapPairs } from "./branch-registers";
import { getSyncState } from "./open-repo";
import { mustNotAttempt } from "./reach";
import { session } from "../shared-wallet/account-registry";
import { assertNuri, escapeLiteral } from "../surface/sparql";
import type { UnionSubject } from "../surface/read-model";
@@ -70,8 +83,63 @@ export type IndexApplication =
| { readonly outcome: "unchanged"; readonly object: Nuri }
/** Resolved, and deliberately not added — a property of the OBJECT, not a failure. */
| { readonly outcome: "skipped"; readonly object: Nuri; readonly reason: string }
/** Nothing was written, and something is wrong enough to say out loud. */
| { readonly outcome: "refused"; readonly why: string };
/** Nothing was written, and something is wrong enough to say out loud. A DETERMINISTIC
* refusal — a misconfigured index, an inbox that is not a document's. Trying again
* changes nothing, so the drain reports it and moves on. */
| { readonly outcome: "refused"; readonly why: string }
/**
* The read did NOT establish anything — it threw, or it returned nothing from a repo
* whose sync this session never confirmed. Not an absence, and not a refusal: THROWS, so
* the drain stops and everything still queued survives the outage.
*
* Its own outcome since 2026-08-26, and the reason is consumption. While deposits were
* re-read for ever the ambiguity was free: absent or unreadable, the answer was "not
* added this time" and a later drain settled it. Once the deposit is popped before the
* arm runs, the two need opposite handling, and picking either one wholesale is wrong in
* a way that was measured: treating every empty read as a FAILURE let one dead reference
* stall every deposit behind it (a stranger can deposit, so that starves an index);
* treating every empty read as an ABSENCE let one cold start destroy the whole queue.
* {@link noRowsMeansAbsent} tells them apart instead of choosing.
*/
| { readonly outcome: "unreadable"; readonly why: string };
/**
* Did a read that came back with NO ROWS actually establish that the document is empty?
*
* This is the question consumption forced, and it has three sources of "no rows", only one
* of which is an absence:
*
* - the read THREW — `onUnreadable` fired. Transient. Not an absence.
* - the document was NEVER ATTEMPTED, because this holder holds no cap for it and
* `readUnion` drops it through `mustNotAttempt` before issuing a query. This looked
* deterministic and is not: for a stranger's PUBLIC document — the reference arm's
* central case — possession is acquired by a network fetch that swallows every error
* and MEMOISES the failure (`public-store.ts`). One outage on that fetch therefore
* presents as "no cap", and it stays that way for the session, so classing it as an
* absence destroyed the deposit AND every re-deposit after it. Not an absence.
* - the repo is PRESENT BUT UNSYNCED — and this is the one that cost round 6. A fresh
* session over a persistent wallet has not received the repo's commits, so the anchored
* query legitimately matches nothing and **raises no error at all**. Classing that as an
* absence let a cold start destroy an entire reference queue, silently, while reporting
* each one as "the object is absent".
*
* So an absence has to be EARNED, and only one shape earns it: a query was actually issued,
* it did not fail, and the repo's sync is confirmed. `open-repo` keeps `synced` and
* `timed-out` apart for exactly this and says so in those words — `synced` means a 0-row
* read MEANS it. `unknown` is the unit-fake path (no `doc_subscribe`, so no `State`
* semantics) where the read is direct and trustworthy.
*
* The cost of the strict answer is LATENCY, never loss: a reference this holder truly can
* never read stops one drain, is already popped, and the next attempt resumes past it. The
* cost of the lax answer was destroyed deposits that no re-deposit could recover. That
* asymmetry is the whole rule — when in doubt, do not call it an absence.
*/
function noRowsMeansAbsent(doc: Nuri, readFailed: boolean): boolean {
if (readFailed) return false;
if (mustNotAttempt(doc)) return false;
const sync = getSyncState(doc);
return sync === "synced" || sync === "unknown";
}
/**
* The values a subject carries for a predicate.
@@ -150,10 +218,13 @@ async function documentAddressedBy(inbox: Nuri): Promise<Nuri | null> {
* document declares it indexes by, whether the reference is already in, what the
* referenced object says for the field — and only then writes ONE entry.
*
* Never removes anything and never rewrites an entry, so re-applying the same deposit —
* which happens on every drain, because deposits are not consumed — lands on the same
* result. Curation is convergent, and its outcome does not depend on the order references
* arrive in.
* Never removes anything and never rewrites an entry, so the same reference arriving TWICE
* lands on the same result. That is still needed now that deposits are consumed, and the
* reason has simply changed: not a drain replaying one deposit, but two DISTINCT deposits
* naming the same object — a re-share, a retry by an application, two depositors who both
* point at it. Upstream keeps the same check for the same reason, with a queue that pops
* (`ContactAlreadyExists`, `engine/verifier/src/inbox_processor.rs`), which is the proof it
* was never a replay compensation. Curation stays convergent and order-independent.
*/
// @provenance applyIndexDeposit kind=invention level=none ref=none — nothing upstream turns a deposited reference into an index entry; the whole arm is this library's own
export async function applyIndexDeposit(
@@ -173,7 +244,22 @@ export async function applyIndexDeposit(
return { outcome: "skipped", object: reference, reason: "self-reference" };
}
const subjects = await readUnion([index]);
// Told whether the read FAILED, rather than left to guess from an empty answer. The two
// demand opposite handling now that deposits are consumed — see the `unreadable` outcome.
// A BOOLEAN, not the error value: `throw null` and `throw undefined` are legal, and a
// sentinel comparison against `null` made exactly those failures invisible.
let indexReadFailed = false;
let indexError: unknown;
const subjects = await readUnion([index], (_doc, error) => {
indexReadFailed = true;
indexError = error;
});
if (indexReadFailed) {
return {
outcome: "unreadable",
why: `${index}: the index itself could not be read: ${String(indexError)}`,
};
}
const declared = declaredFields(subjects, index);
if (declared.length > 1) {
// Ambiguity refuses to curate and leaves READING alone. Picking one would silently
@@ -193,35 +279,63 @@ export async function applyIndexDeposit(
const field = declared[0];
if (field === undefined) {
return {
outcome: "refused",
why:
`${index} declares no index field (${INDEX_FIELD}) — either it is not an index, or ` +
"it could not be read: an unreadable document and an empty one are the same empty " +
"result here. Nothing was written. Retry before concluding it is malformed.",
outcome: noRowsMeansAbsent(index, indexReadFailed) ? "refused" : "unreadable",
why: noRowsMeansAbsent(index, indexReadFailed)
? `${index} declares no index field (${INDEX_FIELD}) — it read cleanly and says ` +
"nothing, so either it is not an index or it was never configured as one. " +
"Nothing was written, and nothing here will write until it declares one."
: `${index} declares no index field (${INDEX_FIELD}), but its repo is not synced ` +
"in this session, so an empty read proves nothing about it. Nothing was written, " +
"and this is NOT a claim that it is misconfigured.",
};
}
if (indexedObjects(subjects, index).has(reference)) {
// Already there. Nothing is re-read and nothing is rewritten — an entry, once made,
// is never touched again.
// is never touched again. Reached by a SECOND deposit naming the same object, not by a
// replay of the first: that one was consumed when it was applied.
return { outcome: "unchanged", object: reference };
}
let resolved: readonly UnionSubject[];
let referenceReadFailed = false;
let referenceError: unknown;
try {
resolved = await readUnion([reference]);
resolved = await readUnion([reference], (_doc, error) => {
referenceReadFailed = true;
referenceError = error;
});
} catch (error) {
return { outcome: "refused", why: `${reference}: ${String(error)}` };
return { outcome: "unreadable", why: `${reference}: ${String(error)}` };
}
// The BROKER failed. Stop the drain: everything still queued has to survive an outage,
// and this deposit is already gone.
if (referenceReadFailed) {
return {
outcome: "unreadable",
why: `${reference}: could not be read: ${String(referenceError)}`,
};
}
// A clean read that found NOTHING — the object is genuinely absent or holds no triples.
// Deterministic: trying again changes nothing, so this is an unapplicable item and the
// deposits behind it keep their turn. Distinguishing this from the case above is the whole
// point of `onUnreadable`: for one day they were the same answer, and treating both as
// failures let a single dead reference stall every deposit behind it — which, since anyone
// may deposit, is a way to starve someone else's index.
if (resolved.length === 0) {
// ABSENT, unreadable, or the read failed — and this layer does not tell them apart,
// because nothing it does depends on the answer: an index only ever grows, so every
// reading leads to the same act, do not add and say so.
if (!noRowsMeansAbsent(reference, referenceReadFailed)) {
return {
outcome: "unreadable",
why:
`${reference}: the read came back empty, but its repo is not synced in this ` +
"session — that proves nothing about the object, so it is not treated as absent",
};
}
return {
outcome: "refused",
why:
`${reference}: the read came back empty — the object is absent, unreadable, or the ` +
"read failed; they are not distinguished here, and a later deposit adds it",
`${reference}: read cleanly from a synced repo and found nothing — the object is ` +
"absent",
};
}
+76 -41
View File
@@ -39,11 +39,12 @@ import {
traceProcessed,
} from "../emulated-verifier/inbox-processor";
import {
applyDeposit,
capOfPayload,
kindOf,
linkPayload,
} from "../emulated-verifier/inbox-dispatch";
import { deliverDeposits } from "../emulated-verifier/inbox-consume";
import type { StoredDeposit } from "../emulated-verifier/inbox-consume";
import {
accountKey,
inboxOwner,
@@ -142,9 +143,14 @@ function summarizePayload(payload: unknown): string {
* Shared by the two readers of an inbox — its owner ({@link read}) and the emulated
* processor draining it for that owner ({@link processForOwner}) — so the two cannot
* come to disagree about what a deposit looks like.
*
* `?d`, the deposit's own subject, is selected because CONSUMING one names it: `post`
* mints a fresh unique subject per deposit, so it is the key that retires that deposit and
* no other, where a `ts` two deposits could share is not. It never reaches an application
* — `depositsFrom` keeps it beside the {@link Deposit} rather than in it.
*/
const DEPOSITS_QUERY = `
SELECT ?payload ?ts ?from WHERE {
SELECT ?d ?payload ?ts ?from WHERE {
?d a <${P.type}> ;
<${P.payload}> ?payload ;
<${P.ts}> ?ts .
@@ -305,18 +311,30 @@ export async function postToDocument(docLike: NuriLike, opts: PostOptions): Prom
* user's store. Reading the raw deposits a second time would mean re-parsing them, so
* what the read already decoded is what the pass works from.
*/
const seenByInbox = new Map<Nuri, unknown[]>();
function declaredIn(inbox: Nuri): unknown[] {
const seenByInbox = new Map<Nuri, StoredHere[]>();
function declaredIn(inbox: Nuri): StoredHere[] {
return seenByInbox.get(inbox) ?? [];
}
/**
* A deposit as it is STORED — its subject and what an application would see of it. The
* subject is what {@link processInbox} and {@link processForOwner} hand to the consuming
* primitive; it is emulation detail and stays inside this package.
*/
interface StoredHere extends StoredDeposit {
readonly deposit: Deposit;
}
/**
* The deposits a raw SELECT result carries, sorted by `ts` ascending — the one place the
* stored shape is turned back into {@link Deposit}s, for both readers of an inbox.
*
* Each keeps the subject it was read under, because a drain has to be able to RETIRE the
* deposit it just applied and a `ts` does not identify one.
*/
function depositsFrom(result: unknown): Deposit[] {
const deposits: Deposit[] = [];
function depositsFrom(result: unknown): StoredHere[] {
const stored: StoredHere[] = [];
for (const row of readBindings(result)) {
const rawPayload = row.payload?.value ?? "null";
let payload: unknown;
@@ -328,10 +346,14 @@ function depositsFrom(result: unknown): Deposit[] {
const tsRaw = row.ts?.value ?? "0";
const ts = Number.parseInt(tsRaw, 10) || 0;
const fromValue = row.from?.value;
deposits.push({ from: fromValue ? fromValue : null, payload, ts });
stored.push({
subject: row.d?.value ?? "",
payload,
deposit: { from: fromValue ? fromValue : null, payload, ts },
});
}
deposits.sort((a, b) => a.ts - b.ts);
return deposits;
stored.sort((a, b) => a.deposit.ts - b.deposit.ts);
return stored;
}
// What a deposit DECLARES itself to be, and what each declaration means, live in
@@ -539,26 +561,35 @@ export async function read(targetInboxLike: NuriLike): Promise<Deposit[]> {
// registry's change signal, which is what makes a view that was empty for want of a cap
// re-read instead of staying stale.
const delivered: Deposit[] = [];
const declared: unknown[] = [];
const declared: StoredHere[] = [];
// The ownership guard ran at entry; the filing happens several awaits later, and filing
// resolves WHO is holding at that moment. So an application switching identity in the
// gap could have this inbox's caps land in the NEW holder's ring. A hazard read off the
// code, not a leak anyone reproduced — see `caps.holderKey`.
//
// Abandoning is the faithful answer: upstream an inbox is processed by ITS owner's
// verifier, and switching user is another session. Nothing is lost — an inbox is not
// consumed by reading, so the next connection under the right identity files them.
// verifier, and switching user is another session. Nothing is lost, and READING is still
// what leaves it that way: this pass only HOLDS caps in memory, and a deposit is retired
// by the applying pass alone (`emulated-verifier/inbox-consume.ts`), which this one
// never reaches. So an abandoned read leaves the queue exactly as it found it.
const stillOwner = getCurrentUser() === owner;
for (const d of deposits) {
if (kindOf(d.payload) !== null) declared.push(d.payload);
if (kindOf(d.payload) !== null) declared.push(d);
const cap = capOfPayload(d.payload);
if (cap) {
if (stillOwner) getCaps().learnFor(ownerKey, cap);
continue;
}
delivered.push(d);
delivered.push(d.deposit);
}
if (declared.length > 0) seenByInbox.set(targetInbox, declared);
// UNCONDITIONAL since 2026-08-26, and the empty case is the whole point: this used to be
// written only when something was declared, so a read that found nothing left the PREVIOUS
// read's list standing. Harmless while nothing was consumed — the arms are idempotent — but
// once deposits are popped it means a later `processInbox` works from a list of deposits
// that are already gone, re-popping and RE-APPLYING them. That refutes "delivered exactly
// once" outright, and the path is real: `processForOwner` drains through `readForHolder`
// and never touches this map, so the owner's own next pass inherits a stale one.
seenByInbox.set(targetInbox, declared);
// Domain-level diagnostic (on top of docs.ts's generic access-path READ log
// of raw triple-rows): how many DEPOSITS were found, and the decoded data of
// each — the exact visibility needed to trace materialization at the owner
@@ -688,9 +719,11 @@ export async function readSyncedForDocument(docLike: NuriLike): Promise<Deposit[
*
* So the holder the guard authorised is the one the filing is checked against, and a switch
* ABANDONS — the same answer {@link read} gives for its own half of this window, and the
* same one `connect.connectedUser` gives for a drain it started. Nothing is lost: an inbox
* is not consumed by being read, so the deposit is still there for its owner's next
* connection or its next push. Filing it FOR the absent owner instead would be a write on
* same one `connect.connectedUser` gives for a drain it started. What that saves is
* everything NOT yet reached: reading consumes nothing, so those deposits are still there
* for the owner's next connection or next push. The one being applied when the identity
* moved is not among them — it was popped first, and `inbox-consume.deliverDeposits`
* reports it ungated rather than pretending it waits. Filing it FOR the absent owner instead would be a write on
* somebody else's store from a published call — which is the deferred processor's
* deliberate, unpublished privilege (`emulated-verifier/inbox-processor.ts`), not this one's.
*/
@@ -710,24 +743,21 @@ export async function processInbox(targetInboxLike: NuriLike): Promise<Deposit[]
// precisely the mis-filing the capture exists to prevent, and it must not have a value.
if (holder === null) return deposits;
const ring = accountKey(holder);
for (const payload of declared) {
// Re-checked per item, not once: an arm reads and writes, so the identity can move
// between two of them just as easily as during the read.
if (getCurrentUser() !== holder) {
logStage(
"ABANDONED " + shortNuri(targetInbox) + " — the identity changed while it was being " +
"processed; its deposits stay for their owner",
);
return deposits;
}
// The `match`, one arm per declared type — `emulated-verifier/inbox-dispatch.ts`.
await applyDeposit(payload, {
// Consume-and-apply, through the ONE primitive both triggers share
// (`emulated-verifier/inbox-consume.ts`). Each deposit is popped before its arm runs and
// is not seen again — this pass no longer relies on being safe to repeat.
//
// The identity is re-checked per item there as it was here, and for the same reason: an
// arm reads and writes, so the holder can move between two of them just as easily as
// during the read. Abandoning now leaves only the deposits NOT yet reached; the ones
// already popped were applied for the holder that was authorised at the time.
await deliverDeposits(declared, {
owner: holder,
ring,
inbox: targetInbox,
ownerHoldsTheSession: true,
ownerHoldsTheSession: () => true,
stillTheHolder: () => getCurrentUser() === holder,
});
}
return deposits;
}
@@ -800,20 +830,25 @@ async function processForOwner(targetInbox: Nuri): Promise<void> {
const deposits = depositsFrom(
await readForHolder(sid, DEPOSITS_QUERY, targetInbox, ownerRing, "inboxProcess"),
);
let applied = 0;
for (const deposit of deposits) {
// The SAME `match` the owner's own session runs, with the holder named instead of
// resolved — and told whether the owner is the one holding the page, because an arm
// The SAME consume-and-apply the owner's own session runs, with the holder named instead
// of resolved — and told whether the owner is the one holding the page, because an arm
// that would write application data into the owner's documents must not run from a
// stranger's session (`emulated-verifier/inbox-dispatch.ts`).
const done = await applyDeposit(deposit.payload, {
// stranger's session, and what such an arm declines must not be consumed either
// (`emulated-verifier/inbox-dispatch.claimsDeposit`).
//
// ONE primitive, not a copy of it: a trigger decides WHO drains and WHEN, never what
// consuming means. `emulated-verifier/inbox-consume.ts` holds the only removal.
const applied = await deliverDeposits(deposits, {
owner: ownerId,
ring: ownerRing,
inbox: targetInbox,
ownerHoldsTheSession: getCurrentUser() === ownerId,
// Asked PER DEPOSIT, not snapshotted once for the run — the identity can arrive or
// leave between two arms, and this decides both what an arm may do and what may be
// popped. Hoisting it out of the loop (briefly, on 2026-08-26) meant a drain that
// started while the owner happened to be connected went on treating every later
// deposit as his, applying application data under whoever held the page by then.
ownerHoldsTheSession: () => getCurrentUser() === ownerId,
});
if (done) applied += 1;
}
traceProcessed(targetInbox, ownerRing, applied);
}
+19 -2
View File
@@ -125,6 +125,7 @@ async function sessionId(): Promise<string | number> {
async function readDoc(
sid: string | number,
doc: Nuri,
onUnreadable?: (doc: Nuri, error: unknown) => void,
): Promise<Array<Record<string, { value: string } | undefined>>> {
try {
const nuri = assertNuri(doc);
@@ -141,6 +142,16 @@ async function readDoc(
return bindings(res);
} catch (error) {
console.error("[read-model] read failed for", doc, error);
// The swallow STAYS — a batch read has always tolerated one bad document, and a caller
// building a list from optional values must not lose the whole read to one hole. What is
// new (2026-08-26) is that the failure is no longer INDISTINGUISHABLE from an empty
// document: a caller that needs to tell them apart says so by passing `onUnreadable`.
//
// It needed telling apart because of inbox consumption. A deposit is now retired before
// its arm runs, so an arm that reads `[]` and concludes "absent, nothing to do" turns a
// broker hiccup into permanent loss — and one that concludes "must be a failure" lets a
// single unresolvable reference stall every deposit behind it. Only the truth serves.
onUnreadable?.(doc, error);
return [];
}
}
@@ -165,7 +176,10 @@ async function readDoc(
* size — a non-empty wallet no longer matters. Reads run in parallel via `Promise.all`.
*/
// @provenance readUnion kind=aligned level=2 ref=sdk/js/lib-wasm/src/lib.rs:sparql_query — one anchored query per document — upstream's own anchoring (`resolve_target_for_sparql`), composed client-side
export async function readUnion(docsLike: NuriLike[]): Promise<UnionSubject[]> {
export async function readUnion(
docsLike: NuriLike[],
onUnreadable?: (doc: Nuri, error: unknown) => void,
): Promise<UnionSubject[]> {
const sid = await sessionId();
// Drop the empties BEFORE validating, not after: this call has always tolerated a
// list with holes in it — a scope index can carry a blank entry, and a caller
@@ -196,7 +210,10 @@ export async function readUnion(docsLike: NuriLike[]): Promise<UnionSubject[]> {
// One anchored query per doc, in parallel, tolerant (a bad doc yields []).
const perDoc = await Promise.all(
reachable.map(async (d) => ({ doc: assertNuri(d), rows: await readDoc(sid, d) })),
reachable.map(async (d) => ({
doc: assertNuri(d),
rows: await readDoc(sid, d, onUnreadable),
})),
);
// Possession gate, kept as defence in depth behind rule 2 above: `reachable`
@@ -137,6 +137,19 @@ function makeFakeNg(faults: Faults) {
const query = a[1] as string;
const anchor = a[2] as string | undefined;
if (!anchor) return undefined;
// Retiring a DEPOSIT: every triple of one subject, whatever predicates it carries
// (`emulated-verifier/inbox-consume.removeDeposit`). Modelled rather than ignored — a
// fake that let this delete fall through would leave every deposit in place, and a
// suite asserting convergence would pass on a queue that never shrinks.
const delDeposit = query.match(/^\s*DELETE\s+WHERE\s*\{\s*<([^>]+)>\s+\?\w+\s+\?\w+\s*\}/);
if (delDeposit) {
const subject = delDeposit[1]!;
for (let i = quads.length - 1; i >= 0; i--) {
const q = quads[i]!;
if (q.s === subject) quads.splice(i, 1);
}
return undefined;
}
// `INSERT DATA { GRAPH <g> { … } }` — the shape the store-ROOT pointer write uses;
// everything else writes the anchored default graph.
const gm = query.match(/GRAPH\s+<([^>]+)>\s*\{([\s\S]*)\}/);
@@ -204,6 +217,8 @@ function makeFakeNg(faults: Faults) {
for (const q of quads) {
if (q.g !== anchor) continue;
const rec = bySubject.get(q.s) ?? {};
// The deposit's own subject — the key a drain names to retire it.
rec.d = q.s;
if (q.p === `${INBOX}:payload`) rec.payload = q.o;
if (q.p === `${INBOX}:ts`) rec.ts = q.o;
if (q.p === `${INBOX}:from`) rec.from = q.o;
@@ -215,6 +230,7 @@ function makeFakeNg(faults: Faults) {
.filter((r) => r.payload !== undefined && r.ts !== undefined)
.map((r) => {
const row: Record<string, { value: string }> = {
d: { value: r.d! },
payload: { value: r.payload! },
ts: { value: r.ts! },
};
@@ -325,7 +325,8 @@ describe("switching identity", () => {
fake._deliver(inTransit);
await converge();
// An inbox is not consumed by being ignored: connecting again drains what was left.
// An inbox is not consumed by being IGNORED — only by being applied. Alice never
// drained this one, so connecting again finds the deposit still queued.
await signIn("alice");
await converge();
@@ -348,15 +349,17 @@ describe("a deposit that cannot be applied", () => {
await converge();
});
expect(reported.filter((l) => /could not apply what is in this inbox/.test(l)).length)
expect(reported.filter((l) => /could not finish draining this inbox/.test(l)).length)
.toBeGreaterThan(0);
// Prefixed by the connected identity, like every other polyfill-layer line.
expect(reported.find((l) => /could not apply what is in this inbox/.test(l)))
expect(reported.find((l) => /could not finish draining this inbox/.test(l)))
.toContain("[alice][polyfill]");
expect(getCaps().capForHolder("alice", first.doc)).toBeUndefined();
// The broker recovers. The observation is still running — one unapplicable item denies
// nothing — and the deposit that failed was never consumed, so both land now.
// nothing — and this deposit was never consumed, which since 2026-08-26 is a claim worth
// being precise about: what failed here is the READ, before anything could be taken out
// of the queue. A deposit that had been popped and whose ARM then failed would be gone.
fake._failReadsOn.delete(aliceInbox);
fake._deliver(second.inTransit);
await converge();
@@ -103,6 +103,19 @@ function makeFakeNg() {
}
return undefined;
}
// Retiring a DEPOSIT: every triple of one subject, whatever predicates it carries
// (`emulated-verifier/inbox-consume.removeDeposit`). Modelled rather than ignored — a
// fake that let this delete fall through would leave every deposit in place, and a
// suite asserting convergence would pass on a queue that never shrinks.
const delDeposit = query.match(/^\s*DELETE\s+WHERE\s*\{\s*<([^>]+)>\s+\?\w+\s+\?\w+\s*\}/);
if (delDeposit) {
const subject = delDeposit[1]!;
for (let i = quads.length - 1; i >= 0; i--) {
const q = quads[i]!;
if (q.s === subject) quads.splice(i, 1);
}
return undefined;
}
// `INSERT DATA { GRAPH <g> { … } }` — the shape the store-ROOT pointer write uses.
// Without this arm the first `<…>` in the body is the GRAPH NAME, so the pointer was
// stored with the graph as its subject and its predicate as its object. The pointer
@@ -163,6 +176,8 @@ function makeFakeNg() {
for (const q of quads) {
if (q.g !== anchor) continue;
const rec = bySubject.get(q.s) ?? {};
// The deposit's own subject — the key a drain names to retire it.
rec.d = q.s;
if (q.p === `${INBOX}:payload`) rec.payload = q.o;
if (q.p === `${INBOX}:ts`) rec.ts = q.o;
if (q.p === `${INBOX}:from`) rec.from = q.o;
@@ -173,7 +188,7 @@ function makeFakeNg() {
bindings: [...bySubject.values()]
.filter((r) => r.payload !== undefined && r.ts !== undefined)
.map((r) => {
const row: Record<string, { value: string }> = { payload: { value: r.payload! }, ts: { value: r.ts! } };
const row: Record<string, { value: string }> = { d: { value: r.d! }, payload: { value: r.payload! }, ts: { value: r.ts! } };
if (r.from !== undefined) row.from = { value: r.from };
return row;
}),
@@ -97,6 +97,19 @@ function makeFakeNg(faults: Faults) {
}
return undefined;
}
// Retiring a DEPOSIT: every triple of one subject, whatever predicates it carries
// (`emulated-verifier/inbox-consume.removeDeposit`). Modelled rather than ignored — a
// fake that let this delete fall through would leave every deposit in place, and a
// suite asserting convergence would pass on a queue that never shrinks.
const delDeposit = query.match(/^\s*DELETE\s+WHERE\s*\{\s*<([^>]+)>\s+\?\w+\s+\?\w+\s*\}/);
if (delDeposit) {
const subject = delDeposit[1]!;
for (let i = quads.length - 1; i >= 0; i--) {
const q = quads[i]!;
if (q.s === subject) quads.splice(i, 1);
}
return undefined;
}
const gm = query.match(/GRAPH\s+<([^>]+)>\s*\{([\s\S]*)\}/);
const body = gm ? gm[2]! : query.replace(/^\s*INSERT DATA\s*\{/, "").replace(/\}\s*$/, "");
const sm = body.match(/<([^>]+)>/);
@@ -160,6 +173,8 @@ function makeFakeNg(faults: Faults) {
for (const q of quads) {
if (q.g !== anchor) continue;
const rec = bySubject.get(q.s) ?? {};
// The deposit's own subject — the key a drain names to retire it.
rec.d = q.s;
if (q.p === `${INBOX}:payload`) rec.payload = q.o;
if (q.p === `${INBOX}:ts`) rec.ts = q.o;
if (q.p === `${INBOX}:from`) rec.from = q.o;
@@ -171,6 +186,7 @@ function makeFakeNg(faults: Faults) {
.filter((r) => r.payload !== undefined && r.ts !== undefined)
.map((r) => {
const row: Record<string, { value: string }> = {
d: { value: r.d! },
payload: { value: r.payload! },
ts: { value: r.ts! },
};
@@ -338,14 +354,15 @@ test("what the drain filed for the owner is DURABLE: he restores it with his inb
await share(aliceDoc, "bob");
await runScheduledInboxProcessingNow();
// EMPTY the inbox, as a consumed queue would be, and drop every in-memory cap — so what
// Bob recovers below can only have come from his own User branch, where the drain filed
// it. Without that, he would simply be draining the queue himself on connection, which
// is the behaviour this service exists to get ahead of.
// The inbox is ALREADY empty — the drain consumed the deposit as it applied it. Until
// 2026-08-26 this test had to empty the queue by hand to reach that state, which is
// exactly the shortcut that fabricates a state the system never produced; now it is
// asserted instead. It matters here because what Bob recovers below must come from his
// own User branch, where the drain filed it: with the deposit still in the queue he
// would simply be draining it himself on connection, which is the behaviour this
// service exists to get ahead of.
const bobInbox = await userInbox("bob", "protected");
for (let k = fake.quads.length - 1; k >= 0; k--) {
if (fake.quads[k]!.g === bobInbox) fake.quads.splice(k, 1);
}
expect(fake.quads.filter((q) => q.g === bobInbox && q.p === `${INBOX}:payload`)).toEqual([]);
resetCaps();
setCurrentUser("alice");
await createEntityDoc("alice", "private"); // re-arms the emulation: a cap exists again
@@ -436,8 +453,10 @@ test("a drain that fails says so in the package's log, and rejects into nobody",
// The wording is the SHARED reporter's since 2026-08-17 (`emulated-verifier/inbox-drain.ts`):
// the timer is no longer the only thing that drains an inbox — the continuous observation
// does too, through the same queue — so the line names the act and not the schedule.
const reported = errors.filter((line) => /could not apply what is in this inbox/.test(line));
// does too, through the same queue — so the line names the act and not the schedule. Since
// 2026-08-26 it also stops promising a retry it cannot make: a deposit already taken is
// consumed, and only what is STILL QUEUED waits for the next attempt.
const reported = errors.filter((line) => /could not finish draining this inbox/.test(line));
expect(reported.length).toBe(1);
// Prefixed by the CONNECTED identity, like every other polyfill-layer line — which is
// what makes a drain running under someone else's session legible in a live trace.
@@ -0,0 +1,461 @@
/**
* An inbox is a QUEUE, and this suite is about the act that had been missing from it: a
* deposit is delivered once and then it is gone.
*
* ── What each test would have looked like before ──────────────────────────
* Until 2026-08-26 this package never consumed anything. Every drain re-read everything
* ever deposited, which produced three symptoms at once: an inbox that grew without bound,
* a malformed deposit re-reported at every push and every connection for as long as the
* page lived, and an idempotence burden on every arm that existed only to survive the
* replay. Each test below fails against that code — the deposit is still there — and fails
* again if the removal is taken back out.
*
* ── Upstream is the specification, and so is its ORDER ────────────────────
* The broker POPS: `take_first_msg` → `MSGS.take_first_value` is `get_first_key_value`
* followed by `del`, so the message leaves the store BEFORE it is delivered; the verifier
* then applies it and a failure is logged and swallowed, never re-enqueued
* (`engine/verifier/src/verifier.rs`). Remove, apply, log, move on. So a deposit whose arm
* refused is gone exactly like one that succeeded, and that is the point of the second test
* below rather than an accident of it.
*
* ── Nobody is handed an inbox address ─────────────────────────────────────
* A depositor names a document and a person (`inbox.share`, `inbox.postToDocument`); the
* address is resolved inside. The addresses the TEST resolves are read off the wallet and
* used only to count what is left in a queue — never given to an actor.
*/
import { test, expect, describe, afterAll, beforeEach } from "bun:test";
import { inbox as inboxSurface, storeRegistry } from "../src/index";
import { resolveAccount, userInbox } from "../src/shared-wallet/account-registry";
import { setCurrentUser } from "../src/shared-wallet/bootstrap";
import { openDocumentInbox } from "../src/emulated-verifier/branch-registers";
import {
cancelScheduledInboxProcessing,
runScheduledInboxProcessingNow,
} from "../src/emulated-verifier/inbox-processor";
import { bootPage, forgetEverything, signIn, type FakeWallet, type Quad } from "./wallet-fake";
import type { Nuri } from "../src/model/types";
const INBOX = "urn:ng-eventually:inbox";
/** A kind in this package's OWN namespace that no arm answers — upstream's `NotImplemented`. */
const UNARMED_KIND = `${INBOX}:nonesuch`;
let quads: Quad[];
let fake: FakeWallet;
beforeEach(() => {
forgetEverything();
quads = [];
fake = bootPage(quads);
});
afterAll(() => {
cancelScheduledInboxProcessing();
forgetEverything();
});
/** What is STILL QUEUED in `inbox` — one entry per deposit that has not been consumed. */
function stillQueuedIn(inbox: Nuri): string[] {
return quads.filter((q) => q.g === inbox && q.p === `${INBOX}:payload`).map((q) => q.o);
}
/** How many deposits have been RETIRED — one `DELETE WHERE { <deposit> ?p ?o }` each. */
function removalCount(): number {
return fake.sparql_update.mock.calls.filter((c) =>
/DELETE\s+WHERE\s*\{\s*<[^>]+>\s+\?\w+\s+\?\w+\s*\}/.test(String(c[1]))
).length;
}
/** Capture what `console.error` is told while `body` runs — where a refusal goes. */
async function reportedDuring(body: () => Promise<void>): Promise<string[]> {
const lines: string[] = [];
const real = console.error;
console.error = ((...args: unknown[]) => {
lines.push(args.map((a) => String(a)).join(" "));
}) as typeof console.error;
try {
await body();
} finally {
console.error = real;
}
return lines;
}
/**
* Alice has been here once, so she is someone a deposit can NAME (`inbox.share` refuses a
* recipient nobody has ever been, and connecting does not provision — an identity acquires
* its account the first time it creates anything).
*/
async function aliceHasBeenHere(): Promise<void> {
await signIn("alice");
await storeRegistry.createEntityDoc("protected");
}
/** Bob shares a document of his with Alice, by name. Answers the document. */
async function bobSharesWithAlice(): Promise<Nuri> {
await signIn("bob");
const doc = await storeRegistry.createEntityDoc("protected");
await inboxSurface.share(doc, "alice");
return doc;
}
/** The Links durably filed on a user's User branch — the emulated `AddLink` records. */
async function documentsGivenTo(id: string): Promise<string[]> {
const store = (await resolveAccount(id))?.docPrivate;
if (!store) return [];
return quads
.filter((q) => q.g === store && q.p === "urn:ng-eventually:shim:link")
.map((q) => q.o.split(":r:")[0]!);
}
// --- a deposit that was applied is GONE -----------------------------------
test("a Link that has been applied is no longer in the queue", async () => {
await aliceHasBeenHere();
const doc = await bobSharesWithAlice();
const aliceInbox = await userInbox("alice", "protected");
// Bob's deposit is waiting, and nothing of it has reached Alice yet.
expect(stillQueuedIn(aliceInbox).length).toBe(1);
expect(await documentsGivenTo("alice")).toEqual([]);
// Alice connects: her own session drains her own inbox.
await signIn("alice");
// Applied — and CONSUMED. The second assertion is the one that was missing: before
// 2026-08-26 the Link was filed and the deposit stayed, so this queue only ever grew.
expect(await documentsGivenTo("alice")).toEqual([doc]);
expect(stillQueuedIn(aliceInbox)).toEqual([]);
});
// --- a deposit whose ARM FAILED is gone too, and said so once --------------
describe("a deposit no arm can apply", () => {
/**
* Alice opens a document with an inbox on it; Bob deposits something that declares
* itself to this package and that no arm answers. He names her DOCUMENT — the address is
* resolved inside, and he never learns it.
*/
async function bobDepositsSomethingUnarmed(): Promise<Nuri> {
await signIn("alice");
// PUBLIC, so Bob can read it — the address rides on the document, so a depositor who
// cannot read it cannot reach its owner. That is the model, not a fixture detail.
const note = await storeRegistry.createEntityDoc("public");
await openDocumentInbox(note);
await signIn("bob");
await inboxSurface.postToDocument(note, { payload: { kind: UNARMED_KIND } });
return note;
}
test("is reported AND consumed — upstream logs the failure and the message is already gone", async () => {
const note = await bobDepositsSomethingUnarmed();
const noteInbox = await (async () => {
const rec = quads.find(
(q) => q.p === "urn:ng-eventually:shim:inboxCap" && q.o.startsWith(note + " "),
);
if (!rec) throw new Error("no AddInboxCap record was written for the note");
return rec.o.split(" ")[1] as Nuri;
})();
expect(stillQueuedIn(noteInbox).length).toBe(1);
const reported = await reportedDuring(async () => {
await signIn("alice");
});
// It said so — a message that could not be applied is never silent.
expect(reported.some((l) => /no arm applies deposits of kind/.test(l))).toBe(true);
// …and it is GONE. This is the whole change: upstream the broker had already removed
// the message before the arm ran, so `log_err!` is the END of it, not a promise to
// try again. The old wording ("it stays in it and the next attempt will try again")
// described a deposit that would be re-reported forever.
expect(stillQueuedIn(noteInbox)).toEqual([]);
expect(reported.some((l) => /will not be retried/.test(l))).toBe(true);
});
test("is not reported a SECOND time, because a second drain finds an empty queue", async () => {
const note = await bobDepositsSomethingUnarmed();
await signIn("alice");
// Alice connects again — a reload, a second tab, any of the things that drain. Before
// consumption this reported the same unapplicable deposit again, and would have gone
// on doing so at every connection for as long as it sat there.
const reportedAgain = await reportedDuring(async () => {
await signIn("alice");
});
expect(reportedAgain.filter((l) => /no arm applies deposits of kind/.test(l))).toEqual([]);
void note;
});
});
// --- a second drain of a drained inbox does nothing ------------------------
test("draining an inbox twice applies once and reports nothing the second time", async () => {
await aliceHasBeenHere();
const doc = await bobSharesWithAlice();
const aliceInbox = await userInbox("alice", "protected");
await signIn("alice");
expect(await documentsGivenTo("alice")).toEqual([doc]);
const reported = await reportedDuring(async () => {
await signIn("alice");
});
// Nothing to do, nothing said, and nothing filed twice. Note WHY that last one is not
// just `addLink`'s de-duplication doing the work: the queue is empty, so no arm ran.
expect(reported).toEqual([]);
expect(stillQueuedIn(aliceInbox)).toEqual([]);
expect(await documentsGivenTo("alice")).toEqual([doc]);
});
// --- ONE primitive, exercised through BOTH triggers -----------------------
/**
* The two triggers must consume through the SAME act, so this asserts on that act rather
* than on each trigger — a fault injected at the ONE removal, and the same consequence
* demanded of both paths.
*
* The consequence is `inbox-consume`'s own contract: a removal that fails STOPS the apply,
* because applying a deposit that stayed in the queue is the replay this exists to end. A
* trigger carrying its own copy of the removal would not be bound by it.
*/
describe("both triggers consume through the one primitive", () => {
/** Make retiring a deposit fail — the single `DELETE WHERE { <deposit> ?p ?o }`. */
function breakTheRemoval(): void {
const real = fake.sparql_update;
fake.sparql_update = (async (...a: unknown[]) => {
if (/DELETE\s+WHERE\s*\{\s*<[^>]+>\s+\?\w+\s+\?\w+\s*\}/.test(String(a[1]))) {
throw new Error("the broker refused to retire this deposit");
}
return real(...a);
}) as typeof fake.sparql_update;
}
test("the OWNER's own drain applies nothing when the deposit cannot be retired", async () => {
await aliceHasBeenHere();
await bobSharesWithAlice();
const aliceInbox = await userInbox("alice", "protected");
breakTheRemoval();
await reportedDuring(async () => {
await signIn("alice");
});
// Nothing applied, and the deposit is still queued — so the next drain will try it
// again, which is the only safe answer when the pop did not happen.
expect(await documentsGivenTo("alice")).toEqual([]);
expect(stillQueuedIn(aliceInbox).length).toBe(1);
});
test("the DEFERRED stand-in applies nothing either, by the same rule", async () => {
await aliceHasBeenHere();
// Alice is NOT connected: Bob's session stands in for her absent one.
await bobSharesWithAlice();
const aliceInbox = await userInbox("alice", "protected");
breakTheRemoval();
await reportedDuring(async () => {
await runScheduledInboxProcessingNow();
});
expect(await documentsGivenTo("alice")).toEqual([]);
expect(stillQueuedIn(aliceInbox).length).toBe(1);
});
test("and with the removal working, the DEFERRED stand-in consumes exactly as the owner does", async () => {
await aliceHasBeenHere();
const doc = await bobSharesWithAlice();
const aliceInbox = await userInbox("alice", "protected");
await runScheduledInboxProcessingNow();
// Same two facts the owner's own drain produces above — one act, two triggers.
expect(await documentsGivenTo("alice")).toEqual([doc]);
expect(stillQueuedIn(aliceInbox)).toEqual([]);
});
});
// --- a pop whose effect did not land must SAY so ---------------------------
test("an I/O failure while filing a Link STOPS the drain, so the rest of the queue survives", async () => {
await aliceHasBeenHere();
// Three deposits waiting for an absent Alice, so "what happened to the OTHERS" is a
// question this can answer at all.
await bobSharesWithAlice();
await bobSharesWithAlice();
await bobSharesWithAlice();
const aliceInbox = await userInbox("alice", "protected");
expect(stillQueuedIn(aliceInbox).length).toBe(3);
// The `AddLink` write fails — a broker that will not take the commit.
const real = fake.sparql_update;
fake.sparql_update = (async (...a: unknown[]) => {
if (/urn:ng-eventually:shim:link/.test(String(a[1]))) {
throw new Error("the broker refused the AddLink commit");
}
return real(...a);
}) as typeof fake.sparql_update;
const reported = await reportedDuring(async () => {
await runScheduledInboxProcessingNow();
});
// Reported ungated, at the granularity of the INBOX — a genuine I/O failure is not an
// unapplicable item, and the drain stops rather than working through the queue.
expect(reported.some((l) => /could not finish draining this inbox/.test(l))).toBe(true);
expect(await documentsGivenTo("alice")).toEqual([]);
// The load-bearing assertion. One deposit was popped before the write failed and is lost;
// the other TWO are untouched and wait for the next attempt. Between 2026-08-26's first
// and second corrections this read 0: `applyLink` caught the throw and returned `false`,
// so the loop went on popping — and most of what `addLink` does before its write is
// deposit-independent infrastructure that fails for the whole queue at once. Catching it
// per deposit therefore destroyed all three while dutifully reporting each one.
expect(stillQueuedIn(aliceInbox).length).toBe(2);
// And there is no WEDGE. Stopping the drain could have traded destruction for permanent
// blockage — a deposit that always throws would sit at the head of the queue for ever and
// no later one would be reached. It cannot, and the ORDER is why: the pop happens BEFORE
// the arm, so the deposit that failed is already gone and the next attempt starts at the
// one behind it. A failing broker costs one deposit per attempt, not the queue.
//
// The attempts are ALICE CONNECTING, not the timer: a deferred window is armed by a
// deposit and is not re-armed by a drain that failed, so the owner's own next connection
// is the recovery path — which is what the emulated stand-in has always documented.
await signIn("alice");
expect(stillQueuedIn(aliceInbox).length).toBe(1);
await signIn("alice");
expect(stillQueuedIn(aliceInbox).length).toBe(0);
});
// --- delivered ONCE means once, across both triggers ----------------------
test("a deposit the deferred drain consumed is not re-applied by the owner's own next pass", async () => {
await aliceHasBeenHere();
await bobSharesWithAlice();
const aliceInbox = await userInbox("alice", "protected");
// Alice READS her inbox without processing it — which is what `inbox.watch` does on every
// push, and it is the only step that records what this session decoded. She is set as the
// holder WITHOUT connecting, deliberately: connecting drains, and a drain would consume
// the deposit before the read could see it.
setCurrentUser("alice");
expect((await inboxSurface.read(aliceInbox)).length + stillQueuedIn(aliceInbox).length)
.toBeGreaterThan(0);
setCurrentUser("bob");
// The deferred stand-in then drains the same inbox through its own path, which never
// touches that record. Before 2026-08-26 the record was only ever WRITTEN when something
// was declared, so it survived the queue it described: Alice's own next pass worked from
// a list of deposits that were already gone, re-popping and re-applying every one.
await runScheduledInboxProcessingNow();
const removalsBefore = removalCount();
await signIn("alice");
expect(removalCount()).toBe(removalsBefore);
expect(stillQueuedIn(aliceInbox)).toEqual([]);
});
// --- the gap between deciding to deliver and delivering --------------------
test("an identity that moves DURING the pop does not get the deposit applied for it", async () => {
await aliceHasBeenHere();
// Alice opens an index and Bob deposits a bare reference into it. That arm is the one
// that reads AMBIENT state, so it is the one an identity switch can mis-file.
await signIn("alice");
const index = await storeRegistry.createEntityDoc("public");
await openDocumentInbox(index);
const object = await storeRegistry.createEntityDoc("public");
await signIn("bob");
await inboxSurface.postToDocument(index, { payload: object });
const indexInbox = (() => {
const rec = quads.find(
(q) => q.p === "urn:ng-eventually:shim:inboxCap" && q.o.startsWith(index + " "),
);
if (!rec) throw new Error("no AddInboxCap record was written for the index");
return rec.o.split(" ")[1] as Nuri;
})();
// Alice signs in, so the reference is CLAIMED — her own session may apply it. Then the
// identity moves in the one place nothing used to look: inside the await that removes the
// deposit. `claimsDeposit` had already said yes, and the arm below reads whoever holds the
// page now, so before the re-check this filed Alice's deposit against Bob's registers.
const real = fake.sparql_update;
fake.sparql_update = (async (...a: unknown[]) => {
const out = await real(...a);
if (/DELETE\s+WHERE\s*\{\s*<[^>]+>\s+\?\w+\s+\?\w+\s*\}/.test(String(a[1]))) {
setCurrentUser("bob");
}
return out;
}) as typeof fake.sparql_update;
const reported = await reportedDuring(async () => {
await signIn("alice");
});
// The deposit is gone — it was popped before the switch was visible — and that is said
// out loud rather than papered over. What must NOT have happened is the arm running for
// the wrong holder.
expect(stillQueuedIn(indexInbox)).toEqual([]);
expect(reported.some((l) => /consumed but not applied/.test(l))).toBe(true);
});
test("an arm that fails WHILE the page switches user is still reported, not swallowed", async () => {
await aliceHasBeenHere();
await bobSharesWithAlice();
const aliceInbox = await userInbox("alice", "protected");
// Both at once: the `AddLink` write fails AND the identity moves. `connect.connectedUser`
// drops an error from a drain whose holder has gone — an ownership refusal there is
// abandoning, not a failure — and that guard cannot tell the two apart. So for one day
// this exact pair destroyed a deposit and printed nothing at all.
const real = fake.sparql_update;
fake.sparql_update = (async (...a: unknown[]) => {
if (/urn:ng-eventually:shim:link/.test(String(a[1]))) {
setCurrentUser("bob");
throw new Error("the broker refused the AddLink commit");
}
return real(...a);
}) as typeof fake.sparql_update;
const reported = await reportedDuring(async () => {
await signIn("alice");
});
// The deposit is gone, and SOMETHING said so. Reported by the only code that knows a
// deposit was consumed, before the error is handed to anyone who might drop it.
expect(stillQueuedIn(aliceInbox)).toEqual([]);
expect(reported.some((l) => /was consumed and could not be applied/.test(l))).toBe(true);
});
test("a deposit that cannot be retired is refused, not applied again at every drain", async () => {
await aliceHasBeenHere();
const doc = await bobSharesWithAlice();
const aliceInbox = await userInbox("alice", "protected");
// A broker that answers the deposits query WITHOUT binding the deposit's subject. `?d` is
// bound by the query so this should not happen, but `depositsFrom` defaults it to "" and
// an empty subject would issue `DELETE WHERE { <> ?p ?o }`. The load-bearing half is that
// it must not be APPLIED either: a deposit applied but never retired is the replay this
// whole change exists to end, repeated at every drain for as long as the row sits there.
const realQuery = fake.sparql_query;
fake.sparql_query = (async (...a: unknown[]) => {
const res = await realQuery(...a);
if (a[3] === aliceInbox && String(a[1]).includes(`${INBOX}:payload`)) {
const r = res as { results?: { bindings?: Array<Record<string, unknown>> } };
for (const row of r.results?.bindings ?? []) delete row.d;
}
return res;
}) as typeof fake.sparql_query;
const reported = await reportedDuring(async () => {
await signIn("alice");
});
expect(reported.some((l) => /carries no subject/.test(l))).toBe(true);
expect(await documentsGivenTo("alice")).toEqual([]);
expect(stillQueuedIn(aliceInbox).length).toBe(1);
void doc;
});
+196 -13
View File
@@ -37,7 +37,7 @@ import {
runScheduledInboxProcessingNow,
} from "../src/emulated-verifier/inbox-processor";
import { stopObservingInboxes } from "../src/emulated-verifier/inbox-observer";
import { resetOpenedRepos } from "../src/emulated-verifier/open-repo";
import { resetOpenedRepos, setOpenTimeoutForTests } from "../src/emulated-verifier/open-repo";
import { resetPublicStoreFetches } from "../src/emulated-verifier/public-store";
import { ENTRY_VALUE, INDEX_FIELD } from "../src/emulated-verifier/index-deposit";
import { postToDocument } from "../src/surface/inbox";
@@ -84,6 +84,19 @@ function makeFakeNg() {
}
return undefined;
}
// Retiring a DEPOSIT: every triple of one subject, whatever predicates it carries
// (`emulated-verifier/inbox-consume.removeDeposit`). Modelled here rather than ignored
// — a fake that dropped this delete would leave every deposit in place and let this
// whole suite pass on a queue that never shrinks.
const delAll = query.match(/^\s*DELETE\s+WHERE\s*\{\s*<([^>]+)>\s+\?\w+\s+\?\w+\s*\}/);
if (delAll) {
const s0 = delAll[1]!;
for (let i = quads.length - 1; i >= 0; i--) {
const q = quads[i]!;
if (q.g === anchor && q.s === s0) quads.splice(i, 1);
}
return undefined;
}
const body = query.replace(/^\s*INSERT DATA\s*\{/, "").replace(/\}\s*$/, "");
const sm = body.match(/<([^>]+)>/);
if (!sm) return undefined;
@@ -140,6 +153,8 @@ function makeFakeNg() {
for (const q of quads) {
if (q.g !== anchor) continue;
const rec = bySubject.get(q.s) ?? {};
// The deposit's own subject — the key a drain names to retire it.
rec.d = q.s;
if (q.p === `${INBOX}:payload`) rec.payload = q.o;
if (q.p === `${INBOX}:ts`) rec.ts = q.o;
if (q.p === `${INBOX}:from`) rec.from = q.o;
@@ -151,6 +166,7 @@ function makeFakeNg() {
.filter((r) => r.payload !== undefined && r.ts !== undefined)
.map((r) => {
const row: Record<string, { value: string }> = {
d: { value: r.d! },
payload: { value: r.payload! },
ts: { value: r.ts! },
};
@@ -200,7 +216,17 @@ function makeFakeNg() {
};
});
return { doc_create, sparql_update, sparql_query, quads };
// `doc_subscribe` is declared and left undefined: absent, `ensureRepoOpen` is the
// documented no-op of the unit-fake path. One test installs a subscription that never
// pushes `State`, to reach the `timed-out` repo state a cold start really produces.
const fakeNg: {
doc_create: typeof doc_create;
sparql_update: typeof sparql_update;
sparql_query: typeof sparql_query;
quads: Quad[];
doc_subscribe?: (...a: unknown[]) => () => void;
} = { doc_create, sparql_update, sparql_query, quads };
return fakeNg;
}
let fake: ReturnType<typeof makeFakeNg>;
@@ -314,18 +340,29 @@ test("a reference nobody can resolve is reported, and costs the index nothing",
await bobSignsInOnce();
const { index, object } = await aliceOpensAnIndex([WHEN]);
// One resolvable reference and one that names nothing. The second must not take the
// first down with it, and must not leave the index short of an entry either.
await bobDepositsInto(index, object, 1);
await bobDepositsInto(index, "did:ng:o:nowhere", 2);
// A reference this holder cannot even attempt — no cap, so `readUnion` drops it before
// issuing a query — and a resolvable one BEHIND it, which is what makes the question
// answerable at all.
await bobDepositsInto(index, "did:ng:o:nowhere", 1);
await bobDepositsInto(index, object, 2);
const reported = await reportedDuring(aliceConnects);
// NOT called an absence, and this is the correction that cost the most to find. "No cap"
// reads as deterministic and is not: for a stranger's public document possession comes
// from a network fetch that swallows its errors and memoises the failure, so one outage
// presents exactly like a document that never existed. Classing it absent destroyed the
// deposit and every re-deposit after it. It stops the drain instead.
expect(reported.some((l) => /did:ng:o:nowhere/.test(l))).toBe(true);
expect(reported.some((l) => /the read came back empty/.test(l))).toBe(true);
// Costs nothing: the resolvable one is in, and nothing was written for the other.
expect(await entryValuesFor(index, object)).toEqual(["2026-01-01"]);
expect(reported.some((l) => /was consumed and could not be applied/.test(l))).toBe(true);
expect(await entryValuesFor(index, "did:ng:o:nowhere")).toEqual([]);
// The price is LATENCY, not loss: the unreadable one was popped, so the next connection
// resumes past it and the resolvable one lands. Nothing wedges, and nothing is destroyed
// that a working broker would have delivered.
expect(await entryValuesFor(index, object)).toEqual([]);
await aliceConnects();
expect(await entryValuesFor(index, object)).toEqual(["2026-01-01"]);
});
test("an object already in the index is passed over — the same reference twice makes ONE entry", async () => {
@@ -333,10 +370,11 @@ test("an object already in the index is passed over — the same reference twice
await bobSignsInOnce();
const { index, object } = await aliceOpensAnIndex([WHEN]);
// Deposits are never consumed, so a curating pass sees every deposit on every run. That
// is affordable only because a reference already in the index is passed over: without
// that check the second deposit writes a second value for the same object, and an entry
// carrying two values is exactly what once made the index SHRINK through additions.
// TWO deposits, each consumed when it is applied — not one deposit replayed. The check
// that a reference already in the index is passed over is what makes this land on ONE
// entry: without it the second deposit writes a second value for the same object, and an
// entry carrying two values is exactly what once made the index SHRINK through additions.
// That is why the check outlived consumption: it never answered the replay.
await bobDepositsInto(index, object, 1);
await bobDepositsInto(index, object, 2);
@@ -447,3 +485,148 @@ test("a consumer payload is not a variant: no arm claims it and nothing is repor
expect(reported.filter((l) => /could not apply a deposit/.test(l))).toEqual([]);
});
// --- a consumed deposit is never discarded in silence ------------------------
test("a reference to an object carrying nothing for the field is REPORTED, not silently dropped", async () => {
inject();
await bobSignsInOnce();
const { index } = await aliceOpensAnIndex([WHEN]);
// A draft: it resolves, and it says nothing for the field the index declares. While
// deposits were re-read for ever this was a benign WAIT — the object might gain the field
// and a later drain would index it — so it was traced behind `debugAccessLog` and nothing
// more. Consumption ends the wait: this deposit is gone once it has been looked at, so the
// discard has to be audible without any flag turned on.
setCurrentUser("alice");
const draft = await createEntityDoc("alice", "public");
// It has to SAY something, or the read comes back empty and this is the `refused` path
// instead — a different outcome with a different report. What is under test is the
// object that resolves fine and simply has no value for the indexed field.
await write(draft, "urn:test:title", "a draft with no date yet");
setCurrentUser(null);
await bobDepositsInto(index, draft, 1);
const reported = await reportedDuring(aliceConnects);
expect(reported.some((l) => /was discarded/.test(l) && new RegExp(`carries no ${WHEN}`).test(l)))
.toBe(true);
expect(await entryValuesFor(index, draft)).toEqual([]);
});
// --- a broker that goes quiet must not walk the whole queue -----------------
test("a broker that cannot read the referenced objects costs ONE deposit, not the queue", async () => {
inject();
await bobSignsInOnce();
const { index } = await aliceOpensAnIndex([WHEN]);
// Three references waiting in the index's inbox.
setCurrentUser("alice");
const objects: Nuri[] = [];
for (let i = 0; i < 3; i++) {
const o = await createEntityDoc("alice", "public");
await write(o, WHEN, `2026-01-0${i + 1}`);
objects.push(o);
}
setCurrentUser(null);
for (let i = 0; i < 3; i++) await bobDepositsInto(index, objects[i]!, i + 1);
const queued = () =>
fake.quads.filter((q) => q.p === `${INBOX}:payload` && q.o.includes("did:ng:o:")).length;
expect(queued()).toBe(3);
// The broker goes quiet on the referenced objects. `read-model.readDoc` SWALLOWS a failed
// read and answers `[]` — so the arm is TOLD, through the `onUnreadable` reporter, rather
// than left to guess from an empty answer. Until 2026-08-26 this was classed as an
// ordinary refusal: the drain reported it and carried on, popping and destroying all three
// while printing "will not be retried" for each.
const realQuery = fake.sparql_query;
fake.sparql_query = (async (...a: unknown[]) => {
if (objects.includes(a[3] as Nuri)) throw new Error("the broker went quiet");
return realQuery(...a);
}) as typeof fake.sparql_query;
const reported = await reportedDuring(aliceConnects);
expect(reported.some((l) => /was consumed and could not be applied/.test(l))).toBe(true);
expect(queued()).toBe(2);
});
test("an EMPTY but readable object is absent — reported, and the queue behind it carries on", async () => {
inject();
await bobSignsInOnce();
const { index, object } = await aliceOpensAnIndex([WHEN]);
// A document Alice really holds and that really reads — it just has no triples. This is
// the genuine clean-empty read, and until 2026-08-26 NOTHING exercised it: the only test
// of this branch pointed at `did:ng:o:nowhere`, which `readUnion`'s possession gate drops
// before any query is issued. So the branch that decides "absent vs unreadable" on a real
// read was uncovered, which is how two opposite misclassifications shipped through it.
setCurrentUser("alice");
const empty = await createEntityDoc("alice", "public");
setCurrentUser(null);
// The empty one FIRST, so "does it stall what is behind it" is a question with an answer.
await bobDepositsInto(index, empty, 1);
await bobDepositsInto(index, object, 2);
const reported = await reportedDuring(aliceConnects);
expect(reported.some((l) => /read cleanly from a synced repo and found nothing/.test(l)))
.toBe(true);
expect(await entryValuesFor(index, empty)).toEqual([]);
expect(await entryValuesFor(index, object)).toEqual(["2026-01-01"]);
});
test("a rejection valued `null` is still a failure, not an absence", async () => {
inject();
await bobSignsInOnce();
const { index, object } = await aliceOpensAnIndex([WHEN]);
await bobDepositsInto(index, object, 1);
// `throw null` is legal, and the first version of this classification tracked the error
// VALUE against a `null` sentinel — so a broker that rejected with `null` was recorded as
// "no failure happened" and its deposit was destroyed as an absence. Tracked by a boolean
// now, which is the only thing that cannot collide with a payload.
const realQuery = fake.sparql_query;
fake.sparql_query = (async (...a: unknown[]) => {
if (a[3] === object) throw null;
return realQuery(...a);
}) as typeof fake.sparql_query;
const reported = await reportedDuring(aliceConnects);
expect(reported.some((l) => /was consumed and could not be applied/.test(l))).toBe(true);
expect(reported.some((l) => /came back clean and empty/.test(l))).toBe(false);
expect(await entryValuesFor(index, object)).toEqual([]);
});
test("an empty read whose repo is NOT SYNCED is not an absence — it stops the drain", async () => {
inject();
await bobSignsInOnce();
const { index, object } = await aliceOpensAnIndex([WHEN]);
setCurrentUser("alice");
const unsynced = await createEntityDoc("alice", "public");
setCurrentUser(null);
await bobDepositsInto(index, unsynced, 1);
await bobDepositsInto(index, object, 2);
// A broker that ACCEPTS the subscription and never pushes `State`. The bounded fallback
// fires, the repo is `timed-out` rather than `synced`, and the anchored read then answers
// 0 rows WITH NO ERROR — which is the shape a cold start takes, not an exotic one:
// `Verifier::load` repopulates `self.repos` from storage, so the repo is present and its
// commits simply have not arrived. Reading that as "the object is absent" consumed and
// destroyed the whole queue while reporting each deposit as absent. `open-repo` keeps
// `synced` and `timed-out` apart precisely so this arm can tell, and now it asks.
setOpenTimeoutForTests(20);
fake.doc_subscribe = () => () => {};
const reported = await reportedDuring(aliceConnects);
expect(reported.some((l) => /is not synced in this session/.test(l))).toBe(true);
expect(reported.some((l) => /came back clean and empty/.test(l))).toBe(false);
// Stopped rather than carried on: the deposit behind it is untouched.
expect(await entryValuesFor(index, object)).toEqual([]);
});
+16
View File
@@ -94,6 +94,19 @@ function makeFakeNg() {
const anchor = a[2] as string | undefined;
if (/GRAPH\s*</.test(query)) return undefined; // explicit-GRAPH write → dropped
if (!anchor) return undefined;
// Retiring a DEPOSIT: every triple of one subject, whatever predicates it carries
// (`emulated-verifier/inbox-consume.removeDeposit`). Modelled rather than ignored — a
// fake that let this delete fall through would leave every deposit in place, and a
// suite asserting convergence would pass on a queue that never shrinks.
const delDeposit = query.match(/^\s*DELETE\s+WHERE\s*\{\s*<([^>]+)>\s+\?\w+\s+\?\w+\s*\}/);
if (delDeposit) {
const subject = delDeposit[1]!;
for (let i = quads.length - 1; i >= 0; i--) {
const q = quads[i]!;
if (q.s === subject) quads.splice(i, 1);
}
return undefined;
}
const g = anchor;
const body = query.replace(/^\s*INSERT DATA\s*\{/, "").replace(/\}\s*$/, "");
const sm = body.match(/<([^>]+)>/);
@@ -150,6 +163,8 @@ function makeFakeNg() {
continue;
}
const rec = bySubject.get(q.s) ?? {};
// The deposit's own subject — the key a drain names to retire it.
rec.d = q.s;
if (q.p === `${INBOX}:payload`) rec.payload = q.o;
if (q.p === `${INBOX}:ts`) rec.ts = q.o;
if (q.p === `${INBOX}:from`) rec.from = q.o;
@@ -159,6 +174,7 @@ function makeFakeNg() {
.filter((r) => r.payload !== undefined && r.ts !== undefined)
.map((r) => {
const row: Record<string, { value: string }> = {
d: { value: r.d! },
payload: { value: r.payload! },
ts: { value: r.ts! },
};
@@ -69,6 +69,19 @@ function makeFakeNg() {
const sparql_update = mock(async (...a: unknown[]) => {
const query = a[1] as string;
const anchor = a[2] as string | undefined;
// Retiring a DEPOSIT: every triple of one subject, whatever predicates it carries
// (`emulated-verifier/inbox-consume.removeDeposit`). Modelled rather than ignored — a
// fake that let this delete fall through would leave every deposit in place, and a
// suite asserting convergence would pass on a queue that never shrinks.
const delDeposit = query.match(/^\s*DELETE\s+WHERE\s*\{\s*<([^>]+)>\s+\?\w+\s+\?\w+\s*\}/);
if (delDeposit) {
const subject = delDeposit[1]!;
for (let i = quads.length - 1; i >= 0; i--) {
const q = quads[i]!;
if (q.s === subject) quads.splice(i, 1);
}
return undefined;
}
const gm = query.match(/GRAPH <([^>]+)>\s*\{([\s\S]*)\}/);
let g: string;
let body: string;
@@ -135,6 +148,8 @@ function makeFakeNg() {
for (const q of quads) {
if (q.g !== anchor) continue;
const rec = bySubject.get(q.s) ?? {};
// The deposit's own subject — the key a drain names to retire it.
rec.d = q.s;
if (q.p === `${INBOX}:payload`) rec.payload = q.o;
if (q.p === `${INBOX}:ts`) rec.ts = q.o;
if (q.p === `${INBOX}:from`) rec.from = q.o;
@@ -144,6 +159,7 @@ function makeFakeNg() {
.filter((r) => r.payload !== undefined && r.ts !== undefined)
.map((r) => {
const row: Record<string, { value: string }> = {
d: { value: r.d! },
payload: { value: r.payload! },
ts: { value: r.ts! },
};
@@ -151,12 +151,21 @@ function makeWallet(quads: Quad[] = []) {
const anchor = a[2] as string | undefined;
const del = query.match(/DELETE\s+WHERE\s*\{([\s\S]*)\}/i);
if (del) {
const pattern = del[1]!.match(/<([^>]+)>\s+<([^>]+)>\s+\?/);
if (pattern && anchor !== undefined) {
if (del && anchor !== undefined) {
// `<s> <p> ?o` clears one register slot; `<s> ?p ?o` retires a whole DEPOSIT
// (`emulated-verifier/inbox-consume.removeDeposit`) — a deposit carries
// `a`/`payload`/`ts` and optionally `from`, so naming predicates would leave some.
const boundPredicate = del[1]!.match(/<([^>]+)>\s+<([^>]+)>\s+\?/);
const wholeSubject = del[1]!.match(/<([^>]+)>\s+\?\w+\s+\?\w+/);
const matches = boundPredicate
? (q: Quad) => q.s === boundPredicate[1] && q.p === boundPredicate[2]
: wholeSubject
? (q: Quad) => q.s === wholeSubject[1]
: null;
if (matches) {
for (let i = quads.length - 1; i >= 0; i--) {
const q = quads[i]!;
if (q.g === anchor && q.s === pattern[1] && q.p === pattern[2]) quads.splice(i, 1);
if (q.g === anchor && matches(q)) quads.splice(i, 1);
}
}
return undefined;
@@ -222,6 +231,8 @@ function makeWallet(quads: Quad[] = []) {
const bySubject = new Map<string, Record<string, string>>();
for (const q of inGraph) {
const rec = bySubject.get(q.s) ?? {};
// The deposit's own subject — the key a drain names to retire it.
rec.d = q.s;
if (q.p === `${INBOX}:payload`) rec.payload = q.o;
if (q.p === `${INBOX}:ts`) rec.ts = q.o;
if (q.p === `${INBOX}:from`) rec.from = q.o;
@@ -231,6 +242,7 @@ function makeWallet(quads: Quad[] = []) {
.filter((r) => r.payload !== undefined && r.ts !== undefined)
.map((r) => {
const row: Record<string, { value: string }> = {
d: { value: r.d! },
payload: { value: r.payload! },
ts: { value: r.ts! },
};
+23 -4
View File
@@ -249,12 +249,27 @@ export function makeWallet(quads: Quad[], options: WalletOptions = {}): FakeWall
const anchor = a[2] as string | undefined;
const del = query.match(/DELETE\s+WHERE\s*\{([\s\S]*)\}/i);
if (del) {
const pattern = del[1]!.match(/<([^>]+)>\s+<([^>]+)>\s+\?/);
if (pattern && anchor !== undefined) {
if (del && anchor !== undefined) {
// Two shapes, and the second one REMOVES A DEPOSIT — so the fake has to model it
// rather than answer "nothing matched" and let a consuming test pass on a queue that
// never shrank.
//
// `<s> <p> ?o` — one register slot, what the address/cap writers clear;
// `<s> ?p ?o` — every triple of one subject, which is how a deposit is retired
// (`emulated-verifier/inbox-consume.removeDeposit`): a deposit
// carries `a`/`payload`/`ts` and optionally `from`, so naming the
// predicates would leave some behind.
const boundPredicate = del[1]!.match(/<([^>]+)>\s+<([^>]+)>\s+\?/);
const wholeSubject = del[1]!.match(/<([^>]+)>\s+\?\w+\s+\?\w+/);
const matches = boundPredicate
? (q: Quad) => q.s === boundPredicate[1] && q.p === boundPredicate[2]
: wholeSubject
? (q: Quad) => q.s === wholeSubject[1]
: null;
if (matches) {
for (let i = quads.length - 1; i >= 0; i--) {
const q = quads[i]!;
if (q.g === anchor && q.s === pattern[1] && q.p === pattern[2]) quads.splice(i, 1);
if (q.g === anchor && matches(q)) quads.splice(i, 1);
}
commit(anchor);
}
@@ -333,6 +348,9 @@ export function makeWallet(quads: Quad[], options: WalletOptions = {}): FakeWall
const bySubject = new Map<string, Record<string, string>>();
for (const q of inGraph) {
const rec = bySubject.get(q.s) ?? {};
// `d` is the deposit's own subject, and it is bound because RETIRING a deposit
// names it. A fake that dropped it would hand back deposits nothing can consume.
rec.d = q.s;
if (q.p === `${INBOX}:payload`) rec.payload = q.o;
if (q.p === `${INBOX}:ts`) rec.ts = q.o;
if (q.p === `${INBOX}:from`) rec.from = q.o;
@@ -342,6 +360,7 @@ export function makeWallet(quads: Quad[], options: WalletOptions = {}): FakeWall
.filter((r) => r.payload !== undefined && r.ts !== undefined)
.map((r) => {
const row: Record<string, { value: string }> = {
d: { value: r.d! },
payload: { value: r.payload! },
ts: { value: r.ts! },
};