fix: un dépôt est délivré une fois et retiré, comme en amont

This commit is contained in:
Sylvain Duchesne
2026-08-26 14:11:56 +02:00
parent abfb619c76
commit f6a48a4507
24 changed files with 1486 additions and 176 deletions
@@ -579,8 +579,11 @@ export async function enumerateMyInboxes(): Promise<InboxEnumeration> {
* upstream an inbox is consumed, and processing a message *applies* it. Applying a
* Link means writing it here.
*
* Idempotent — re-applying the same Link is a no-op, so re-processing an inbox
* (a second tab, a reconnect) costs nothing.
* Idempotent — filing the same Link twice is a no-op. Kept after deposits became consumed
* (2026-08-26), because the case it answers was never the replay: the SAME cap legitimately
* arrives as two distinct deposits when a document is shared twice or a key is rotated, and
* both are consumed and both applied. Without it that user's User branch carries the same
* `ng:link` twice.
*
* `forHolder` names WHOSE User branch this lands on when it is not the connected
* identity's — the emulated inbox processor
@@ -598,17 +601,24 @@ export async function addLink(cap: ReadCap, forHolder?: PrincipalId): Promise<vo
if (!store) return;
if ((await readLinks(forHolder)).includes(cap)) return;
const s = await session();
try {
await registerUpdate(
s.sessionId,
`INSERT DATA { <${USER_BRANCH_SUBJECT}> <${P.link}> "${escapeLiteral(cap)}" }`,
store,
"addLink",
holderRing(forHolder),
);
} catch (error) {
console.error(accessLogPrefix() + " addLink failed:", error);
}
// PROPAGATES since 2026-08-26. It used to catch and log, which was survivable only while
// nothing was consumed: the deposit stayed, and the next connection filed the cap again.
// Now the deposit is popped BEFORE this runs, so a swallowed failure is a cap lost for
// good — and worse, `applyLink` went on returning `true`, so the drain's trace counted a
// Link it had not filed.
//
// Its ONE caller does NOT catch, deliberately: most of what happens above this line
// (`ensureAccount`, `readLinks`, `session`) is deposit-INDEPENDENT infrastructure that
// fails for the whole queue at once, so catching per deposit meant popping and destroying
// every remaining one while reporting each. The throw travels to the drain, which reports
// the deposit it lost, reports the inbox, and stops — preserving everything not yet taken.
await registerUpdate(
s.sessionId,
`INSERT DATA { <${USER_BRANCH_SUBJECT}> <${P.link}> "${escapeLiteral(cap)}" }`,
store,
"addLink",
holderRing(forHolder),
);
}
/**
@@ -82,8 +82,8 @@ import type { Nuri } from "../model/types";
function reportUndrained(inbox: Nuri, error: unknown): void {
console.error(
accessLogPrefix() +
" connected, but this queue could not be drained — a share deposited in it is not " +
"applied yet, and connecting again will try it again: " +
" connected, but this queue could not be drained — whatever it had already taken out " +
"is consumed and gone; what is still in it waits for the next attempt: " +
shortNuri(inbox) +
":",
error,
@@ -120,12 +120,19 @@ const inFlight = new Map<string, Promise<void>>();
* ── Why a queue is NOT the connection, corrected 2026-08-16 ───────────────
* This said "there is no third category here" and rejected on an undrained inbox too. The
* reasoning was that an unapplied share and a lost restore look the same to a person. What
* it missed is what happens NEXT: an inbox is not consumed by failing, so a queue that
* cannot be applied is still on the list at the following connection, and the one after
* it missed is what happens NEXT. At the time nothing was ever consumed, so a queue that
* could not be applied was still on the list at the following connection, and the one after
* that. One unapplicable item therefore did not delay a share — it **locked the person out
* of the application, permanently**, and got worse rather than better with time. Reported
* live by a consuming application, three sign-ins out of three.
*
* Consuming (2026-08-26) removes that particular ratchet — a deposit whose arm fails is now
* gone, so it cannot deny the same session twice — but it does NOT restore the old rule.
* Rejecting here would still be wrong, and now for a worse reason: the item is already
* consumed by the time its arm fails, so denying the session would trade a permanent
* lockout for a permanent lockout AND a lost deposit. Applying one item stays data, and
* data is reported, never a refusal to connect.
*
* That is strictly worse than the silence the 2026-08-13 shape replaced, and it collapsed
* a distinction that has to stay open: **failing to REACH the queues** (the session cannot
* proceed at all — reject, unchanged) against **failing to apply what is in one** (data,
@@ -135,8 +142,11 @@ const inFlight = new Map<string, Promise<void>>();
* restore step is a plain `await` and still rejects, so a broker that cannot answer fails
* the connection there, before any queue is reached. Only the per-inbox drain is caught.
*
* Not silence: {@link reportUndrained} is loud and ungated, and the caps a share carries
* are not lost — the deposit stays in the queue, so the next connection applies it.
* Not silence: {@link reportUndrained} is loud and ungated. What it no longer promises,
* since deposits became consumed on 2026-08-26, is that nothing was lost: the deposit the
* drain was applying when it fell over is GONE, and only what is still queued reaches the
* next connection. The message says exactly that, and so does the test
* `an I/O failure while filing a Link STOPS the drain, so the rest of the queue survives`.
*
* Not fixed with a retry, a timeout or a flag on purpose: deciding *what to do* about a
* broker that cannot answer belongs to the caller, and it can only decide if it is told.
@@ -223,6 +233,12 @@ export async function connectedUser(): Promise<void> {
// is abandoning, which this function has always called "not a failure", and
// reporting it would put a broker-looking error in the log every time a page
// switches user. The loop is about to return for the same reason.
//
// This guard cannot tell an abandonment from a genuine failure that happened to
// coincide with a user switch, and it does not have to: since 2026-08-26 a deposit
// that was consumed and not applied is reported by `inbox-consume.deliverDeposits`
// BEFORE the error is thrown, so what this drops is the duplicate, never the only
// notice of a loss. Dropping it here was silent for exactly one day.
if (!stillConnected()) return;
reportUndrained(inbox, error);
}
@@ -0,0 +1,212 @@
/**
* Consuming an inbox — the ONE act that retires a deposit, and the ONE loop that performs it.
*
* ── Upstream pops, and the order is not an accident ───────────────────────
* Read at all three of its sites, because the order is the whole contract:
*
* 1. the broker REMOVES the message from the queue — `ClientEvent::InboxPopRequest` →
* `inbox_pop_for_user` → `take_first_msg` → `MSGS.take_first_value`, which is
* `get_first_key_value` followed by `del` (`engine/repo/src/kcv_storage.rs:557`,
* `engine/storage-rocksdb/src/kcv_storage.rs:239-243`). The removal happens in the
* STORE, before anything has looked at the content;
* 2. only then is it delivered (`ClientMessageContentV0::InboxReceive { from_queue: true }`,
* `engine/net/src/actors/client/client_event.rs:68`);
* 3. the verifier applies it, and a failure is LOGGED AND SWALLOWED —
* `log_err!("Error during process_inbox {e}")` — never re-enqueued. It then asks for
* the next one (`engine/verifier/src/verifier.rs:1680-1715`).
*
* So: **remove, then apply, then log whatever went wrong, then move on.** A deposit is
* delivered exactly once whether or not its arm succeeded, and an arm that throws loses it
* — upstream that message left the store before the arm ever ran. Removing FIRST is
* therefore not a risk this module takes, it is the behaviour it reproduces; and it makes
* the reverse mistake impossible, since a removal that fails stops the apply instead of
* letting an applied deposit stay in the queue.
*
* ── Why a deliverability question exists here and not upstream ────────────
* Upstream an inbox holds nothing but typed messages, and WHO may pop one is settled by
* reader registration rather than by content: `inbox_pop_for_user(user)` pops only from
* inboxes that user is a registered reader of, so a message is never popped by anybody but
* its own recipient. Here an inbox holds a MIX — this package's deposits and a consumer's
* own payloads, which `surface/inbox.ts` hands back untouched — so "may this be popped
* now?" has to be asked of the content. That is `inbox-dispatch.claimsDeposit`, and it is
* asked BEFORE the removal for the same reason the broker checks registration before
* `take_first_msg`: what is not deliverable to this consumer was never popped for it.
*
* ── One removal, and it is here ───────────────────────────────────────────
* Both triggers this package drains from — the owner's own session (`inbox.processInbox`)
* and the deferred stand-in for an absent owner (`inbox.processForOwner`, scheduled by
* `emulated-verifier/inbox-processor.ts`) — call {@link deliverDeposits} and nothing else.
* Neither carries a copy of the removal: the triggers differ in WHO drains and WHEN, never
* in what consuming means. `grep -rn "DELETE" packages/polyfill/src/` shows the only
* deposit removal in the package is {@link removeDeposit} below.
*/
import { registerUpdate } from "./register-write";
import { applyDeposit, claims, claimsDeposit } from "./inbox-dispatch";
import type { Applying } from "./inbox-dispatch";
import { getStoreRegistryDeps } from "../shared-wallet/bootstrap";
import { accessLogPrefix, logStage, shortNuri } from "../shared-wallet/access-log";
/**
* One deposit as it is STORED: the RDF subject `inbox.post` minted for it, and what it
* carries. The subject is the removal key — `post` gives every deposit a fresh unique
* subject, so naming it retires that deposit and no other, where a `ts` could collide.
*
* Deliberately NOT the published {@link import("../surface/inbox").Deposit}: a subject is
* emulation detail (upstream a queued message has no address of its own, it has a
* position), so it stays inside the package and an application never learns it.
*/
export interface StoredDeposit {
/** The RDF subject the deposit lives under, in its inbox document's default graph. */
readonly subject: string;
/** What the deposit carries — handed to the dispatch as-is. */
readonly payload: unknown;
}
/**
* Retire ONE deposit from its inbox. **The only removal in this package.**
*
* `DELETE WHERE { <subject> ?p ?o }` — every triple of that one deposit, whichever
* predicates it happens to carry (`from` is optional). Two established forms are combined
* here, and neither half is invented:
*
* - `DELETE WHERE` with a VARIABLE predicate (`<s> ?p ?o`) is the shape verified against
* the real broker for removing a whole subject
* (`docs/decisions/sparql-delete-for-orm-objects.md`);
* - it is ANCHORED with no `GRAPH <…>` wrapper, like every other register write in this
* package (`public-store.exposeReadCap`, `branch-registers.publishInboxAddress`). That
* is not a shortcut, it is where the data IS: `inbox.post` writes the anchored DEFAULT
* graph and `DEPOSITS_QUERY` reads it, so a `GRAPH`-wrapped delete would name a graph
* the deposits were never written to. The decision above shows the wrapped form because
* its subject is an ORM object in a named graph; the anchor plays that role here.
*
* A `;`-joined compound update is exercised nowhere in this library, so this stays one
* statement.
*
* Through `registerUpdate`, and that is deliberate: retiring a deposit is the verifier's
* own bookkeeping, not an application write — upstream it happens inside the broker's
* store, and no application makes it. That door skips the OWNERSHIP guard, which a drain
* running for an absent owner could not pass, and keeps `assertMayReach` against the named
* ring: whoever consumes must hold the inbox's cap, and `processForOwner` names the
* owner's ring exactly as it does for every other step it takes.
*
* A failure PROPAGATES. Nothing is applied for a deposit that could not be retired — the
* caller's loop stops and its drain reports itself where drains report — because applying
* one that stayed in the queue is the replay this module exists to end.
*/
// @provenance removeDeposit kind=aligned level=1 ref=engine/broker/src/server_storage/core/inbox.rs:take_first_msg — upstream the store REMOVES the message as it hands it over (`MSGS.take_first_value` = `get_first_key_value` + `del`); here the queue is RDF in a document, so the same act is a DELETE of the deposit's own subject
async function removeDeposit(stored: StoredDeposit, ctx: Applying): Promise<void> {
const { sessionId } = await getStoreRegistryDeps().getSession();
await registerUpdate(
sessionId,
`DELETE WHERE { <${stored.subject}> ?p ?o }`,
ctx.inbox,
"inboxConsume",
ctx.ring,
);
}
/**
* Deliver every deposit in `stored` to the dispatch, consuming each as it goes. Answers
* how many produced an EFFECT — which is what a drain traces, and all any caller wants.
*
* The loop both triggers share, so that "what consuming means" cannot come to differ
* between the owner's own drain and the deferred one. Per deposit, in this order:
*
* 1. **claimed?** — a payload no arm answers for is not popped at all, and the loop
* moves on leaving it in the inbox. That covers a consumer's own data (not a message,
* so nothing is reported) and a deposit whose arm belongs to the owner's own session
* while a stranger's is running (see `inbox-dispatch.claimsDeposit`);
* 2. **remove** — the pop, before anything is applied, as upstream;
* 3. **apply** — the `match`, one arm per declared type. A refusal inside an arm is
* reported on the log stream and the deposit is already gone, which is upstream's
* `log_err!` + move on, and the end of the re-report-forever loop this package had.
*
* Sequential rather than parallel, for the reason every drain in this package is: an arm
* writes what it applies (`branch-registers.addLink`, on the owner's User branch), so two
* in flight over one queue interleave their reads and writes.
*/
// @provenance deliverDeposits kind=aligned level=1 ref=engine/verifier/src/verifier.rs:inbox — upstream's pop-apply-log-next loop, over a queue that is a document here rather than a column family
export async function deliverDeposits(
stored: readonly StoredDeposit[],
ctx: Applying,
): Promise<number> {
let applied = 0;
for (const deposit of stored) {
// The holder can move between two arms as easily as during the read that found these.
// Stopping leaves every deposit not yet reached in the queue for its owner — which is
// the whole reason this is asked BEFORE the pop rather than after it.
if (!holderStillStands(ctx)) {
logStage(
"ABANDONED " + shortNuri(ctx.inbox) + " — the identity changed while it was being " +
"processed; its remaining deposits stay for their owner",
);
return applied;
}
if (!claimsDeposit(deposit.payload, ctx)) continue;
// No subject, no pop. `?d` is bound by the deposits query, so this should be
// unreachable — but `depositsFrom` defaults it to "" rather than dropping the row, and
// an empty subject would issue `DELETE WHERE { <> ?p ?o }`: a malformed statement whose
// effect on a real broker nobody has established. Refusing to APPLY as well is the
// load-bearing half — applying without retiring is precisely the replay this module
// exists to end, and it would repeat at every drain for as long as the row sat there.
if (deposit.subject === "") {
console.error(
accessLogPrefix() + " a deposit in " + shortNuri(ctx.inbox) + " carries no subject, " +
"so it cannot be retired — refusing to apply it rather than applying it again at " +
"every drain. Nothing was consumed and nothing was written.",
);
continue;
}
await removeDeposit(deposit, ctx);
// Popping is an AWAIT, and the arm below reads AMBIENT state — so the identity can move
// across exactly this gap, between the decision to deliver and the delivery itself.
// Re-asked rather than assumed: applying now would file this deposit for whoever holds
// the page instead of for its owner, which is the mis-filing the whole `Applying`
// context exists to prevent.
//
// BOTH guards, and that matters: `processInbox` names its owner as a constant
// (`ownerHoldsTheSession: () => true`) because it only ever runs in that owner's own
// session, so the claim check alone cannot see the move — only `stillTheHolder` can.
// Checking one and not the other left this hole open, which is how it was missed.
//
// Reported ungated and then ABANDONED: the deposit is gone and cannot be got back, and
// whatever is behind it in the queue belongs to a session that is no longer here.
if (!holderStillStands(ctx) || !claims(deposit.payload, ctx)) {
console.error(
accessLogPrefix() + " a deposit in " + shortNuri(ctx.inbox) + " was consumed but not " +
"applied: the identity changed while it was being taken out of the queue, and " +
"applying it now would file it for the wrong holder. It cannot be recovered; what " +
"is still queued stays for its owner.",
);
return applied;
}
// The arm's own I/O failure is REPORTED HERE, at the source, and only then allowed to
// travel. It must stop the drain — everything not yet popped has to survive — but a
// throw is at the mercy of whoever catches it, and one caller legitimately swallows:
// `connect.connectedUser` drops the error when the identity has moved, because an
// ownership refusal there is abandoning and not a failure. That guard cannot tell the
// two apart, so an arm that failed WHILE the page switched user destroyed a deposit and
// printed nothing at all. Reporting before rethrowing closes that without touching a
// policy that is right for its own case: the loss is announced by the only code that
// knows a deposit was consumed.
try {
if (await applyDeposit(deposit.payload, ctx)) applied += 1;
} catch (error) {
console.error(
accessLogPrefix() + " a deposit in " + shortNuri(ctx.inbox) + " was consumed and " +
"could not be applied — it cannot be recovered; what is still queued stays for " +
"the next attempt:",
error,
);
throw error;
}
}
return applied;
}
/** Is the identity this drain was authorised for still the one holding the page? A drain
* that NAMED its holder rather than resolving it has nothing to lose and always stands. */
function holderStillStands(ctx: Applying): boolean {
return ctx.stillTheHolder === undefined || ctx.stillTheHolder();
}
@@ -85,15 +85,39 @@ export interface Applying {
/**
* Whether {@link owner} is ALSO the identity holding the session.
*
* It decides what an arm may do, and the line is not arbitrary. A drain running under
* It decides what is DELIVERED, and the line is not arbitrary. A drain running under
* SOMEONE ELSE's session (the deferred stand-in for an absent owner,
* `emulated-verifier/inbox-processor.ts`) usurps the owner's identity, and it does so
* for the verifier's own bookkeeping only — a register write on the owner's User
* branch. Writing APPLICATION data into the owner's documents from a stranger's page is
* a different act, and this package does not take it: the deposit is not consumed, so
* it stays for the owner's own next connection, which is the real path.
* a different act, and this package does not take it.
*
* Since 2026-08-26 it also decides what is CONSUMED, and the two must stay the same
* decision: what a stranger's drain declines to apply, it must not pop either, or the
* owner would lose a deposit to a session that was never going to apply it. So this is
* read by {@link claimsDeposit}, before the removal, and not by the arm afterwards.
*
* A PREDICATE rather than a value, because it is asked once per deposit: a drain runs
* across many awaits and the owner can arrive or leave inside it. A drain that captured
* it once would keep treating every later deposit as the owner's long after he stopped
* holding the page.
*/
readonly ownerHoldsTheSession: boolean;
readonly ownerHoldsTheSession: () => boolean;
/**
* Is {@link owner} STILL the identity this drain was authorised for? Consulted before
* each deposit, never once for the run.
*
* Only a drain that resolved its holder from the ambient session needs it —
* `inbox.processInbox`, whose guard ran at entry while its arms read and write many
* awaits later, so the identity can move mid-run and its remaining deposits must be left
* for whoever owns them. It matters more now than it did: a deposit reached after the
* switch would be POPPED under the wrong identity, not merely mis-filed.
*
* Omitted by a drain that NAMED its holder instead of resolving it
* (`inbox.processForOwner`, which runs under a stranger's session on purpose and would
* abandon every run if it asked). Absent means "nothing can invalidate this run".
*/
readonly stillTheHolder?: () => boolean;
}
/**
@@ -132,21 +156,84 @@ export function linkPayload(cap: ReadCap): { kind: string; cap: ReadCap } {
* item is reported and denies nobody their session, nor the OTHER deposits their turn. A
* genuine I/O failure is a different thing and still propagates — the caller reports the
* whole inbox and moves to the next one.
*
* **The deposit is already gone when this runs**, and the wording says so. It was popped
* before the arm was entered (`inbox-consume.deliverDeposits`), exactly as upstream logs
* `Error during process_inbox` for a message the broker took out of the queue before
* delivering it. So this is reported ONCE, not at every push and every connection for as
* long as the deposit sits there — which is what it used to promise, in these words.
*/
function report(ctx: Applying, why: string): void {
console.error(
accessLogPrefix() + " could not apply a deposit in " + shortNuri(ctx.inbox) +
" — it stays in it and the next attempt will try again: " + why,
" — it has been consumed and will not be retried: " + why,
);
}
/**
* May this deposit be POPPED for the consumer described by `ctx`? Asked before anything is
* removed, and the reason it has to exist is a divergence, not a preference.
*
* Upstream nothing asks it: an inbox holds only typed messages, and WHO may take one is
* settled by reader registration rather than content — `inbox_pop_for_user(user)` walks
* only the inboxes that user reads, so a message is never popped by anyone but its
* recipient. Here one inbox holds a MIX, so the same question has to be put to the payload.
*
* Two answers are "no", and neither is a failure:
*
* - **a consumer's own payload.** It declares no kind of this package's, so no arm claims
* it, and nothing is reported: it is not an unapplied message, it is not a message. It
* is also the thing `inbox.read` and `inbox.watch` hand back, so consuming it would
* take from an application the data it deposited for itself.
* - **an arm that belongs to the owner's OWN session, while a stranger's is running.**
* The deferred stand-in usurps an absent owner's identity for the verifier's own
* bookkeeping only; writing application data into that owner's documents is an act it
* does not take (see {@link Applying.ownerHoldsTheSession}). Upstream such a pop simply
* never happens — only the recipient pops. So the deposit is left in the queue for the
* owner's next connection, which is the real path, and is NOT consumed by a drain that
* was never going to apply it. Getting this wrong is the one way consuming could lose
* data outright: a stranger's timer would retire a reference the owner never indexed.
*
* The trace lives here rather than in the arm, because this is now where the decision is.
*/
// @provenance claimsDeposit kind=divergent level=1 ref=engine/broker/src/server_broker.rs:inbox_pop_for_user — upstream deliverability is reader REGISTRATION and the content is never consulted; here an inbox mixes this package's deposits with a consumer's own, so it is read off the payload
export function claimsDeposit(payload: unknown, ctx: Applying): boolean {
const claimed = claims(payload, ctx);
if (!claimed && kindOf(payload) === REFERENCE_KIND) {
logStage(
"WAITING " + shortNuri(ctx.inbox) + " — a reference deposit is applied by its own " +
"owner's session; it stays for " + ctx.owner,
);
}
return claimed;
}
/**
* The same question with NO trace — for the re-ask that follows the pop.
*
* Separate because the trace is only true the FIRST time. Asked again after the deposit has
* been removed, {@link claimsDeposit} announced "it stays for <owner>" about something that
* had just been destroyed, one line above the report saying so. A predicate that logs is
* fine where it is asked once; this one is asked twice, and only the first ask is a wait.
*/
export function claims(payload: unknown, ctx: Applying): boolean {
const kind = kindOf(payload);
if (kind === null) return false;
if (kind === REFERENCE_KIND && !ctx.ownerHoldsTheSession()) return false;
return true;
}
/**
* Apply ONE deposit for the owner of its inbox. Answers whether it produced an effect —
* which is what a drain counts, and all any caller has ever needed from it.
*
* Nothing is consumed here, by any arm: an inbox stays re-readable, so every abandon and
* every refusal above is harmless — what was not applied is still there for the next push
* or the next connection.
* **Called only for a deposit already consumed**, and only through
* `inbox-consume.deliverDeposits`, which asked {@link claimsDeposit} first and popped it
* second. Every arm below therefore runs on a deposit that has left its inbox: a refusal
* is final and is reported once, and there is no next attempt to leave anything for. The
* arm that must NOT run under a stranger's session is not guarded here — it is not
* delivered here at all, which is the only way a refusal and a non-delivery stay
* distinguishable now that one of them destroys.
*/
// @provenance applyDeposit kind=aligned level=1 ref=engine/verifier/src/inbox_processor.rs:process_inbox — the same `match` over the declared content, one arm per variant, `NotImplemented` for the rest
export async function applyDeposit(payload: unknown, ctx: Applying): Promise<boolean> {
@@ -190,6 +277,21 @@ async function applyLink(payload: unknown, ctx: Applying): Promise<boolean> {
return false;
}
getCaps().learnFor(ctx.ring, cap);
// PROPAGATES, deliberately, and this arm is where that was got wrong twice in one day.
// `addLink` used to swallow, so a cap that was never filed was reported as applied. The
// first correction caught the throw HERE and reported per deposit — which was worse: most
// of what `addLink` does before its write (`ensureAccount`, `readLinks`, `session`) is
// deposit-INDEPENDENT infrastructure that fails for the whole queue at once, so catching
// it meant popping and destroying every remaining deposit one by one while reporting each.
//
// A genuine I/O failure is not an unapplicable item: it stops the drain, the caller
// reports the whole inbox, and everything not yet popped is preserved for the next
// attempt. That is the rule stated beside {@link report}, and BOTH arms follow it — but
// the reference arm only since its `unreadable` outcome was split out of `refused` on the
// same day. It had been the opposite: `read-model.readDoc` swallows a failed read and
// answers `[]`, so an outage arrived as an ordinary refusal and the drain walked the
// whole queue destroying it. Correcting one arm and leaving the other is precisely how
// that survived a round of review.
await addLink(cap, ctx.owner);
return true;
}
@@ -197,20 +299,14 @@ async function applyLink(payload: unknown, ctx: Applying): Promise<boolean> {
/**
* The **reference** arm: resolve it and look — see `index-deposit.ts` for the whole of it.
*
* Only in the owner's OWN session, for the reason {@link Applying.ownerHoldsTheSession}
* gives: an entry is application data in the owner's document, and a drain running under a
* stranger's session writes the verifier's registers and nothing else. Left waiting is not
* left silent, and it is not a failure either — the deposit is not consumed, so the owner's
* next connection applies it, which is the path this package documents.
* Only ever entered in the owner's OWN session, and {@link claimsDeposit} is what makes
* that true: under a stranger's session this deposit is not delivered, so it is not popped
* either and it waits for its owner. The test used to stand HERE and return `false`, which
* was harmless while nothing was consumed and would now retire a reference nobody applied.
* That is why deliverability had to become a question asked before the pop rather than a
* branch inside the arm.
*/
async function applyReference(reference: Nuri, ctx: Applying): Promise<boolean> {
if (!ctx.ownerHoldsTheSession) {
logStage(
"WAITING " + shortNuri(ctx.inbox) + " — a reference deposit is applied by its own " +
"owner's session; it stays for " + ctx.owner,
);
return false;
}
const application = await applyIndexDeposit(reference, ctx.inbox);
switch (application.outcome) {
case "applied":
@@ -222,14 +318,33 @@ async function applyReference(reference: Nuri, ctx: Applying): Promise<boolean>
case "unchanged":
return false;
case "skipped":
// A property of the OBJECT, not a failure: it resolved, and there is nothing to
// index it by. Traced, not reported — see `index-deposit.ts`.
logStage(
"NOT INDEXED " + shortNuri(application.object) + " — " + application.reason,
// A property of the OBJECT rather than a failure — it resolved, and there is nothing
// to index it by. It was TRACED and not reported while deposits were re-read for
// ever: an object that carried nothing for the field yet would be indexed by a later
// drain once it did. Consumption ends that, so the deposit is discarded here and now,
// and a discard may not be gated behind `debugAccessLog` — a diagnostic may be
// opt-in, something the caller can no longer recover may not.
report(
ctx,
"a reference to " + shortNuri(application.object) + " was discarded — " +
application.reason,
);
return false;
case "refused":
// DETERMINISTIC: a misconfigured index, an inbox that is not a document's. Trying
// again changes nothing, so this is an unapplicable item — reported, and the deposits
// behind it keep their turn.
report(ctx, application.why);
return false;
case "unreadable":
// The read established NOTHING — it threw, or it answered nothing from a repo whose
// sync this session never confirmed (`index-deposit.noRowsMeansAbsent`). THROWS, so
// the drain stops and everything still queued survives — the same rule the Link arm
// follows. Treating this as an ordinary refusal let a single outage walk the whole
// queue, popping and destroying every reference while reporting each as absent.
// `inbox-consume.deliverDeposits` reports the deposit this one cost before letting
// the throw travel. An object that is genuinely absent is `refused`, not this: it
// costs its own deposit and the queue behind it carries on.
throw new Error(application.why);
}
}
@@ -37,13 +37,18 @@ const running = new Map<Nuri, Promise<void>>();
* `inbox.watch` use — and it is NOT gated by the access-log flag: a diagnostic may be
* opt-in, a failure may not.
*
* One failed item denies nothing, here as at connection: the inbox is not consumed by
* failing, the deposit stays in it, and the next push or the next connection applies it.
* One failed item denies nothing, here as at connection. What it no longer promises is a
* RETRY: since 2026-08-26 a deposit is popped before its arm runs
* (`emulated-verifier/inbox-consume.ts`), so whatever had already been delivered when this
* drain fell over is gone, exactly as upstream loses a message the broker took out of the
* queue. What is still QUEUED is untouched and the next push or connection reaches it —
* which is a different sentence, and the wording says so rather than promising the old one.
*/
function reportFailure(inbox: Nuri, error: unknown): void {
console.error(
accessLogPrefix() + " could not apply what is in this inbox — its deposits stay in it " +
"and the next attempt will try again: " + shortNuri(inbox) + ":",
accessLogPrefix() + " could not finish draining this inbox — whatever it had already " +
"taken is consumed; what is still queued waits for the next attempt: " +
shortNuri(inbox) + ":",
error,
);
}
@@ -187,9 +187,12 @@ async function applyWhatArrived(obs: Observation, inbox: Nuri): Promise<void> {
// is not theirs. (Landing after it, the run ABANDONS quietly instead and returns; both
// leave the deposit where it is.) Abandoning is what this package has always called
// "not a failure", and reporting it would put a broker-looking error in the log every
// time a page switches user. Nothing is lost: an inbox is not consumed by being
// abandoned, so the deposit is still there for its owner's next connection. Same rule,
// same words, as the drain loop in `connect.connectedUser`.
// time a page switches user. Nothing is lost: abandoning happens BEFORE the deposit it
// stopped at is taken out of the queue (`emulated-verifier/inbox-consume.ts` asks
// whether the holder still stands, then pops), so that deposit and every one behind it
// are still there for its owner's next connection. What had already been popped was
// applied for the holder that was authorised at the time. Same rule, same words, as the
// drain loop in `connect.connectedUser`.
if (!current(obs)) return;
throw error;
}
@@ -388,7 +391,9 @@ async function enumerate(obs: Observation): Promise<void> {
*
* It never rejects. Failing to WATCH is reported where failures go and denies nobody their
* session — the same rule the drain loop above it follows, and for the same reason: what
* has not been applied stays in its queue for the next attempt.
* has not been DELIVERED stays in its queue for the next attempt. (Delivered, not applied:
* since 2026-08-26 a deposit is popped before its arm runs, so one whose arm then failed is
* gone — see `emulated-verifier/inbox-consume.ts`. Failing to WATCH takes nothing out.)
*/
// @provenance startObservingInboxes kind=aligned level=1 ref=engine/verifier/src/verifier.rs:inbox — upstream a connected verifier is handed each inbox message as it arrives and applies it; here the arrival signal is a document push, and the set of inboxes is the User branch's `AddInboxCap` records
export async function startObservingInboxes(): Promise<void> {
@@ -19,8 +19,11 @@
* seconds later. Alice deposits into Bob's inbox; twenty seconds later this package drains
* Bob's inbox, while the connected session is still Alice's. When Bob IS connected in this
* page, the observation has already applied it long before the window closes and the timer
* finds nothing left to do — harmless (draining is idempotent) and not worth a special
* case: the two answer different questions, and only this one answers the absent owner.
* finds an EMPTY queue — since 2026-08-26 that is literally true rather than merely
* harmless: the observation CONSUMED the deposit, so there is nothing left to see, where
* before the timer re-read it and relied on every arm being idempotent. Not worth a special
* case either way: the two answer different questions, and only this one answers the absent
* owner.
*
* ── That is identity usurpation, and it is deliberate ─────────────────────
* It is only possible because one shared wallet holds every virtual identity, and it is
@@ -24,11 +24,22 @@
* fields and corrupt each other's entries.
*
* ── AN INDEX ONLY EVER GROWS ──────────────────────────────────────────────
* Nothing here removes an entry, and nothing here can express a removal. That is what
* makes the failure story trivial: since the only write is an addition, a read that comes
* back empty — object gone, unreadable, or the broker silent — can only ever mean "not
* added this time", and a later deposit adds it. Nothing has to tell an absence from a
* failure, so nothing can get that wrong. The price, stated plainly because someone will
* Nothing here removes an entry, and nothing here can express a removal, so the only write
* is an addition and curation stays convergent whatever order references arrive in.
*
* That used to make the failure story trivial too: a read coming back empty — object gone,
* unreadable, or the broker silent — meant "not added this time" whichever it was, so
* nothing had to tell an absence from a failure. **That is no longer true, and it is the
* sharpest consequence of consuming.** An absence is deterministic and costs its own deposit;
* a silent broker is transient and must stop the drain, or one outage walks the queue and
* destroys every reference in it. `read-model.readDoc` answers `[]` for both, so this module
* passes it an `onUnreadable` reporter and asks which one happened.
*
* What it no longer means, since deposits became consumed on 2026-08-26, is "and a later
* drain adds it". THIS deposit is gone once it has been looked at, so a broker that was
* silent for the one second the reference was examined costs that reference permanently.
* The recourse is a NEW deposit, which someone has to make; nothing retries on its own.
* Every such case is reported ungated, so it is at least never silent. The price, stated plainly because someone will
* need it: an entry cannot be taken out of an index, by anyone, including its owner.
*
* Harmless is not the same as invisible: every reference that could not be resolved is
@@ -46,6 +57,8 @@
import { readUnion } from "../surface/read-model";
import { sparqlUpdate } from "../surface/docs";
import { readInboxCapPairs } from "./branch-registers";
import { getSyncState } from "./open-repo";
import { mustNotAttempt } from "./reach";
import { session } from "../shared-wallet/account-registry";
import { assertNuri, escapeLiteral } from "../surface/sparql";
import type { UnionSubject } from "../surface/read-model";
@@ -70,8 +83,63 @@ export type IndexApplication =
| { readonly outcome: "unchanged"; readonly object: Nuri }
/** Resolved, and deliberately not added — a property of the OBJECT, not a failure. */
| { readonly outcome: "skipped"; readonly object: Nuri; readonly reason: string }
/** Nothing was written, and something is wrong enough to say out loud. */
| { readonly outcome: "refused"; readonly why: string };
/** Nothing was written, and something is wrong enough to say out loud. A DETERMINISTIC
* refusal — a misconfigured index, an inbox that is not a document's. Trying again
* changes nothing, so the drain reports it and moves on. */
| { readonly outcome: "refused"; readonly why: string }
/**
* The read did NOT establish anything — it threw, or it returned nothing from a repo
* whose sync this session never confirmed. Not an absence, and not a refusal: THROWS, so
* the drain stops and everything still queued survives the outage.
*
* Its own outcome since 2026-08-26, and the reason is consumption. While deposits were
* re-read for ever the ambiguity was free: absent or unreadable, the answer was "not
* added this time" and a later drain settled it. Once the deposit is popped before the
* arm runs, the two need opposite handling, and picking either one wholesale is wrong in
* a way that was measured: treating every empty read as a FAILURE let one dead reference
* stall every deposit behind it (a stranger can deposit, so that starves an index);
* treating every empty read as an ABSENCE let one cold start destroy the whole queue.
* {@link noRowsMeansAbsent} tells them apart instead of choosing.
*/
| { readonly outcome: "unreadable"; readonly why: string };
/**
* Did a read that came back with NO ROWS actually establish that the document is empty?
*
* This is the question consumption forced, and it has three sources of "no rows", only one
* of which is an absence:
*
* - the read THREW — `onUnreadable` fired. Transient. Not an absence.
* - the document was NEVER ATTEMPTED, because this holder holds no cap for it and
* `readUnion` drops it through `mustNotAttempt` before issuing a query. This looked
* deterministic and is not: for a stranger's PUBLIC document — the reference arm's
* central case — possession is acquired by a network fetch that swallows every error
* and MEMOISES the failure (`public-store.ts`). One outage on that fetch therefore
* presents as "no cap", and it stays that way for the session, so classing it as an
* absence destroyed the deposit AND every re-deposit after it. Not an absence.
* - the repo is PRESENT BUT UNSYNCED — and this is the one that cost round 6. A fresh
* session over a persistent wallet has not received the repo's commits, so the anchored
* query legitimately matches nothing and **raises no error at all**. Classing that as an
* absence let a cold start destroy an entire reference queue, silently, while reporting
* each one as "the object is absent".
*
* So an absence has to be EARNED, and only one shape earns it: a query was actually issued,
* it did not fail, and the repo's sync is confirmed. `open-repo` keeps `synced` and
* `timed-out` apart for exactly this and says so in those words — `synced` means a 0-row
* read MEANS it. `unknown` is the unit-fake path (no `doc_subscribe`, so no `State`
* semantics) where the read is direct and trustworthy.
*
* The cost of the strict answer is LATENCY, never loss: a reference this holder truly can
* never read stops one drain, is already popped, and the next attempt resumes past it. The
* cost of the lax answer was destroyed deposits that no re-deposit could recover. That
* asymmetry is the whole rule — when in doubt, do not call it an absence.
*/
function noRowsMeansAbsent(doc: Nuri, readFailed: boolean): boolean {
if (readFailed) return false;
if (mustNotAttempt(doc)) return false;
const sync = getSyncState(doc);
return sync === "synced" || sync === "unknown";
}
/**
* The values a subject carries for a predicate.
@@ -150,10 +218,13 @@ async function documentAddressedBy(inbox: Nuri): Promise<Nuri | null> {
* document declares it indexes by, whether the reference is already in, what the
* referenced object says for the field — and only then writes ONE entry.
*
* Never removes anything and never rewrites an entry, so re-applying the same deposit —
* which happens on every drain, because deposits are not consumed — lands on the same
* result. Curation is convergent, and its outcome does not depend on the order references
* arrive in.
* Never removes anything and never rewrites an entry, so the same reference arriving TWICE
* lands on the same result. That is still needed now that deposits are consumed, and the
* reason has simply changed: not a drain replaying one deposit, but two DISTINCT deposits
* naming the same object — a re-share, a retry by an application, two depositors who both
* point at it. Upstream keeps the same check for the same reason, with a queue that pops
* (`ContactAlreadyExists`, `engine/verifier/src/inbox_processor.rs`), which is the proof it
* was never a replay compensation. Curation stays convergent and order-independent.
*/
// @provenance applyIndexDeposit kind=invention level=none ref=none — nothing upstream turns a deposited reference into an index entry; the whole arm is this library's own
export async function applyIndexDeposit(
@@ -173,7 +244,22 @@ export async function applyIndexDeposit(
return { outcome: "skipped", object: reference, reason: "self-reference" };
}
const subjects = await readUnion([index]);
// Told whether the read FAILED, rather than left to guess from an empty answer. The two
// demand opposite handling now that deposits are consumed — see the `unreadable` outcome.
// A BOOLEAN, not the error value: `throw null` and `throw undefined` are legal, and a
// sentinel comparison against `null` made exactly those failures invisible.
let indexReadFailed = false;
let indexError: unknown;
const subjects = await readUnion([index], (_doc, error) => {
indexReadFailed = true;
indexError = error;
});
if (indexReadFailed) {
return {
outcome: "unreadable",
why: `${index}: the index itself could not be read: ${String(indexError)}`,
};
}
const declared = declaredFields(subjects, index);
if (declared.length > 1) {
// Ambiguity refuses to curate and leaves READING alone. Picking one would silently
@@ -193,35 +279,63 @@ export async function applyIndexDeposit(
const field = declared[0];
if (field === undefined) {
return {
outcome: "refused",
why:
`${index} declares no index field (${INDEX_FIELD}) — either it is not an index, or ` +
"it could not be read: an unreadable document and an empty one are the same empty " +
"result here. Nothing was written. Retry before concluding it is malformed.",
outcome: noRowsMeansAbsent(index, indexReadFailed) ? "refused" : "unreadable",
why: noRowsMeansAbsent(index, indexReadFailed)
? `${index} declares no index field (${INDEX_FIELD}) — it read cleanly and says ` +
"nothing, so either it is not an index or it was never configured as one. " +
"Nothing was written, and nothing here will write until it declares one."
: `${index} declares no index field (${INDEX_FIELD}), but its repo is not synced ` +
"in this session, so an empty read proves nothing about it. Nothing was written, " +
"and this is NOT a claim that it is misconfigured.",
};
}
if (indexedObjects(subjects, index).has(reference)) {
// Already there. Nothing is re-read and nothing is rewritten — an entry, once made,
// is never touched again.
// is never touched again. Reached by a SECOND deposit naming the same object, not by a
// replay of the first: that one was consumed when it was applied.
return { outcome: "unchanged", object: reference };
}
let resolved: readonly UnionSubject[];
let referenceReadFailed = false;
let referenceError: unknown;
try {
resolved = await readUnion([reference]);
resolved = await readUnion([reference], (_doc, error) => {
referenceReadFailed = true;
referenceError = error;
});
} catch (error) {
return { outcome: "refused", why: `${reference}: ${String(error)}` };
return { outcome: "unreadable", why: `${reference}: ${String(error)}` };
}
// The BROKER failed. Stop the drain: everything still queued has to survive an outage,
// and this deposit is already gone.
if (referenceReadFailed) {
return {
outcome: "unreadable",
why: `${reference}: could not be read: ${String(referenceError)}`,
};
}
// A clean read that found NOTHING — the object is genuinely absent or holds no triples.
// Deterministic: trying again changes nothing, so this is an unapplicable item and the
// deposits behind it keep their turn. Distinguishing this from the case above is the whole
// point of `onUnreadable`: for one day they were the same answer, and treating both as
// failures let a single dead reference stall every deposit behind it — which, since anyone
// may deposit, is a way to starve someone else's index.
if (resolved.length === 0) {
// ABSENT, unreadable, or the read failed — and this layer does not tell them apart,
// because nothing it does depends on the answer: an index only ever grows, so every
// reading leads to the same act, do not add and say so.
if (!noRowsMeansAbsent(reference, referenceReadFailed)) {
return {
outcome: "unreadable",
why:
`${reference}: the read came back empty, but its repo is not synced in this ` +
"session — that proves nothing about the object, so it is not treated as absent",
};
}
return {
outcome: "refused",
why:
`${reference}: the read came back empty — the object is absent, unreadable, or the ` +
"read failed; they are not distinguished here, and a later deposit adds it",
`${reference}: read cleanly from a synced repo and found nothing — the object is ` +
"absent",
};
}