feat(client): per-document reactive subscription (doc_subscribe), drop polling

Expose subscribeDoc(nuri, onChange) / subscribeDocs(nuris, onChange) wrapping the
real ng.doc_subscribe — per-document, event-driven (initial State + a Patch per
commit, local or broker-synced from a remote peer), returning a sync unsubscribe.
subscribeDocs isolates per doc (a failing/unsynced doc never aborts the others),
so it sidesteps the ORM fan-out hang (never orm_start_graph(graphs:[…])).

Replace the setInterval polling in inbox.watch() and discovery.watchIndex() with
doc_subscribe — same public contract, now push not poll. NextGraph is subscription-
first; no polling remains.

Verified against the real broker (@data harness spike): the doc_subscribe callback
marshals across the iframe RPC (@ng-org/web strips the callback and drives it via a
MessagePort — no DataCloneError) and fires on the initial push and on a real write.
Lib bun test 91 pass, tsc clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Sylvain Duchesne
2026-07-06 23:04:32 +02:00
parent 7ebb03a3f3
commit c0498a6ebc
7 changed files with 396 additions and 22 deletions
+32 -9
View File
@@ -38,6 +38,7 @@
*/
import * as inbox from "./inbox";
import { subscribeDoc } from "./subscribe";
import { ensureAccount, reservedAccount } from "./store-registry";
import { getCaps } from "./polyfill";
import type { Nuri, PrincipalId } from "./types";
@@ -156,18 +157,25 @@ export async function readIndex(): Promise<IndexEntry[]> {
}
/**
* Watch the discovery index. Polls {@link readIndex} and fires `onEntries`
* whenever the index changes. Returns an unsubscribe. Fires once immediately.
* (Deduplication is applied on each read.)
* Watch the discovery index — **event-driven, not polled**. Subscribes to the
* index document via {@link subscribeDoc} (the platform's `doc_subscribe` push):
* `onEntries` fires once on the initial state push and again on every subsequent
* change to the index document — a local submission OR a broker-synced remote one.
* Returns an unsubscribe. (Deduplication is applied on each read.)
*
* The `intervalMs` option is accepted for signature compatibility but IGNORED:
* there is no polling. The index is a single document, so this is immune to the
* ORM fan-out hang (see {@link subscribeDoc}).
*/
export function watchIndex(
onEntries: (entries: IndexEntry[]) => void,
opts?: { intervalMs?: number },
_opts?: { intervalMs?: number },
): () => void {
let stopped = false;
let lastCount = -1;
const intervalMs = opts?.intervalMs ?? 1000;
const tick = async (): Promise<void> => {
let unsubscribe: (() => void) | null = null;
const refresh = async (): Promise<void> => {
if (stopped) return;
try {
const entries = await readIndex();
@@ -179,10 +187,25 @@ export function watchIndex(
console.error("[discovery] watchIndex read failed:", error);
}
};
void tick();
const handle = setInterval(() => void tick(), intervalMs);
// The index document NURI is resolved async (ensureAccount); subscribe once it
// is known. The initial State push fires the first read (onEntries fires once),
// each later Patch a re-read.
void (async () => {
try {
const anchor = await indexInboxNuri();
if (stopped) return;
unsubscribe = subscribeDoc(anchor, () => void refresh());
} catch (error) {
console.error("[discovery] watchIndex subscribe failed:", error);
}
})();
return () => {
stopped = true;
clearInterval(handle);
if (unsubscribe) {
unsubscribe();
unsubscribe = null;
}
};
}
+22 -11
View File
@@ -25,6 +25,7 @@
*/
import { sparqlUpdate, sparqlQuery } from "./docs";
import { subscribeDoc } from "./subscribe";
import { getCurrentUser, getStoreRegistryDeps } from "./polyfill";
import { escapeLiteral } from "./sparql";
import type { Nuri, PrincipalId } from "./types";
@@ -186,22 +187,31 @@ export async function read(targetInbox: Nuri): Promise<Deposit[]> {
export const materialize = read;
/**
* Subscription over an inbox. Polls {@link read} on an interval and invokes
* `onDeposits` with the full current deposit list whenever it changes (grows).
* Returns an unsubscribe function. The polyfill has no reactive inbox
* subscription, so this polls; against real NextGraph it follows the recipient's
* own inbox processing. `onDeposits` fires once immediately.
* Subscription over an inbox — **event-driven, not polled**. Subscribes to the
* inbox document via {@link subscribeDoc} (the platform's `doc_subscribe` push):
* `onDeposits` fires once on the initial state push and again on every subsequent
* change to the inbox document — a local deposit OR a broker-synced remote one.
* Returns an unsubscribe function.
*
* On each push it re-reads the full deposit list ({@link read}) and invokes
* `onDeposits` only when the deposit count changed (grew), keeping the same
* "fires on change" contract the polling watcher had — same callback signature
* and same behaviour, just event-driven instead of `setInterval`.
*
* The `intervalMs` option is accepted for signature compatibility but IGNORED:
* there is no polling. (The inbox document is a single doc, so this is immune to
* the ORM fan-out hang — see {@link subscribeDoc}.)
*/
export function watch(
targetInbox: Nuri,
onDeposits: (deposits: Deposit[]) => void,
opts?: { intervalMs?: number },
_opts?: { intervalMs?: number },
): () => void {
let stopped = false;
let lastCount = -1;
const intervalMs = opts?.intervalMs ?? 1000;
const tick = async (): Promise<void> => {
// Re-read on every push; fire onDeposits only when the set changed (grew).
const refresh = async (): Promise<void> => {
if (stopped) return;
try {
const deposits = await read(targetInbox);
@@ -214,10 +224,11 @@ export function watch(
}
};
void tick(); // fire once immediately
const handle = setInterval(() => void tick(), intervalMs);
// Subscribe to the inbox document: the initial State push fires the first read
// (so onDeposits fires once immediately, as before), each later Patch a re-read.
const unsubscribe = subscribeDoc(targetInbox, () => void refresh());
return () => {
stopped = true;
clearInterval(handle);
unsubscribe();
};
}
+2
View File
@@ -18,6 +18,8 @@ export * as inbox from "./inbox";
export * as discovery from "./discovery";
export type { IndexEntry, SubmitOptions } from "./discovery";
export * as docs from "./docs";
export { subscribeDoc, subscribeDocs } from "./subscribe";
export type { DocChange, Unsubscribe } from "./subscribe";
export * as readModel from "./read-model";
export type { UnionSubject } from "./read-model";
export * as storeRegistry from "./store-registry";
+149
View File
@@ -0,0 +1,149 @@
/**
* Reactive single-document subscription — the polyfill's typed wrapper over the
* platform's `doc_subscribe` primitive. This is the canonical NextGraph reactive
* read at the document granularity: subscribe once, get the initial state pushed,
* then a push on every subsequent commit to that document — whether the write was
* local (this session) or a broker-synced remote change. NO POLLING.
*
* ── Why call the REAL injected `ng` directly (never `makeNg`) ──────────────
* Same hard constraint as `docs.ts`: the public `ng` is a JS `Proxy` over
* `@ng-org/web`'s iframe-RPC proxy. `doc_subscribe` is a STREAMED method — the
* `@ng-org/web` RPC strips the callback (by positional index) BEFORE it posts to
* the iframe and drives it locally via a `MessageChannel` port (the function is
* never structured-cloned, so no `DataCloneError`). Layering our own Proxy on top
* risks re-wrapping that surface; reaching the real `ng` held in the config avoids
* the double-proxy exactly as the raw `docs` primitives do. Do not import from
* `./ng-proxy`.
*
* ── The primitive shape (verified against nextgraph-rs) ────────────────────
* `ng.doc_subscribe(repo_o: string, session_id, callback)`
* (`sdk/js/lib-wasm/src/lib.rs:1907`) is **per-document** — one repo NURI, one
* callback. It is `async`, resolving to a JS **unsubscribe function**. The
* callback is invoked `callback(appResponse)` with a serialized `AppResponse`:
* `{ V0: { State | Patch | TabInfo | ... } }`. It pushes an initial `State`
* (plus a `TabInfo`) on subscribe, then a `Patch` per verified commit on the
* branch. Returning `true` from the callback also cancels; we cancel by calling
* the returned unsubscribe fn.
*
* ── Why per-document, never `orm_start_graph(graphs:[…])` ──────────────────
* A single not-yet-synced repo in an ORM graph fan-out makes `RepoNotFound` abort
* the WHOLE subscription (`initialize.rs:125-128`), so the readyPromise never
* resolves → the ~75s hang. `doc_subscribe` is per-branch/per-doc and has no
* fan-out: an absent doc breaks only its own subscription. {@link subscribeDocs}
* builds a set of these with per-doc error isolation to preserve that property.
*/
import { getConfig, getStoreRegistryDeps } from "./polyfill";
import type { Nuri } from "./types";
/**
* A push from the platform to a document subscriber. Loosely typed: the raw
* serialized `AppResponse` (`{ V0: { State | Patch | TabInfo | ... } }`). The
* consumer typically ignores the payload and uses the push purely as a
* change SIGNAL (re-query on change — the read-model pattern), so this stays
* permissive rather than modelling every AppResponse variant.
*/
export type DocChange = unknown;
/** An unsubscribe function — idempotent (calling it twice is a no-op). */
export type Unsubscribe = () => void;
async function sessionId(): Promise<string> {
return (await getStoreRegistryDeps().getSession()).sessionId;
}
/**
* Subscribe to ONE document. `onChange` fires on the initial state push and on
* every subsequent change to that doc (local write OR broker-synced remote
* change). Returns an unsubscribe function.
*
* The wrapper is synchronous-returning (an unsubscribe fn) even though the
* underlying `ng.doc_subscribe` is async: the real unsubscribe is captured when
* the promise resolves; if the caller unsubscribes before setup completes, the
* cancellation is honoured as soon as the real unsubscribe is available (and no
* further `onChange` fires after unsubscribe).
*
* Calls the REAL injected `ng.doc_subscribe` directly (never `makeNg`).
*/
export function subscribeDoc(nuri: Nuri, onChange: (r: DocChange) => void): Unsubscribe {
const { ng } = getConfig();
let stopped = false;
let realUnsub: (() => void) | null = null;
const cb = (resp: DocChange): void => {
if (stopped) return;
try {
onChange(resp);
} catch (error) {
console.error("[subscribe] onChange handler threw for", nuri, error);
}
};
// Kick off the async subscription. Errors are isolated to this doc (they never
// reject a shared batch — see subscribeDocs). If setup fails, this doc simply
// never fires; the caller's unsubscribe stays a safe no-op.
void (async () => {
try {
const sid = await sessionId();
const unsub = (await ng.doc_subscribe(nuri, sid, cb)) as (() => void) | undefined;
if (stopped) {
// Unsubscribed before setup resolved — cancel immediately.
if (typeof unsub === "function") unsub();
return;
}
realUnsub = typeof unsub === "function" ? unsub : null;
} catch (error) {
console.error("[subscribe] doc_subscribe failed for", nuri, error);
}
})();
return () => {
if (stopped) return;
stopped = true;
if (realUnsub) {
try {
realUnsub();
} catch (error) {
console.error("[subscribe] unsubscribe failed for", nuri, error);
}
realUnsub = null;
}
};
}
/**
* Subscribe to a SET of documents, one {@link subscribeDoc} per NURI, with
* PER-DOC error isolation. `onChange(nuri, r)` fires for whichever doc changed.
* Returns a single unsubscribe that tears down all of them.
*
* The per-doc isolation is the point: a bad / not-yet-synced doc breaks only its
* own subscription and NEVER aborts the others (this is precisely what avoids the
* ORM fan-out hang — do NOT replace this with `orm_start_graph(graphs:[…])`). The
* set is deduplicated; an empty set returns a no-op unsubscribe.
*/
export function subscribeDocs(
nuris: Nuri[],
onChange: (nuri: Nuri, r: DocChange) => void,
): Unsubscribe {
const unique = [...new Set(nuris.filter(Boolean))];
const unsubs = unique.map((nuri) => {
// Each subscription is independent: subscribeDoc already isolates its own
// async setup failure (logged, never thrown), so one bad doc cannot abort the
// construction of the others here.
try {
return subscribeDoc(nuri, (r) => onChange(nuri, r));
} catch (error) {
console.error("[subscribe] subscribeDocs: failed to subscribe", nuri, error);
return () => {};
}
});
return () => {
for (const u of unsubs) {
try {
u();
} catch (error) {
console.error("[subscribe] subscribeDocs: unsubscribe failed", error);
}
}
};
}