5 Commits

Author SHA1 Message Date
Sylvain Duchesne e2ed970cbd feat!: curer n'est plus un appel, c'est ce que fait le traitement de l'inbox 2026-08-20 11:04:31 +02:00
Sylvain Duchesne 2ce2113157 fix: le polyfill est un peer, pas un chemin vers la copie de travail d'un développeur 2026-08-17 11:48:17 +02:00
Sylvain Duchesne 821ea997fe docs: dire pourquoi le tag est nu, pour qu'il cesse de l'être au bon moment 2026-08-17 10:53:32 +02:00
Sylvain Duchesne aeb8c7d157 docs: l'engagement de la couche d'indexation, et ses deux déclarations
Le dépôt gagne sa doctrine et ses contrats, dans le régime bidirectionnel.

Il publie son engagement — ce qu'un consommateur peut attendre de l'indexation —
et déclare ce qu'il consomme lui-même, du polyfill et de ng-e2e-helpers. Les
deux déclarations sont écrites depuis les appels réels, pas depuis ce que la
surface offre : un usage non déclaré est la faute du consommateur en cas de
rupture, et une surface offerte mais non déclarée reste librement modifiable.

Version 1.0.0, pas 0.1.0 : sous semver, 0.x ne promet rien du tout, donc le
majeur ne porte son signal qu'à partir de 1. Ce dépôt étant sur main, c'est une
version pleine et non une pré-version.
2026-08-17 10:10:23 +02:00
Sylvain Duchesne f4050b95c0 test(e2e): éprouvé contre un vrai broker, et ce qu'on y apprend
69 tests unitaires, trois rounds d'auto-critique, et rien n'avait jamais tourné
contre un vrai broker. Sept parcours, vingt vérifications, à travers
ng-e2e-helpers — rien de réimplémenté.

La forme d'écriture est ACCEPTÉE par oxigraph, établie en relisant le document
et non parce que la mise à jour n'a pas levé : après curation, readUnion rend
deux sujets, celui de l'index et celui de l'objet de Bob, l'entrée portant sa
valeur. C'était l'une des deux inconnues.

L'autre est REPRODUITE, et c'est un défaut : un openInbox qui échoue en cours de
createIndex laisse un document orphelin. L'appel rejette et ne rend rien, mais
le document existe dans le store public du propriétaire, porte le descripteur,
et refuse les dépôts. Le paquet n'expose pas openInbox, donc il ne peut ni le
réparer ni le supprimer — orphelin permanent. Injecté pour être atteint : rien
de ce que contrôle un appelant ne fait échouer un vrai openDocumentInbox.

Et quatre endroits où la suite unitaire prouve moins qu'elle ne l'annonce, tous
vérifiés. Le plus net : les treize tests d'adaptateur ne chargent JAMAIS le vrai
polyfill. Preuve dure — la copie installée avait perdu un fichier qu'importe
surface/inbox.ts, et 69 sur 69 passaient quand même. Aucun des deux côtés n'a
tort ; c'est l'affirmation « l'adaptateur fonctionne » qui n'était pas testée.
Rien n'a été affaibli, l'e2e est ce qui la teste enfin.

Les trois autres sont de la même nature — une doublure trop faible plutôt qu'un
code faux : ses NURI n'ont pas la forme réelle, elle n'écrit pas la machinerie
que le vrai document porte, et étant une seule Map elle ne peut par construction
jamais révéler un retard de cohérence.

Trois exécutions, 27/27 chacune, autour de 58 secondes, aucune reprise.
2026-08-17 10:02:13 +02:00
29 changed files with 2203 additions and 139 deletions
+5
View File
@@ -4,3 +4,8 @@ dist/
.DS_Store
bun.lockb
bun.lock
e2e/.dist/
*.ngw
# Per-developer contract access map — canonical identities are committed, local paths are not
.project/contracts.local.yaml
+11
View File
@@ -0,0 +1,11 @@
<!-- generated — edit .project/concepts/*/_overview.md vocabulary blocks, then run lint --fix; do not edit this file -->
## Project vocabulary — canonical terms: use VERBATIM in any language, marked `like this`
```text
«indexing»
`index` an ordinary public document that holds one entry per indexed object, plus its own field declaration (never: catalogue, registry, listing)
`deposit` a bare object reference left in an index document's inbox — open to anyone, and never an instruction (never: message, submission, request)
`curate` the owner resolving the references deposited on its index and adding what it can (never: process, ingest, sync)
`entry` what an index holds for one indexed object — its NURI and its value for the index's field (never: row, record, item)
```
+42
View File
@@ -0,0 +1,42 @@
---
type: overview
summary: An index is an ordinary public document that only ever grows — what this repo publishes to applications, and what it consumes from NextGraph
triggers:
keywords: [index, indexing, curate, curation, deposit, depositor, entry, descriptor, reference, only-grows]
paths:
- "src/**"
- "test/**"
- "e2e/**"
- "README.md"
vocabulary:
- term: index
gloss: an ordinary public document that holds one entry per indexed object, plus its own field declaration
not: [catalogue, registry, listing]
- term: deposit
gloss: a bare object reference left in an index document's inbox — open to anyone, and never an instruction
not: [message, submission, request]
- term: curate
gloss: the owner resolving the references deposited on its index and adding what it can
not: [process, ingest, sync]
- term: entry
gloss: what an index holds for one indexed object — its NURI and its value for the index's field
not: [row, record, item]
---
# indexing — an index built on top of NextGraph, and the boundaries around it
NextGraph has no indexing concept and will not grow one, so this is a construction **above** it, in its own repository, and the dependency runs one way only: this repo depends on `@ng-eventually/polyfill`, and the polyfill must never learn anything about indexing.
An index is an **ordinary document** in its creator's public store. What makes it an index is that an application references its NURI in its own source. Anyone may hand it a reference by depositing into its inbox; its owner resolves those references itself and adds what it finds. **An index only ever grows** — no removal was ever built, and none is planned.
## Roles at the repository boundary
This repo is a **provider** of `indexing-layer`, which the Festipod application consumes, and a **consumer** of two engagements published by `ng-eventually-js`: `polyfill-surface` and `ng-e2e-helpers`. Each pair lives in its own interface folder: the engagement is pulled and never hand-edited, our declaration beside it is ours to keep current.
Frictions are the main path for telling a provider what we need. They go in our `usage_` leaf, the signal goes out of band, and the entry is pruned once the engagement absorbs it.
## Read first
- `indexing-layer/contract_indexing-layer` — what an application may rely on from `@ng-helpers/indexing`.
- `polyfill-surface/usage_ng-helpers` — the exact polyfill entries this layer stands on, and what it had to build for want of them.
- `ng-e2e-helpers/usage_ng-helpers` — what our end-to-end suite calls, and the peer-dependency constraint it must respect.
@@ -0,0 +1,138 @@
---
type: contract
summary: The API @ng-helpers/indexing exposes to an application — creating an index, depositing references into it, reading it back; curating is not on it: it is what an index's inbox being processed does
---
# contract_indexing-layer — `@ng-helpers/indexing`
## Scope
This package builds an **index** on top of NextGraph: an ordinary public document that holds one entry per indexed object, keyed by that object's NURI and carrying its value for a single declared field.
It covers creating an index, handing one a reference to an object (open to anyone), and reading the entries back in order. Resolving those references and adding what can be added is covered too, but never as a call: it is what happens when the index's inbox is processed.
It does not cover NextGraph itself — documents, identity, sharing, inboxes, transport — all of which reach it through a port you supply. It does not cover search, filtering, pagination, or querying by anything but the index's own field. It **never removes anything**, anywhere — an engagement, not a missing feature.
### Deployment requirements
An application using this package must:
- have a NextGraph session already open under the identity it wants to act as, and build the port from it — `polyfillPort({ sessionId })`, where `sessionId` is what `@ng-eventually/polyfill`'s own `init(…)` hands its callback;
- **await `indexing(port)` once at startup and keep what it produces.** That handle is one identity's and is also that identity's connection: awaiting it is what curates, dropping it is what stops;
- reach a broker, since every operation here is a document read, a document write or an inbox deposit;
- **supply `@ng-eventually/polyfill` itself.** This package declares it a *peer*: the application names it among its own dependencies, and that copy must be the one its own code calls — that package requires exactly one instance of itself in an application, for reasons its own contract states;
- **hardcode the index's NURI in its own source, and be the creator's own application if the index is ever to fill.** One requirement, not two: nothing marks a document as an index, so the reference an application carries is the only way anyone reaches it, and its creator's connections are the only thing that curates it. An index nobody hardcodes is unreachable; one whose creator never comes back stays as it was, however many references it is handed.
One handle is one identity: the port carries a session, no call takes an identifier, and two users mean two handles.
**Obtaining it.** Not published to npm or any other host, and not built output: the entry point is TypeScript source, so whatever builds the application compiles it. `@ng-eventually/polyfill` arrives the same way. What this contract fixes is the version you pin and what you must provide alongside it.
## Surface
Full typed shape: the package's `types` entry, `@ng-helpers/indexing`. The load-bearing signatures:
```ts
// ── wiring: one handle, one identity, and that identity's connection ─────────
export function polyfillPort(options: PolyfillPortOptions): NextGraphPort;
export interface PolyfillPortOptions { readonly sessionId: string | number }
/** Produces this identity's handle — and before resolving, goes through the inbox of
* every index it owns, leaving each watched for as long as the handle lives. */
export function indexing(port: NextGraphPort): Promise<Indexing>;
// ── addressing (re-exported so you import them from here) ────────────────────
export type Nuri = `did:ng:${string}`;
export type NuriLike = Nuri | string;
export type { PrincipalId, UnionSubject, NextGraphPort, IncomingDeposit, ObjectResolution };
// ── the three acts an application performs ───────────────────────────────────
export interface Indexing {
/** Produces a new index in THIS identity's public store, its inbox open, and the NURI
* to hardcode. Any user may. `field` is the predicate an indexed object must carry,
* declared once and for good; an empty or blank one throws. */
createIndex(field: string): Promise<Nuri>;
/** Deposits a bare reference into the index's inbox. Open to ANYONE. Produces nothing:
* no receipt, and no inbox address is ever handed out. Throws if there is no inbox. */
refer(index: NuriLike, object: NuriLike): Promise<void>;
/** Produces the entries, ordered by value. Refuses a document that declares no index
* field rather than producing an empty list. Sugar over `readUnion([index])`. */
read(index: NuriLike): Promise<IndexEntry[]>;
}
// ── what an index holds, and what travels from a depositor to a curator ──────
export interface IndexEntry { readonly object: Nuri; readonly value: string }
export interface IndexDescriptor { readonly field: string }
export type IndexDeposit = Nuri; // the reference IS the whole payload
export function decodeReference(payload: unknown): Nuri | null; // untrusted input
// ── the IRIs, for a reader going straight to `readUnion` ─────────────────────
export const INDEX_FIELD: string; // on the index's own subject: the field it indexes by
export const ENTRY_VALUE: string; // on an entry: that object's value for the field
```
## Guarantees
**An index is an ordinary public document, and nothing marks it as one.** It lives in its creator's public store, so any reader opens it from the reference alone; its creator owns it, and any user may create one.
**What becomes of what was created: the index is curated at its creator's next connection, and on each deposit while the creator is connected.** Awaiting `indexing(port)` is that connection — it goes through the inbox of every index the identity owns, backlog and all, and leaves each watched, so a deposit made from then on is applied as it lands. There is nothing to call, schedule or configure, and no way to aim curating at one index. Only the owner could anyway: nobody else reads that inbox, and nobody else writes that document.
**The field is declared once, inside the document, and cannot be changed.** `createIndex` refuses an empty or blank one at the door: nothing here deletes, so an index created on a useless field is useless for good. Declaring it in the document rather than in an application's source stops two applications curating one index on two fields.
**`createIndex` opens the index's inbox itself, and brings the new index under observation.** Only the owner can open one, and creation is the one moment the owner is present; and since the search at connection ran before this document existed, what was just created is added to what the session watches.
**Depositing is open to anyone; writing is the owner's alone.** `refer` is a deposit into the index document's inbox — not a write — so a stranger can contribute to an index they do not own. The deposit is a **bare reference**: no operation, no index reference (the inbox address identifies the index), no copy of the indexed value. What the object itself says is what goes in.
**A BET, named as one.** That a document can have an inbox is aligned with NextGraph: a repository takes an inbox capability, at most one, driven by a real commit upstream. **What a deposit carries, and what processing one does, are ours.** Upstream's inbox content type declares `Link`, `Patch` and four others as bare names with no payload at all — reserved words, not shapes — and the only two kinds carrying data are unrelated to indexing; that set is closed, with no trait, table of handlers or hook. An index deposit therefore has a shape NextGraph has not defined. When upstream defines those variants, this layer moves with them, and a `major` is how you hear about it.
**An index ONLY EVER GROWS.** No call removes an entry, for anyone including the owner, and none is planned: this package cannot express a removal at all, and it was deliberately never built rather than left for later. Do not design around a future delete — the only answer to "this entry must go" is a fresh index.
**Curating is convergent and order-independent.** Deposits are never retired, so every run reads every deposit ever made to that index — linear in its history — and re-applying one lands on the same result; an already-indexed object is passed over. Neither the order references arrived in nor the number of notifications a burst produced changes anything: runs on one index never overlap, and an arrival during a run earns exactly one more run after it.
**None of this can deny you anything.** A session that could not read its public store, watch an index, or go through one still hands you a working handle: reading an index and depositing into one never depended on that work. Every such failure is on this package's log stream, and so is every reference that could not be resolved — harmless is not the same as invisible. Nothing is lost either way: the deposits stay in their inbox for the next notification or connection.
**Reading is per-entry tolerant.** `read` returns entries ordered by value, ties broken on the object NURI, so two readers always see the same order. Values are compared **as strings** — an index whose field holds ISO-8601 dates comes out in chronological order. A subject that is not a NURI is passed over rather than thrown on, and only own properties are read: one stray triple cannot make every real entry unreadable.
**An entry carrying several values keeps the smallest, deterministically** — which two runs racing each other can produce, and which keeps the entry visible with every reader agreeing on it.
**The document's own declaration is read strictly for curating and leniently for reading.** `read` refuses a document that declares no field at all rather than answering "an empty index": an unreadable document and an empty one arrive as the same empty result, so an empty answer would be a failure wearing the shape of a fact — retry before concluding it is malformed. An index declaring SEVERAL fields stops being curated, loudly and permanently, and stays readable: picking one would leave a single list ordered by two properties, since entries already written are never re-read. That cannot be undone — curate into a fresh index.
**Reading needs nothing from this package.** An application that knows the NURI can call the polyfill's `readUnion([index])` and get one subject per indexed object, keyed by its NURI, plus the index's own subject declaring its field, which `read` drops. `INDEX_FIELD` and `ENTRY_VALUE` are published for that reader.
**Every inbox payload is untrusted.** Anyone may deposit anything; `decodeReference` returns `null` for whatever is not a reference, and such a payload is passed over rather than crashing the run.
## Non-guarantees
**Nothing reports curating to you.** No report, no outcome list, no callback: an application that cannot ask for it has nowhere to receive the result. A reference that could not be resolved is warned about on the log stream; an object carrying nothing for the field, one carrying several values, a self-reference and a payload that is not a reference are not reported at all. Reading the index is how you find out.
**No timing.** A deposit is in the index once its creator's session has been through that inbox; nothing says how long that takes or lets you wait, and if the creator is not connected it waits for as long as that lasts. There is no queue depth and no ordering between a deposit and a read.
**No refresh.** An already-indexed object is never re-read, so one whose value changes later keeps its original indefinitely.
**No private data.** Only objects the curator can open itself are indexed; one the owner cannot read is not added.
**A handle is one identity for its whole life, and nothing detaches the inboxes it watches.** An application that changes identity within one page must build a new handle and drop the old one, which goes on watching under a session that holds nothing.
**Connecting reads this identity's whole public store** — a store read plus one read per document, every time a handle is built, because nothing marks a document as an index. An index whose read did not answer in that moment is not found, silently, and is curated at the next connection instead.
**The narrow behaviours are open questions, not promises.** An object carrying nothing for the field is not added; one carrying several values is not added; a raced entry keeps the smallest value. Each is implemented in its narrowest form rather than generalised, and each may change.
**No stable error text.** What a throw or a log line reads is for a human. Do not parse it or branch on it.
**No cross-broker reach.** A NURI resolves for users of one broker.
**No depositor authentication or rate limit.** Anyone may deposit any number of payloads into any inbox.
## Change policy
**Semver, and majors are the normal case.** This layer sits on a polyfill itself converging on a NextGraph that does not ship yet, several of its behaviours are open questions above, and one part of it is a bet. Settling any of those narrows this surface, so the major number moves often — that frequency is the honest signal about this package, not an apology.
- **major** — an exported symbol is removed or renamed, **or** an existing call narrows: it throws where it returned, or reports a state you did not have to handle before. Settling an open question counts, and so does anything the bet forces. A signature change a caller must react to counts; one that only accepts more than before does not.
- **minor** — a symbol is added and nothing existing moves: a new read helper, a new optional option.
- **patch** — a fix that changes neither the exported surface nor anything under `## Guarantees`, throw text included.
**A tag says where it comes from.** A release cut on `main` carries a **full version** (`2.0.0`); work on a branch carries a **pre-release** of the version it heads for (`2.1.0-dev.3`), which sorts below it by construction, and between two pre-releases of the same version nothing is promised. Nothing you pinned is ever withdrawn: a pre-release keeps resolving once the full version appears alongside it. The tag is bare — `v2.0.0` — because this repository publishes exactly one engagement; should a second ever ship here, tags take the package name from then on (`indexing/v…`).
**`2.0.0` took the curating call off this surface, and made obtaining a handle asynchronous.** `Indexing.curate(index)` is gone, and with it `CurationReport`, `CurationOutcome` and `SkipReason`, which nothing published produces any more; `indexing(port)` now returns a promise, because obtaining a handle is what goes through this identity's inboxes and a caller has to be able to await it. Both are removals under the rule above, hence the major. The reason is not tidiness: a published `curate(index)` asked every application to decide who owns an index and when curating runs, and neither is an application's decision — the owner is the only one who can, and "when" is "whenever a deposit arrives, or has been waiting". **Migrating**: delete every call to `curate`, and `await` the `indexing(port)` you already make. If you read `CurationReport` for what happened, read the index instead, and the log for what did not resolve.
`1.0.1` and `1.0.0` keep resolving and neither is forced to upgrade, `1.0.0` having been uninstallable from anywhere but one working copy. This engagement is cut on `main`, so `2.0.0` is what you pin, and your `usage_` leaf anchors `against:` on that exact string — `against: @ng-helpers/indexing@2.0.0`.
There is no changelog file and no deprecation window: **the sections above are the release note.** Diff this leaf between two pulls, `## Guarantees` and `## Non-guarantees` before `## Surface`, because that is where a narrowing shows up first.
@@ -0,0 +1,49 @@
---
type: usage
summary: What this repo's end-to-end suite calls from ng-e2e-helpers, the peer-dependency constraint it must respect, and the gaps it fills itself
against: ng-e2e-helpers@1.0.0-dev.1
---
# usage_ng-helpers — `@ng-helpers/indexing`'s end-to-end suite on `ng-e2e-helpers`
The consumer is this repository's end-to-end suite: one run that builds a small indexing application, serves it, gets real people into it through the real broker, and drives depositing and curating between them.
Everything generic — the wallet lifecycle, the broker crossing, per-run profiles, bounds, the report shape, the recognition of the known browser failure modes — comes from the engagement and is **used, never reimplemented**. What is specific to this repository is the page that carries our application and the name its runs mint, and nothing else.
## Consumed surface
**Bounds**`within`, `armSuiteDeadline`, `closeQuietly`, `firstLine`.
**Measurement**`measured`.
**Browser and profiles**`launchWatchedContext`, `closeContext`, `newPage`, and the type `RunProfile`.
**Wallet**`mintWalletProfile` and the type `WalletCredentials`. Each run mints its own.
**Broker crossing**`setupBrokerPage`.
**Serving**`serveOnEphemeralPort`, for the application bundle.
**Known failure modes**`browserTrouble`, wired as the suite's `diagnose`.
**Report**`declareSuite` and the type `Prerequisite`.
**Constants**`BROKER_ROUND_TRIP_MS`, `NEW_PAGE_MS`.
Everything else the engagement offers is NOT consumed here: the screen inventory and its types, `completeBrokerLogin`, `emptyProfileContext`, `importWalletFile`, `exportWalletBytes`/`exportWalletFile`, `mintWalletBytes`, `mintWalletProfileKeepingContext`, `createWalletInContext`, `newRunProfile`, `isAlive`, `browserLost`, `lossDeclared`, `enclosingBound`, `frameTrouble`, the exported error classes, and the remaining `*_MS` constants. It is safely evolvable as far as this suite is concerned.
## Constraints
**The browser types are DERIVED from the helpers, never imported from `playwright` here.** The engagement declares Playwright a peer dependency, and this package reaches it as symlinked files — so TypeScript resolves the helpers' `playwright` from where those files really live. Importing the driver in this repository as well produced two structurally different copies of `BrowserContext`, and a context this suite had opened could not be handed back to the helper that opens contexts. Taking the types from the calls that return them leaves exactly one set, and a version skew can no longer express itself as a type error in code that is correct.
**One wallet per run, minted, never carried.** The run's wallet name is stable and its identity is not; nothing survives a run, and no result depends on a previous one.
**Nothing generic is reimplemented here.** Where a helper exists, it is called. That is a standing rule for this suite, not a preference — the crossing alone has cost days of misdiagnosis upstream, and a local copy of it would not carry those lessons.
**Every wait is entered bounded.** No page or frame operation runs outside `within` or a helper that bounds it itself.
## Frictions
**Nothing bounds a call into the application iframe.** `frame.evaluate` carries no timeout of its own, so this suite wraps every bridge call itself. The engagement offers no way to obtain that bound, so each consumer re-derives the same wrapper — and the derivation is not free: the calls this suite reached for outside its own wrapper are exactly the ones that can still hang it. A bounded `evaluate` here would delete the wrapper and close the gap in one move.
**"Measured and bounded" is one intent and two calls.** Sizing a bound from its own measurement is the discipline the engagement itself prescribes, yet every step in this suite has to compose `measured(what, ms, (bound) => within(what, bound, task))` by hand. Two consumers writing the same three-line helper is the tell that the pair belongs on the engagement.
@@ -0,0 +1,59 @@
---
type: usage
summary: The nine polyfill entries the indexing layer stands on, the constraints it holds itself to, and what it had to build for want of a published helper
against: "@ng-eventually/polyfill@1.0.0-dev.1"
---
# usage_ng-helpers — `@ng-helpers/indexing` on `polyfill-surface`
The consumer is the indexing layer: an index is an ordinary public document, contributions reach it through its inbox, and its owner's session curates it by going through that inbox. NextGraph has no indexing concept, so nothing of what a deposit *means* belongs upstream — the polyfill's inbox stays generic and carries opaque payloads, and this layer decides what they say.
The whole runtime dependency passes through **one file**, the adapter that builds our `NextGraphPort`. Everything else in the package is written against that port, so a change to the engagement breaks exactly one file and nothing else. The types are imported type-only, so they are literally the published ones rather than a copy that can drift.
## Consumed surface
**Placement**`storeRegistry.createEntityDoc("public")`, for the index document itself; `storeRegistry.listMyEntityDocs("public")`, to find again, at each connection, the indexes this identity owns; `storeRegistry.openDocumentInbox(doc)`, at creation to open one and afterwards to resolve its address.
**Reading**`readUnion([doc])` and the type `UnionSubject`. Used both for the index document and for resolving a deposited reference.
**Writing**`docs.sparqlUpdate(sessionId, update, anchor)`, with the document named ONCE as the anchor so the statement carries no `GRAPH <…>` wrapper. It is the only write this package makes, and it is always an `INSERT DATA` of literal triples.
**Inbox**`inbox.postToDocument(doc, { payload })` for depositing, `inbox.readForDocument(doc)` for the owner going through it, `inbox.watch(address, onDeposits)` so that a deposit made while the owner is connected is applied as it lands, and the shape of `Deposit` (`from` / `payload` / `ts`), which our `IncomingDeposit` mirrors.
**Types**`Nuri`, `NuriLike`, `PrincipalId`, `UnionSubject`.
**Bootstrap, in the end-to-end application only**`configure`, this package's `init` (for the `sessionId` its callback delivers), and `ensureIdentity`.
Everything else on the engagement is offered and NOT consumed: `watchShape`, `useShape`, `subscribeDoc`/`subscribeDocs`, `docs.docCreate`, `docs.sparqlQuery`, `storeRegistry.resolveScopeGraph`/`resolveWriteGraph`, `inbox.share`/`post`/`read`/`readSynced`/`readSyncedForDocument`/`processInbox`, `ng`, `initNg`. It is safely evolvable as far as this layer is concerned.
## Constraints
**One port is one identity.** The polyfill's session is one user's and no call takes an identifier, so an `Indexing` handle is one person's. Two users mean two handles — which is also what keeps our multi-actor tests honest: a depositor obtains the index NURI the way an application does, never through a shared variable.
**`sessionId` is relayed, never converted.** We carry it at the engagement's own `string | number` and hand it back untouched.
**The write is add-only, and structurally so.** There is no delete builder anywhere in this package, and the only statement it can compose is an anchored `INSERT DATA`. Our own tests execute that SPARQL against an engine that refuses anything else, so a removal is unrunnable rather than merely undetected. This constrains what we ask of the engagement: we need exactly one write primitive and no more.
**The inbox is opened at creation, from one place — and resolved, never opened, afterwards.** The engagement disclaims coalescing `openDocumentInbox` across pages, so the only call that can CREATE one is in `createIndex`, under its owner, at the moment the document is created. Every later call is on a document whose inbox already exists, where the same entry resolves the address instead; the engagement guarantees that idempotence within a page, which is the level our watching lives at.
**One session watches its own indexes, and nothing stops it.** `inbox.watch`'s unsubscribe is deliberately dropped: our watching lasts exactly as long as the identity is connected, which is what the engagement does with the inboxes it watches on its own account. It also means the inbox address never leaves the one adapter function that resolves it — everything above names a document.
**Our watching relies on subscriptions coexisting on one document.** `inbox.watch` opens a `subscribeDoc` on the inbox document, and the engagement already watches every inbox this identity may read. Before subscriptions coexisted, one of those two would have silenced the other with nothing raised anywhere. Verified in the resolved copy: the fan-out holds a set of listeners.
**Every payload out of an inbox is untrusted input.** Anyone may deposit anything, so nothing read from a deposit reaches a query before being checked; a non-reference is reported, never thrown on.
**An empty read is never treated as "empty".** Nothing in this layer reads `[]` from `readUnion` as "a valid index that happens to hold nothing" — the descriptor check refuses a document declaring no field, and that refusal aborts curation before a single write.
## Frictions
**The engagement does not say what `readUnion` does with a document it cannot read.** It says what it returns for a document it can, and it says that a rejection means "unknown, never absent" — but not whether an unreadable document inside the list comes back as a rejection or is swallowed into the result. We assume the worst (swallowed, therefore indistinguishable from empty) and code defensively around it. A sentence in `## Guarantees` settling this would replace a guess we are carrying in every read path.
**No published NURI type guard.** `## Guarantees` states plainly that no type guard is published, so this layer carries its own — and it needs one, because a NURI arrives here from an untrusted inbox deposit and must be checked before it can be written between angle brackets. The check we wrote is a guess at what the engagement considers a valid `Nuri`, and a wrong guess is either a rejected legitimate reference or an injected one.
**No published escaping helpers.** The engagement lists no `escapeIri`/`escapeLiteral`, and we compose SPARQL against `docs.sparqlUpdate` — so this package carries its own escaping rather than reach into the provider's internals. That is a security-relevant duplication of something the provider certainly already has: two implementations of the same rule, one of which is not the one the provider tests.
**Watching a document's inbox needs an address, and the only call that hands one out is the one that creates one.** The engagement is explicit that an application never resolves an inbox address, and it publishes `inbox.watch(address, …)` with no `watchForDocument(doc, …)` beside `readForDocument`. So the one thing an owner cannot avoid — being told about deposits on its own document — is also the one place this layer must hold an address, obtained from `openDocumentInbox`, whose other job is to create. A `watchForDocument(doc, onDeposits)` would close that gap and keep the "never resolve an address" rule whole.
**The version declared and the version the guarantees describe disagree.** The resolved copy's `package.json` says `1.0.0-dev.1`, while the engagement dates the continuous inbox observation and the coexisting subscriptions to `1.0.0-dev.2`. The code has them, so nothing is broken; but a pin cannot express what we actually depend on, and `against:` above names the string we resolve rather than the one the guarantees belong to.
**No published way to write triples above the raw SPARQL primitive.** `docs.sparqlUpdate` is the level we had to align on, which is why the two frictions above exist at all. A published "add these triples to this document" would remove the query composition, the escaping and the NURI validation from this layer in one move.
+35
View File
@@ -0,0 +1,35 @@
# Inter-repo contracts. `publish:` is this project's engagement toward its consumers —
# listing a leaf here IS the act of publishing it; an unlisted `contract_` leaf is a draft.
#
# This project is BOTH a provider and a consumer, and the two faces are isolated at document
# level: `indexing-layer` never mentions what we consume, and our `usage_` leaves never
# mention what we promise. What backs a published guarantee is internal doctrine.
#
# PROVIDES indexing-layer (concepts/indexing/indexing-layer/)
# consumer: the Festipod application, in its own repo. It has not declared a
# usage leaf, and we do not author one on its behalf — an interface with no
# declared consumer simply runs in the one-document mode.
#
# CONSUMES polyfill-surface and ng-e2e-helpers, both published by `ng-eventually-js`.
# We pull each engagement and author the `usage_ng-helpers.md` beside it.
#
# `pullFrom:` values are CANONICAL remote identities, because this file travels with the
# branch. Per-developer local access lives in `.project/contracts.local.yaml`, which is
# gitignored and must never be committed.
publish:
# paths are relative to `.project/`
indexing-layer: concepts/indexing/indexing-layer/contract_indexing-layer.md
consume:
- contract: polyfill-surface
type: git
pullFrom: git@gitea.reconnexion.apps.gueraud.net:Reconnexion/ng-eventually.git/.project/concepts/app-contract/polyfill-surface/contract_polyfill-surface.md
ref: caps-p1a-and-virtual-user-boundary
into: concepts/indexing/polyfill-surface/
- contract: ng-e2e-helpers
type: git
pullFrom: git@gitea.reconnexion.apps.gueraud.net:Reconnexion/ng-eventually.git/.project/concepts/e2e-harness/ng-e2e-helpers/contract_ng-e2e-helpers.md
ref: caps-p1a-and-virtual-user-boundary
into: concepts/indexing/ng-e2e-helpers/
+3
View File
@@ -0,0 +1,3 @@
## Project vocabulary (always loaded)
@.project/VOCABULARY.md
+15 -5
View File
@@ -23,6 +23,8 @@ That boundary is held by one file. `src/polyfill-adapter.ts` is the only place t
**A deposit is a bare reference. Nothing else.** It states no claim and gives no instruction: no operation, no index reference (the inbox address already identifies the index), no copy of the indexed value. When the owner curates, it resolves the reference and opens the object itself — which it can, because indexing is limited to public data for now. What the object says is what goes in. This is the shape NextGraph already uses upstream, where a `SocialQueryRequest` carries a reference to an RDF definition and the recipient composes its own update; a payload carrying an operation would be a licence for anyone to rewrite someone else's document.
**Curating is not an act an application performs.** There is no `curate` to call, and there never will be: it would ask an application to decide who owns an index and when curation runs, and neither is its decision. Curation is what happens when the index's inbox is processed — **at its creator's next connection, and on each deposit while the creator is connected**. `await indexing(port)` IS that connection: it goes through the inbox of every index that identity owns, and leaves each one watched. `src/observation.ts` is where that lives.
**Reading needs nothing new.** An application that knows the NURI calls `readUnion([indexNuri])` and gets the entries as subjects. `Indexing.read` is sugar over exactly that, dropping the index's own declaration subject.
## An index only ever grows
@@ -47,20 +49,22 @@ The lesson is worth keeping: **"nothing removes" is a claim about the write path
What enforces the write half is no longer a pattern over source. `test/adapter.test.ts` runs the real adapter on `test/fake-polyfill.ts`, an in-memory polyfill whose SPARQL is **executed** by an engine that understands one statement — an anchored `INSERT DATA` of literal triples — and refuses everything else. A removal is therefore not *detected*, it is **unrunnable**: `DELETE WHERE …` fails on the first keyword, a second statement smuggled after the closing brace fails on the trailing text, a keyword hidden inside a literal stays inside the literal because a parser tokenises where a regex only matches, and splitting the keyword across concatenated strings buys nothing, since it is one string by the time it arrives. The regex over `src/` in `test/units.test.ts` stays as a cheap tripwire that names the file early; it is not the proof.
**A failed resolve is still a failure, and still surfaces.** Harmless is not the same as invisible. Every reference that could not be resolved comes back as an `unresolved` outcome in the curation report and is warned about — a failure that looks exactly like a normal outcome teaches nobody anything.
**A failed resolve is still a failure, and still surfaces.** Harmless is not the same as invisible. Nobody asked for the run, so there is nobody to hand a report to: every reference that could not be resolved is warned about on this package's log stream, and so is a run, a watch or a store read that could not happen at all. None of them denies the application anything — reading an index and depositing into one never depended on that work, and the deposits stay in their inbox for the next connection.
## Open questions
Deliberately not settled. Each is implemented in its narrowest form and reported rather than generalised.
- **An object that carries nothing for the index's field.** Narrow behaviour: it is not added, and reported as `skipped: "no-field"`. There is no key to index it by, and inventing one — a placeholder, the deposit's timestamp — would put something in the index that the object does not say. Whether it should instead be indexed under an absent key, or refused louder, is open.
- **An object that carries several values for the field.** Not added, reported as `skipped: "several-values"`. Which of them the entry would hold has not been decided.
- **An object that carries nothing for the index's field.** Narrow behaviour: it is not added, and nothing says so. There is no key to index it by, and inventing one — a placeholder, the deposit's timestamp — would put something in the index that the object does not say. Whether it should instead be indexed under an absent key, or refused louder, is open.
- **An object that carries several values for the field.** Not added either. Which of them the entry would hold has not been decided.
- **An already-indexed object is never re-read.** Curation skips it outright, so an object whose field value changes later keeps its original value in the index. Refreshing it would be a write nobody asked for, and it is the same question as removal.
- **Which value a raced entry should keep.** Two curation runs racing each other can leave an entry with two values; the smallest is chosen so that readers agree and the entry stays visible. That the entry must survive is settled; *which* of the two it should hold is not.
Note what this means, since the two points above are easy to read as one: an entry's value **can** change after it is made. Not by re-reading the object — that never happens — but because a *second* value arriving for the same entry can be the smaller one, and `read()` returns the smallest. Index an object at `2026-05-05`, let a raced run add `2026-01-01`, and `read()` answers `2026-01-01`. What never changes is the set of entries and the fact that each stays visible; the value one of them reports is settled by "smallest wins", not by arrival order.
- **How an index recovers from an ambiguous declaration.** Today it does not: curation refuses for good and the only way forward is a fresh index. Since nothing here removes anything, giving it a way back needs a mechanism that does not exist yet.
- **Deposits are never retired.** Every curation run sees every deposit ever made. That is affordable because re-applying one is a no-op, but it is linear in the history. How a curator retires an applied deposit is open — `inbox.processInbox` may be the answer, but its semantics are not published.
- **How an owner finds its own indexes.** Nothing marks a document as an index, so connecting reads that identity's whole public store and looks at each document. It is the only question the surface can ask, and it does not scale with a large store. A marker on the document, or a listing narrowed by shape, would both settle it — and both are decisions about what an index *is*.
- **Nothing stops a handle.** `inbox.watch`'s unsubscribe is dropped, so a session watches its own indexes until the page goes. An application that changes identity in one page has to drop the old handle and know that it goes on watching.
- **What an entry holds besides the object reference and the field value**, and **how several index kinds would coexist**, are both untouched.
## Layout
@@ -72,6 +76,8 @@ Deliberately not settled. Each is implemented in its narrowest form and reported
| `src/deposit.ts` | The deposit's shape: a bare reference |
| `src/index-document.ts` | An index's declaration and its entries |
| `src/curator.ts` | Resolving references and adding what is there |
| `src/observation.ts` | When that happens: this identity's indexes, caught up and watched |
| `src/coalescing.ts` | Runs a job, never twice at once, and grants exactly one more run |
| `src/indexing.ts` | The public surface, bound to one identity |
| `src/sparql.ts` | The one statement this package writes — no deletion exists |
| `test/fake-nextgraph.ts` | An in-memory NextGraph behind `NextGraphPort`, enforcing the polyfill's published guarantees |
@@ -80,12 +86,16 @@ Deliberately not settled. Each is implemented in its narrowest form and reported
## Depends on
`@ng-eventually/polyfill`, by local path (`file:../ng-eventually-js/packages/polyfill`), which expects that repository to sit beside this one.
`@ng-eventually/polyfill`, declared as a **peer** dependency: an application using this package supplies it, so exactly one copy of it exists in that application. That is a requirement of the polyfill itself, which keeps its state in the package — two copies mean two subscription registries and two current identities, and nothing detects it.
For this repository's own tests and typecheck it is *also* a `devDependency` by local path (`file:../ng-eventually-js/packages/polyfill`), which expects that repository to sit beside this one. A dev dependency is not installed by a consumer, so this local path never reaches one. `ng-e2e-helpers` is a `devDependency` by local path on the same expectation.
## Running it
```sh
bun install
npm install # or: pnpm install
bunx tsc --noEmit -p tsconfig.json
bun test
```
**`bun install` does not work in this repository** (checked with bun 1.3.9): bun resolves a mandatory peer dependency against the npm registry whatever local path provides it, and `@ng-eventually/polyfill` is published to no registry, so the install stops on `GET https://registry.npmjs.org/@ng-eventually%2fpolyfill - 404`. `npm install` and `pnpm install` both resolve it from the sibling checkout. `bun test` itself is unaffected — it is only the installer that cannot express this.
+91
View File
@@ -0,0 +1,91 @@
/**
* The contract between the application page and the suite that drives it.
*
* It is declared ONCE and imported by both sides — `indexing-app.ts` implements it,
* `run.ts` calls it — so a method that changes shape breaks the typecheck instead of
* failing at run time inside a browser, where the only symptom would be `undefined is
* not a function` three minutes into a broker crossing.
*
* Everything crossing `frame.evaluate` must be structured-cloneable, which is why every
* member below takes and returns plain strings, numbers and object literals. A `Nuri` is
* a template-literal string type upstream (`did:ng:${string}`), so it crosses as itself;
* it is declared `string` here because a value that has been through structured clone
* carries no proof of its shape, and pretending otherwise is how an unvalidated string
* ends up typed as a reference.
*/
import type { IndexEntry, UnionSubject } from "../src/index";
/** What the leak probe observed — see `run.ts`'s last journey. */
export interface BrokenInboxOutcome {
/** The message `createIndex` rejected with, or `null` if it did not reject. */
readonly rejected: string | null;
/** What `createIndex` returned, on the impossible branch where it did not reject. */
readonly returned: string | null;
/** The documents that appeared in this identity's public store despite the failure. */
readonly appeared: readonly string[];
}
/**
* The acts this application can perform — and ONLY acts an application can perform.
*
* There is no back door onto the library's internals here. The one method that is not
* something an application does (`createIndexWithBrokenInbox`) injects a failure and is
* named for it, because the alternative — leaving the question unanswered — is worse
* than a probe that says what it is.
*/
export interface IndexingBridge {
/** `connecting` → `ready`, or `failed`. */
status(): string;
/** Why the boot failed, or `null`. */
error(): string | null;
/** Who this page signed in as. */
whoami(): string;
/**
* The index this deployment was BUILT to contribute to, read off its own configuration.
*
* An index is an ordinary document; what makes it an index is that an application
* references its NURI in its own source (`src/indexing.ts`). This page is configured
* through its URL rather than through a compiled-in constant, which is the same thing
* one build step earlier — and it is how the reference reaches a SECOND identity
* without the suite handing it over through a variable no application would have.
*/
configuredIndex(): string | null;
/** Create an index in this identity's public store, indexing by `field`. */
createIndex(field: string): Promise<string>;
/** Publish a public document carrying one value for one predicate. */
publishObject(predicate: string, value: string): Promise<string>;
/** Hand the CONFIGURED index a reference to an object. Anyone may. */
referConfigured(object: string): Promise<void>;
/** Hand a NAMED index a reference — used where no identity boundary is crossed. */
referTo(index: string, object: string): Promise<void>;
/**
* Connect again — obtain a fresh `Indexing` handle, which is what a page load does.
*
* There is no curating act to drive: an index is curated at its creator's next
* connection and on each deposit while the creator is connected. This is the first
* of the two, driven deliberately so a journey has a point at which the catching
* up is over; the second needs nothing from anyone.
*/
reconnect(): Promise<void>;
/** The index's entries, ordered by value. */
read(index: string): Promise<IndexEntry[]>;
/** What a document literally holds, straight off `readUnion` — the write-form probe. */
readRaw(doc: string): Promise<UnionSubject[]>;
/** This identity's public documents. How an owner discovers a document it did not keep. */
listPublicDocs(): Promise<string[]>;
/**
* `createIndex` with its inbox step made to fail — everything else real, against the
* real broker. Answers whether a half-created index is left behind.
*/
createIndexWithBrokenInbox(field: string): Promise<BrokenInboxOutcome>;
}
declare global {
interface Window {
__indexing: IndexingBridge;
}
}
+86
View File
@@ -0,0 +1,86 @@
/**
* What is SPECIFIC to this repository in the end-to-end setup: the page that carries the
* application, and the name of the wallet its runs mint.
*
* Everything generic — the wallet lifecycle, the broker crossing, per-run profiles,
* bounds, the report shape, the recognition of the known browser failure modes — lives in
* `ng-e2e-helpers` and is used, never reimplemented. That package knows nothing about
* this one and must keep knowing nothing about it: it talks about NextGraph itself, so it
* outlives both the polyfill and this indexing layer.
*/
import { execSync } from "node:child_process";
import * as fs from "node:fs";
import * as path from "node:path";
import { fileURLToPath } from "node:url";
import {
mintWalletProfile,
serveOnEphemeralPort,
type RunProfile,
type WalletCredentials,
} from "ng-e2e-helpers";
const here = path.dirname(fileURLToPath(import.meta.url));
/**
* The throwaway credentials each run mints its own wallet with.
*
* A NAME, not an identity that survives: every run gets a profile of its own and mints
* this wallet into it, so two runs sharing the name share nothing else — which is what
* lets this suite run beside another repository's at the same time, against the same
* broker, without a lock. The password sits here in the clear because it opens a wallet
* that exists for the length of one run and is deleted with the profile holding it.
*/
export const WALLET: WalletCredentials = {
name: "ng-helpers-e2e",
password: "ng-helpers-e2e",
};
/** This run's physical user, in a profile of its own. */
export function mintRunWallet(suite: string): Promise<RunProfile> {
return mintWalletProfile(suite, WALLET);
}
/** `bun build` is a local bundle; a minute is already many times what it takes. */
const BUILD_MS = 60_000;
const ENTRY = path.resolve(here, "indexing-app.ts");
const BUNDLE_OUT = path.resolve(here, ".dist", "indexing-app.js");
export function buildApp(): void {
fs.mkdirSync(path.dirname(BUNDLE_OUT), { recursive: true });
execSync(`bun build ${ENTRY} --outfile ${BUNDLE_OUT} --bundle --format=esm`, {
stdio: "pipe",
cwd: path.resolve(here, ".."),
timeout: BUILD_MS,
});
}
/**
* Serve the application the way a deployment would.
*
* An unknown path 404s rather than answering with the page: a catch-all makes a request
* for a file nobody serves look like a perfectly good download, and hides exactly the
* kind of mistake a served asset can carry.
*/
export function serveApp(): Promise<{ url: string; close: () => void }> {
const bundle = fs.readFileSync(BUNDLE_OUT, "utf-8");
const html =
`<!DOCTYPE html><html lang="en"><head><meta charset="utf-8">` +
`<title>ng-helpers indexing — e2e</title></head><body>` +
`<script type="module" src="/indexing-app.js"></script></body></html>`;
return serveOnEphemeralPort((req, res) => {
const route = (req.url ?? "/").split("?")[0];
if (route === "/indexing-app.js") {
res.writeHead(200, { "Content-Type": "application/javascript; charset=utf-8" });
res.end(bundle);
} else if (route === "/" || route === "/index.html") {
res.writeHead(200, { "Content-Type": "text/html; charset=utf-8" });
res.end(html);
} else {
res.writeHead(404, { "Content-Type": "text/plain; charset=utf-8" });
res.end("not served");
}
});
}
+219
View File
@@ -0,0 +1,219 @@
/**
* The application the end-to-end suite drives — written the way a consumer of
* `@ng-helpers/indexing` writes one, and nothing more.
*
* ── Why an application and not a bag of library calls ──────────────────────
* The 80 unit tests in `test/` run against a fake this repository wrote. They prove the
* indexing RULES are consistent; they cannot prove that NextGraph does what the fake
* pretends, because the fake is the thing being asked. This page closes that gap by
* putting the real broker underneath: it imports `@ng-eventually/polyfill` for real,
* crosses the real broker, and calls `indexing(polyfillPort(...))` exactly as an
* application would.
*
* It reaches nothing private. Every import below is a published entry — of the polyfill
* (`configure`, `ensureIdentity`, `init`, `readUnion`, `storeRegistry`) or of this
* package (`indexing`, `polyfillPort`). If something here is awkward, it is awkward for
* every consumer, which is the second reason to write it this way.
*
* ── The one thing here no application does ─────────────────────────────────
* `createIndexWithBrokenInbox` injects a failure into the inbox step of `createIndex`.
* That is a probe, it is named for what it is, and it exists because the question it
* answers — does a failed `openInbox` leave a document behind? — cannot be reached from
* outside: nothing a caller controls makes a real `openDocumentInbox` fail on demand.
* Everything around the injection is real, including the broker and the document.
*/
import {
configure,
ensureIdentity,
init,
readUnion,
storeRegistry,
type Nuri,
type UnionSubject,
} from "@ng-eventually/polyfill";
import { ng as realNg, init as realInit } from "@ng-org/web";
import { indexing, polyfillPort } from "../src/index";
import type { IndexEntry, Indexing, NextGraphPort } from "../src/index";
import type { BrokenInboxOutcome, IndexingBridge } from "./bridge";
// ── bootstrap: the one polyfill-era call, then the SDK-shaped ones ──────────
//
// `sharedWallet` is declared because the access gate wants somewhere to point when it
// has to render, and never used: this suite always enters through the broker's redirect,
// where the wallet is already open in the run's profile. Nothing is served at that path.
configure({
ng: realNg,
useShape: () => undefined, // this application reads through `readUnion`, not the ORM
init: realInit,
sharedWallet: { fileUrl: "/wallet-never-served.ngw", password: "" },
});
// The library's `init`, not the injected one: it settles the identity BEFORE handing the
// page to the broker, so the round-trip leaves with `?ng-id=` in the address it carries.
// The callback is this application's own business — it keeps the session because
// `polyfillPort` takes a session id, exactly as the real SDK's primitives do.
const sessionReady = new Promise<{ session_id: string }>((resolve) => {
init(
(event: { status: string; session?: { session_id: string } }) => {
if (event.status === "loggedin" && event.session) resolve(event.session);
},
true,
[],
);
});
// ── this application's state ───────────────────────────────────────────────
const state: { status: string; error: string | null; who: string } = {
status: "connecting",
error: null,
who: "",
};
let api: Indexing | null = null;
let port: NextGraphPort | null = null;
/** The index this deployment contributes to, read off its own configuration. */
function configuredIndex(): string | null {
return new URLSearchParams(window.location.search).get("index");
}
async function boot(): Promise<void> {
// One await, and it covers everything: the identity settles, the connection work runs,
// and the identity comes back. The application keeps it only to show it.
state.who = await ensureIdentity();
const session = await sessionReady;
port = polyfillPort({ sessionId: session.session_id });
// One await, and curation is part of it: obtaining the handle processes the inboxes
// of the indexes this identity owns and leaves them watched. This application never
// curates anything, and has nothing to call if it wanted to.
api = await indexing(port);
state.status = "ready";
}
void boot().catch((e: unknown) => {
state.status = "failed";
state.error = String((e as Error)?.message ?? e);
});
/** The library, once the page is up. Throws with the boot's own reason if it is not. */
function ready(): Indexing {
if (api === null) {
throw new Error(`[e2e] the application is not ready (${state.status}): ${state.error ?? "still connecting"}`);
}
return api;
}
function readyPort(): NextGraphPort {
if (port === null) {
throw new Error(`[e2e] the application is not ready (${state.status}): ${state.error ?? "still connecting"}`);
}
return port;
}
/**
* Wait for a document to appear in this identity's public store.
*
* A store listing is a read like any other, and a document written a moment ago is not
* owed to be in it instantly. Polling is therefore what an owner would actually do, and
* it is bounded: an empty answer at the end is evidence, not a hang.
*/
async function publicDocsAfter(
before: ReadonlySet<string>,
budgetMs: number,
): Promise<readonly string[]> {
const deadline = Date.now() + budgetMs;
let appeared: readonly string[] = [];
for (;;) {
const now = await storeRegistry.listMyEntityDocs("public");
appeared = now.filter((d) => !before.has(d));
if (appeared.length > 0 || Date.now() >= deadline) return appeared;
await new Promise((r) => setTimeout(r, 500));
}
}
// ── the acts ───────────────────────────────────────────────────────────────
const bridge: IndexingBridge = {
status: () => state.status,
error: () => state.error,
whoami: () => state.who,
configuredIndex,
async createIndex(field: string): Promise<string> {
return ready().createIndex(field);
},
/**
* Publish a public document carrying one value for one predicate.
*
* It goes through the SAME primitive the curator writes an entry with
* (`addLiteralProperty`), with the document as its own subject. That makes it the
* CONTROL for the write-form question: if this round-trips and an index entry does
* not, the difference is the foreign subject and nothing else.
*/
async publishObject(predicate: string, value: string): Promise<string> {
const p = readyPort();
const doc = await p.createPublicDocument();
await p.addLiteralProperty(doc, doc, predicate, value);
return doc;
},
async referConfigured(object: string): Promise<void> {
const index = configuredIndex();
if (index === null) {
throw new Error("[e2e] this application was not configured with an index reference");
}
await ready().refer(index, object);
},
async referTo(index: string, object: string): Promise<void> {
await ready().refer(index, object);
},
async reconnect(): Promise<void> {
api = await indexing(readyPort());
},
async read(index: string): Promise<IndexEntry[]> {
return ready().read(index);
},
async readRaw(doc: string): Promise<UnionSubject[]> {
return readUnion([doc]);
},
async listPublicDocs(): Promise<string[]> {
const docs: Nuri[] = await storeRegistry.listMyEntityDocs("public");
return [...docs];
},
async createIndexWithBrokenInbox(field: string): Promise<BrokenInboxOutcome> {
const p = readyPort();
const before = new Set<string>(await storeRegistry.listMyEntityDocs("public"));
// Everything real except the inbox step. The failure is injected at the exact moment
// the question is about: after the document exists and carries its descriptor, before
// anyone can deposit into it.
const broken = await indexing({
...p,
openInbox: async (): Promise<void> => {
throw new Error("[e2e] injected: the inbox could not be opened");
},
});
let rejected: string | null = null;
let returned: string | null = null;
try {
returned = await broken.createIndex(field);
} catch (e: unknown) {
rejected = String((e as Error)?.message ?? e);
}
return { rejected, returned, appeared: await publicDocsAfter(before, 15_000) };
},
};
window.__indexing = bridge;
+619
View File
@@ -0,0 +1,619 @@
/**
* `@ng-helpers/indexing` against the REAL broker.
*
* ── What this suite is for ─────────────────────────────────────────────────
* The unit suite proves the indexing rules are consistent with a fake this repository
* wrote. It cannot prove NextGraph behaves the way that fake pretends, because the fake
* is the very thing in question. Two claims in particular had never met a broker:
*
* 1. **The write form.** An entry is a triple whose SUBJECT is another document — the
* indexed object — written into the index document's anchored default graph. The
* polyfill's own suites only ever write a document's own subject into itself, so
* nothing had ever asked oxigraph whether a FOREIGN subject survives the round trip.
* `publishObject` here writes the self-subject form with the same primitive, which
* makes it the control: if one round-trips and the other does not, the difference is
* the foreign subject and nothing else.
*
* 2. **A half-created index.** `createIndex` creates a document, writes its descriptor,
* then opens its inbox. If the last step fails the caller gets an exception and no
* reference — but the document exists. The last journey injects that failure and
* asks the broker what was left behind.
*
* ── Two identities, and how the index reference reaches the second ─────────
* The whole point of an index is that STRANGERS contribute to it. So Bob must reach
* Alice's index — and he must reach it the way an application would, not through a
* variable in this file. An index is an ordinary document whose NURI an application
* references in its own source (`src/indexing.ts`), so Bob's page is CONFIGURED with it,
* through its URL: one build step earlier, that is a compiled-in constant. What must
* never happen — and does not happen here — is an inbox address crossing the identity
* boundary through a channel no deployment has.
*
* ── Reading a failure ──────────────────────────────────────────────────────
* A named deadline, or a message `ng-e2e-helpers` recognises as a browser or frame
* failure, is the HOST. A failed check carrying an unexpected value is this code. The
* report says which, and the run is repeated rather than anything being loosened.
*/
import {
BROKER_ROUND_TRIP_MS,
NEW_PAGE_MS,
armSuiteDeadline,
browserTrouble,
closeContext,
closeQuietly,
declareSuite,
firstLine,
launchWatchedContext,
measured,
newPage,
setupBrokerPage,
within,
type Prerequisite,
type RunProfile,
} from "ng-e2e-helpers";
import { ENTRY_VALUE, INDEX_FIELD } from "../src/index";
import { WALLET, buildApp, mintRunWallet, serveApp } from "./harness-page";
/**
* The browser types, taken from the helpers that RETURN them rather than imported from
* `playwright` directly.
*
* `ng-e2e-helpers` declares Playwright a PEER dependency — the consumer owns the version,
* because browser binaries have to match the driver. Its files reach this repository as
* symlinks, so TypeScript resolves its `playwright` from where those files really live,
* and importing the driver here as well produced two structurally different copies of
* `BrowserContext`: a context this file had opened could not be handed back to the helper
* that opens contexts. Derived, there is exactly one set of these types — whichever copy
* the helpers speak — and a version skew can no longer express itself as a type error in
* code that is correct.
*/
type BrowserContext = Awaited<ReturnType<typeof launchWatchedContext>>;
type Page = Awaited<ReturnType<typeof newPage>>;
type Frame = Awaited<ReturnType<typeof setupBrokerPage>>;
// ── the domain this suite indexes by ───────────────────────────────────────
//
// A date, so the suite exercises the case the package is built around: an index "by a
// date" is just an index whose field is a date predicate, and ISO-8601 sorts as a string.
const PUBLISHED_AT = "urn:ng-helpers-e2e:published-at";
/** A predicate an index does NOT curate on — for the object that carries nothing usable. */
const UNRELATED = "urn:ng-helpers-e2e:unrelated";
// ── bounds ─────────────────────────────────────────────────────────────────
//
// Sized to be generous rather than tight. A bound exists to turn a hang into a named
// failure; sized to the median it would instead fail on a slow-but-healthy broker, which
// is the one thing it must never do. The wall clocks of the three reported runs are the
// measurement these should be re-sized from.
/** The bridge appearing on the page — a bundle evaluating, no broker involved. */
const BRIDGE_UP_MS = 60_000;
/** `ensureIdentity` + the session: an identity settled and the connection work run. */
const READY_MS = 180_000;
/** One sign-in: a page, the broker round trip, and the application booting behind it. */
const SIGN_IN_MS = NEW_PAGE_MS + BROKER_ROUND_TRIP_MS + READY_MS;
/** One call across the bridge. The slowest here are curations, which round-trip per deposit. */
const BRIDGE_MS = 4 * 60_000;
/** One journey. The longest holds two sign-ins' worth of work behind it. */
const JOURNEY_MS = 10 * 60_000;
/** The whole run. A budget that cannot interrupt anything is not a budget. */
const SUITE_MS = 30 * 60_000;
// ── the report ─────────────────────────────────────────────────────────────
let actors: BrowserContext | null = null;
const { check, journey, finish } = declareSuite({
label: "ng-helpers indexing e2e",
journeyBound: JOURNEY_MS,
diagnose: async () => (actors === null ? null : browserTrouble("actors", actors)),
journeys: [
{
name: "Alice signs in and creates an index",
checks: [
"Alice signs in and the application knows who she is",
"creating an index answers with a document reference",
"the index document declares the field it indexes by",
],
},
{
name: "Bob signs in configured with Alice's index, and publishes an object",
checks: [
"Bob signs in, configured with the index his application contributes to",
"Bob publishes a public object carrying the indexed field",
"Bob's object reads back carrying the value he wrote",
],
},
{
name: "Bob hands the index a reference, and Alice's next connection curates it",
checks: [
"a stranger's deposit into the index's inbox reached its owner and became an entry",
"the entry is stored under Bob's object's own reference as its subject",
"the indexed value was read off Bob's object, and never travelled in his deposit",
],
},
{
name: "The index reads back, for its owner and for a stranger",
checks: [
"Alice reads exactly one entry, and it is Bob's object",
"Bob, who does not own the index, reads the same entry",
"curating a second time changes nothing, and the index still holds one entry",
],
},
{
name: "An object carrying nothing for the field is not indexed",
checks: [
"the unrelated object is not indexed, and the index still holds exactly one entry",
],
},
{
name: "An index whose inbox cannot be opened leaves a document behind",
checks: [
"createIndex refuses when the inbox cannot be opened",
"a document was nevertheless created in the owner's public store",
"the leaked document carries a descriptor but accepts no deposit",
],
},
{
name: "A hostile value crosses the round trip as one inert literal",
checks: [
"the object reads back the hostile value byte for byte",
"the index holds it as one entry, and its own descriptor is untouched",
],
},
],
});
/** A named step that is both measured and bounded — `evaluate` carries no timeout of its own. */
function step<T>(what: string, ms: number, task: () => Promise<T>): Promise<T> {
return measured(what, ms, (bound) => within(what, bound, task));
}
// ── an actor ───────────────────────────────────────────────────────────────
interface Actor {
readonly id: string;
readonly frame: Frame;
readonly page: Page;
}
/**
* Sign an actor in, and wait for its application to be up.
*
* `?ng-id=` is the one channel that survives the broker round trip (the access gate's
* resolution order). `index` rides the same query string when the actor's deployment is
* built to contribute to one.
*/
async function signIn(
ctx: BrowserContext,
appUrl: string,
id: string,
index: string | null,
): Promise<Actor> {
const opened: { page: Page | null } = { page: null };
const query =
`?ng-id=${encodeURIComponent(id)}` +
(index === null ? "" : `&index=${encodeURIComponent(index)}`);
try {
return await measured(`${id}'s sign-in`, SIGN_IN_MS, (bound) =>
within(`${id} to sign in`, bound, async () => {
const page = await measured(`a page for ${id}`, NEW_PAGE_MS, () => newPage(id, ctx));
opened.page = page;
page.on("pageerror", (e) => console.error(`[${id} pageerror]`, e.message));
page.on("console", (m) => {
if (m.type() === "error") console.error(`[${id} console]`, m.text());
});
const frame = await measured(`${id}'s broker round trip`, BROKER_ROUND_TRIP_MS, () =>
setupBrokerPage(page, `${appUrl}/${query}`, WALLET.password),
);
await waitReady(id, frame);
return { id, frame, page };
}),
);
} catch (e) {
if (opened.page !== null) {
await closeQuietly(`${id}'s abandoned sign-in page`, () => opened.page!.close());
}
throw e;
}
}
/** Wait for the application to be up, and say why with ITS reason when it is not. */
async function waitReady(id: string, frame: Frame): Promise<void> {
await step(`${id}'s application bundle`, BRIDGE_UP_MS, () =>
frame.waitForFunction(() => window.__indexing !== undefined, undefined, {
timeout: BRIDGE_UP_MS,
}),
);
await step(`${id}'s identity and session`, READY_MS, () =>
frame.waitForFunction(() => window.__indexing.status() !== "connecting", undefined, {
timeout: READY_MS,
}),
);
const status = await frame.evaluate(() => window.__indexing.status());
if (status !== "ready") {
const why = await frame.evaluate(() => window.__indexing.error());
throw new Error(`[e2e] ${id}'s application did not start (${status}): ${why ?? "no reason given"}`);
}
}
/**
* A journey cannot start without the actor it drives — reported as that, not discovered
* as a timeout on an innocent call.
*
* It takes a THUNK, not the actor: read eagerly, the value would be captured as it was
* before any sign-in happened, and every journey would report an actor that is standing
* right there as missing.
*/
function actorIsUp(id: string, actor: () => Actor | null): Prerequisite {
return () => (actor() === null ? `${id} never signed in` : null);
}
// ── the run ────────────────────────────────────────────────────────────────
async function main(): Promise<void> {
armSuiteDeadline("ng-helpers indexing e2e", SUITE_MS, () =>
finish("the suite exceeded its wall clock"),
);
console.log("[e2e] building the application...");
buildApp();
// This run's own physical user, in a directory of its own — so another repository's
// suite can drive the same broker at the same time without either noticing.
console.log("[e2e] minting this run's wallet...");
const wallet: RunProfile = await mintRunWallet("the indexing suite (e2e/run.ts)");
const stamp = Date.now().toString(36);
const ALICE = `alice-${stamp}`;
const BOB = `bob-${stamp}`;
let ctx: BrowserContext | null = null;
let closeServer: (() => void) | null = null;
try {
const served = await serveApp();
closeServer = served.close;
console.log(`[e2e] application served at ${served.url}`);
ctx = await launchWatchedContext("actors", wallet.dir);
actors = ctx;
let alice: Actor | null = null;
let bob: Actor | null = null;
let index: string | null = null;
let bobsObject: string | null = null;
const aliceIsUp = actorIsUp(ALICE, () => alice);
const bobIsUp = actorIsUp(BOB, () => bob);
const indexExists: Prerequisite = () =>
index === null ? "Alice never created an index" : null;
await journey({
name: "Alice signs in and creates an index",
run: async () => {
alice = await signIn(ctx!, served.url, ALICE, null);
const who = await alice.frame.evaluate(() => window.__indexing.whoami());
check("Alice signs in and the application knows who she is", who.length > 0, `who=${who}`);
index = await step("Alice creating an index", BRIDGE_MS, () =>
alice!.frame.evaluate((f) => window.__indexing.createIndex(f), PUBLISHED_AT),
);
check(
"creating an index answers with a document reference",
typeof index === "string" && index.startsWith("did:ng:"),
`index=${index}`,
);
// The descriptor's round trip — and the first thing the fake could have been
// lying about: the index document is found by an EXACT match on its own NURI as
// a subject, so a broker that returns a subject shaped differently breaks every
// read of every index.
const raw = await step("Alice reading the index document", BRIDGE_MS, () =>
alice!.frame.evaluate((d) => window.__indexing.readRaw(d), index!),
);
const self = raw.find((s) => s.subject === index);
const declared = self?.props[INDEX_FIELD] ?? [];
check(
"the index document declares the field it indexes by",
declared.length === 1 && declared[0] === PUBLISHED_AT && self?.graph === index,
`subjects=${raw.length} self=${self === undefined ? "(not found)" : "found"} ` +
`graph=${self?.graph} declared=${JSON.stringify(declared)}`,
);
},
});
await journey({
name: "Bob signs in configured with Alice's index, and publishes an object",
needs: [indexExists],
run: async () => {
bob = await signIn(ctx!, served.url, BOB, index);
const configured = await bob.frame.evaluate(() => window.__indexing.configuredIndex());
check(
"Bob signs in, configured with the index his application contributes to",
configured === index,
`configured=${configured}`,
);
bobsObject = await step("Bob publishing an object", BRIDGE_MS, () =>
bob!.frame.evaluate(
([p, v]) => window.__indexing.publishObject(p!, v!),
[PUBLISHED_AT, "2026-08-17T09:00:00Z"],
),
);
check(
"Bob publishes a public object carrying the indexed field",
typeof bobsObject === "string" && bobsObject.startsWith("did:ng:"),
`object=${bobsObject}`,
);
// The CONTROL for the write form: the same primitive, the document as its own
// subject. This is the shape the polyfill's own suites already exercise.
const raw = await step("Bob reading his own object", BRIDGE_MS, () =>
bob!.frame.evaluate((d) => window.__indexing.readRaw(d), bobsObject!),
);
const self = raw.find((s) => s.subject === bobsObject);
check(
"Bob's object reads back carrying the value he wrote",
(self?.props[PUBLISHED_AT] ?? []).includes("2026-08-17T09:00:00Z"),
`subjects=${raw.length} props=${JSON.stringify(self?.props ?? {})}`,
);
},
});
await journey({
name: "Bob hands the index a reference, and Alice's next connection curates it",
needs: [
aliceIsUp,
bobIsUp,
indexExists,
() => (bobsObject === null ? "Bob never published an object" : null),
],
run: async () => {
// Bob names his OWN object, and the index he was configured with. Nothing about
// the value travels: the deposit is the reference and nothing else.
await step("Bob depositing a reference", BRIDGE_MS, () =>
bob!.frame.evaluate((o) => window.__indexing.referConfigured(o), bobsObject!),
);
// NOBODY CURATES: there is nothing on the surface to call. Alice's page connects
// again — what a page load does — and her session processes the inboxes of the
// indexes she owns, this one among them.
await step("Alice connecting again", BRIDGE_MS, () =>
alice!.frame.evaluate(() => window.__indexing.reconnect()),
);
// THE WRITE FORM, answered. An entry is a triple whose subject is another
// document, written into this one's anchored default graph. "The write did not
// throw" is not the same claim as "oxigraph stored it": this reads it back.
const raw = await step("Alice reading the index document back", BRIDGE_MS, () =>
alice!.frame.evaluate((d) => window.__indexing.readRaw(d), index!),
);
const entry = raw.find((s) => s.subject === bobsObject);
// The deposit is proven ARRIVED by its only possible effect: nobody but Alice
// reads that inbox, so an entry for Bob's object means his deposit crossed the
// identity boundary and her session found it. That the post did not throw is a
// weaker claim entirely — it says the call returned, and nothing more.
check(
"a stranger's deposit into the index's inbox reached its owner and became an entry",
entry !== undefined,
`from=${BOB} subjects=${JSON.stringify(raw.map((s) => s.subject))}`,
);
check(
"the entry is stored under Bob's object's own reference as its subject",
entry !== undefined && Object.hasOwn(entry.props, ENTRY_VALUE),
`subjects=${JSON.stringify(raw.map((s) => s.subject))}`,
);
// The value never travelled: a deposit is the reference and nothing else, so its
// presence here means the curation read it off Bob's object itself.
check(
"the indexed value was read off Bob's object, and never travelled in his deposit",
(entry?.props[ENTRY_VALUE] ?? []).includes("2026-08-17T09:00:00Z"),
`entry=${JSON.stringify(entry?.props ?? {})}`,
);
},
});
await journey({
name: "The index reads back, for its owner and for a stranger",
needs: [aliceIsUp, bobIsUp, indexExists],
run: async () => {
const mine = await step("Alice reading the index", BRIDGE_MS, () =>
alice!.frame.evaluate((i) => window.__indexing.read(i), index!),
);
check(
"Alice reads exactly one entry, and it is Bob's object",
mine.length === 1 && mine[0]?.object === bobsObject,
`entries=${JSON.stringify(mine)}`,
);
// A public index is read by whoever holds its reference — including someone who
// owns neither it nor anything in it. This is the act an application performs.
const theirs = await step("Bob reading the index he does not own", BRIDGE_MS, () =>
bob!.frame.evaluate((i) => window.__indexing.read(i), index!),
);
check(
"Bob, who does not own the index, reads the same entry",
theirs.length === 1 && theirs[0]?.object === bobsObject,
`entries=${JSON.stringify(theirs)}`,
);
// Deposits are never retired, so every run sees every deposit again. Convergence
// is what makes that affordable.
await step("Alice connecting a second time", BRIDGE_MS, () =>
alice!.frame.evaluate(() => window.__indexing.reconnect()),
);
const still = await step("Alice reading the index again", BRIDGE_MS, () =>
alice!.frame.evaluate((i) => window.__indexing.read(i), index!),
);
check(
"connecting a second time changes nothing, and the index still holds one entry",
still.length === 1 && still[0]?.object === bobsObject,
`entries=${JSON.stringify(still)}`,
);
},
});
await journey({
name: "An object carrying nothing for the field is not indexed",
needs: [aliceIsUp, bobIsUp, indexExists],
run: async () => {
// PRESENT but carrying nothing for the field — which is a different answer from
// an object that cannot be read at all, and the reason this object carries a
// predicate rather than being empty: an empty document reads exactly like an
// unreadable one, and resolves as `unresolved`, not `skipped`.
const other = await step("Bob publishing an unrelated object", BRIDGE_MS, () =>
bob!.frame.evaluate(
([p, v]) => window.__indexing.publishObject(p!, v!),
[UNRELATED, "nothing to index by"],
),
);
await step("Bob depositing the unrelated reference", BRIDGE_MS, () =>
bob!.frame.evaluate((o) => window.__indexing.referConfigured(o), other),
);
await step("Alice connecting after the unrelated reference", BRIDGE_MS, () =>
alice!.frame.evaluate(() => window.__indexing.reconnect()),
);
const entries = await step("Alice reading the index once more", BRIDGE_MS, () =>
alice!.frame.evaluate((i) => window.__indexing.read(i), index!),
);
check(
"the unrelated object is not indexed, and the index still holds exactly one entry",
entries.length === 1 && !entries.some((e) => e.object === other),
`entries=${JSON.stringify(entries)}`,
);
},
});
await journey({
name: "An index whose inbox cannot be opened leaves a document behind",
needs: [aliceIsUp],
run: async () => {
const outcome = await step("Alice creating an index whose inbox fails", BRIDGE_MS, () =>
alice!.frame.evaluate(
(f) => window.__indexing.createIndexWithBrokenInbox(f),
PUBLISHED_AT,
),
);
check(
"createIndex refuses when the inbox cannot be opened",
outcome.rejected !== null && outcome.returned === null,
`rejected=${outcome.rejected} returned=${outcome.returned}`,
);
check(
"a document was nevertheless created in the owner's public store",
outcome.appeared.length === 1,
`appeared=${JSON.stringify(outcome.appeared)}`,
);
// What the leaked document IS: an index in every respect but the one that makes
// it usable. Alice found it in her own store — the only way anyone can, since
// `createIndex` threw its reference away.
const leaked = outcome.appeared[0];
if (leaked === undefined) {
check(
"the leaked document carries a descriptor but accepts no deposit",
false,
"no document appeared, so there was nothing to inspect",
);
return;
}
const raw = await step("Alice reading the leaked document", BRIDGE_MS, () =>
alice!.frame.evaluate((d) => window.__indexing.readRaw(d), leaked),
);
const declares = (raw.find((s) => s.subject === leaked)?.props[INDEX_FIELD] ?? []).includes(
PUBLISHED_AT,
);
const refused = await step("Alice trying to deposit into it", BRIDGE_MS, async () => {
try {
await alice!.frame.evaluate(
([i, o]) => window.__indexing.referTo(i!, o!),
[leaked, bobsObject ?? leaked],
);
return null;
} catch (e) {
return firstLine(e);
}
});
check(
"the leaked document carries a descriptor but accepts no deposit",
declares && refused !== null,
`declares=${declares} deposit=${refused ?? "(accepted)"}`,
);
},
});
// LAST, deliberately: if the escaping below turned out not to hold, the damage would
// be to this index, and every check above has already been taken.
await journey({
name: "A hostile value crosses the round trip as one inert literal",
needs: [aliceIsUp, bobIsUp, indexExists],
run: async () => {
// `src/sparql.ts` carries this package's OWN escaping, because the polyfill
// publishes none. Until now it had only ever been judged by a fake whose SPARQL
// reader was written from the same assumptions — a pair that agrees with itself
// proves nothing about oxigraph. This value closes every construct the escaping
// is responsible for: the literal's own quote, a backslash, the whitespace
// escapes, and a complete injected UPDATE that would empty the index if the
// quote ever escaped its literal.
const hostile =
'a "quoted" part, a \\ backslash, a\nnewline, a\ttab, ' +
'" } ; DROP ALL ; INSERT DATA { <urn:ng-helpers-e2e:pwned> <urn:ng-helpers-e2e:pwned> "';
const object = await step("Bob publishing a hostile value", BRIDGE_MS, () =>
bob!.frame.evaluate(
([p, v]) => window.__indexing.publishObject(p!, v!),
[PUBLISHED_AT, hostile],
),
);
const raw = await step("Bob reading the hostile object", BRIDGE_MS, () =>
bob!.frame.evaluate((d) => window.__indexing.readRaw(d), object),
);
const stored = raw.find((s) => s.subject === object)?.props[PUBLISHED_AT] ?? [];
check(
"the object reads back the hostile value byte for byte",
stored.length === 1 && stored[0] === hostile,
`stored=${JSON.stringify(stored)}`,
);
await step("Bob depositing the hostile reference", BRIDGE_MS, () =>
bob!.frame.evaluate((o) => window.__indexing.referConfigured(o), object),
);
await step("Alice connecting after the hostile reference", BRIDGE_MS, () =>
alice!.frame.evaluate(() => window.__indexing.reconnect()),
);
// Read the index document RAW: it must still declare its own field. An injected
// `DROP ALL` that had taken effect would show up exactly here, as a descriptor
// that is no longer there — and `read()` alone could not tell that apart from an
// ordinary failure.
const after = await step("Alice reading the index after the hostile entry", BRIDGE_MS, () =>
alice!.frame.evaluate((d) => window.__indexing.readRaw(d), index!),
);
const entry = after.find((s) => s.subject === object)?.props[ENTRY_VALUE] ?? [];
const descriptor = after.find((s) => s.subject === index)?.props[INDEX_FIELD] ?? [];
check(
"the index holds it as one entry, and its own descriptor is untouched",
entry.length === 1 && entry[0] === hostile && descriptor.includes(PUBLISHED_AT),
`entry=${JSON.stringify(entry)} descriptor=${JSON.stringify(descriptor)}`,
);
},
});
} finally {
if (ctx !== null) await closeContext("actors", ctx);
if (closeServer !== null) {
await closeQuietly("the application server", async () => closeServer!());
}
wallet.discard();
}
finish(null);
}
void main().catch((e: unknown) => {
console.error("[e2e] fatal:", (e as Error)?.stack ?? e);
finish(firstLine(e));
});
+9 -4
View File
@@ -1,23 +1,28 @@
{
"name": "@ng-helpers/indexing",
"version": "0.0.0",
"version": "2.0.0",
"private": true,
"type": "module",
"description": "An indexing layer built on top of NextGraph, via @ng-eventually/polyfill. An index is an ordinary public document; contributions reach it through its inbox; its owner curates it.",
"description": "An indexing layer built on top of NextGraph, via @ng-eventually/polyfill. An index is an ordinary public document; contributions reach it through its inbox; processing that inbox is what curates it, and that happens at its creator's connections.",
"main": "./src/index.ts",
"types": "./src/index.ts",
"exports": {
".": "./src/index.ts"
},
"dependencies": {
"@ng-eventually/polyfill": "file:../ng-eventually-js/packages/polyfill"
"peerDependencies": {
"@ng-eventually/polyfill": "*"
},
"devDependencies": {
"@ng-eventually/polyfill": "file:../ng-eventually-js/packages/polyfill",
"@ng-org/web": "0.1.2-alpha.13",
"@types/bun": "latest",
"ng-e2e-helpers": "file:../ng-eventually-js/packages/ng-e2e-helpers",
"playwright": "1.61.1",
"typescript": "^5.6.0"
},
"scripts": {
"test": "bun test",
"test:e2e": "bun run e2e/run.ts",
"typecheck": "bunx tsc --noEmit -p tsconfig.json"
}
}
+41
View File
@@ -0,0 +1,41 @@
/**
* Runs a job, and never runs it twice at once: an ask that arrives while it is
* running earns exactly ONE more run afterwards, however many arrive.
*
* A burst of deposits produces a burst of notifications, and each one means the same
* thing — "look at this inbox again". Curation reads the whole inbox every time, so
* a run started after the last deposit landed already covers every deposit before
* it: running once per notification would re-read the same inbox N times to reach
* the same place. One more run is enough, and one more run is also NECESSARY — a
* deposit that lands while a run is in flight may have arrived after that run read
* the inbox, and dropping the ask would leave it unprocessed until the next
* connection.
*
* `run` must not reject: this returns the caller's own promise and nothing here
* turns a rejection into a report. Its one caller wraps a failing run in its own
* reporting before handing it over.
*/
export function coalescing(run: () => Promise<void>): () => Promise<void> {
let inFlight: Promise<void> | null = null;
let asked = false;
async function drain(): Promise<void> {
try {
do {
asked = false;
await run();
} while (asked);
} finally {
inFlight = null;
}
}
return function ask(): Promise<void> {
if (inFlight !== null) {
asked = true;
return inFlight;
}
inFlight = drain();
return inFlight;
};
}
+15 -2
View File
@@ -117,8 +117,7 @@ export function entryValue(subject: UnionSubject): string | undefined {
* declaration above it turned ambiguous.
*/
export function assertIndexDocument(subjects: readonly UnionSubject[], index: Nuri): void {
const self = subjects.find((s) => s.subject === index);
if (valuesOf(self, INDEX_FIELD).length > 0) return;
if (declaresIndexField(subjects, index)) return;
throw new Error(
`${index} declares no index field (${INDEX_FIELD}) — either it is not an index, ` +
"or it could not be read: an unreadable document and an empty one are the same " +
@@ -126,6 +125,20 @@ export function assertIndexDocument(subjects: readonly UnionSubject[], index: Nu
);
}
/**
* The same question as `assertIndexDocument`, asked without an opinion on the
* answer: is this document an index?
*
* Separate because the two callers want opposite things from a "no". A reader
* naming a document it believes to be an index wants the refusal. A session
* looking through its own public store for the indexes it owns wants a plain
* `false`: most of what it looks at is not an index and never claimed to be.
*/
export function declaresIndexField(subjects: readonly UnionSubject[], doc: Nuri): boolean {
const self = subjects.find((s) => s.subject === doc);
return valuesOf(self, INDEX_FIELD).length > 0;
}
/**
* The field to CURATE on — and here it must be unambiguous.
*
+4 -4
View File
@@ -7,8 +7,9 @@
*
* An index is an ORDINARY document in its creator's public store. What makes it
* an index is that an application references its NURI in its own source. Anyone
* may hand it a reference by depositing into its inbox; its owner resolves those
* references itself and adds what it finds.
* may hand it a reference by depositing into its inbox; its creator's session
* resolves those references and adds what it finds see `observation.ts` for what
* makes that happen, and `curator.ts` for what it does.
*
* An index only ever grows see `curator.ts`.
*/
@@ -31,9 +32,8 @@ export type {
export { decodeReference } from "./deposit";
export type { IndexDeposit } from "./deposit";
// What an index holds, and what curating it reports
// What an index holds
export type { IndexDescriptor, IndexEntry } from "./index-document";
export type { CurationOutcome, CurationReport, SkipReason } from "./curator";
// The IRIs written into an index document, for a reader going straight to `readUnion`
export { ENTRY_VALUE, INDEX_FIELD } from "./vocabulary";
+34 -23
View File
@@ -1,6 +1,6 @@
import type { NextGraphPort, Nuri, NuriLike } from "./port";
import { asNuri } from "./nuri";
import { curate, type CurationReport } from "./curator";
import { observeOwnIndexes } from "./observation";
import {
assertIndexDocument,
entriesOf,
@@ -19,42 +19,39 @@ import {
export interface Indexing {
/**
* Creates an index, in THIS identity's public store; the creator owns it. Any
* user may create one.
* user may create one. Produces the index's NURI, and an inbox on it, open.
*
* `field` is the predicate an indexed object must carry, declared once, here.
* An index "by a date" is just an index whose field is a date predicate there
* is no separate kind of index, and the entries of such an index come out in
* chronological order because ISO-8601 sorts as a string.
*
* The returned NURI is what an application hardcodes in its own source: that
* reference is the only thing that makes this ordinary document an index, and
* the only way anyone reaches it.
* WHAT BECOMES OF IT is its creator's business, in both directions at once. The
* returned NURI is what an application hardcodes in its own source: that
* reference is the only thing that makes this ordinary document an index, and the
* only way anyone reaches it. And this session is what curates it from now on
* while it lasts, and again at its creator's next connection. An index has no
* life of its own: it lives through the application that names it and the
* creator who connects.
*/
createIndex(field: string): Promise<Nuri>;
/**
* Hands an index a reference to an object. Open to ANYONE it is a deposit in
* the index document's inbox, not a write.
* the index document's inbox, not a write. Produces nothing: an application names
* a document or a person, never an inbox, and there is no receipt to hold on to.
*
* The reference is the whole message: it claims nothing and instructs nothing,
* it just invites the index's owner to look. Call it when the object is
* created, and again whenever anyone notices the index may not have it yet
* including a third party. Nothing lands in the index until its owner curates.
* including a third party. What the deposit becomes is up to the index's creator:
* it enters the index the moment their session processes that inbox.
*/
refer(index: NuriLike, object: NuriLike): Promise<void>;
/**
* Resolves the references this index has received and adds what it can. Only
* the index's OWNER gets anything: nobody else reads its inbox, and nobody else
* may write it.
*
* Check the returned outcomes for `unresolved` those references were not
* added.
*/
curate(index: NuriLike): Promise<CurationReport>;
/**
* The index's entries, ordered by value.
* The index's entries, ordered by value. Refuses a document that declares no
* index field rather than producing an empty list.
*
* Sugar only. Reading an index needs NOTHING new from NextGraph: an application
* that knows the NURI can call `readUnion([indexNuri])` and get these entries as
@@ -64,7 +61,22 @@ export interface Indexing {
read(index: NuriLike): Promise<IndexEntry[]>;
}
export function indexing(port: NextGraphPort): Indexing {
/**
* This identity's handle, and its connection.
*
* Awaiting it processes the inbox of every index this identity owns the deposits
* that piled up while it was away and leaves those inboxes watched, so a deposit
* made from now on is curated as it lands. That is the whole of when curation
* happens; there is nothing to call, schedule or configure, and no way to aim it at
* one index.
*
* It never rejects for that work: an identity that could not be caught up still
* gets its handle, because reading an index and depositing into one do not depend
* on it. What went wrong is reported on this package's log stream.
*/
export async function indexing(port: NextGraphPort): Promise<Indexing> {
const observation = await observeOwnIndexes(port);
return {
async createIndex(field: string): Promise<Nuri> {
// Refused at the door, because a field cannot be corrected afterwards:
@@ -82,6 +94,9 @@ export function indexing(port: NextGraphPort): Indexing {
// its inbox, and this is the one moment the owner is here — so it is opened
// at creation rather than left for a later call to remember.
await port.openInbox(index);
// The store was searched before this document existed, so this session would
// otherwise ignore its own new index until the next connection.
await observation.include(index);
return index;
},
@@ -90,10 +105,6 @@ export function indexing(port: NextGraphPort): Indexing {
await port.depositTo(asNuri(index), asNuri(object));
},
curate(index: NuriLike): Promise<CurationReport> {
return curate(port, index);
},
async read(index: NuriLike): Promise<IndexEntry[]> {
const nuri = asNuri(index);
const subjects = await readIndexDocument(port, nuri);
+128
View File
@@ -0,0 +1,128 @@
import type { NextGraphPort, Nuri, NuriLike } from "./port";
import { asNuri } from "./nuri";
import { coalescing } from "./coalescing";
import { curate } from "./curator";
import { declaresIndexField, readIndexDocument } from "./index-document";
/**
* Curating an index is what happens when its inbox is processed. Nothing calls it.
*
* ## Why there is no `curate(index)` to call
*
* An application that could call it would have to answer two questions it has no
* business answering who owns the index, and when curation runs. Both answers are
* already fixed by the design: the owner is the only one who CAN (nobody else reads
* the inbox, nobody else writes the document), and "when" is "whenever a deposit
* arrives, or has been waiting". A published call would only let an application get
* those wrong.
*
* So the trigger is the identity's own session: obtaining an `Indexing` handle is a
* connection, and a connection processes what is waiting for it and keeps
* processing what arrives. That is the same shape the polyfill gives its own
* inboxes watched for as long as the identity is connected, backlog applied at
* connection and this layer holds itself to it.
*
* ## Only the owner, and the layer does not have to check
*
* Reading an index's inbox is refused to anyone else, so a session curates exactly
* the indexes it owns and could not do otherwise if it tried. Which is why the
* search below is a search of THIS identity's own public store.
*
* ## Nothing here denies anything
*
* A session that could not look for its indexes, could not watch one, or could not
* process one reports it and carries on: reading an index and depositing into one
* need none of this, and an index only ever grows, so a run that did not happen
* costs a deposit nothing it is still in the inbox, and the next notification or
* the next connection applies it. The polyfill states the same rule for its own
* inboxes: failing to apply one denies nothing.
*/
export interface IndexObservation {
/**
* Brings one more index under observation its inbox is processed now, and again
* on every deposit. For an index this session has just created: it did not exist
* when the store was searched, and its creator is right here.
*
* Asking twice for the same index changes nothing.
*/
include(index: NuriLike): Promise<void>;
}
/**
* Processes the inboxes of every index this identity owns, and keeps processing
* them. Resolves once the search is done and what it found has been caught up.
*/
export async function observeOwnIndexes(port: NextGraphPort): Promise<IndexObservation> {
const observed = new Set<Nuri>();
async function include(indexLike: NuriLike): Promise<void> {
const index = asNuri(indexLike);
if (observed.has(index)) return;
observed.add(index);
const processInbox = coalescing(async () => {
try {
await curate(port, index);
} catch (error) {
console.error(
`[ng-helpers/indexing] ${index}: its inbox could not be processed — ${String(error)}`,
);
}
});
// WATCH FIRST, then process. The other order has a gap: a deposit landing
// between the read and the watch is seen by neither, and waits for the next
// connection. This order overlaps instead of gapping — a deposit landing in
// between is processed twice — which costs nothing, because curation resolves
// the reference again and lands on the same result.
try {
await port.watchDeposits(index, processInbox);
} catch (error) {
console.error(
`[ng-helpers/indexing] ${index}: deposits into it will not be noticed until the next ` +
`connection — its inbox could not be watched: ${String(error)}`,
);
}
await processInbox();
}
const observation: IndexObservation = { include };
let mine: readonly Nuri[];
try {
mine = await port.listPublicDocuments();
} catch (error) {
console.error(
"[ng-helpers/indexing] this identity's indexes were not found, so none of them is being " +
`curated in this session — its public store could not be listed: ${String(error)}`,
);
return observation;
}
await Promise.all(
mine.map(async (doc) => {
if (await isAnIndex(port, doc)) await include(doc);
}),
);
return observation;
}
/**
* Does this document declare an index field? A document that could not be READ
* answers `false`, silently and on purpose.
*
* There is nothing to report: upstream hands a failed read back as an empty one, so
* "not an index" and "could not tell" are the same answer here, and warning about
* every ordinary document that did not answer would bury the failures that mean
* something. The cost is bounded by the invariant this package is built on an
* index only ever grows, so an index missed at this connection is curated at the
* next one, with its deposits still in its inbox.
*/
async function isAnIndex(port: NextGraphPort, doc: Nuri): Promise<boolean> {
try {
return declaresIndexField(await readIndexDocument(port, doc), doc);
} catch {
return false;
}
}
+24
View File
@@ -2,6 +2,7 @@ import { docs, inbox, readUnion, storeRegistry } from "@ng-eventually/polyfill";
import type {
IncomingDeposit,
NextGraphPort,
Nuri,
NuriLike,
ObjectResolution,
UnionSubject,
@@ -42,6 +43,12 @@ export function polyfillPort(options: PolyfillPortOptions): NextGraphPort {
return storeRegistry.createEntityDoc("public");
},
async listPublicDocuments(): Promise<readonly Nuri[]> {
// The same store `createPublicDocument` writes into, listed back: an index is
// an ordinary document there, and there is no narrower question to ask.
return storeRegistry.listMyEntityDocs("public");
},
async resolveObject(doc: NuriLike): Promise<ObjectResolution> {
// The decision itself lives in `resolution.ts`, where it is unit-tested:
// here it would be reachable only through a live broker, and an untested
@@ -92,5 +99,22 @@ export function polyfillPort(options: PolyfillPortOptions): NextGraphPort {
async readDeposits(doc: NuriLike): Promise<readonly IncomingDeposit[]> {
return inbox.readForDocument(asNuri(doc));
},
async watchDeposits(doc: NuriLike, onDeposits: () => Promise<void>): Promise<void> {
// The one call that hands out an inbox address, and the reason this is the only
// line in the package that holds one. It is idempotent within a page — the
// engagement says so, and says that firing one per component is supported — so
// asking again for a document whose inbox `createIndex` already opened resolves
// that inbox rather than adding a second.
const address = await storeRegistry.openDocumentInbox(asNuri(doc));
// `inbox.watch` hands back an unsubscribe, and this layer deliberately drops it:
// a session watches its own indexes for its whole life, exactly as the polyfill
// watches the inboxes it holds for as long as the identity stays connected.
//
// The callback is declared `void` upstream, so what it returns is ignored and the
// processing runs on its own. That is why it must never reject: nothing over
// there would catch it. It reports its own failures instead.
inbox.watch(address, onDeposits);
},
};
}
+31
View File
@@ -52,6 +52,18 @@ export interface NextGraphPort {
*/
createPublicDocument(): Promise<Nuri>;
/**
* Every document this identity has in its PUBLIC store. Backs onto
* `storeRegistry.listMyEntityDocs("public")`.
*
* It is how an owner finds its own indexes again after a page load, and it has to
* be a full listing because NOTHING marks a document as an index the index's own
* declaration, read from the document, is the only tell. Upstream throws rather
* than hand back a listing whose documents could not be opened, so an answer here
* is a listing, never a shrug.
*/
listPublicDocuments(): Promise<readonly Nuri[]>;
/** Resolves a reference deposited into an index: the two-state answer above. */
resolveObject(doc: NuriLike): Promise<ObjectResolution>;
@@ -114,4 +126,23 @@ export interface NextGraphPort {
* inbox, you may only READ your own. Backs onto `inbox.readForDocument(doc)`.
*/
readDeposits(doc: NuriLike): Promise<readonly IncomingDeposit[]>;
/**
* Calls back for as long as this session lives, every time something is deposited
* into this document's inbox. OWNER only, for the same reason `readDeposits` is:
* being told what landed in an inbox is reading it.
*
* Backs onto `inbox.watch(address, onDeposits)`, whose address comes from
* `storeRegistry.openDocumentInbox(doc)` the one call that hands one out, and
* which is idempotent within a page. The address never leaves this layer: a
* document is what everything above names, exactly as upstream intends.
*
* `onDeposits` takes nothing: what arrived is not read from the callback but from
* the inbox itself, which is re-read whole. That is not an omission a deposit is
* never an instruction, so the only thing a notification can say is "look again".
*
* Nothing stops it. This session watches its own indexes for as long as it lasts,
* which is what the polyfill does with the inboxes it watches on its own account.
*/
watchDeposits(doc: NuriLike, onDeposits: () => Promise<void>): Promise<void>;
}
+40 -17
View File
@@ -1,5 +1,6 @@
import { expect, test } from "bun:test";
import { indexing } from "../src/indexing";
import { curate } from "../src/curator";
import { ENTRY_VALUE, INDEX_FIELD } from "../src/vocabulary";
import type { NextGraphPort, Nuri } from "../src/port";
import { DESTRUCTIVE, blankLiterals, installFakePolyfill } from "./fake-polyfill";
@@ -85,25 +86,47 @@ async function publish(port: NextGraphPort, field: string, value: string): Promi
// --- the whole loop, through the real adapter -----------------------------
test("the real adapter carries the whole loop: create, publish, refer, curate, read", async () => {
const index = await as("alice", (alice) => indexing(alice).createIndex(FIELD));
const index = await as("alice", async (alice) => (await indexing(alice)).createIndex(FIELD));
const article = await as("bob", async (bob) => {
const object = await publish(bob, FIELD, "2026-07-08");
await indexing(bob).refer(index, object);
await (await indexing(bob)).refer(index, object);
return object;
});
const report = await as("alice", (alice) => indexing(alice).curate(index));
const report = await as("alice", (alice) => curate(alice, index));
expect(report.outcomes).toEqual([{ result: "indexed", object: article, value: "2026-07-08" }]);
const entries = await as("alice", (alice) => indexing(alice).read(index));
const entries = await as("alice", async (alice) => (await indexing(alice)).read(index));
expect(entries).toEqual([{ object: article, value: "2026-07-08" }]);
});
test("a deposit made while the owner is connected is curated as it lands, with nobody asking", async () => {
const seen = await as("alice", async (alice) => {
const api = await indexing(alice);
const index = await api.createIndex(FIELD);
// An owner may deposit into her own index: `refer` is open to anyone, and here it
// keeps both sides on one session, which is all this fake models at a time.
const object = await publish(alice, FIELD, "2026-09-09");
await api.refer(index, object);
// NOTHING CALLS CURATION. The session is told a deposit landed on an inbox it
// watches, and processes that inbox itself — through the real adapter, so the
// address resolution and `inbox.watch` are the ones an application would get.
await world.deliverNotifications();
return { object, subjects: await alice.readDocument(index) };
});
expect(seen.subjects.find((s) => s.subject === seen.object)?.props[ENTRY_VALUE]).toEqual([
"2026-09-09",
]);
});
test("two indexes are two documents, each owned by whoever created it", async () => {
const [first, second] = await as("alice", async (alice) => [
await indexing(alice).createIndex(FIELD),
await indexing(alice).createIndex(FIELD),
await (await indexing(alice)).createIndex(FIELD),
await (await indexing(alice)).createIndex(FIELD),
]);
expect(first).not.toBe(second);
@@ -116,10 +139,10 @@ test("two indexes are two documents, each owned by whoever created it", async ()
// --- the two answers a resolve may give, and why they must stay apart -----
test("a reference that could not be READ comes back unresolved", async () => {
const index = await as("alice", (alice) => indexing(alice).createIndex(FIELD));
const index = await as("alice", async (alice) => (await indexing(alice)).createIndex(FIELD));
const article = await as("bob", async (bob) => {
const object = await publish(bob, FIELD, "2026-07-08");
await indexing(bob).refer(index, object);
await (await indexing(bob)).refer(index, object);
return object;
});
@@ -128,7 +151,7 @@ test("a reference that could not be READ comes back unresolved", async () => {
// the failure as a FACT about the object is what must not happen.
world.breakReadsOf(article, "broker unreachable");
try {
const report = await as("alice", (alice) => indexing(alice).curate(index));
const report = await as("alice", (alice) => curate(alice, index));
expect(report.outcomes).toEqual([
{ result: "unresolved", object: article, reason: expect.stringContaining("absent") },
]);
@@ -138,10 +161,10 @@ test("a reference that could not be READ comes back unresolved", async () => {
});
test("an object that really carries nothing for the field is SKIPPED — a different answer", async () => {
const index = await as("alice", (alice) => indexing(alice).createIndex(FIELD));
const index = await as("alice", async (alice) => (await indexing(alice)).createIndex(FIELD));
const unrelated = await as("bob", async (bob) => {
const object = await publish(bob, "http://schema.org/name", "Anemone");
await indexing(bob).refer(index, object);
await (await indexing(bob)).refer(index, object);
return object;
});
@@ -149,7 +172,7 @@ test("an object that really carries nothing for the field is SKIPPED — a diffe
// keeping these two apart: replace `resolutionFromRead(subjects)` with
// `{ state: "present", subjects }` and the unreadable object above is reported
// here's answer instead — a broker failure filed as a property of the object.
const report = await as("alice", (alice) => indexing(alice).curate(index));
const report = await as("alice", (alice) => curate(alice, index));
expect(report.outcomes).toEqual([
{ result: "skipped", object: unrelated, reason: "no-field" },
]);
@@ -187,8 +210,8 @@ test("a document whose owner never opened an inbox REFUSES the deposit", async (
});
test("anyone may deposit into an index, only its owner may read what was deposited", async () => {
const index = await as("alice", (alice) => indexing(alice).createIndex(FIELD));
await as("bob", (bob) => indexing(bob).refer(index, "did:ng:o:some-object"));
const index = await as("alice", async (alice) => (await indexing(alice)).createIndex(FIELD));
await as("bob", async (bob) => (await indexing(bob)).refer(index, "did:ng:o:some-object"));
const own = await as("alice", (alice) => alice.readDeposits(index));
expect(own.map((deposit) => deposit.payload)).toEqual(["did:ng:o:some-object"]);
@@ -202,18 +225,18 @@ test("anyone may deposit into an index, only its owner may read what was deposit
// --- what the adapter actually wrote --------------------------------------
test("readDocument returns what was written, and refuses a document that is no index", async () => {
const index = await as("alice", (alice) => indexing(alice).createIndex(FIELD));
const index = await as("alice", async (alice) => (await indexing(alice)).createIndex(FIELD));
const subjects = await as("alice", (alice) => alice.readDocument(index));
expect(subjects).toEqual([{ subject: index, graph: index, props: { [INDEX_FIELD]: [FIELD] } }]);
const ordinary = await as("alice", (alice) => alice.createPublicDocument());
await expect(as("alice", (alice) => indexing(alice).read(ordinary))).rejects.toThrow(
await expect(as("alice", async (alice) => (await indexing(alice)).read(ordinary))).rejects.toThrow(
/declares no index field/,
);
});
test("the write is the polyfill's canonical anchored form: the document named once, as the anchor", async () => {
const index = await as("alice", (alice) => indexing(alice).createIndex(FIELD));
const index = await as("alice", async (alice) => (await indexing(alice)).createIndex(FIELD));
const writes = world.calls.filter((call) => call.entry === "docs.sparqlUpdate");
const last = writes.at(-1);
expect(last?.args[1]).toBe(`INSERT DATA { <${index}> <${INDEX_FIELD}> "${FIELD}" }`);
+97 -1
View File
@@ -27,11 +27,29 @@ import { asNuri } from "../src/nuri";
* (`openDocumentInbox`: doing so publishes the document's address);
* - a document in a public store is readable by whoever knows its NURI;
* - a document that cannot be read REJECTS, and a rejection means "unknown",
* never "absent".
* never "absent";
* - being TOLD what landed in an inbox is reading it, so watching one is refused to
* anyone but the document's owner, exactly as opening one is.
*
* ## Telling a watcher crosses the network, so it is a step of its own
*
* A deposit is stored the moment it is made that is the durable fact, and it is
* what the owner's next connection finds. Notifying a session that is watching goes
* over the wire, and this fake holds those notifications until a test calls
* {@link FakeNextGraph.deliverNotifications}. A test that never calls it is a test
* in which the owner has not been told yet: a real state, and precisely the one the
* catch-up at connection exists for.
*/
type Properties = Map<string, string[]>;
/** One session watching one document's inbox. */
interface Watch {
readonly doc: Nuri;
readonly user: string;
readonly onDeposits: () => Promise<void>;
}
interface StoredDocument {
readonly nuri: Nuri;
readonly owner: string;
@@ -44,6 +62,12 @@ export class FakeNextGraph {
readonly #documents = new Map<string, StoredDocument>();
/** Documents the broker currently cannot answer about. See `breakReadsOf`. */
readonly #unreachable = new Map<string, string>();
/** Every live watch, across every identity — a session watching its own inbox. */
#watches: Watch[] = [];
/** Notifications the broker has not handed over yet. See `deliverNotifications`. */
#undelivered: Watch[] = [];
/** Why a store listing cannot answer, when a test has made it fail. */
#listingFailure: string | undefined;
#documentCount = 0;
#clock = 0;
@@ -83,6 +107,14 @@ export class FakeNextGraph {
async readDeposits(doc: NuriLike): Promise<readonly IncomingDeposit[]> {
return network.#readDeposits(user, asNuri(doc));
},
async watchDeposits(doc: NuriLike, onDeposits: () => Promise<void>): Promise<void> {
network.#watchDeposits(user, asNuri(doc), onDeposits);
},
async listPublicDocuments(): Promise<readonly Nuri[]> {
return network.#listDocuments(user);
},
};
}
@@ -100,6 +132,40 @@ export class FakeNextGraph {
this.#unreachable.delete(asNuri(doc));
}
/**
* Hands over every inbox notification the broker was holding, and waits for the
* sessions watching to finish with them including notifications those very runs
* provoke, so this returns with nothing left in flight.
*/
async deliverNotifications(): Promise<void> {
while (this.#undelivered.length > 0) {
const batch = this.#undelivered;
this.#undelivered = [];
for (const watch of batch) await watch.onDeposits();
}
}
/**
* This identity's page is gone: every watch its sessions had opened stops, and
* anything the broker was about to tell them is dropped. The polyfill's watching
* lasts exactly as long as an identity stays connected, and so does this.
*
* Nothing durable is lost the deposits are in their inboxes, which is what makes
* the catch-up at the next connection enough on its own.
*/
disconnect(user: string): void {
this.#undelivered = this.#undelivered.filter((watch) => watch.user !== user);
this.#watches = this.#watches.filter((watch) => watch.user !== user);
}
/**
* The store can no longer say which documents an identity has. Upstream throws
* rather than answer a listing it could not establish, so this does too.
*/
breakListing(reason: string): void {
this.#listingFailure = reason;
}
/** A NURI shaped like any other, that no document was ever created for. */
neverCreatedNuri(): Nuri {
return "did:ng:o:doc-never-created" as Nuri;
@@ -216,6 +282,36 @@ export class FakeNextGraph {
}
this.#clock += 1;
stored.deposits.push({ from: user, payload, ts: this.#clock });
// Stored first, told afterwards: the deposit is a fact even if nobody is ever
// told, which is what makes the catch-up at connection sufficient on its own.
for (const watch of this.#watches) {
if (watch.doc === doc) this.#undelivered.push(watch);
}
}
#watchDeposits(user: string, doc: Nuri, onDeposits: () => Promise<void>): void {
const stored = this.#require(doc);
if (stored.owner !== user) {
throw new Error(
`${user} may not watch the inbox of ${doc}: being told what landed in an inbox ` +
"is reading it, and you may only READ your own",
);
}
// Watching resolves the inbox address, and the call that resolves one opens it
// when there is none — the same idempotent call `openInbox` makes.
stored.deposits ??= [];
this.#watches.push({ doc, user, onDeposits });
}
#listDocuments(user: string): readonly Nuri[] {
if (this.#listingFailure !== undefined) {
throw new Error(`cannot list the public store: ${this.#listingFailure}`);
}
const mine: Nuri[] = [];
for (const stored of this.#documents.values()) {
if (stored.owner === user) mine.push(stored.nuri);
}
return mine;
}
#readDeposits(user: string, doc: Nuri): readonly IncomingDeposit[] {
+77 -2
View File
@@ -40,7 +40,11 @@ import type { Nuri, UnionSubject } from "../src/port";
* - a document with no inbox READS as `[]` a state, not an error
* (`depositsForDocument`);
* - opening an inbox is refused to anyone but the document's owner
* (`openDocumentInbox`);
* (`openDocumentInbox`), and so is watching one, since being told what landed in an
* inbox is reading it;
* - watching lasts exactly as long as the identity stays connected: signing in as
* somebody else stops every watch the previous identity had opened
* (`contract_polyfill-surface`, "Guarantees");
* - `readUnion` swallows a failing document into `[]` (`readDoc`'s
* `try {…} catch { return [] }`), and may also reject outright;
* - `readUnion` builds each subject's props as a plain object literal filled by
@@ -271,6 +275,13 @@ export interface FakePolyfill {
/** `readUnion` REJECTS outright — its session-level failure, not a per-document one. */
breakReadUnion(reason: string): void;
healReadUnion(): void;
/**
* Hands over every inbox notification the broker was holding, and waits for the
* watching session to finish with each the callback is declared `void` upstream,
* so production never waits for it, and this does only because a test needs a point
* at which the work is over.
*/
deliverNotifications(): Promise<void>;
/** Every subject in a document, read from outside the adapter. */
contentsOf(doc: string): { subject: string; predicate: string; values: string[] }[];
}
@@ -284,6 +295,10 @@ export function installFakePolyfill(): FakePolyfill {
const documents = new Map<string, StoredDocument>();
const unreachable = new Map<string, string>();
const calls: RecordedCall[] = [];
/** address → the document whose inbox it is. Nothing else resolves one. */
const inboxAddresses = new Map<string, string>();
let watches: { readonly doc: string; readonly onDeposits: (d: Deposit[]) => unknown }[] = [];
let undelivered: { readonly doc: string; readonly onDeposits: (d: Deposit[]) => unknown }[] = [];
let unionFailure: string | undefined;
let currentUser = "nobody";
let documentCount = 0;
@@ -386,6 +401,32 @@ export function installFakePolyfill(): FakePolyfill {
const from = Object.hasOwn(opts, "from") ? (opts.from ?? null) : currentUser;
clock += 1;
stored.deposits.push({ from, payload: opts.payload ?? null, ts: clock });
// Stored first, told afterwards — and told over the wire, which is why the
// notification waits for `deliverNotifications` rather than firing inline.
for (const watch of watches) if (watch.doc === stored.nuri) undelivered.push(watch);
},
watch(targetInbox: unknown, onDeposits: unknown): () => void {
const doc = inboxAddresses.get(String(targetInbox));
if (doc === undefined) {
// "`inbox.post` refuses a target that is not an inbox" — so does watching one,
// and nothing outside `openDocumentInbox` ever hands an address out.
throw new Error(`[fake-polyfill] not an inbox address: ${String(targetInbox)}`);
}
if (require(doc).owner !== currentUser) {
throw new Error(
`${currentUser} may not watch the inbox of ${doc}: you may DEPOSIT into ` +
"anyone's inbox, you may only READ your own",
);
}
if (typeof onDeposits !== "function") {
throw new Error("[fake-polyfill] inbox.watch takes a callback");
}
const watch = { doc, onDeposits: onDeposits as (d: Deposit[]) => unknown };
watches.push(watch);
return () => {
watches = watches.filter((w) => w !== watch);
};
},
async readForDocument(doc: unknown): Promise<Deposit[]> {
@@ -422,7 +463,23 @@ export function installFakePolyfill(): FakePolyfill {
);
}
stored.deposits ??= [];
return `${stored.nuri}:inbox`;
// Idempotent within a page: asking again for a document that already has one
// resolves that same address rather than adding a second inbox.
const address = `${stored.nuri}:inbox`;
inboxAddresses.set(address, stored.nuri);
return address;
},
async listMyEntityDocs(scope: unknown): Promise<Nuri[]> {
if (scope !== "public" && scope !== "mine") {
throw new Error(`[fake-polyfill] unknown scope ${JSON.stringify(scope)}`);
}
// "listMyEntityDocs returns a listing whose documents you can open, or it throws."
const mine: Nuri[] = [];
for (const stored of documents.values()) {
if (stored.owner === currentUser) mine.push(stored.nuri);
}
return mine;
},
};
@@ -492,8 +549,26 @@ export function installFakePolyfill(): FakePolyfill {
return {
calls,
signIn(user: string) {
// "It lasts exactly as long as that identity stays connected — changing identity
// or clearing it stops it." This fake has ONE signed-in identity at a time, as a
// page does, so a watch cannot outlive the identity that opened it. Signing in as
// the same identity again is not a change, and stops nothing.
if (user !== currentUser) {
watches = [];
undelivered = [];
}
currentUser = user;
},
async deliverNotifications() {
while (undelivered.length > 0) {
const batch = undelivered;
undelivered = [];
for (const watch of batch) {
const stored = documents.get(watch.doc);
await watch.onDeposits([...(stored?.deposits ?? [])]);
}
}
},
sessionId() {
return `session:${currentUser}`;
},
+227
View File
@@ -0,0 +1,227 @@
import { expect, mock, test } from "bun:test";
import { indexing, type Indexing } from "../src/indexing";
import { coalescing } from "../src/coalescing";
import type { Nuri } from "../src/port";
import { FakeNextGraph, publishObject } from "./fake-nextgraph";
/**
* WHEN an index is curated the engagement `createIndex` makes about what becomes
* of what it created: the index is curated at its creator's next connection, and on
* each deposit while the creator is connected.
*
* Nothing below calls curation, because there is nothing to call. Every test here
* drives the two acts an application really has connecting (obtaining a handle)
* and depositing and asserts what the index holds afterwards.
*
* The case space is the creator's presence crossed with the deposit's timing:
* away when it was made, connected when it was made, and connected on an index
* that a previous session created. Plus the three that must NOT happen: a stranger
* connecting curates nothing, a document that is no index is left alone, and a
* session that could not look for its indexes is still a working handle.
*/
const PUBLISHED_AT = "http://schema.org/datePublished";
function hardcodedInAppSource(nuri: Nuri): Nuri {
return nuri;
}
/**
* Alice creates an index, and then her page closes. That is the state most of these
* tests start from: an index exists, its creator is away, and nothing is watching
* it so a deposit made now can only be seen at her next connection.
*/
async function aliceCreatesAnIndexAndLeaves(network: FakeNextGraph): Promise<Nuri> {
const alice = await indexing(network.portFor("alice"));
const index = await alice.createIndex(PUBLISHED_AT);
network.disconnect("alice");
return hardcodedInAppSource(index);
}
// --- the deposits that piled up while the creator was away ----------------
test("an index is curated at its creator's next connection, with nobody asking", async () => {
const network = new FakeNextGraph();
const index = await aliceCreatesAnIndexAndLeaves(network);
// Bob deposits while Alice is away: her session is never told, and the deposit
// waits in the inbox where only she can see it.
const bobPort = network.portFor("bob");
const article = await publishObject(bobPort, PUBLISHED_AT, "2026-03-04");
await (await indexing(bobPort)).refer(index, article);
const carol = await indexing(network.portFor("carol"));
expect(await carol.read(index)).toEqual([]);
// Alice comes back. This is the whole of it: obtaining her handle IS the trigger.
const alice = await indexing(network.portFor("alice"));
expect(await alice.read(index)).toEqual([{ object: article, value: "2026-03-04" }]);
expect(await carol.read(index)).toEqual([{ object: article, value: "2026-03-04" }]);
});
test("a whole backlog is caught up, across every index the creator owns", async () => {
const network = new FakeNextGraph();
const alicePort = network.portFor("alice");
const bobPort = network.portFor("bob");
const first = await aliceCreatesAnIndexAndLeaves(network);
const second = await aliceCreatesAnIndexAndLeaves(network);
// An ordinary public document of Alice's, which is no index at all.
await publishObject(alicePort, PUBLISHED_AT, "2026-01-01");
const bob = await indexing(bobPort);
const early = await publishObject(bobPort, PUBLISHED_AT, "2026-01-02");
const late = await publishObject(bobPort, PUBLISHED_AT, "2026-05-06");
await bob.refer(first, early);
await bob.refer(first, late);
await bob.refer(second, late);
const alice = await indexing(alicePort);
expect((await alice.read(first)).map((e) => e.value)).toEqual(["2026-01-02", "2026-05-06"]);
expect(await alice.read(second)).toEqual([{ object: late, value: "2026-05-06" }]);
});
// --- the deposits that arrive while the creator is looking ----------------
test("a deposit made while the creator is connected is curated as it lands", async () => {
const network = new FakeNextGraph();
const alice = await indexing(network.portFor("alice"));
const index = hardcodedInAppSource(await alice.createIndex(PUBLISHED_AT));
const bobPort = network.portFor("bob");
const article = await publishObject(bobPort, PUBLISHED_AT, "2026-03-04");
await (await indexing(bobPort)).refer(index, article);
// The index was created in THIS session, so the store search never saw it: what
// brings it under observation is `createIndex` itself.
await network.deliverNotifications();
expect(await alice.read(index)).toEqual([{ object: article, value: "2026-03-04" }]);
});
test("an index from a previous session is watched too, not merely caught up once", async () => {
const network = new FakeNextGraph();
const index = await aliceCreatesAnIndexAndLeaves(network);
const bobPort = network.portFor("bob");
const bob = await indexing(bobPort);
// Alice comes back to an index she created before, with nothing waiting in it.
const alice = await indexing(network.portFor("alice"));
expect(await alice.read(index)).toEqual([]);
// …and only now does Bob deposit. Nothing but the watch can carry this one.
const article = await publishObject(bobPort, PUBLISHED_AT, "2026-07-08");
await bob.refer(index, article);
await network.deliverNotifications();
expect(await alice.read(index)).toEqual([{ object: article, value: "2026-07-08" }]);
});
test("a burst of deposits settles on the same index, whatever order they are told in", async () => {
const network = new FakeNextGraph();
const alice = await indexing(network.portFor("alice"));
const index = hardcodedInAppSource(await alice.createIndex(PUBLISHED_AT));
const bobPort = network.portFor("bob");
const bob = await indexing(bobPort);
for (const date of ["2026-03-04", "2026-01-31", "2025-12-25"]) {
await bob.refer(index, await publishObject(bobPort, PUBLISHED_AT, date));
}
await network.deliverNotifications();
expect((await alice.read(index)).map((e) => e.value)).toEqual([
"2025-12-25",
"2026-01-31",
"2026-03-04",
]);
});
// --- what connecting must NOT do -----------------------------------------
test("connecting curates nothing for anyone but the creator", async () => {
const network = new FakeNextGraph();
const index = await aliceCreatesAnIndexAndLeaves(network);
const bobPort = network.portFor("bob");
const bob = await indexing(bobPort);
const article = await publishObject(bobPort, PUBLISHED_AT, "2026-03-04");
await bob.refer(index, article);
// Bob connects again, and Carol connects: neither owns the index, so neither can
// read its inbox — and connecting must not try, nor fail, nor write anything.
const carol = await indexing(network.portFor("carol"));
await indexing(bobPort);
await network.deliverNotifications();
expect(await carol.read(index)).toEqual([]);
});
test("a public document that is no index is left alone — no inbox, no entry", async () => {
const network = new FakeNextGraph();
const alicePort = network.portFor("alice");
const ordinary = await alicePort.createPublicDocument();
await indexing(alicePort);
// Had connecting treated every public document as an index, it would have opened
// an inbox on this one — which is exactly what makes a deposit possible.
const bob = await indexing(network.portFor("bob"));
await expect(bob.refer(ordinary, "did:ng:o:doc-9")).rejects.toThrow(/has no inbox/);
});
test("a session that could not look for its indexes is still a working handle", async () => {
const network = new FakeNextGraph();
const index = await aliceCreatesAnIndexAndLeaves(network);
const bobPort = network.portFor("bob");
const article = await publishObject(bobPort, PUBLISHED_AT, "2026-03-04");
await (await indexing(bobPort)).refer(index, article);
network.breakListing("broker unreachable");
const reported = mock((..._args: unknown[]) => {});
const original = console.error;
console.error = reported;
let alice: Indexing;
try {
alice = await indexing(network.portFor("alice"));
} finally {
console.error = original;
}
// Reading an index and depositing into one need none of that work, so nothing is
// denied — but the failure is on the log, because a silent one teaches nobody.
expect(await alice.read(index)).toEqual([]);
await alice.refer(index, article);
expect(reported).toHaveBeenCalledTimes(1);
expect(String(reported.mock.calls[0]?.[0])).toContain("public store could not be listed");
});
// --- the primitive that keeps a burst from piling up ----------------------
test("coalescing never runs twice at once, and grants exactly one more run", async () => {
const trace: string[] = [];
const ask = coalescing(async () => {
trace.push("start");
// Yields, so the asks below really do arrive while a run is in flight — which
// is the only situation this primitive exists for.
await Promise.resolve();
trace.push("end");
});
const first = ask();
const during = [ask(), ask(), ask()];
await Promise.all([first, ...during]);
// Three asks during one run earn ONE more run between them, not three — and not
// none, since a deposit that landed after the first run read the inbox would
// otherwise wait for the next connection.
expect(trace).toEqual(["start", "end", "start", "end"]);
// Runs never overlap: no "start" ever follows a "start".
expect(trace.join(" ")).not.toContain("start start");
// And an ask that arrives once everything is quiet is a run of its own.
await ask();
expect(trace.filter((step) => step === "start")).toHaveLength(3);
});
+56 -44
View File
@@ -1,5 +1,6 @@
import { expect, test } from "bun:test";
import { indexing, type Indexing } from "../src/indexing";
import { curate } from "../src/curator";
import type { Nuri } from "../src/port";
import { ENTRY_VALUE, INDEX_FIELD } from "../src/vocabulary";
import { FakeNextGraph, publishObject } from "./fake-nextgraph";
@@ -20,32 +21,43 @@ function hardcodedInAppSource(nuri: Nuri): Nuri {
type Port = ReturnType<FakeNextGraph["portFor"]>;
function world(): {
async function world(): Promise<{
network: FakeNextGraph;
alice: Indexing;
bob: Indexing;
carol: Indexing;
ports: { alice: Port; bob: Port; carol: Port };
} {
}> {
const network = new FakeNextGraph();
const ports = {
alice: network.portFor("alice"),
bob: network.portFor("bob"),
carol: network.portFor("carol"),
};
// Three connections, none of which owns an index yet: there is nothing to catch up
// on and nothing to watch. What each of them does next is what these tests are about.
return {
network,
alice: indexing(ports.alice),
bob: indexing(ports.bob),
carol: indexing(ports.carol),
alice: await indexing(ports.alice),
bob: await indexing(ports.bob),
carol: await indexing(ports.carol),
ports,
};
}
/**
* These tests exercise the curation RULES, so they run the curator itself rather
* than wait for an inbox notification: what a run makes of a deposit is what is
* under test, not when the run happens. `inbox-processing.test.ts` covers the when.
*
* The deposits below therefore sit in their inbox, told to nobody, which is exactly
* the state an owner's next connection finds.
*/
// --- creating an index ----------------------------------------------------
test("any user creates an index in their public store, and it declares its field", async () => {
const { alice, ports } = world();
const { alice, ports } = await world();
const index = await alice.createIndex(PUBLISHED_AT);
@@ -59,7 +71,7 @@ test("any user creates an index in their public store, and it declares its field
});
test("reading a document that declares no index field is refused, not answered empty", async () => {
const { alice, ports } = world();
const { alice, ports } = await world();
const ordinary = await ports.alice.createPublicDocument();
await expect(alice.read(ordinary)).rejects.toThrow(/declares no index field/);
});
@@ -67,7 +79,7 @@ test("reading a document that declares no index field is refused, not answered e
// --- the whole loop, across three people ----------------------------------
test("a stranger refers an object, the owner curates, and anyone reads the result", async () => {
const { alice, bob, carol, ports } = world();
const { alice, bob, carol, ports } = await world();
// Alice creates the index and its NURI goes into the application's source.
const indexNuri = hardcodedInAppSource(await alice.createIndex(PUBLISHED_AT));
@@ -80,7 +92,7 @@ test("a stranger refers an object, the owner curates, and anyone reads the resul
// Nothing is in the index until its owner acts.
expect(await carol.read(indexNuri)).toEqual([]);
const report = await alice.curate(indexNuri);
const report = await curate(ports.alice, indexNuri);
expect(report.outcomes).toEqual([{ result: "indexed", object: article, value: "2026-03-04" }]);
// Carol knows only the NURI from the application's source, and gets the entry.
@@ -96,11 +108,11 @@ test("a stranger refers an object, the owner curates, and anyone reads the resul
});
test("an entry is a subject keyed by the object's NURI, so reading needs nothing new", async () => {
const { alice, bob, ports } = world();
const { alice, bob, ports } = await world();
const indexNuri = hardcodedInAppSource(await alice.createIndex(PUBLISHED_AT));
const article = await publishObject(ports.bob, PUBLISHED_AT, "2026-03-04");
await bob.refer(indexNuri, article);
await alice.curate(indexNuri);
await curate(ports.alice, indexNuri);
// What `readUnion([indexNuri])` hands an application that never loaded this
// package: the index's own subject, plus one subject per indexed object.
@@ -113,17 +125,17 @@ test("an entry is a subject keyed by the object's NURI, so reading needs nothing
// --- only the owner curates ----------------------------------------------
test("nobody but the index's owner can curate it: the inbox is refused to others", async () => {
const { alice, bob, ports } = world();
const { alice, bob, ports } = await world();
const indexNuri = hardcodedInAppSource(await alice.createIndex(PUBLISHED_AT));
const article = await publishObject(ports.bob, PUBLISHED_AT, "2026-03-04");
await bob.refer(indexNuri, article);
await expect(bob.curate(indexNuri)).rejects.toThrow(/may only READ your own/);
await expect(curate(ports.bob, indexNuri)).rejects.toThrow(/may only READ your own/);
expect(await alice.read(indexNuri)).toEqual([]);
});
test("nobody but the owner writes an index, whatever they know about it", async () => {
const { alice, ports } = world();
const { alice, ports } = await world();
const indexNuri = hardcodedInAppSource(await alice.createIndex(PUBLISHED_AT));
await expect(
@@ -133,7 +145,7 @@ test("nobody but the owner writes an index, whatever they know about it", async
});
test("an index whose owner never opened an inbox refuses a deposit rather than losing it", async () => {
const { bob, ports } = world();
const { bob, ports } = await world();
// A public document that was never made into an index: no inbox was opened.
const notAnIndex = hardcodedInAppSource(await ports.alice.createPublicDocument());
await expect(bob.refer(notAnIndex, "did:ng:o:doc-9")).rejects.toThrow(/has no inbox/);
@@ -142,14 +154,14 @@ test("an index whose owner never opened an inbox refuses a deposit rather than l
// --- adding is idempotent -------------------------------------------------
test("the same reference deposited twice produces one entry", async () => {
const { alice, bob, ports } = world();
const { alice, bob, ports } = await world();
const indexNuri = hardcodedInAppSource(await alice.createIndex(PUBLISHED_AT));
const article = await publishObject(ports.bob, PUBLISHED_AT, "2026-03-04");
await bob.refer(indexNuri, article);
await bob.refer(indexNuri, article);
const report = await alice.curate(indexNuri);
const report = await curate(ports.alice, indexNuri);
expect(report.outcomes).toEqual([
{ result: "indexed", object: article, value: "2026-03-04" },
{ result: "unchanged", object: article },
@@ -158,15 +170,15 @@ test("the same reference deposited twice produces one entry", async () => {
});
test("curating twice changes nothing the second time — deposits are not consumed", async () => {
const { alice, bob, ports } = world();
const { alice, bob, ports } = await world();
const indexNuri = hardcodedInAppSource(await alice.createIndex(PUBLISHED_AT));
const article = await publishObject(ports.bob, PUBLISHED_AT, "2026-03-04");
await bob.refer(indexNuri, article);
await alice.curate(indexNuri);
await curate(ports.alice, indexNuri);
const before = await alice.read(indexNuri);
const second = await alice.curate(indexNuri);
const second = await curate(ports.alice, indexNuri);
expect(second.outcomes).toEqual([{ result: "unchanged", object: article }]);
expect(await alice.read(indexNuri)).toEqual(before);
});
@@ -174,18 +186,18 @@ test("curating twice changes nothing the second time — deposits are not consum
// --- a read that cannot answer must never cost the index anything ---------
test("a reference the broker cannot resolve is reported, and adds nothing", async () => {
const { network, alice, bob, ports } = world();
const { network, alice, bob, ports } = await world();
const indexNuri = hardcodedInAppSource(await alice.createIndex(PUBLISHED_AT));
const first = await publishObject(ports.bob, PUBLISHED_AT, "2026-01-01");
await bob.refer(indexNuri, first);
await alice.curate(indexNuri);
await curate(ports.alice, indexNuri);
const second = await publishObject(ports.bob, PUBLISHED_AT, "2026-02-02");
await bob.refer(indexNuri, second);
network.breakReadsOf(second, "broker unreachable");
const report = await alice.curate(indexNuri);
const report = await curate(ports.alice, indexNuri);
const unresolved = report.outcomes.filter((o) => o.result === "unresolved");
expect(unresolved).toHaveLength(1);
expect(unresolved[0]).toMatchObject({ object: second });
@@ -195,11 +207,11 @@ test("a reference the broker cannot resolve is reported, and adds nothing", asyn
});
test("an already-indexed object survives its own reads failing, and is not even re-read", async () => {
const { network, alice, bob, carol, ports } = world();
const { network, alice, bob, carol, ports } = await world();
const indexNuri = hardcodedInAppSource(await alice.createIndex(PUBLISHED_AT));
const article = await publishObject(ports.bob, PUBLISHED_AT, "2026-01-01");
await bob.refer(indexNuri, article);
await alice.curate(indexNuri);
await curate(ports.alice, indexNuri);
// A passer-by nudges the index about an entry she found IN IT. Carol obtains
// the reference the only way she could in a real application — by reading the
@@ -212,24 +224,24 @@ test("an already-indexed object survives its own reads failing, and is not even
network.breakReadsOf(article, "broker unreachable");
await carol.refer(indexNuri, noticed!.object);
const report = await alice.curate(indexNuri);
const report = await curate(ports.alice, indexNuri);
expect(report.outcomes.every((o) => o.result === "unchanged")).toBe(true);
expect(await alice.read(indexNuri)).toEqual([{ object: article, value: "2026-01-01" }]);
});
test("a failed resolve is self-correcting: the next curation adds what it could not", async () => {
const { network, alice, bob, ports } = world();
const { network, alice, bob, ports } = await world();
const indexNuri = hardcodedInAppSource(await alice.createIndex(PUBLISHED_AT));
const article = await publishObject(ports.bob, PUBLISHED_AT, "2026-05-06");
await bob.refer(indexNuri, article);
network.breakReadsOf(article, "broker unreachable");
expect((await alice.curate(indexNuri)).outcomes[0]?.result).toBe("unresolved");
expect((await curate(ports.alice, indexNuri)).outcomes[0]?.result).toBe("unresolved");
expect(await alice.read(indexNuri)).toEqual([]);
// The deposit is still there, so nothing has to be re-deposited.
network.healReadsOf(article);
expect((await alice.curate(indexNuri)).outcomes[0]).toEqual({
expect((await curate(ports.alice, indexNuri)).outcomes[0]).toEqual({
result: "indexed",
object: article,
value: "2026-05-06",
@@ -238,11 +250,11 @@ test("a failed resolve is self-correcting: the next curation adds what it could
});
test("a reference to something that was never created is reported, not silently dropped", async () => {
const { network, alice, bob } = world();
const { network, alice, bob, ports } = await world();
const indexNuri = hardcodedInAppSource(await alice.createIndex(PUBLISHED_AT));
await bob.refer(indexNuri, network.neverCreatedNuri());
const report = await alice.curate(indexNuri);
const report = await curate(ports.alice, indexNuri);
expect(report.outcomes).toHaveLength(1);
expect(report.outcomes[0]?.result).toBe("unresolved");
expect(await alice.read(indexNuri)).toEqual([]);
@@ -251,39 +263,39 @@ test("a reference to something that was never created is reported, not silently
// --- an object that does not fit the index --------------------------------
test("an object carrying nothing for the index's field is not added", async () => {
const { alice, bob, ports } = world();
const { alice, bob, ports } = await world();
const indexNuri = hardcodedInAppSource(await alice.createIndex(PUBLISHED_AT));
// Exists, is public, is readable — but says nothing about the field this index
// is built on. OPEN QUESTION: this is the narrow behaviour, not a settled policy.
const object = await publishObject(ports.bob, NAME, "an object with no date");
await bob.refer(indexNuri, object);
const report = await alice.curate(indexNuri);
const report = await curate(ports.alice, indexNuri);
expect(report.outcomes).toEqual([{ result: "skipped", object, reason: "no-field" }]);
expect(await alice.read(indexNuri)).toEqual([]);
});
test("an object carrying several values for the field is not added", async () => {
const { alice, bob, ports } = world();
const { alice, bob, ports } = await world();
const indexNuri = hardcodedInAppSource(await alice.createIndex(PUBLISHED_AT));
const object = await publishObject(ports.bob, PUBLISHED_AT, "2026-01-01");
await ports.bob.addLiteralProperty(object, object, PUBLISHED_AT, "2026-09-09");
await bob.refer(indexNuri, object);
const report = await alice.curate(indexNuri);
const report = await curate(ports.alice, indexNuri);
expect(report.outcomes).toEqual([{ result: "skipped", object, reason: "several-values" }]);
expect(await alice.read(indexNuri)).toEqual([]);
});
test("a payload that is not a reference is reported as foreign and changes nothing", async () => {
const { alice, bob, ports } = world();
const { alice, bob, ports } = await world();
const indexNuri = hardcodedInAppSource(await alice.createIndex(PUBLISHED_AT));
const article = await publishObject(ports.bob, PUBLISHED_AT, "2026-03-04");
await bob.refer(indexNuri, article);
// Anyone may deposit anything into an inbox, so untrusted payloads do arrive.
await ports.bob.depositTo(indexNuri, { drop: "everything" });
const report = await alice.curate(indexNuri);
const report = await curate(ports.alice, indexNuri);
expect(report.outcomes).toEqual([
{ result: "indexed", object: article, value: "2026-03-04" },
{ result: "foreign", reason: "payload is not a reference" },
@@ -292,11 +304,11 @@ test("a payload that is not a reference is reported as foreign and changes nothi
});
test("an index referred to itself is skipped, so its declaration cannot become an entry", async () => {
const { alice, bob } = world();
const { alice, bob, ports } = await world();
const indexNuri = hardcodedInAppSource(await alice.createIndex(PUBLISHED_AT));
await bob.refer(indexNuri, indexNuri);
const report = await alice.curate(indexNuri);
const report = await curate(ports.alice, indexNuri);
expect(report.outcomes).toEqual([
{ result: "skipped", object: indexNuri, reason: "self-reference" },
]);
@@ -306,7 +318,7 @@ test("an index referred to itself is skipped, so its declaration cannot become a
// --- indexing by a date is an instance of indexing by a field -------------
test("an index whose field is a date reads back in chronological order", async () => {
const { alice, bob, carol, ports } = world();
const { alice, bob, carol, ports } = await world();
const indexNuri = hardcodedInAppSource(await alice.createIndex(PUBLISHED_AT));
const march = await publishObject(ports.bob, PUBLISHED_AT, "2026-03-04");
@@ -317,7 +329,7 @@ test("an index whose field is a date reads back in chronological order", async (
await bob.refer(indexNuri, march);
await bob.refer(indexNuri, december);
await bob.refer(indexNuri, january);
await alice.curate(indexNuri);
await curate(ports.alice, indexNuri);
expect((await carol.read(indexNuri)).map((e) => e.value)).toEqual([
"2025-12-25",
@@ -327,7 +339,7 @@ test("an index whose field is a date reads back in chronological order", async (
});
test("two indexes over the same objects, on different fields, do not interfere", async () => {
const { alice, bob, ports } = world();
const { alice, bob, ports } = await world();
const byDate = hardcodedInAppSource(await alice.createIndex(PUBLISHED_AT));
const byName = hardcodedInAppSource(await alice.createIndex(NAME));
@@ -336,8 +348,8 @@ test("two indexes over the same objects, on different fields, do not interfere",
await bob.refer(byDate, object);
await bob.refer(byName, object);
await alice.curate(byDate);
await alice.curate(byName);
await curate(ports.alice, byDate);
await curate(ports.alice, byName);
expect(await alice.read(byDate)).toEqual([{ object, value: "2026-03-04" }]);
expect(await alice.read(byName)).toEqual([{ object, value: "Anemone" }]);
+35 -30
View File
@@ -1,5 +1,6 @@
import { expect, mock, test } from "bun:test";
import { indexing } from "../src/indexing";
import { curate } from "../src/curator";
import { entriesOf, entryValue } from "../src/index-document";
import { resolutionFromFailure, resolutionFromRead } from "../src/resolution";
import type { Nuri, UnionSubject } from "../src/port";
@@ -64,11 +65,11 @@ test("an entry whose value is the empty string is still an entry", () => {
test("adding a second value to an entry cannot make it disappear", async () => {
const network = new FakeNextGraph();
const ownerPort = network.portFor("alice");
const owner = indexing(ownerPort);
const owner = await indexing(ownerPort);
const index = await owner.createIndex(FIELD);
const article = await publishObject(network.portFor("bob"), FIELD, "2026-01-01");
await indexing(network.portFor("bob")).refer(index, article);
await owner.curate(index);
await (await indexing(network.portFor("bob"))).refer(index, article);
await curate(ownerPort, index);
// A pure ADD — the only write this package has. Before the fix this emptied
// `read()` while both triples sat in the document.
@@ -80,12 +81,12 @@ test("adding a second value to an entry cannot make it disappear", async () => {
test("a raced double-add settles, and does not make every later run re-add", async () => {
const network = new FakeNextGraph();
const ownerPort = network.portFor("alice");
const owner = indexing(ownerPort);
const owner = await indexing(ownerPort);
const index = await owner.createIndex(FIELD);
const bobPort = network.portFor("bob");
const article = await publishObject(bobPort, FIELD, "2026-01-01");
await indexing(bobPort).refer(index, article);
await owner.curate(index);
await (await indexing(bobPort)).refer(index, article);
await curate(ownerPort, index);
// What two curation runs racing each other leave behind: the object's owner
// edited it between their reads, so each added its own value.
@@ -94,7 +95,7 @@ test("a raced double-add settles, and does not make every later run re-add", asy
// The entry is still there, and the curator recognises it as already indexed —
// before the fix it was invisible, so every run added yet another value.
const report = await owner.curate(index);
const report = await curate(ownerPort, index);
expect(report.outcomes).toEqual([{ result: "unchanged", object: article }]);
expect(await owner.read(index)).toEqual([{ object: article, value: "2026-01-01" }]);
});
@@ -104,13 +105,13 @@ test("a raced double-add settles, and does not make every later run re-add", asy
test("a second declared field stops curation LOUDLY and costs no entry", async () => {
const network = new FakeNextGraph();
const ownerPort = network.portFor("alice");
const owner = indexing(ownerPort);
const owner = await indexing(ownerPort);
const bobPort = network.portFor("bob");
const index = await owner.createIndex(FIELD);
for (const date of ["2026-01-01", "2026-02-02", "2026-03-03"]) {
await indexing(bobPort).refer(index, await publishObject(bobPort, FIELD, date));
await (await indexing(bobPort)).refer(index, await publishObject(bobPort, FIELD, date));
}
await owner.curate(index);
await curate(ownerPort, index);
expect(await owner.read(index)).toHaveLength(3);
// One add-only write through the published surface — and the SMALLER string, the
@@ -121,28 +122,28 @@ test("a second declared field stops curation LOUDLY and costs no entry", async (
// unreadable because the declaration above it turned ambiguous.
expect(await owner.read(index)).toHaveLength(3);
// Curating refuses, and says why instead of quietly picking one.
await expect(owner.curate(index)).rejects.toThrow(/declares 2 index fields/);
await expect(curate(ownerPort, index)).rejects.toThrow(/declares 2 index fields/);
});
test("a mixed-field index is never produced: curation refuses before adding anything", async () => {
const network = new FakeNextGraph();
const ownerPort = network.portFor("alice");
const owner = indexing(ownerPort);
const owner = await indexing(ownerPort);
const bobPort = network.portFor("bob");
const NAME = "http://schema.org/name";
const index = await owner.createIndex(NAME);
const first = await publishObject(bobPort, NAME, "Anemone");
await indexing(bobPort).refer(index, first);
await owner.curate(index);
await (await indexing(bobPort)).refer(index, first);
await curate(ownerPort, index);
// "…/datePublished" < "…/name", so under "smallest wins" the new field took over
// while `first` kept its old value forever — one list ordered by two properties.
await ownerPort.addLiteralProperty(index, index, INDEX_FIELD, FIELD);
const second = await publishObject(bobPort, FIELD, "2026-02-02");
await indexing(bobPort).refer(index, second);
await (await indexing(bobPort)).refer(index, second);
await expect(owner.curate(index)).rejects.toThrow(/refusing to curate rather than pick one/);
await expect(curate(ownerPort, index)).rejects.toThrow(/refusing to curate rather than pick one/);
expect(await owner.read(index)).toEqual([{ object: first, value: "Anemone" }]);
});
@@ -150,11 +151,11 @@ test("an index declaring no field at all is still refused", async () => {
const network = new FakeNextGraph();
const ownerPort = network.portFor("alice");
const ordinary = await ownerPort.createPublicDocument();
await expect(indexing(ownerPort).read(ordinary)).rejects.toThrow(/declares no index field/);
await expect((await indexing(ownerPort)).read(ordinary)).rejects.toThrow(/declares no index field/);
});
test("a field that could never match an object is refused at creation", async () => {
const owner = indexing(new FakeNextGraph().portFor("alice"));
const owner = await indexing(new FakeNextGraph().portFor("alice"));
// It cannot be corrected later — nothing here deletes — so it is refused now.
await expect(owner.createIndex("")).rejects.toThrow(/cannot be changed later/);
await expect(owner.createIndex(" ")).rejects.toThrow(/cannot be changed later/);
@@ -164,7 +165,8 @@ test("a field that could never match an object is refused at creation", async ()
test("a field colliding with Object.prototype neither crashes nor is silently mis-read", async () => {
const network = new FakeNextGraph();
const owner = indexing(network.portFor("alice"));
const ownerPort = network.portFor("alice");
const owner = await indexing(ownerPort);
const bobPort = network.portFor("bob");
for (const field of ["constructor", "toString", "valueOf", "hasOwnProperty"]) {
@@ -178,10 +180,10 @@ test("a field colliding with Object.prototype neither crashes nor is silently mi
// An object that merely LACKS it must still resolve cleanly: reading the field
// off a plain object literal would otherwise hand back an inherited function.
const lacks = await publishObject(bobPort, "http://schema.org/name", "unrelated");
await indexing(bobPort).refer(index, lacks);
await indexing(bobPort).refer(index, carries);
await (await indexing(bobPort)).refer(index, lacks);
await (await indexing(bobPort)).refer(index, carries);
const report = await owner.curate(index);
const report = await curate(ownerPort, index);
expect(report.outcomes[0]).toEqual({ result: "skipped", object: lacks, reason: "no-field" });
expect(report.outcomes[1]?.result).toBe("unresolved");
expect(await owner.read(index)).toEqual([]);
@@ -212,17 +214,18 @@ test("a read that threw resolves as unresolved, naming the error", () => {
test("an unresolved reference is warned about, not only reported", async () => {
const network = new FakeNextGraph();
const owner = indexing(network.portFor("alice"));
const ownerPort = network.portFor("alice");
const owner = await indexing(ownerPort);
const index = await owner.createIndex(FIELD);
const article = await publishObject(network.portFor("bob"), FIELD, "2026-01-01");
await indexing(network.portFor("bob")).refer(index, article);
await (await indexing(network.portFor("bob"))).refer(index, article);
network.breakReadsOf(article, "broker unreachable");
const warn = mock((..._args: unknown[]) => {});
const original = console.warn;
console.warn = warn;
try {
await owner.curate(index);
await curate(ownerPort, index);
} finally {
console.warn = original;
}
@@ -233,17 +236,18 @@ test("an unresolved reference is warned about, not only reported", async () => {
test("a normal run warns about nothing", async () => {
const network = new FakeNextGraph();
const owner = indexing(network.portFor("alice"));
const ownerPort = network.portFor("alice");
const owner = await indexing(ownerPort);
const index = await owner.createIndex(FIELD);
const article = await publishObject(network.portFor("bob"), FIELD, "2026-01-01");
await indexing(network.portFor("bob")).refer(index, article);
await (await indexing(network.portFor("bob"))).refer(index, article);
const warn = mock((..._args: unknown[]) => {});
const original = console.warn;
console.warn = warn;
let report;
try {
report = await owner.curate(index);
report = await curate(ownerPort, index);
} finally {
console.warn = original;
}
@@ -257,13 +261,14 @@ test("a normal run warns about nothing", async () => {
test("an index that could not be read is refused, and says so without blaming the document", async () => {
const network = new FakeNextGraph();
const owner = indexing(network.portFor("alice"));
const ownerPort = network.portFor("alice");
const owner = await indexing(ownerPort);
const index = await owner.createIndex(FIELD);
network.breakReadsOf(index, "broker unreachable");
// The real `readUnion` turns a failed read into `[]`, so the failure arrives
// looking like a blank document. Whatever the shape, nothing may be written.
await expect(owner.curate(index)).rejects.toThrow();
await expect(curate(ownerPort, index)).rejects.toThrow();
await expect(owner.read(index)).rejects.toThrow();
network.healReadsOf(index);
+12 -6
View File
@@ -9,7 +9,6 @@ import {
polyfillPort,
ENTRY_VALUE,
INDEX_FIELD,
type CurationReport,
type IndexEntry,
type NextGraphPort,
} from "../src/index";
@@ -22,21 +21,28 @@ test("the published surface carries the whole loop, end to end", async () => {
const ownerPort: NextGraphPort = network.portFor("alice");
const strangerPort: NextGraphPort = network.portFor("bob");
const owner = indexing(ownerPort);
const stranger = indexing(strangerPort);
const owner = await indexing(ownerPort);
const stranger = await indexing(strangerPort);
const index = await owner.createIndex(PUBLISHED_AT);
const article = await publishObject(strangerPort, PUBLISHED_AT, "2026-07-08");
await stranger.refer(index, article);
const report: CurationReport = await owner.curate(index);
expect(report.index).toBe(index);
expect(report.outcomes).toEqual([{ result: "indexed", object: article, value: "2026-07-08" }]);
// NOBODY CURATES — there is nothing on this surface to call. Alice is connected,
// so her session is told a deposit landed and processes that inbox itself.
await network.deliverNotifications();
const entries: IndexEntry[] = await owner.read(index);
expect(entries).toEqual([{ object: article, value: "2026-07-08" }]);
});
test("the published surface offers no way to run, aim or schedule curation", async () => {
const handle = await indexing(new FakeNextGraph().portFor("alice"));
// Read off the handle rather than from a list: an application gets these three
// acts and nothing else, and curation is not one of them.
expect(Object.keys(handle).sort()).toEqual(["createIndex", "read", "refer"]);
});
test("the published surface exposes the deposit decoder and the two IRIs it writes", () => {
expect(decodeReference("did:ng:o:doc-1")).toBe("did:ng:o:doc-1");
expect(decodeReference({ object: "did:ng:o:doc-1" })).toBeNull();
+1 -1
View File
@@ -12,5 +12,5 @@
"isolatedModules": true,
"noEmit": true
},
"include": ["src", "test"]
"include": ["src", "test", "e2e"]
}