Sylvain/ng-helpers
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat!: le paquet crée un index et lui ajoute une référence, rien de plus

Browse Source
This commit is contained in:
Sylvain Duchesne
2026-08-21 19:56:11 +02:00
parent c2f9ff4674
commit ff78a70f14
26 changed files with 735 additions and 2651 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.project/concepts/indexing/indexing-layer/contract_indexing-layer.md
+49 -65
View File
@@ -1,6 +1,6 @@
---
type: contract
summary: What @ng-helpers/indexing engages to do — create an index, take a reference anyone deposits, and make it an entry once the index's creator connects; the index itself is an ordinary document anyone queries
summary: What @ng-helpers/indexing engages to do — create an index, and hand one a reference anyone may deposit; the index itself is an ordinary document anyone queries, and what it receives becomes entries through the layer below
---
# contract_indexing-layer — `@ng-helpers/indexing`
@@ -9,68 +9,58 @@ summary: What @ng-helpers/indexing engages to do — create an index, take a ref
This package builds an **index** on top of NextGraph: an ordinary public document that holds one entry per indexed object, keyed by that object's NURI and carrying its value for a single declared field.
It covers creating an index and depositing a reference to an object into one (open to anyone). What becomes of that reference is covered too, but never as a call — see `## Guarantees`. Reading is not covered: the document is ordinary, and `## Surface` has the shape to query it.
It covers two acts: creating an index, and handing one a reference to an object (open to anyone). It is a thin helper, and its whole job is to spare you the address an index receives on — you name documents, never an inbox.
It does not cover NextGraph itself — documents, identity, sharing, transport, all of which reach it through a port you supply — nor search, filtering, pagination, or querying by anything but the index's field.
It does not cover what becomes of a reference you hand over: making entries of what an index receives belongs to the layer below, and there is no call here that does it, aims it or asks about it. Nor reading — the document is ordinary, and `## Surface` has the shape to query it. Nor NextGraph itself: documents, identity, sharing, transport, all reached through the session you hand it. Nor search, filtering, pagination, or querying by anything but the index's field.
### Deployment requirements
An application using this package must:
- have a NextGraph session open under the identity it wants to act as, and build the port from it — `polyfillPort({ sessionId })`, where `sessionId` is what `@ng-eventually/polyfill`'s own `init(…)` hands its callback;
- **await `indexing(port)` once at startup and keep what it produces.** That handle is one identity's — no call takes an identifier, so two users mean two handles — and it is also that identity's connection: awaiting it is what makes entries appear, dropping it is what stops them;
- reach a broker — nothing here is answered locally;
- **supply `@ng-eventually/polyfill` itself**, as a *peer*: the application names it among its own dependencies, and that copy must be the one its own code calls — that package requires exactly one instance of itself;
- **connect as an index's creator if that index is ever to fill.** Unconditional, whoever holds its reference: an index whose creator never returns stays as it was, however many references it is handed.
- have a NextGraph session open under the identity it wants to act as, and hand this package its **session id** — what `@ng-eventually/polyfill`'s own `init(…)` hands its callback, relayed and never converted;
- **supply `@ng-eventually/polyfill` itself**, as a *peer*: the application names it among its own dependencies, and that copy must be the one its own code calls — that package requires exactly one instance of itself. It is also what turns what an index receives into entries, so there is nothing extra to install or configure for that;
- reach a broker — nothing here is answered locally.
**Holding an index's reference.** An index is reached by its NURI, held however the application holds any other reference — per user, per context, or read out of a document it opens anyway. That reference is the only way anyone reaches it, and losing it loses the index. **Hardcoding it is what a single GLOBAL index needs, and only that case**; anything narrower is discovered.
A handle is one identity's and holds nothing: build one at startup and keep it, or build one where you need it.
**Obtaining it.** Not published to npm and not built output: the entry point is TypeScript source, so whatever builds the application compiles it — and `@ng-eventually/polyfill` arrives the same way.
**Holding an index's reference.** An index is reached by its NURI, held however the application holds any other reference — per user, per context, or read out of a document it opens anyway. It is the only way anyone reaches the index, and losing it loses the index. **Hardcoding it is what a single GLOBAL index needs, and only that case**; anything narrower is discovered.
**Obtaining it.** Not published to npm and not built output: the entry point is TypeScript source, so whatever builds the application compiles it — as does `@ng-eventually/polyfill`.
## Surface
Full typed shape: the package's `types` entry. The load-bearing signatures:
Full typed shape: the package's `types` entry. All of it, and it is deliberately this small:
```ts
// wiring
export function polyfillPort(options: PolyfillPortOptions): NextGraphPort;
export interface PolyfillPortOptions { readonly sessionId: string | number }
/** This identity's handle, and its connection: see `## Guarantees`. */
export function indexing(port: NextGraphPort): Promise<Indexing>;
/** This identity's handle. It reaches nothing on its own — only the two acts do.
* `sessionId` is what
* `@ng-eventually/polyfill`'s own `init(…)` hands its callback, relayed, never converted. */
export function indexing(sessionId: string | number): Indexing;
// addressing (import these from here)
export type Nuri = `did:ng:${string}`;
export type NuriLike = Nuri | string;
export type { UnionSubject, NextGraphPort, IncomingDeposit, ObjectResolution };
// the three acts
export interface Indexing {
/** A new index in THIS identity's public store, and its NURI. Any user may. `field`
* is the predicate an indexed object must carry; an empty or blank one throws. */
createIndex(field: string): Promise<Nuri>;
/** Deposits a bare reference to an object into the index. Open to ANYONE. Throws
* when the document cannot take one, rather than losing it. */
refer(index: NuriLike, object: NuriLike): Promise<void>;
/** Convenience: the entries, ordered by value, without the index's own subject.
* Refuses a document declaring no field. NOT an engagement — see below. */
read(index: NuriLike): Promise<IndexEntry[]>;
create(field: string): Promise<Nuri>;
/** Hands the index a bare reference to an object. Open to ANYONE. Throws when the
* document cannot take one, rather than losing it. Added now, visible later. */
add(index: NuriLike, object: NuriLike): Promise<void>;
}
// what an index holds, and what a depositor sends
export interface IndexEntry { readonly object: Nuri; readonly value: string }
export interface IndexDescriptor { readonly field: string }
export type IndexDeposit = Nuri; // the reference IS the whole payload
export function decodeReference(payload: unknown): Nuri | null; // untrusted input
// addressing, as the two acts above speak it
export type Nuri = `did:ng:${string}`;
export type NuriLike = Nuri | string;
// the IRIs it is written with
// the IRIs an index is written with — how you recognise what you read
export const INDEX_FIELD: string; // "urn:ng-helpers:index:field"
export const ENTRY_VALUE: string; // "urn:ng-helpers:index:value"
```
**What an index document holds** — two shapes, and this layer writes nothing else:
**There is no call here that reads an index, and that absence is the engagement.** An index is an ordinary document: its contents come back the way any other document's do, and the two IRIs above are the whole of what you need to make sense of them. A helper of ours would only teach a shape you would have to unlearn.
- on the **index's own subject**, `INDEX_FIELD` carries the predicate an indexed object must carry, as a **literal**, not a URI;
- on **each entry**, whose subject is the indexed object's own `did:ng:` NURI, `ENTRY_VALUE` carries that object's value for the field, as a literal. One subject may carry more than one, which two sessions racing each other produce.
**What an index document holds** — two shapes:
- on the **index's own subject**, `INDEX_FIELD` carries the predicate an indexed object must carry, as a **literal**, not a URI. `create` writes it, and it is the only thing this package ever writes;
- on **each entry**, whose subject is the indexed object's own `did:ng:` NURI, `ENTRY_VALUE` carries that object's value for the field, as a literal. One subject may carry more than one.
That is everything reading takes: an anchored `SELECT ?object ?value WHERE { ?object <urn:ng-helpers:index:value> ?value }` returns the entries, for a stranger owning neither the index nor the objects exactly as for its creator; `readUnion([index])` returns the same subjects plus the index's own.
@@ -78,43 +68,31 @@ That is everything reading takes: an anchored `SELECT ?object ?value WHERE { ?ob
**An index is an ordinary public document, and nothing marks it as one.** It lives in its creator's public store, so any reader opens it from the reference alone; its creator owns it, and any user may create one.
**A reference deposited into an index becomes an entry once the index's creator is connected.** Awaiting `indexing(port)` is that connection: across every index that identity owns, what was deposited while it was away becomes an entry then, and what arrives from that moment on becomes one as it lands. Nothing to call, schedule or configure, and no way to aim it at one index — that session serves all of them.
**A reference you hand an index becomes one of its entries.** Not by anything on this surface: the layer below processes what an index receives, and that is what an entry is made by. Nothing to call, schedule or configure — and nothing to wait on, which is why `add` returns as soon as the reference is lodged rather than when it can be read back.
**The field is declared once, inside the document, and cannot be changed.** `createIndex` refuses an empty or blank one at the door; an index created on a useless field is useless for good. A document that ends up declaring several — which nothing on this surface can do, only a direct write to it — stops gaining entries, loudly and permanently.
**The field is declared once, at creation, and this package never changes it.** `create` refuses an empty or blank one at the door; an index created on a useless field is useless for good.
**A new index is ready the moment `createIndex` produces it** — deposit into it straight away; nothing to open or register.
**A new index is ready the moment `create` produces it** — hand it a reference straight away; nothing to open or register.
**Depositing is open to anyone; writing is the creator's alone.** `refer` is a deposit, so a stranger contributes to an index they could not write. What is deposited is a **bare reference** — no operation, no claim, no copy of the value: what the object itself says, when its entry is made, is what goes in.
**Handing a reference over is open to anyone; writing an index is its creator's alone.** `add` is not a write, so a stranger contributes to an index they could not write a triple into. What travels is a **bare reference** — no operation, no claim, no copy of the value, and no index either, since the one it is handed to identifies it. What the object itself says, when its entry is made, is what goes in. And it is a refusal rather than a silent loss: `add` throws when the document cannot take a reference, and refuses anything that is not a NURI before it goes anywhere.
**An index ONLY EVER GROWS.** No call removes an entry, for anyone including the creator: this package cannot express a removal at all. The answer to "this entry must go" is a fresh index.
**Handing the same reference over twice is safe, and so is doing it a hundred times.** A reference is an invitation to look, never an instruction. Do it when the object is created, and again whenever anyone notices the index may not have it yet — including a third party who only read the index.
**The same references produce the same index, whatever order they arrived in.** One reference deposited a hundred times leaves one entry, and an object already indexed is passed over rather than read again. The cost: no deposit is ever retired, so the work behind an index is linear in its history.
**None of this can deny you anything.** A session that could not find its indexes, catch one up, or stay posted about one still hands you a working handle, and nothing deposited is lost: the next connection makes its entry. Such failures, and every reference that could not be resolved, are on this package's log stream.
**Everything deposited is untrusted.** Anyone may deposit anything; `decodeReference` returns `null` for whatever is not a reference, and such a payload is passed over rather than stopping the rest.
**Nothing on this surface removes anything.** No call takes an entry out of an index, for anyone including its creator: this package cannot express a removal at all. The answer to "this entry must go" is a fresh index.
## Non-guarantees
**Reading is not an engagement.** `read` is sugar over the shape above; its ordering, its tie-breaks and what it makes of a subject carrying several values are free to change. The order an index comes out in is the caller's decision — if it matters, query the document and order the result.
**Nothing tells you what became of a reference, or when.** No report, no callback, nothing to wait on, no ordering between handing one over and reading the index. Whether an entry is made at all, what value it ends up holding, and how long that takes are not engaged here — reading the index is how you find out. `add` checks that a reference is a NURI and nothing more: not that the object exists, that it is readable, or that it carries the index's field.
**Nothing tells you what became of a reference, or when.** No report, no callback, nothing to wait on, no ordering between a deposit and a read; while an index's creator stays away, a deposit waits as long as that lasts. A reference that could not be resolved is warned about on the log stream; an object carrying nothing for the field, one carrying several, a self-reference and a payload that is no reference are not reported at all. Reading the index is how you find out.
**No order, and nobody to ask for one.** Nothing here hands you an index's contents, so the order they come out in is whatever your own query says — and a subject carrying several values is yours to make something of.
**No refresh.** An already-indexed object is never read again, so one whose value changes later keeps its original indefinitely.
**No private data.** Only objects the index's creator can open are indexed; one it cannot read is not added.
**A handle is one identity for its whole life, and nothing releases what it holds.** An application that changes identity within one page must build a new handle and drop the old one, which goes on listening under a session that holds nothing.
**Connecting reads this identity's whole public store** — a store read plus one read per document, every time a handle is built. An index whose read did not answer then is not found, silently; its entries are made at the next connection.
**The narrow behaviours are open questions.** An object carrying nothing for the field is not added; one carrying several values is not added. Each may change.
**A handle holds nothing, and there is nothing to release.** It is one identity's for its whole life — an application changing identity within one page builds another, which is free.
**No cross-broker reach.** A NURI resolves for users of one broker.
**No depositor authentication and no rate limit.** Anyone may deposit any number of payloads into any index.
**No depositor authentication and no rate limit.** Anyone may hand any index any number of references, and what an index receives is untrusted: telling a reference from the rest is the business of whoever processes it.
**A BET, named as one — this layer's, not yours.** That a document can receive deposits at all is aligned with NextGraph; **what a deposit carries, and what receiving one does, are not** — upstream has defined no such shape and offers no hook to extend the one it has. When it does, this layer moves with it, under a `major`.
**A BET, named as one — this layer's, not yours.** That a document can receive at all is aligned with NextGraph; **what is handed over, and what receiving it does, are not** — upstream defines no such shape and offers no hook to extend the one it has. When it does, this layer moves with it, under a `major`.
## Change policy
@@ -124,12 +102,18 @@ That is everything reading takes: an anchored `SELECT ?object ?value WHERE { ?ob
- **minor** — a symbol is added and nothing existing moves: a new read helper, a new optional option.
- **patch** — a fix that changes neither the exported surface nor anything under `## Guarantees`, throw text included.
**A tag says where it comes from.** A release cut on `main` carries a **full version** (`3.0.0`); work on a branch carries a **pre-release** of the version it heads for (`3.1.0-dev.3`), which sorts below it, and between two pre-releases nothing is promised. Nothing you pinned is ever withdrawn. The tag is bare — `v3.0.0`.
**A tag says where it comes from.** A release cut on `main` carries a **full version** (`4.0.0`); work on a branch carries a **pre-release** of the version it heads for (`4.1.0-dev.3`), which sorts below it, and between two pre-releases nothing is promised. Nothing you pinned is ever withdrawn. The tag is bare — `v4.0.0`.
**`3.0.0` stops engaging on reading, and drops a re-export that never existed.** The guarantees describing what `read` does — ordering, the tie-break on NURI, a subject with several values, a leniently-read declaration — are gone, replaced by the storage shape under `## Surface`. `PrincipalId` leaves it too: listed as re-exported, never exported by `src/index.ts`. **Nothing to migrate and no call behaves differently** — `read` still orders exactly as before. A **major** under the rule above: the surface loses a symbol, and `## Guarantees` loses statements you may no longer rely on.
**`4.0.0` cuts this package to two acts.** Everything that made entries of what an index received has left it: that work is done by the layer below, on the inbox an index already has, so an index goes on filling without any of the calls listed here.
**`2.0.0` removed `Indexing.curate(index)` — and `CurationReport`, `CurationOutcome`, `SkipReason` with it — and made `indexing(port)` a promise.** **Migrating from `1.x`**: delete every call to `curate`, `await` the `indexing(port)` you already make, and where you read a `CurationReport` read the index instead. `2.0.1` was prose and one relaxed deployment requirement; nothing to migrate. `1.0.1` and `1.0.0` keep resolving.
- **`indexing(port)` → `indexing(sessionId)`, and it no longer returns a promise.** Pass the session id you were passing to `polyfillPort`, and delete the `polyfillPort({ sessionId })` call: this package does that wiring itself now, so `polyfillPort` and `PolyfillPortOptions` go with it. `await indexing(…)` still compiles — there is simply nothing left for it to wait for.
- **`createIndex(field)` → `create(field)`**, and **`refer(index, object)` → `add(index, object)`.** Same behaviour, same throws; `add` is the verb every index carries, and it already says added-now-visible-later.
- **`read(index)` is gone — read the document.** `readUnion([index])` gives you its subjects: skip the one whose subject is the index itself, take `ENTRY_VALUE` off the rest, or run the anchored `SELECT` under `## Surface`. Ordering is yours now; the call you are replacing ordered by value, broke ties on the object's NURI, and refused a document declaring no field.
- **`decodeReference` and `IndexDeposit` are gone**, with the types that served only the calls above: `NextGraphPort`, `IndexEntry`, `IndexDescriptor`, `ObjectResolution`, `IncomingDeposit`, `UnionSubject`. `Nuri`, `NuriLike` and `Indexing` stay.
- **Nothing replaces the guarantees about processing, and nothing needs to.** `3.0.0` engaged that awaiting a handle applied what an identity's indexes had received, kept applying it, caught up a backlog, resolved each reference, passed over an object carrying nothing or too much, and refused an index declaring two fields. None of that is engaged here any more. If your application relied on a handle being what made an index fill, it no longer is — and no longer has to be: the index fills whether that handle exists or not.
Cut on `main`: pin `3.0.0`, and anchor your `usage_` leaf on `against: @ng-helpers/indexing@3.0.0`.
**`3.0.0` stopped engaging on what `read` returned** — its ordering, its tie-breaks, what it made of a subject carrying several values — replacing those guarantees with the storage shape under `## Surface`, and dropped `PrincipalId`, listed as re-exported and never exported. **`2.0.0` removed `Indexing.curate(index)`**, with `CurationReport`, `CurationOutcome` and `SkipReason`. `2.0.1`, `1.0.1` and `1.0.0` keep resolving.
Cut on `main`: pin `4.0.0`, and anchor your `usage_` leaf on `against: @ng-helpers/indexing@4.0.0`.
**No changelog file and no deprecation window: the sections above are the release note.** Diff `## Guarantees`, `## Non-guarantees` and `## Surface` between two pulls.
README.md
+20 -43
View File
@@ -6,80 +6,57 @@ Helpers built **on top of** NextGraph. The first — and for now the only — on
NextGraph has no indexing concept, at any level, and will not grow one. This is not scaffolding standing in for something that will arrive upstream; it is a construction *above* NextGraph, and it stays in its own repository so that the two are never confused.
So the dependency runs one way and only one way: **this repo depends on `@ng-eventually/polyfill`, and the polyfill must never learn anything about indexing.** No index vocabulary, no index type, no inbox variant that knows what a deposit means. The polyfill's inbox stays generic and carries opaque deposits; what a deposit *means* is decided here.
So the dependency runs one way and only one way: **this repo depends on `@ng-eventually/polyfill`, never the reverse.**
That boundary is held by one file. `src/polyfill-adapter.ts` is the only place that imports the polyfill at runtime; everything else is written against `NextGraphPort` (`src/port.ts`), a small interface describing what this layer needs. Only entries listed in that package's published `contract_polyfill-surface` are used — nothing is reached for inside it.
That boundary is held by one file. `src/polyfill-adapter.ts` is the only place that imports the polyfill at runtime; everything else is written against `NextGraphPort` (`src/port.ts`), a four-operation interface describing what this layer needs. Only entries listed in that package's published `contract_polyfill-surface` are used — nothing is reached for inside it.
## What an index is
**An ordinary document.** Nothing marks it as an index. What makes it one is that an application references its NURI in its own source.
**An ordinary document.** Nothing marks it as an index. What makes it one is that it declares a field, and that an application references its NURI in its own source.
- Any user may create one, and its creator owns it.
- It lives in the creator's **public store**, so any reader can open it from the reference alone.
- It declares, on its own subject, the **field** it indexes by — a predicate, since the objects are RDF. An index "by a date" is simply an index whose field is a date predicate: there is no separate kind of index. Entries come out in chronological order because ISO-8601 sorts as a string.
- Each entry is a subject keyed by the indexed object's NURI, carrying that object's value for the field.
**Contributing is a deposit, not a write.** Creating an object that belongs in an index means depositing into the index document's inbox — `inbox.postToDocument`, which is exactly "reach this document's owner" and is open to anyone. Nobody but the owner ever writes the index.
**Contributing is a deposit, not a write.** Handing an index a reference means depositing into the index document's inbox — `inbox.postToDocument`, which is exactly "reach this document's owner" and is open to anyone. Nobody but the owner ever writes the index.
**A deposit is a bare reference. Nothing else.** It states no claim and gives no instruction: no operation, no index reference (the inbox address already identifies the index), no copy of the indexed value. When the owner curates, it resolves the reference and opens the object itself — which it can, because indexing is limited to public data for now. What the object says is what goes in. This is the shape NextGraph already uses upstream, where a `SocialQueryRequest` carries a reference to an RDF definition and the recipient composes its own update; a payload carrying an operation would be a licence for anyone to rewrite someone else's document.
**What travels is a bare reference. Nothing else.** It states no claim and gives no instruction: no operation, no index reference (the inbox address already identifies the index), no copy of the indexed value. Whoever processes it opens the object itself and takes what the object says. This is the shape NextGraph already uses upstream, where a `SocialQueryRequest` carries a reference to an RDF definition and the recipient composes its own update; a payload carrying an operation would be a licence for anyone to rewrite someone else's document.
**Curating is not an act an application performs.** There is no `curate` to call, and there never will be: it would ask an application to decide who owns an index and when curation runs, and neither is its decision. Curation is what happens when the index's inbox is processed — **at its creator's next connection, and on each deposit while the creator is connected**. `await indexing(port)` IS that connection: it goes through the inbox of every index that identity owns, and leaves each one watched. `src/observation.ts` is where that lives.
**Reading needs nothing new.** An application that knows the NURI calls `readUnion([indexNuri])` and gets the entries as subjects, or runs an anchored `SELECT`. There is no read helper here, and that absence is deliberate: one would only teach a shape a caller has to unlearn.
**Reading needs nothing new.** An application that knows the NURI calls `readUnion([indexNuri])` and gets the entries as subjects. `Indexing.read` is sugar over exactly that, dropping the index's own declaration subject.
## What this package does, and what it deliberately does not
## An index only ever grows
Two acts — `create(field)` and `add(index, object)` — and its whole job is to hide that an index is implemented by an inbox. An application names documents and never an address.
**Nothing is ever removed from an index — by anyone, including its owner.** An index only ever grows. There is no removal function, and there never was one: removal was deliberately **never built**, not built and then withdrawn, and nothing is planned. This is written down here rather than left to be deduced from a missing function, because someone who needs an entry gone should learn that it was never possible instead of hunting for the call that does it. Today the only answer to that need is a fresh index.
**Nothing here makes an entry of what an index receives.** No resolving a reference, no opening the object, no writing an entry, no reading a document or an inbox, no going through a store to find one's own indexes, nothing watching. That work belongs to the layer that processes an index's inbox, which is where it now lives — and this package has no call that does it, aims it, or asks about it. A reference handed to an index becomes an entry without anything on this surface being involved.
It is what makes the failure story trivial. Since the only write is an addition, a reference that does not resolve — the object is gone, or unreadable, or the broker simply did not answer — can only ever mean *not added this time*. It cannot damage what is already there, and a later deposit adds it. Nothing has to tell an absence from a failure, so nothing can get that wrong.
That is why `indexing(sessionId)` is not a promise and reaches nothing: a handle is a session id and two acts.
That distinction is worth stating, because getting it wrong is a defect this family of code has already produced: in `ng-eventually-js`, `resolveAccount` returned `null` on a failed lookup, so a read that *failed* came back as "this account does not exist" — a failure disguised as an absence, which silently broke document sharing (fixed in `8c8ade7` and `e32b6d0`).
**An index still only ever grows, and this package still cannot express a removal.** `src/sparql.ts` builds exactly one statement — an anchored `INSERT DATA` — and there is no delete builder. What enforces that is not a pattern over source: `test/adapter.test.ts` runs the real adapter on `test/fake-polyfill.ts`, an in-memory polyfill whose SPARQL is **executed** by an engine that understands one statement and refuses everything else. A removal is therefore not *detected*, it is **unrunnable**: `DELETE WHERE …` fails on the first keyword, a second statement smuggled after the closing brace fails on the trailing text, a keyword hidden inside a literal stays inside the literal because a parser tokenises where a regex only matches, and splitting the keyword across concatenated strings buys nothing, since it is one string by the time it arrives. The regex over `src/` in `test/units.test.ts` stays as a cheap tripwire that names the file early; it is not the proof.
`NextGraphPort` has no operation that removes, and nothing in `src/` can express a deletion — but **the write path was never where this went wrong.** Three rounds of adversarial review broke the guarantee five times, and not one of the breaks involved deleting anything. Every one was on the **read** path:
- reading an entry required *exactly one* value, so a subject carrying two read as **absent** — and a second `addLiteralProperty`, the only write this package has, made an entry vanish while both triples sat in the document. Two curation runs racing each other produce exactly that;
- the same rule on the declared field was worse: one add-only write of a second `INDEX_FIELD` made the descriptor unreadable and took **every entry in the document** with it, permanently;
- one subject that is not a NURI threw out of `entriesOf` and made every real entry unreadable at once;
- a field named `constructor` or `toString` read back an inherited function from the plain object `readUnion` builds, crashing curation for every remaining deposit;
- and the fix for the second of these silently **corrupted** the index instead: "smallest wins" switched the field while entries already written kept their old one, so `read()` returned a single list "ordered by value" whose values came from two different properties.
What holds now: entries take **at least one value, smallest wins**, deterministically. Reading is **per-entry tolerant** — a stray subject is skipped, never thrown on — and reads **own properties only**. Reading an index asks only *is this an index?*; the field is required only to **curate**, and an ambiguous one refuses loudly rather than picking, because a quiet wrong answer is worse than a loud stop. That refusal is permanent, which is the honest price of having no delete, and the message says so instead of suggesting a retry.
The lesson is worth keeping: **"nothing removes" is a claim about the write path, and an invariant about what a reader can *see* has to be checked on the read path too.**
What enforces the write half is no longer a pattern over source. `test/adapter.test.ts` runs the real adapter on `test/fake-polyfill.ts`, an in-memory polyfill whose SPARQL is **executed** by an engine that understands one statement — an anchored `INSERT DATA` of literal triples — and refuses everything else. A removal is therefore not *detected*, it is **unrunnable**: `DELETE WHERE …` fails on the first keyword, a second statement smuggled after the closing brace fails on the trailing text, a keyword hidden inside a literal stays inside the literal because a parser tokenises where a regex only matches, and splitting the keyword across concatenated strings buys nothing, since it is one string by the time it arrives. The regex over `src/` in `test/units.test.ts` stays as a cheap tripwire that names the file early; it is not the proof.
**A failed resolve is still a failure, and still surfaces.** Harmless is not the same as invisible. Nobody asked for the run, so there is nobody to hand a report to: every reference that could not be resolved is warned about on this package's log stream, and so is a run, a watch or a store read that could not happen at all. None of them denies the application anything — reading an index and depositing into one never depended on that work, and the deposits stay in their inbox for the next connection.
**The lesson that came out of the removed half is worth keeping**, because it was learned the expensive way: *"nothing removes" is a claim about the write path, and an invariant about what a reader can see has to be checked on the read path too.* Three rounds of adversarial review broke the guarantee five times and not one break involved deleting anything — every one was on the read path (an entry requiring exactly one value read as absent when a race gave it two; the same rule on the declared field took every entry with it; one non-NURI subject threw out of the whole listing; a field named `constructor` read back an inherited function). It belongs with whoever holds the read path now.
## Open questions
Deliberately not settled. Each is implemented in its narrowest form and reported rather than generalised.
Deliberately not settled.
- **An object that carries nothing for the index's field.** Narrow behaviour: it is not added, and nothing says so. There is no key to index it by, and inventing one — a placeholder, the deposit's timestamp — would put something in the index that the object does not say. Whether it should instead be indexed under an absent key, or refused louder, is open.
- **An object that carries several values for the field.** Not added either. Which of them the entry would hold has not been decided.
- **An already-indexed object is never re-read.** Curation skips it outright, so an object whose field value changes later keeps its original value in the index. Refreshing it would be a write nobody asked for, and it is the same question as removal.
- **Which value a raced entry should keep.** Two curation runs racing each other can leave an entry with two values; the smallest is chosen so that readers agree and the entry stays visible. That the entry must survive is settled; *which* of the two it should hold is not.
Note what this means, since the two points above are easy to read as one: an entry's value **can** change after it is made. Not by re-reading the object — that never happens — but because a *second* value arriving for the same entry can be the smaller one, and `read()` returns the smallest. Index an object at `2026-05-05`, let a raced run add `2026-01-01`, and `read()` answers `2026-01-01`. What never changes is the set of entries and the fact that each stays visible; the value one of them reports is settled by "smallest wins", not by arrival order.
- **How an index recovers from an ambiguous declaration.** Today it does not: curation refuses for good and the only way forward is a fresh index. Since nothing here removes anything, giving it a way back needs a mechanism that does not exist yet.
- **Deposits are never retired.** Every curation run sees every deposit ever made. That is affordable because re-applying one is a no-op, but it is linear in the history. How a curator retires an applied deposit is open — `inbox.processInbox` may be the answer, but its semantics are not published.
- **How an owner finds its own indexes.** Nothing marks a document as an index, so connecting reads that identity's whole public store and looks at each document. It is the only question the surface can ask, and it does not scale with a large store. A marker on the document, or a listing narrowed by shape, would both settle it — and both are decisions about what an index *is*.
- **Nothing stops a handle.** `inbox.watch`'s unsubscribe is dropped, so a session watches its own indexes until the page goes. An application that changes identity in one page has to drop the old handle and know that it goes on watching.
- **What an entry holds besides the object reference and the field value**, and **how several index kinds would coexist**, are both untouched.
- **Nothing marks a document as an index** except the field it declares. That is enough to recognise one you already hold a reference to, and not enough to discover one.
The questions about what becomes of a reference — an object carrying nothing for the field, or several values; whether an already-indexed object is ever re-read; what a raced entry keeps; whether a processed deposit is ever retired — are no longer this package's to answer, and moved with the code that answered them.
## Layout
| Path | What it is |
| --- | --- |
| `src/index.ts` | Everything published: one function, two acts, two IRIs |
| `src/indexing.ts` | The two acts, bound to one identity |
| `src/port.ts` | What this layer needs from NextGraph, and nothing more |
| `src/polyfill-adapter.ts` | The only runtime import of `@ng-eventually/polyfill` |
| `src/deposit.ts` | The deposit's shape: a bare reference |
| `src/index-document.ts` | An index's declaration and its entries |
| `src/curator.ts` | Resolving references and adding what is there |
| `src/observation.ts` | When that happens: this identity's indexes, caught up and watched |
| `src/coalescing.ts` | Runs a job, never twice at once, and grants exactly one more run |
| `src/indexing.ts` | The public surface, bound to one identity |
| `src/nuri.ts` | What a reference is, checked before it goes anywhere |
| `src/sparql.ts` | The one statement this package writes — no deletion exists |
| `src/vocabulary.ts` | The two IRIs an index document is written with |
| `test/fake-nextgraph.ts` | An in-memory NextGraph behind `NextGraphPort`, enforcing the polyfill's published guarantees |
| `test/fake-polyfill.ts` | An in-memory `@ng-eventually/polyfill` that **executes** the SPARQL — an engine that can only add |
| `test/adapter.test.ts` | Runs the real adapter on it: behaviour, every query emitted, and every method driven |
e2e/bridge.ts
+33 -14
View File
@@ -14,13 +14,27 @@
* ends up typed as a reference.
*/
import type { IndexEntry, UnionSubject } from "../src/index";
import type { UnionSubject } from "@ng-eventually/polyfill";
/**
* One entry, as THIS APPLICATION decodes one out of the index document.
*
* It is declared here and not imported, because `@ng-helpers/indexing` publishes no
* such type and no call that produces one: an index is an ordinary document, so its
* contents are whatever `readUnion` hands back, read with the two published IRIs.
* A subject carrying several values shows up as several rows — the document's own
* truth, and the application's business to make something of.
*/
export interface IndexRow {
readonly object: string;
readonly value: string;
}
/** What the leak probe observed — see `run.ts`'s last journey. */
export interface BrokenInboxOutcome {
/** The message `createIndex` rejected with, or `null` if it did not reject. */
/** The message `create` rejected with, or `null` if it did not reject. */
readonly rejected: string | null;
/** What `createIndex` returned, on the impossible branch where it did not reject. */
/** What `create` returned, on the impossible branch where it did not reject. */
readonly returned: string | null;
/** The documents that appeared in this identity's public store despite the failure. */
readonly appeared: readonly string[];
@@ -75,20 +89,25 @@ export interface IndexingBridge {
/** Publish a public document carrying one value for one predicate. */
publishObject(predicate: string, value: string): Promise<string>;
/** Hand the CONFIGURED index a reference to an object. Anyone may. */
referConfigured(object: string): Promise<void>;
addToConfigured(object: string): Promise<void>;
/** Hand a NAMED index a reference — used where no identity boundary is crossed. */
referTo(index: string, object: string): Promise<void>;
addTo(index: string, object: string): Promise<void>;
/**
* Connect again — obtain a fresh `Indexing` handle, which is what a page load does.
* Obtain a fresh `Indexing` handle, which is what a page load does.
*
* There is no curating act to drive: an index is curated at its creator's next
* connection and on each deposit while the creator is connected. This is the first
* of the two, driven deliberately so a journey has a point at which the catching
* up is over; the second needs nothing from anyone.
* It reaches NOTHING — a handle is a session id and two acts, and building one talks
* to nobody. That is precisely what one journey asserts: taking a handle again changes
* nothing an index holds. It is NOT a settle point and cannot be used as one; what a
* journey waits on after a deposit is the INDEX, read like any other document (`run.ts`,
* `settled`).
*/
reconnect(): Promise<void>;
/** The index's entries, ordered by value. */
read(index: string): Promise<IndexEntry[]>;
rebuildHandle(): Promise<void>;
/**
* The index's entries, ordered by value — decoded BY THIS APPLICATION, out of an
* ordinary `readUnion` of the index document, with nothing from the library but the
* two IRIs it publishes. That the suite can do this at all is the claim under test.
*/
read(index: string): Promise<IndexRow[]>;
/** What a document literally holds, straight off `readUnion` — the write-form probe. */
readRaw(doc: string): Promise<UnionSubject[]>;
@@ -105,7 +124,7 @@ export interface IndexingBridge {
listPublicDocs(): Promise<string[]>;
/**
* `createIndex` with its inbox step made to fail — everything else real, against the
* `create` with its inbox step made to fail — everything else real, against the
* real broker. Answers whether a half-created index is left behind.
*/
createIndexWithBrokenInbox(field: string): Promise<BrokenInboxOutcome>;
e2e/indexing-app.ts
+79 -41
View File
@@ -7,20 +7,27 @@
* indexing RULES are consistent; they cannot prove that NextGraph does what the fake
* pretends, because the fake is the thing being asked. This page closes that gap by
* putting the real broker underneath: it imports `@ng-eventually/polyfill` for real,
* crosses the real broker, and calls `indexing(polyfillPort(...))` exactly as an
* application would.
* crosses the real broker, and calls `indexing(sessionId)` exactly as an application
* would.
*
* It reaches nothing private. Every import below is a published entry — of the polyfill
* (`configure`, `ensureIdentity`, `init`, `readUnion`, `storeRegistry`) or of this
* package (`indexing`, `polyfillPort`). If something here is awkward, it is awkward for
* every consumer, which is the second reason to write it this way.
* What this application takes from `@ng-helpers/indexing` is its WHOLE published
* surface: `indexing(sessionId)`, the two acts on what it hands back, and the two IRIs.
* Nothing here makes an entry of a deposit, and nothing here can ask for one: that is
* the business of the layer below, and this page is where that shows or does not.
* Everything else here is the polyfill (`configure`, `ensureIdentity`, `init`,
* `readUnion`, `docs`, `storeRegistry`) — including reading the index, which is an
* ordinary `readUnion` of an ordinary document. If something here is awkward, it is
* awkward for every consumer, which is the second reason to write it this way.
*
* The one thing here no application does
* `createIndexWithBrokenInbox` injects a failure into the inbox step of `createIndex`.
* That is a probe, it is named for what it is, and it exists because the question it
* answers — does a failed `openInbox` leave a document behind? — cannot be reached from
* outside: nothing a caller controls makes a real `openDocumentInbox` fail on demand.
* Everything around the injection is real, including the broker and the document.
* The two things here no application does, and why they reach past the surface
* `createIndexWithBrokenInbox` injects a failure into the inbox step of `create`: the
* question it answers — does a failed `openInbox` leave a document behind? — cannot be
* reached from outside, because nothing a caller controls makes a real
* `openDocumentInbox` fail on demand. And `publishObject` writes through the very
* primitive `create` declares an index's field with, which is what makes it the CONTROL
* for the write-form question. Both therefore build the package's INTERNAL port, and
* both are named for what they are. Everything around them is real, including the
* broker.
*/
import {
@@ -35,9 +42,13 @@ import {
} from "@ng-eventually/polyfill";
import { ng as realNg, init as realInit } from "@ng-org/web";
import { indexing, polyfillPort } from "../src/index";
import type { IndexEntry, Indexing, NextGraphPort } from "../src/index";
import type { BrokenInboxOutcome, IndexingBridge, SelectOutcome } from "./bridge";
// The whole published surface of the package under test.
import { ENTRY_VALUE, indexing, type Indexing } from "../src/index";
// NOT published, and reached only by the two probes above — see the header.
import { indexingOn } from "../src/indexing";
import type { NextGraphPort } from "../src/port";
import { polyfillPort } from "../src/polyfill-adapter";
import type { BrokenInboxOutcome, IndexRow, IndexingBridge, SelectOutcome } from "./bridge";
// bootstrap: the one polyfill-era call, then the SDK-shaped ones
//
@@ -54,7 +65,7 @@ configure({
// The library's `init`, not the injected one: it settles the identity BEFORE handing the
// page to the broker, so the round-trip leaves with `?ng-id=` in the address it carries.
// The callback is this application's own business — it keeps the session because
// `polyfillPort` takes a session id, exactly as the real SDK's primitives do.
// `indexing(sessionId)` takes one, exactly as the real SDK's primitives do.
const sessionReady = new Promise<{ session_id: string }>((resolve) => {
init(
(event: { status: string; session?: { session_id: string } }) => {
@@ -74,7 +85,6 @@ const state: { status: string; error: string | null; who: string } = {
};
let api: Indexing | null = null;
let port: NextGraphPort | null = null;
/** The index this deployment contributes to, read off its own configuration. */
function configuredIndex(): string | null {
@@ -86,11 +96,11 @@ async function boot(): Promise<void> {
// and the identity comes back. The application keeps it only to show it.
state.who = await ensureIdentity();
const session = await sessionReady;
port = polyfillPort({ sessionId: session.session_id });
// One await, and curation is part of it: obtaining the handle processes the inboxes
// of the indexes this identity owns and leaves them watched. This application never
// curates anything, and has nothing to call if it wanted to.
api = await indexing(port);
// The session id, and nothing else — this application never names a port, and never
// awaits anything here: a handle reaches nothing. A reference this application hands
// an index becomes an entry because the layer below processes that index's inbox,
// and there is nothing to call, schedule or configure for it.
api = indexing(session.session_id);
state.status = "ready";
}
@@ -107,11 +117,13 @@ function ready(): Indexing {
return api;
}
function readyPort(): NextGraphPort {
if (port === null) {
throw new Error(`[e2e] the application is not ready (${state.status}): ${state.error ?? "still connecting"}`);
}
return port;
/**
* The package's INTERNAL port, for the two probes that need one. Never used by an act
* this application performs as an application — see the header for why each needs it.
*/
async function probePort(): Promise<NextGraphPort> {
const session = await sessionReady;
return polyfillPort({ sessionId: session.session_id });
}
/**
@@ -159,6 +171,10 @@ function render(result: unknown): string {
* `surface/read-model.ts`). A binding whose term carries no string `value` is dropped
* rather than guessed at; `raw` beside it is what keeps that honest.
*/
function compare(a: string, b: string): number {
return a < b ? -1 : a > b ? 1 : 0;
}
function rowsOf(result: unknown): Array<Record<string, string>> {
if (result === null || typeof result !== "object") return [];
const answered = result as {
@@ -183,42 +199,64 @@ const bridge: IndexingBridge = {
configuredIndex,
async createIndex(field: string): Promise<string> {
return ready().createIndex(field);
return ready().create(field);
},
/**
* Publish a public document carrying one value for one predicate.
*
* It goes through the SAME primitive the curator writes an entry with
* It goes through the SAME primitive `create` declares an index's field with
* (`addLiteralProperty`), with the document as its own subject. That makes it the
* CONTROL for the write-form question: if this round-trips and an index entry does
* not, the difference is the foreign subject and nothing else.
*/
async publishObject(predicate: string, value: string): Promise<string> {
const p = readyPort();
const p = await probePort();
const doc = await p.createPublicDocument();
await p.addLiteralProperty(doc, doc, predicate, value);
return doc;
},
async referConfigured(object: string): Promise<void> {
async addToConfigured(object: string): Promise<void> {
const index = configuredIndex();
if (index === null) {
throw new Error("[e2e] this application was not configured with an index reference");
}
await ready().refer(index, object);
await ready().add(index, object);
},
async referTo(index: string, object: string): Promise<void> {
await ready().refer(index, object);
async addTo(index: string, object: string): Promise<void> {
await ready().add(index, object);
},
async reconnect(): Promise<void> {
api = await indexing(readyPort());
async rebuildHandle(): Promise<void> {
const session = await sessionReady;
api = indexing(session.session_id);
},
async read(index: string): Promise<IndexEntry[]> {
return ready().read(index);
/**
* The index read BY THIS APPLICATION, with nothing the package publishes but
* `ENTRY_VALUE` — the claim "an index is an ordinary document" performed rather than
* repeated. `readUnion([index])` is the same call this page makes on any other
* document; the index's own subject is the one that is not an entry, told apart by
* being the document itself.
*/
async read(index: string): Promise<IndexRow[]> {
const subjects = await readUnion([index]);
const rows: IndexRow[] = [];
for (const subject of subjects) {
if (subject.subject === index) continue;
for (const value of subject.props[ENTRY_VALUE] ?? []) {
rows.push({ object: subject.subject, value });
}
}
// Ordered by value, ties broken on the object, so two readers of the same document
// see the same order. The package used to do this; an application does it in four
// lines, and gets to choose differently.
rows.sort((a, b) =>
a.value === b.value ? compare(a.object, b.object) : compare(a.value, b.value),
);
return rows;
},
async readRaw(doc: string): Promise<UnionSubject[]> {
@@ -249,13 +287,13 @@ const bridge: IndexingBridge = {
},
async createIndexWithBrokenInbox(field: string): Promise<BrokenInboxOutcome> {
const p = readyPort();
const p = await probePort();
const before = new Set<string>(await storeRegistry.listMyEntityDocs("public"));
// Everything real except the inbox step. The failure is injected at the exact moment
// the question is about: after the document exists and carries its descriptor, before
// anyone can deposit into it.
const broken = await indexing({
const broken = indexingOn({
...p,
openInbox: async (): Promise<void> => {
throw new Error("[e2e] injected: the inbox could not be opened");
@@ -265,7 +303,7 @@ const bridge: IndexingBridge = {
let rejected: string | null = null;
let returned: string | null = null;
try {
returned = await broken.createIndex(field);
returned = await broken.create(field);
} catch (e: unknown) {
rejected = String((e as Error)?.message ?? e);
}
e2e/run.ts
+157 -41
View File
@@ -14,7 +14,7 @@
* makes it the control: if one round-trips and the other does not, the difference is
* the foreign subject and nothing else.
*
* 2. **A half-created index.** `createIndex` creates a document, writes its descriptor,
* 2. **A half-created index.** `create` creates a document, writes its descriptor,
* then opens its inbox. If the last step fails the caller gets an exception and no
* reference — but the document exists. The last journey injects that failure and
* asks the broker what was left behind.
@@ -28,6 +28,23 @@
* never happen — and does not happen here — is an inbox address crossing the identity
* boundary through a channel no deployment has.
*
* Waiting, because nothing here says when processing is done
* The two acts are `create` and `add`; what becomes of what is added is the business of
* the layer below, which applies a deposit when the owner's session is PUSHED one. There
* is no call that forces it, no receipt, and no `await` that covers it — by design. So a
* journey that has deposited WAITS for the index to hold the entry, reading it the way an
* application reads one (`settled`), and only then asserts. Before that wait existed the
* checks read the instant after the deposit and seven of them reported an index that was
* merely not written YET.
*
* The one claim that wait cannot carry: "an object carrying nothing for the field is not
* indexed". A deposit that is deliberately not indexed writes nothing, so no reading tells
* "examined and skipped" from "not examined yet", and there is nothing to wait for that is
* not a sleep. That journey reads straight away and its check is therefore weaker than its
* name — recorded here rather than hidden. What partly holds it up is order: that deposit
* precedes the hostile one, and the hostile one IS waited for, so by the end of the run the
* queue holding it has demonstrably been applied at least once.
*
* Reading a failure
* A named deadline, or a message `ng-e2e-helpers` recognises as a browser or frame
* failure, is the HOST. A failed check carrying an unexpected value is this code. The
@@ -77,7 +94,7 @@ type Frame = Awaited<ReturnType<typeof setupBrokerPage>>;
// A date, so the suite exercises the case the package is built around: an index "by a
// date" is just an index whose field is a date predicate, and ISO-8601 sorts as a string.
const PUBLISHED_AT = "urn:ng-helpers-e2e:published-at";
/** A predicate an index does NOT curate on — for the object that carries nothing usable. */
/** A predicate an index is NOT built on — for the object that carries nothing usable. */
const UNRELATED = "urn:ng-helpers-e2e:unrelated";
// bounds
@@ -93,9 +110,48 @@ const BRIDGE_UP_MS = 60_000;
const READY_MS = 180_000;
/** One sign-in: a page, the broker round trip, and the application booting behind it. */
const SIGN_IN_MS = NEW_PAGE_MS + BROKER_ROUND_TRIP_MS + READY_MS;
/** One call across the bridge. The slowest here are curations, which round-trip per deposit. */
/** One call across the bridge. The slowest cross the broker once per deposit. */
const BRIDGE_MS = 4 * 60_000;
/** One journey. The longest holds two sign-ins' worth of work behind it. */
/**
* How often the index is READ while waiting for a deposit to have become an entry.
*
* Not a sleep standing in for the wait: it is the interval at which the CONDITION is asked,
* and the wait ends on the first reading that holds. `indexing-app.ts` polls its own store
* at the same interval, for the same reason.
*/
const SETTLE_POLL_MS = 500;
/**
* How long a deposit has to become an entry before the wait gives up and says what the
* index held instead.
*
* MEASURED on a green run (`E2E_TIMINGS=1`, 2026-08-21, one sample each): 0.6s for Bob's
* first deposit becoming an entry, 0.7s for the hostile one, and 0.0s for the entry
* reaching a stranger's own session — that last one had already converged by the time it
* was asked. The layer below is push-driven, so what is waited on is one push and one small
* write, not a broker crossing; that is why these are sub-second while a deposit or a
* publish measures 0.1–0.9s.
*
* Bounded at 60s ≈ 85x the slowest of them, generous on purpose. It must never fire on a
* slow-but-healthy broker, and it is what turns "the entry never came" into a named failure
* carrying the last reading — instead of a check that merely read too early, which is what
* the seven failures it was written for looked like.
*/
const SETTLE_MS = 60_000;
/**
* One journey. The longest holds two sign-ins' worth of work behind it.
*
* NOT the sum of the steps it encloses, and that is deliberate — the same arbitration
* `packages/polyfill/e2e/notebook.ts` records for its own: the longest journey here sums to
* 16 min of step bounds (four bridge calls), and a journey bounded above that would outlast
* the SUITE's own clock, so one hung journey would take the summary down with it. Every step
* inside a journey already carries a bound and names itself, so this catches only a hang in
* code no step wraps. What DOES have to hold is that this enclosure cannot fire BEFORE the
* settle point it now encloses, or a settle that gave up would report as an anonymous
* journey timeout: measured on a green run, the longest journey holding a settle takes 1.6s
* and the longest of all (a sign-in) 9.8s, so a journey that spends SETTLE_MS (60s) waiting
* still has more than eight minutes of this bound left — the settle is always the one that
* reports.
*/
const JOURNEY_MS = 10 * 60_000;
/** The whole run. A budget that cannot interrupt anything is not a budget. */
const SUITE_MS = 30 * 60_000;
@@ -126,7 +182,7 @@ const { check, journey, finish } = declareSuite({
],
},
{
name: "Bob hands the index a reference, and Alice's next connection curates it",
name: "Bob hands the index a reference, and it becomes an entry of Alice's index",
checks: [
"a stranger's deposit into the index's inbox reached its owner and became an entry",
"the entry is stored under Bob's object's own reference as its subject",
@@ -204,6 +260,54 @@ function step<T>(what: string, ms: number, task: () => Promise<T>): Promise<T> {
return measured(what, ms, (bound) => within(what, bound, task));
}
/**
* Read the index until it HOLDS what the checks about to run expect, and hand that reading
* back to them.
*
* ── Why the harness needs this at all ───────────────────────────────────────
* Nothing on this package's surface says "processing is done", deliberately: the two acts
* are `create` and `add`, and what becomes of what is added belongs to the layer below,
* which applies a deposit when the owner's session is PUSHED one. There is no call that
* forces it and no receipt to hold. So a check that reads the instant after a deposit reads
* too early — which is exactly what seven of these checks were reporting.
*
* ── A wait is not a retry of the assertion ──────────────────────────────────
* `holds` asks one thing only: has the deposit ARRIVED. The checks then assert their own
* claims ONCE, on the reading this returns. A loop that re-ran the checks until one of them
* passed would turn a flapping result into a green one and report the reading that happened
* to suit it.
*
* ── It reads the way an application reads ───────────────────────────────────
* `read` is an ordinary `readUnion` of an ordinary document, or an anchored SELECT — the
* only two ways anyone has of reading an index, the harness included. Nothing here reaches
* for a way to hurry the layer below, because a consumer has none either.
*
* On expiry it throws, naming what never arrived and WHAT THE INDEX HELD INSTEAD: the
* journey's remaining checks are then reported unreached with that reason, so the run still
* reports its thirty rows and the reason is the observation rather than a bare "timeout".
*/
async function settled<T>(
what: string,
read: () => Promise<T>,
holds: (seen: T) => boolean,
describe: (seen: T) => string,
): Promise<T> {
return measured(what, SETTLE_MS, async (bound) => {
const deadline = Date.now() + bound;
for (;;) {
const seen = await within(what, bound, read);
if (holds(seen)) return seen;
if (Date.now() >= deadline) {
throw new Error(
`[e2e] ${what}: nothing arrived within ${(bound / 1000).toFixed(0)}s — ` +
`the last reading was ${describe(seen)}`,
);
}
await new Promise((r) => setTimeout(r, SETTLE_POLL_MS));
}
});
}
// an actor
interface Actor {
@@ -397,7 +501,7 @@ async function main(): Promise<void> {
});
await journey({
name: "Bob hands the index a reference, and Alice's next connection curates it",
name: "Bob hands the index a reference, and it becomes an entry of Alice's index",
needs: [
aliceIsUp,
bobIsUp,
@@ -408,21 +512,23 @@ async function main(): Promise<void> {
// Bob names his OWN object, and the index he was configured with. Nothing about
// the value travels: the deposit is the reference and nothing else.
await step("Bob depositing a reference", BRIDGE_MS, () =>
bob!.frame.evaluate((o) => window.__indexing.referConfigured(o), bobsObject!),
bob!.frame.evaluate((o) => window.__indexing.addToConfigured(o), bobsObject!),
);
// NOBODY CURATES: there is nothing on the surface to call. Alice's page connects
// again — what a page load does — and her session processes the inboxes of the
// indexes she owns, this one among them.
await step("Alice connecting again", BRIDGE_MS, () =>
alice!.frame.evaluate(() => window.__indexing.reconnect()),
);
// THE WRITE FORM, answered. An entry is a triple whose subject is another
// document, written into this one's anchored default graph. "The write did not
// throw" is not the same claim as "oxigraph stored it": this reads it back.
const raw = await step("Alice reading the index document back", BRIDGE_MS, () =>
alice!.frame.evaluate((d) => window.__indexing.readRaw(d), index!),
// NOBODY PROCESSES ANYTHING HERE, and nothing says when it is done: there is no
// such call on the surface, and no receipt. The reference becomes an entry because
// the layer below applies the inbox of the index Alice owns, when her session is
// pushed it — so the suite WAITS for the index to hold it, reading the document the
// way any application reads one.
//
// THE WRITE FORM, answered by the same reading. An entry is a triple whose subject
// is another document, written into this one's anchored default graph. "The write
// did not throw" is not the same claim as "oxigraph stored it": this reads it back.
const raw = await settled(
"Bob's deposit becoming an entry of Alice's index",
() => alice!.frame.evaluate((d) => window.__indexing.readRaw(d), index!),
(subjects) => subjects.some((s) => s.subject === bobsObject),
(subjects) => `subjects=${JSON.stringify(subjects.map((s) => s.subject))}`,
);
const entry = raw.find((s) => s.subject === bobsObject);
// The deposit is proven ARRIVED by its only possible effect: nobody but Alice
@@ -440,7 +546,7 @@ async function main(): Promise<void> {
`subjects=${JSON.stringify(raw.map((s) => s.subject))}`,
);
// The value never travelled: a deposit is the reference and nothing else, so its
// presence here means the curation read it off Bob's object itself.
// presence here means whatever processed that inbox read it off Bob's object itself.
check(
"the indexed value was read off Bob's object, and never travelled in his deposit",
(entry?.props[ENTRY_VALUE] ?? []).includes("2026-08-17T09:00:00Z"),
@@ -464,8 +570,15 @@ async function main(): Promise<void> {
// A public index is read by whoever holds its reference — including someone who
// owns neither it nor anything in it. This is the act an application performs.
const theirs = await step("Bob reading the index he does not own", BRIDGE_MS, () =>
bob!.frame.evaluate((i) => window.__indexing.read(i), index!),
//
// Waited for on its OWN account: the journey above settled ALICE's session, and
// Bob's is a different one reading a document he does not own. "The owner sees it"
// and "a stranger sees it" are two arrivals, and only one of them has happened.
const theirs = await settled(
"the entry reaching a stranger's session",
() => bob!.frame.evaluate((i) => window.__indexing.read(i), index!),
(rows) => rows.some((r) => r.object === bobsObject),
(rows) => `entries=${JSON.stringify(rows)}`,
);
check(
"Bob, who does not own the index, reads the same entry",
@@ -475,8 +588,8 @@ async function main(): Promise<void> {
// Deposits are never retired, so every run sees every deposit again. Convergence
// is what makes that affordable.
await step("Alice connecting a second time", BRIDGE_MS, () =>
alice!.frame.evaluate(() => window.__indexing.reconnect()),
await step("Alice loading her page a second time", BRIDGE_MS, () =>
alice!.frame.evaluate(() => window.__indexing.rebuildHandle()),
);
const still = await step("Alice reading the index again", BRIDGE_MS, () =>
alice!.frame.evaluate((i) => window.__indexing.read(i), index!),
@@ -504,13 +617,17 @@ async function main(): Promise<void> {
),
);
await step("Bob depositing the unrelated reference", BRIDGE_MS, () =>
bob!.frame.evaluate((o) => window.__indexing.referConfigured(o), other),
);
await step("Alice connecting after the unrelated reference", BRIDGE_MS, () =>
alice!.frame.evaluate(() => window.__indexing.reconnect()),
bob!.frame.evaluate((o) => window.__indexing.addToConfigured(o), other),
);
// NOTHING TO WAIT FOR, and it is worth saying rather than dressing up. A deposit
// that is deliberately not indexed writes nothing, so the index holds afterwards
// exactly what it held before and no reading can tell "examined and skipped" from
// "not examined yet". A wait for the state the check expects would return on its
// first reading and prove nothing; a wait for a duration would be a sleep. So this
// reads straight away, and the check is a weaker claim than it reads as — see the
// suite's header. What DOES hold it: the deposit is applied before the hostile one
// below, and that one is waited for.
const entries = await step("Alice reading the index once more", BRIDGE_MS, () =>
alice!.frame.evaluate((i) => window.__indexing.read(i), index!),
);
@@ -564,7 +681,7 @@ async function main(): Promise<void> {
const refused = await step("Alice trying to deposit into it", BRIDGE_MS, async () => {
try {
await alice!.frame.evaluate(
([i, o]) => window.__indexing.referTo(i!, o!),
([i, o]) => window.__indexing.addTo(i!, o!),
[leaked, bobsObject ?? leaked],
);
return null;
@@ -614,18 +731,17 @@ async function main(): Promise<void> {
);
await step("Bob depositing the hostile reference", BRIDGE_MS, () =>
bob!.frame.evaluate((o) => window.__indexing.referConfigured(o), object),
bob!.frame.evaluate((o) => window.__indexing.addToConfigured(o), object),
);
await step("Alice connecting after the hostile reference", BRIDGE_MS, () =>
alice!.frame.evaluate(() => window.__indexing.reconnect()),
);
// Read the index document RAW: it must still declare its own field. An injected
// `DROP ALL` that had taken effect would show up exactly here, as a descriptor
// that is no longer there — and `read()` alone could not tell that apart from an
// ordinary failure.
const after = await step("Alice reading the index after the hostile entry", BRIDGE_MS, () =>
alice!.frame.evaluate((d) => window.__indexing.readRaw(d), index!),
// Waited for as the first deposit was, and read RAW: the index must still declare
// its own field. An injected `DROP ALL` that had taken effect would show up exactly
// here, as a descriptor that is no longer there — and `read()` alone could not tell
// that apart from an ordinary failure.
const after = await settled(
"the hostile deposit becoming an entry",
() => alice!.frame.evaluate((d) => window.__indexing.readRaw(d), index!),
(subjects) => subjects.some((s) => s.subject === object),
(subjects) => `subjects=${JSON.stringify(subjects.map((s) => s.subject))}`,
);
const entry = after.find((s) => s.subject === object)?.props[ENTRY_VALUE] ?? [];
const descriptor = after.find((s) => s.subject === index)?.props[INDEX_FIELD] ?? [];
package.json
+2 -2
View File
@@ -1,9 +1,9 @@
{
"name": "@ng-helpers/indexing",
"version": "3.0.0",
"version": "4.0.0",
"private": true,
"type": "module",
"description": "An indexing layer built on top of NextGraph, via @ng-eventually/polyfill. An index is an ordinary public document; anyone may hand it a reference to an object, and that reference becomes an entry once the index's creator is connected.",
"description": "An indexing helper built on top of NextGraph, via @ng-eventually/polyfill. An index is an ordinary public document; anyone may hand it a reference to an object, and that reference becomes an entry through the layer below — this package hides that an index is implemented by an inbox.",
"main": "./src/index.ts",
"types": "./src/index.ts",
"exports": {
src/coalescing.ts
-41
View File
@@ -1,41 +0,0 @@
/**
* Runs a job, and never runs it twice at once: an ask that arrives while it is
* running earns exactly ONE more run afterwards, however many arrive.
*
* A burst of deposits produces a burst of notifications, and each one means the same
* thing — "look at this inbox again". Curation reads the whole inbox every time, so
* a run started after the last deposit landed already covers every deposit before
* it: running once per notification would re-read the same inbox N times to reach
* the same place. One more run is enough, and one more run is also NECESSARY — a
* deposit that lands while a run is in flight may have arrived after that run read
* the inbox, and dropping the ask would leave it unprocessed until the next
* connection.
*
* `run` must not reject: this returns the caller's own promise and nothing here
* turns a rejection into a report. Its one caller wraps a failing run in its own
* reporting before handing it over.
*/
export function coalescing(run: () => Promise<void>): () => Promise<void> {
let inFlight: Promise<void> | null = null;
let asked = false;
async function drain(): Promise<void> {
try {
do {
asked = false;
await run();
} while (asked);
} finally {
inFlight = null;
}
}
return function ask(): Promise<void> {
if (inFlight !== null) {
asked = true;
return inFlight;
}
inFlight = drain();
return inFlight;
};
}
src/curator.ts
-167
View File
@@ -1,167 +0,0 @@
import type { NextGraphPort, Nuri, NuriLike, UnionSubject } from "./port";
import { asNuri } from "./nuri";
import { decodeReference } from "./deposit";
import { descriptorOf, entriesOf, readIndexDocument, valuesOf } from "./index-document";
import { ENTRY_VALUE } from "./vocabulary";
/**
* Applying the references an index has received. Runs as the index's OWNER — only
* the owner reads the inbox, and only the owner writes the document.
*
* ## The curator does not act on a claim, it resolves the reference and looks
*
* A deposit is a bare reference. For each one the curator opens the object
* itself — which it can, because indexing is limited to PUBLIC data for now:
*
* - the object is **present** → it is added to the index, under the value it
* holds for the index's field, unless it is already there;
* - **anything else** → nothing at all happens.
*
* So there is no ownership check, and none is needed. A deposit is an invitation
* to re-examine, not an instruction: the truth is what the object says, not what
* the depositor says. A stranger who deposits a reference achieves exactly what
* the owner would have — which is what lets a passer-by who notices a missing
* entry nudge the index into re-checking it.
*
* ## AN INDEX ONLY EVER GROWS
*
* Nothing here removes an entry. Not "removal is guarded", not "removal needs
* proof" — there is no removal, and `NextGraphPort` has no operation that could
* express one.
*
* That is what makes the failure story trivial. Since the only write is an
* addition, a read that comes back empty — whether the object is gone, or
* unreadable, or the broker simply did not answer — can only ever mean "not added
* this time". It cannot damage what is already there, and a later deposit adds
* it. Nothing has to tell an absence from a failure, so nothing can get that
* wrong. Compare the defect this avoids: a failed read resolving as an absence,
* removed from `ng-eventually-js` in 8c8ade7 and e32b6d0 — "le silence n'est plus
* possible que sur une absence VÉRIFIÉE".
*
* The cost is stated plainly, because someone will eventually need it: an entry
* cannot be taken out of an index, by anyone, including the index's owner.
*
* ## A failure still has to be seen
*
* Harmless is not the same as invisible. Every reference that could not be
* resolved comes back as an `unresolved` outcome AND is warned about, because a
* failure that looks exactly like a normal outcome teaches nobody anything.
*
* ## Deposits are not consumed
*
* Nothing here retires an applied deposit, so every run sees every deposit again.
* That is affordable only because a deposit carries no instruction: re-applying
* one re-resolves the reference and lands on the same result. Curation is
* convergent, and its outcome does not depend on the order references arrive in.
*/
export type CurationOutcome =
/** Present, and now in the index under this value. */
| { readonly result: "indexed"; readonly object: Nuri; readonly value: string }
/** Present, and already in the index. Nothing was written. */
| { readonly result: "unchanged"; readonly object: Nuri }
/** Resolved, and deliberately not added. */
| { readonly result: "skipped"; readonly object: Nuri; readonly reason: SkipReason }
/**
* The reference could not be resolved — gone, unreadable, or the read failed;
* this layer does not tell those apart. Nothing was written, and nothing that
* was already in the index was touched. A later deposit will add it.
*/
| { readonly result: "unresolved"; readonly object: Nuri; readonly reason: string }
/** A payload in the inbox that is not a reference at all. */
| { readonly result: "foreign"; readonly reason: string };
export type SkipReason =
/** Present, but carries nothing for the index's field. OPEN QUESTION — see README. */
| "no-field"
/** Present, but carries several values for the field: which one would the entry hold? */
| "several-values"
/** A reference to the index document itself. */
| "self-reference";
export interface CurationReport {
readonly index: Nuri;
/**
* One per deposit, in the order they were made. Check it for `unresolved`:
* those references were not added and are worth a retry or a look.
*/
readonly outcomes: readonly CurationOutcome[];
}
export async function curate(port: NextGraphPort, indexLike: NuriLike): Promise<CurationReport> {
const index = asNuri(indexLike);
const subjects = await readIndexDocument(port, index);
const descriptor = descriptorOf(subjects, index);
// What the index already holds, kept up to date as this run writes, so that the
// same reference deposited twice produces one entry.
const indexed = new Set<Nuri>(entriesOf(subjects, index).map((e) => e.object));
const deposits = await port.readDeposits(index);
const outcomes: CurationOutcome[] = [];
for (const deposit of deposits) {
const object = decodeReference(deposit.payload);
if (object === null) {
outcomes.push({ result: "foreign", reason: "payload is not a reference" });
continue;
}
if (object === index) {
outcomes.push({ result: "skipped", object, reason: "self-reference" });
continue;
}
if (indexed.has(object)) {
// Already there. Nothing is re-read and nothing is rewritten — an entry, once
// made, is never touched again.
outcomes.push({ result: "unchanged", object });
continue;
}
const resolution = await port.resolveObject(object);
if (resolution.state === "unresolved") {
console.warn(
`[ng-helpers/indexing] ${index}: reference not added — ${object}: ${resolution.reason}`,
);
outcomes.push({ result: "unresolved", object, reason: resolution.reason });
continue;
}
const values = fieldValues(resolution.subjects, descriptor.field);
if (values.length === 0) {
// NARROW BEHAVIOUR, and an open question: an object carrying nothing for the
// field is simply not added. There is no key to index it by, and inventing
// one (a placeholder, the deposit time) would put something in the index that
// the object does not say.
outcomes.push({ result: "skipped", object, reason: "no-field" });
continue;
}
const value = values[0];
if (values.length > 1 || value === undefined) {
outcomes.push({ result: "skipped", object, reason: "several-values" });
continue;
}
await port.addLiteralProperty(index, object, ENTRY_VALUE, value);
indexed.add(object);
outcomes.push({ result: "indexed", object, value });
}
return { index, outcomes };
}
/**
* Every value the object holds for the field, across all the subjects it carries.
*
* Goes through `valuesOf`, which reads OWN properties only. The field is named by
* the index document, so `constructor` and `toString` are fields an index can
* genuinely declare — and reading one off a plain object literal hands back an
* inherited FUNCTION where a list of values belongs.
*/
function fieldValues(subjects: readonly UnionSubject[], field: string): readonly string[] {
const values: string[] = [];
for (const subject of subjects) {
values.push(...valuesOf(subject, field));
}
return values;
}
src/deposit.ts
-37
View File
@@ -1,37 +0,0 @@
import type { Nuri } from "./port";
import { isNuri } from "./nuri";
/**
* What a depositor puts in an index document's inbox: **a bare reference, and
* nothing else**.
*
* Not an instruction — not "add this row", not even "I published this". The
* curator resolves the reference and looks; what it finds is what decides. This
* is the shape NextGraph already uses upstream: a `SocialQueryRequest` carries
* `definition_commit_body_ref` — a REFERENCE — and the recipient's
* `inbox_processor` composes its own SPARQL. Since anyone may deposit into any
* index, a payload that carried an operation would be a licence to rewrite
* someone else's document.
*
* It carries no index reference either. The deposit is addressed to the index
* document's inbox, and upstream an inbox belongs to exactly one repo — the
* address IS the identification. The polyfill says the same of its own inbox:
* "Tagging deposits with their document would be an invention consumers would
* have to unlearn at migration."
*
* And it carries no indexed data. Indexing is limited to PUBLIC objects
* precisely so that the curator can open the object itself; a value copied into
* the deposit would let a depositor put something in the index that the object
* does not say.
*
* What is left is the NURI. That is the whole payload.
*/
export type IndexDeposit = Nuri;
/**
* Anyone may deposit anything into an index's inbox, so every payload is
* untrusted input. Returns `null` for everything that is not a reference.
*/
export function decodeReference(payload: unknown): Nuri | null {
return isNuri(payload) ? payload : null;
}
src/index-document.ts
-223
View File
@@ -1,223 +0,0 @@
import type { NextGraphPort, Nuri, NuriLike, UnionSubject } from "./port";
import { asNuri, isNuri } from "./nuri";
import { ENTRY_VALUE, INDEX_FIELD } from "./vocabulary";
/**
* An index is an ORDINARY document. Nothing in NextGraph marks it as one; what
* makes it an index is that an application references its NURI in its own source,
* and that it holds the two things below.
*
* - **A descriptor**, on the document's own NURI as subject: the FIELD this index
* indexes by — a predicate, since the objects are RDF.
* - **One entry per indexed object**: subject = the object's NURI, carrying the
* value that object holds for the field.
*
* ## Why the field is declared IN the document
*
* The curator has to know which field to read off an object. The only other place
* to keep it is the application's source next to the NURI — and then two
* applications pointing at the same index could curate it on two different fields
* and corrupt each other's entries. Declared in the document, the index answers
* for itself and its creator's choice is the only one.
*
* The cost is one extra subject in `readUnion([indexNuri])`, told apart from the
* entries by being the index's own NURI. `read` drops it; a reader going straight
* to `readUnion` sees it.
*/
export interface IndexDescriptor {
/** The predicate an indexed object must carry. */
readonly field: string;
}
/** One row of an index, as an application reads it. */
export interface IndexEntry {
/** The indexed object — hand this straight back to `readUnion`. */
readonly object: Nuri;
/** What that object holds for the index's field. */
readonly value: string;
}
export async function writeDescriptor(
port: NextGraphPort,
index: Nuri,
descriptor: IndexDescriptor,
): Promise<void> {
await port.addLiteralProperty(index, index, INDEX_FIELD, descriptor.field);
}
/**
* The values a subject carries for a predicate.
*
* Reads OWN properties only, because `props` arrives as a plain object literal
* (`readUnion` builds `const props: Record<string, string[]> = {}`), so it
* inherits from `Object.prototype`. An index whose field is `constructor`,
* `toString` or `valueOf` would otherwise find a FUNCTION where a list of values
* belongs — verified: spreading it throws `TypeError: Spread syntax requires
* ...iterable[Symbol.iterator] to be a function`, which aborted curation for
* every remaining deposit and could not be undone, since the field cannot change.
*
* The non-string filter is the same caution: this data crosses a process boundary,
* and a value that is not a string has no business being compared with `<`.
*
* The two guards are deliberately redundant — either alone stops the crash, as
* mutation testing confirms (removing just one fails nothing; removing both fails
* a test). They are kept because they answer different questions: `hasOwn` asks
* whether the subject really carries this predicate, `Array.isArray` whether what
* came back has the shape the type promises.
*/
export function valuesOf(
subject: UnionSubject | undefined,
predicate: string,
): readonly string[] {
const props = subject?.props;
if (props === undefined || !Object.hasOwn(props, predicate)) return [];
const values: unknown = props[predicate];
if (!Array.isArray(values)) return [];
return values.filter((value): value is string => typeof value === "string");
}
/**
* The smallest of several values, or `undefined` when there are none.
*
* The one tie-break rule this package has, used everywhere a document may carry
* more values than expected. It is deterministic and order-independent, so every
* reader of the same document reaches the same answer — which is the property that
* matters, far more than WHICH value wins.
*/
function smallestOf(values: readonly string[]): string | undefined {
let chosen: string | undefined;
for (const value of values) {
if (chosen === undefined || value < chosen) chosen = value;
}
return chosen;
}
/**
* The value of an ENTRY: it exists as soon as its subject carries AT LEAST ONE.
*
* This is what makes "an index only ever grows" true, and requiring exactly one
* was a hole big enough to drive the whole guarantee through. Two values for one
* object — which two curation runs racing each other produce, since each only ever
* ADDS — made the entry read as ABSENT. The index could therefore SHRINK through
* nothing but additions: the invariant defeated by the very operation meant to
* uphold it, with no delete anywhere in sight.
*/
export function entryValue(subject: UnionSubject): string | undefined {
return smallestOf(valuesOf(subject, ENTRY_VALUE));
}
/**
* Is this document an index at all? Answering needs only that it declares SOME
* field — not which one.
*
* Reading entries is deliberately separated from curating them, because the two
* need different things and conflating them cost every entry in the document. An
* entry already written is a fact; it does not become unreadable because the
* declaration above it turned ambiguous.
*/
export function assertIndexDocument(subjects: readonly UnionSubject[], index: Nuri): void {
if (declaresIndexField(subjects, index)) return;
throw new Error(
`${index} declares no index field (${INDEX_FIELD}) — either it is not an index, ` +
"or it could not be read: an unreadable document and an empty one are the same " +
"empty result here. Nothing was written. Retry before concluding it is malformed.",
);
}
/**
* The same question as `assertIndexDocument`, asked without an opinion on the
* answer: is this document an index?
*
* Separate because the two callers want opposite things from a "no". A reader
* naming a document it believes to be an index wants the refusal. A session
* looking through its own public store for the indexes it owns wants a plain
* `false`: most of what it looks at is not an index and never claimed to be.
*/
export function declaresIndexField(subjects: readonly UnionSubject[], doc: Nuri): boolean {
const self = subjects.find((s) => s.subject === doc);
return valuesOf(self, INDEX_FIELD).length > 0;
}
/**
* The field to CURATE on — and here it must be unambiguous.
*
* Both earlier attempts at this were wrong, in opposite directions, and the
* second was worse than the first:
*
* - demanding exactly one and throwing out of the READ path meant a single
* add-only write of a second `INDEX_FIELD` made every entry in the document
* unreadable, permanently, since nothing here deletes;
* - "smallest wins" kept it readable but silently CORRUPTED it: entries already
* written are never re-read, so they keep their old field's values while new
* ones arrive under the new field, and `read()` returns a single list "ordered
* by value" whose values come from two different properties. A quiet wrong
* answer, which is worse than a loud failure.
*
* So: ambiguity refuses to CURATE, loudly, and leaves reading alone. Entries
* already in the index stay visible and correct; nothing new is added until a
* human looks. That the refusal is permanent is the honest price of having no
* delete — the message says so instead of pretending a retry will help.
*/
export function descriptorOf(subjects: readonly UnionSubject[], index: Nuri): IndexDescriptor {
const self = subjects.find((s) => s.subject === index);
const declared = valuesOf(self, INDEX_FIELD);
if (declared.length > 1) {
throw new Error(
`${index} declares ${declared.length} index fields (${declared.map((f) => JSON.stringify(f)).join(", ")}) ` +
"— refusing to curate rather than pick one, because entries already written under " +
"the other field are never re-read, and mixing them would leave one list ordered " +
"by two different properties. Reading the existing entries still works. This " +
"cannot be undone (nothing here deletes): curate into a fresh index.",
);
}
const field = declared[0];
if (field === undefined) {
throw new Error(
`${index} declares no index field (${INDEX_FIELD}) — either it is not an index, ` +
"or it could not be read: an unreadable document and an empty one are the same " +
"empty result here. Nothing was written. Retry before concluding it is malformed.",
);
}
return { field };
}
/**
* The entries, ordered by value. Ties are broken on the object NURI so that two
* readers of the same index always see the same order.
*
* Values are compared AS STRINGS — this layer does not know what the field means.
* An index whose field holds ISO-8601 dates therefore comes out in chronological
* order, which is the whole reason to index by a date.
*
* A subject that is not a NURI is SKIPPED, never thrown on. An index document is
* an ordinary document and its owner may put anything in it; a single stray
* triple must not be able to make every real entry unreadable at once. Reading is
* per-entry tolerant for the same reason the write path has no delete — what a
* reader could already see has to keep being visible.
*/
export function entriesOf(subjects: readonly UnionSubject[], index: Nuri): IndexEntry[] {
const entries: IndexEntry[] = [];
for (const subject of subjects) {
if (subject.subject === index) continue;
if (!isNuri(subject.subject)) continue;
const value = entryValue(subject);
if (value === undefined) continue;
entries.push({ object: asNuri(subject.subject), value });
}
entries.sort((a, b) => (a.value === b.value ? cmp(a.object, b.object) : cmp(a.value, b.value)));
return entries;
}
function cmp(a: string, b: string): number {
return a < b ? -1 : a > b ? 1 : 0;
}
export async function readIndexDocument(
port: NextGraphPort,
index: NuriLike,
): Promise<readonly UnionSubject[]> {
return port.readDocument(asNuri(index));
}
src/index.ts
+25 -28
View File
@@ -1,39 +1,36 @@
/**
* `@ng-helpers/indexing` — an index built on top of NextGraph.
*
* NextGraph has no indexing concept and will not grow one, so nothing here
* belongs in `@ng-eventually/polyfill`: that package's inbox stays generic and
* carries opaque deposits, and knows nothing of what a deposit means.
* NextGraph has no indexing concept and will not grow one, so nothing here belongs
* in `@ng-eventually/polyfill`: that package's inbox stays generic and carries
* opaque deposits.
*
* An index is an ORDINARY document in its creator's public store. What makes it
* an index is that an application references its NURI in its own source. Anyone
* may hand it a reference by depositing into its inbox; its creator's session
* resolves those references and adds what it finds — see `observation.ts` for what
* makes that happen, and `curator.ts` for what it does.
* An index is an ORDINARY document in its creator's public store. What makes it an
* index is that it declares a field, and that an application references its NURI in
* its own source. Anyone may hand it a reference; that reference becomes an entry
* because the layer below processes the index's inbox.
*
* An index only ever grows — see `curator.ts`.
* ## What this package is, and is not
*
* A helper that hides that an index is implemented by an inbox — two acts, and the
* addresses stay out of sight. It does not make an entry of what an index receives,
* and it does not read: an index is an ordinary document, so whoever holds its NURI
* queries it with the very calls any other document takes — `readUnion([index])`, or
* a SPARQL `SELECT` anchored on it — using the two IRIs below to recognise what it
* finds. A helper of ours would only teach a shape that has to be unlearned.
*
* ## Everything published is below, and this file wires nothing
*
* One function, the two acts on the handle it produces, and the two IRIs an index is
* written with.
*/
// Wiring
export { indexing } from "./indexing";
// The one function, built on the real polyfill where the real polyfill is known
export { indexing } from "./polyfill-adapter";
export type { Indexing } from "./indexing";
export { polyfillPort } from "./polyfill-adapter";
export type { PolyfillPortOptions } from "./polyfill-adapter";
export type {
NextGraphPort,
IncomingDeposit,
ObjectResolution,
Nuri,
NuriLike,
UnionSubject,
} from "./port";
// What travels from a depositor to a curator: a bare reference
export { decodeReference } from "./deposit";
export type { IndexDeposit } from "./deposit";
// Addressing, as this layer's two acts speak it
export type { Nuri, NuriLike } from "./port";
// What an index holds
export type { IndexDescriptor, IndexEntry } from "./index-document";
// The IRIs written into an index document, for a reader going straight to `readUnion`
// The IRIs an index document is written with, for whoever queries one
export { ENTRY_VALUE, INDEX_FIELD } from "./vocabulary";
src/indexing.ts
+31 -63
View File
@@ -1,13 +1,6 @@
import type { NextGraphPort, Nuri, NuriLike } from "./port";
import { asNuri } from "./nuri";
import { observeOwnIndexes } from "./observation";
import {
assertIndexDocument,
entriesOf,
readIndexDocument,
writeDescriptor,
type IndexEntry,
} from "./index-document";
import { INDEX_FIELD } from "./vocabulary";
/**
* Everything this package does, bound to one identity.
@@ -15,6 +8,10 @@ import {
* A port carries an identity — the polyfill's session is one user's and no call
* takes an identifier — so an `Indexing` is one person's handle. Two people mean
* two handles, which is also what keeps a multi-actor test honest.
*
* TWO acts, and no third. What becomes of what is added is not one of them, and
* neither is reading: an index is an ordinary document, so its contents come back
* through `readUnion` or a SPARQL `SELECT`, recognised by the two published IRIs.
*/
export interface Indexing {
/**
@@ -26,93 +23,64 @@ export interface Indexing {
* is no separate kind of index, and the entries of such an index come out in
* chronological order because ISO-8601 sorts as a string.
*
* WHAT BECOMES OF IT is its creator's business, in both directions at once. The
* returned NURI is what an application hardcodes in its own source: that
* reference is the only thing that makes this ordinary document an index, and the
* only way anyone reaches it. And this session is what curates it — from now on
* while it lasts, and again at its creator's next connection. An index has no
* life of its own: it lives through the application that names it and the
* creator who connects.
* The returned NURI is what an application hardcodes in its own source: that
* reference is the only thing that makes this ordinary document an index, and
* the only way anyone reaches it.
*/
createIndex(field: string): Promise<Nuri>;
create(field: string): Promise<Nuri>;
/**
* Hands an index a reference to an object. Open to ANYONE — it is a deposit in
* the index document's inbox, not a write. Produces nothing: an application names
* a document or a person, never an inbox, and there is no receipt to hold on to.
*
* The reference is the whole message: it claims nothing and instructs nothing,
* it just invites the index's owner to look. Call it when the object is
* created, and again whenever anyone notices the index may not have it yet —
* including a third party. What the deposit becomes is up to the index's creator:
* it enters the index the moment their session processes that inbox.
* ADDED NOW, VISIBLE LATER — the verb is the one every index carries (`addDocument`,
* `addObjects`), and those systems are asynchronous too. The reference is the whole
* message: it claims nothing and instructs nothing, it just invites the index's
* owner to look. Call it when the object is created, and again whenever anyone
* notices the index may not have it yet — including a third party.
*/
refer(index: NuriLike, object: NuriLike): Promise<void>;
/**
* The index's entries, ordered by value. Refuses a document that declares no
* index field rather than producing an empty list.
*
* Sugar only. Reading an index needs NOTHING new from NextGraph: an application
* that knows the NURI can call `readUnion([indexNuri])` and get these entries as
* subjects — one per indexed object, keyed by its NURI — plus the index's own
* subject declaring its field, which this function drops.
*/
read(index: NuriLike): Promise<IndexEntry[]>;
add(index: NuriLike, object: NuriLike): Promise<void>;
}
/**
* This identity's handle, and its connection.
* This identity's handle, built on a port.
*
* Awaiting it processes the inbox of every index this identity owns — the deposits
* that piled up while it was away — and leaves those inboxes watched, so a deposit
* made from now on is curated as it lands. That is the whole of when curation
* happens; there is nothing to call, schedule or configure, and no way to aim it at
* one index.
* INTERNAL. What an application gets is `indexing(sessionId)` in `index.ts`, which
* builds the port for it — the port is the seam that keeps the two acts testable
* against an in-memory NextGraph, not a concept a caller has to learn.
*
* It never rejects for that work: an identity that could not be caught up still
* gets its handle, because reading an index and depositing into one do not depend
* on it. What went wrong is reported on this package's log stream.
* Synchronous, and it does nothing but bind the port. Only the two acts below talk
* to anything.
*/
export async function indexing(port: NextGraphPort): Promise<Indexing> {
const observation = await observeOwnIndexes(port);
export function indexingOn(port: NextGraphPort): Indexing {
return {
async createIndex(field: string): Promise<Nuri> {
async create(field: string): Promise<Nuri> {
// Refused at the door, because a field cannot be corrected afterwards:
// nothing here deletes, so an index created on a useless field is useless
// for good. An empty string is the sharp case — it would match no object.
if (field.length === 0 || field.trim().length === 0) {
throw new Error(
"createIndex: the field must be the predicate an indexed object carries, " +
"create: the field must be the predicate an indexed object carries, " +
"and it cannot be changed later — this package never removes anything.",
);
}
const index = await port.createPublicDocument();
await writeDescriptor(port, index, { field });
// The declaration, on the document's own subject: the one triple this package
// writes, and what makes an ordinary document an index. Whoever reads the
// document recognises it by `INDEX_FIELD`.
await port.addLiteralProperty(index, index, INDEX_FIELD, field);
// An index nobody can deposit into is not an index. Only its owner can open
// its inbox, and this is the one moment the owner is here — so it is opened
// at creation rather than left for a later call to remember.
await port.openInbox(index);
// The store was searched before this document existed, so this session would
// otherwise ignore its own new index until the next connection.
await observation.include(index);
return index;
},
async refer(index: NuriLike, object: NuriLike): Promise<void> {
// The payload IS the reference. Nothing wraps it, nothing annotates it.
async add(index: NuriLike, object: NuriLike): Promise<void> {
// The payload IS the reference. Nothing wraps it, nothing annotates it — and
// it carries no index either, because the inbox it lands in identifies one.
await port.depositTo(asNuri(index), asNuri(object));
},
async read(index: NuriLike): Promise<IndexEntry[]> {
const nuri = asNuri(index);
const subjects = await readIndexDocument(port, nuri);
// Only "is this an index?", deliberately NOT "which field does it curate
// on?". Reading entries never needed the field, and making it need one is
// what let an ambiguous declaration make every entry unreadable for good.
assertIndexDocument(subjects, nuri);
return entriesOf(subjects, nuri);
},
};
}
src/observation.ts
-128
View File
@@ -1,128 +0,0 @@
import type { NextGraphPort, Nuri, NuriLike } from "./port";
import { asNuri } from "./nuri";
import { coalescing } from "./coalescing";
import { curate } from "./curator";
import { declaresIndexField, readIndexDocument } from "./index-document";
/**
* Curating an index is what happens when its inbox is processed. Nothing calls it.
*
* ## Why there is no `curate(index)` to call
*
* An application that could call it would have to answer two questions it has no
* business answering — who owns the index, and when curation runs. Both answers are
* already fixed by the design: the owner is the only one who CAN (nobody else reads
* the inbox, nobody else writes the document), and "when" is "whenever a deposit
* arrives, or has been waiting". A published call would only let an application get
* those wrong.
*
* So the trigger is the identity's own session: obtaining an `Indexing` handle is a
* connection, and a connection processes what is waiting for it and keeps
* processing what arrives. That is the same shape the polyfill gives its own
* inboxes — watched for as long as the identity is connected, backlog applied at
* connection — and this layer holds itself to it.
*
* ## Only the owner, and the layer does not have to check
*
* Reading an index's inbox is refused to anyone else, so a session curates exactly
* the indexes it owns and could not do otherwise if it tried. Which is why the
* search below is a search of THIS identity's own public store.
*
* ## Nothing here denies anything
*
* A session that could not look for its indexes, could not watch one, or could not
* process one reports it and carries on: reading an index and depositing into one
* need none of this, and an index only ever grows, so a run that did not happen
* costs a deposit nothing — it is still in the inbox, and the next notification or
* the next connection applies it. The polyfill states the same rule for its own
* inboxes: failing to apply one denies nothing.
*/
export interface IndexObservation {
/**
* Brings one more index under observation — its inbox is processed now, and again
* on every deposit. For an index this session has just created: it did not exist
* when the store was searched, and its creator is right here.
*
* Asking twice for the same index changes nothing.
*/
include(index: NuriLike): Promise<void>;
}
/**
* Processes the inboxes of every index this identity owns, and keeps processing
* them. Resolves once the search is done and what it found has been caught up.
*/
export async function observeOwnIndexes(port: NextGraphPort): Promise<IndexObservation> {
const observed = new Set<Nuri>();
async function include(indexLike: NuriLike): Promise<void> {
const index = asNuri(indexLike);
if (observed.has(index)) return;
observed.add(index);
const processInbox = coalescing(async () => {
try {
await curate(port, index);
} catch (error) {
console.error(
`[ng-helpers/indexing] ${index}: its inbox could not be processed — ${String(error)}`,
);
}
});
// WATCH FIRST, then process. The other order has a gap: a deposit landing
// between the read and the watch is seen by neither, and waits for the next
// connection. This order overlaps instead of gapping — a deposit landing in
// between is processed twice — which costs nothing, because curation resolves
// the reference again and lands on the same result.
try {
await port.watchDeposits(index, processInbox);
} catch (error) {
console.error(
`[ng-helpers/indexing] ${index}: deposits into it will not be noticed until the next ` +
`connection — its inbox could not be watched: ${String(error)}`,
);
}
await processInbox();
}
const observation: IndexObservation = { include };
let mine: readonly Nuri[];
try {
mine = await port.listPublicDocuments();
} catch (error) {
console.error(
"[ng-helpers/indexing] this identity's indexes were not found, so none of them is being " +
`curated in this session — its public store could not be listed: ${String(error)}`,
);
return observation;
}
await Promise.all(
mine.map(async (doc) => {
if (await isAnIndex(port, doc)) await include(doc);
}),
);
return observation;
}
/**
* Does this document declare an index field? A document that could not be READ
* answers `false`, silently and on purpose.
*
* There is nothing to report: upstream hands a failed read back as an empty one, so
* "not an index" and "could not tell" are the same answer here, and warning about
* every ordinary document that did not answer would bury the failures that mean
* something. The cost is bounded by the invariant this package is built on — an
* index only ever grows, so an index missed at this connection is curated at the
* next one, with its deposits still in its inbox.
*/
async function isAnIndex(port: NextGraphPort, doc: Nuri): Promise<boolean> {
try {
return declaresIndexField(await readIndexDocument(port, doc), doc);
} catch {
return false;
}
}
src/polyfill-adapter.ts
+38 -60
View File
@@ -1,25 +1,23 @@
import { docs, inbox, readUnion, storeRegistry } from "@ng-eventually/polyfill";
import type {
IncomingDeposit,
NextGraphPort,
Nuri,
NuriLike,
ObjectResolution,
UnionSubject,
} from "./port";
import { docs, inbox, storeRegistry } from "@ng-eventually/polyfill";
import type { NextGraphPort, NuriLike } from "./port";
import { indexingOn, type Indexing } from "./indexing";
import { asNuri } from "./nuri";
import { buildInsertTriple } from "./sparql";
import { resolutionFromFailure, resolutionFromRead } from "./resolution";
/**
* The ONLY place in this package that touches `@ng-eventually/polyfill` at
* runtime. Everything else is written against `NextGraphPort`, so the indexing
* rules never learn a polyfill call — and the polyfill never learns anything
* about indexing.
* runtime. Everything else is written against `NextGraphPort`, so the two acts
* never learn a polyfill call — and the polyfill never learns anything about
* indexing.
*
* Only entries listed in `contract_polyfill-surface` are used here; nothing is
* reached for inside that package. Because this file compiles against the real
* module, a change to the published surface breaks THIS file and nothing else.
*
* NOT PUBLISHED. `indexing(sessionId)` builds the port itself, so an application
* never names one: the port is the seam that keeps the two acts testable without a
* broker, and every call it exposes is either one of those acts or something the
* polyfill already publishes to whoever wants it.
*/
export interface PolyfillPortOptions {
/**
@@ -43,30 +41,6 @@ export function polyfillPort(options: PolyfillPortOptions): NextGraphPort {
return storeRegistry.createEntityDoc("public");
},
async listPublicDocuments(): Promise<readonly Nuri[]> {
// The same store `createPublicDocument` writes into, listed back: an index is
// an ordinary document there, and there is no narrower question to ask.
return storeRegistry.listMyEntityDocs("public");
},
async resolveObject(doc: NuriLike): Promise<ObjectResolution> {
// The decision itself lives in `resolution.ts`, where it is unit-tested:
// here it would be reachable only through a live broker, and an untested
// "empty means unresolved" quietly becomes "empty means the object has no
// field" — a failure filed as a fact about the object.
let subjects: readonly UnionSubject[];
try {
subjects = await readUnion([asNuri(doc)]);
} catch (error) {
return resolutionFromFailure(error);
}
return resolutionFromRead(subjects);
},
async readDocument(doc: NuriLike): Promise<readonly UnionSubject[]> {
return readUnion([asNuri(doc)]);
},
async addLiteralProperty(
doc: NuriLike,
subject: string,
@@ -75,7 +49,7 @@ export function polyfillPort(options: PolyfillPortOptions): NextGraphPort {
): Promise<void> {
const graph = asNuri(doc);
// An INSERT and nothing else. There is deliberately no delete anywhere in
// this package, so no failure here can leave an index short of an entry.
// this package, so no failure here can leave an index short of anything.
//
// The document is named ONCE, as the anchor: `sparqlUpdate(sid, update, anchor)`
// scopes the write to that repo's default graph, so the statement carries no
@@ -86,7 +60,9 @@ export function polyfillPort(options: PolyfillPortOptions): NextGraphPort {
async openInbox(doc: NuriLike): Promise<void> {
// Without this nobody can deposit: `postToDocument` resolves the document's
// address and throws when its owner never opened one.
// address and throws when its owner never opened one. It is also the one call
// that hands an inbox address out, and this layer deliberately drops it — a
// document is what everything above names, exactly as upstream intends.
await storeRegistry.openDocumentInbox(asNuri(doc));
},
@@ -95,26 +71,28 @@ export function polyfillPort(options: PolyfillPortOptions): NextGraphPort {
// anyone — which is what lets a stranger contribute to someone else's index.
await inbox.postToDocument(asNuri(doc), { payload });
},
async readDeposits(doc: NuriLike): Promise<readonly IncomingDeposit[]> {
return inbox.readForDocument(asNuri(doc));
},
async watchDeposits(doc: NuriLike, onDeposits: () => Promise<void>): Promise<void> {
// The one call that hands out an inbox address, and the reason this is the only
// line in the package that holds one. It is idempotent within a page — the
// engagement says so, and says that firing one per component is supported — so
// asking again for a document whose inbox `createIndex` already opened resolves
// that inbox rather than adding a second.
const address = await storeRegistry.openDocumentInbox(asNuri(doc));
// `inbox.watch` hands back an unsubscribe, and this layer deliberately drops it:
// a session watches its own indexes for its whole life, exactly as the polyfill
// watches the inboxes it holds for as long as the identity stays connected.
//
// The callback is declared `void` upstream, so what it returns is ignored and the
// processing runs on its own. That is why it must never reject: nothing over
// there would catch it. It reports its own failures instead.
inbox.watch(address, onDeposits);
},
};
}
/**
* This identity's handle, on the real polyfill — the package's one published
* function, re-exported by `index.ts`.
*
* It lives HERE, beside the port it builds, because this is the only file that knows
* `@ng-eventually/polyfill` exists at runtime; `index.ts` says what is published and
* wires nothing.
*
* `sessionId` is what the polyfill's own `init(…)` hands its callback — upstream's
* type, relayed and never converted. A session IS one identity, so the handle is one
* person's: two users mean two handles, which is also what keeps a multi-actor test
* honest.
*
* It reaches nothing on its own — build it once and keep it, or build one wherever
* you need it; only the two acts on it talk to anything.
*/
export function indexing(sessionId: string | number): Indexing {
// The port stays inside. It is the seam that keeps the two acts testable without
// a broker, and an application has no use for it: every call it exposes is either
// an act of the published surface or something the polyfill already publishes.
return indexingOn(polyfillPort({ sessionId }));
}
src/port.ts
+14 -91
View File
@@ -1,45 +1,23 @@
/**
* What this layer needs from NextGraph, and nothing more.
*
* FOUR operations, because this package does exactly two things: it creates an
* index document that declares its field and can receive deposits, and it deposits
* a reference into one. Nothing here reads a document, an inbox or a store —
* making an index's deposits into its entries belongs to the layer below, and an
* operation kept "in case" would be an invitation to do that work here again.
*
* Every member below is backed by published entries of `@ng-eventually/polyfill`
* (see that package's `contract_polyfill-surface`). The mapping lives in
* `polyfill-adapter.ts`; the rest of this package never imports the polyfill at
* runtime, so the indexing rules stay testable against an in-memory port.
* runtime, so the two acts stay testable against an in-memory port.
*
* The types are imported *from* the polyfill (type-only, erased at build) so they
* are literally the published ones rather than a copy that could drift.
*/
import type { Nuri, NuriLike, PrincipalId, UnionSubject } from "@ng-eventually/polyfill";
import type { Nuri, NuriLike } from "@ng-eventually/polyfill";
export type { Nuri, NuriLike, PrincipalId, UnionSubject };
/** One deposit read out of a document's inbox. Mirrors the published `Deposit`. */
export interface IncomingDeposit {
/** The depositor, when identified; `null` for an anonymous deposit. */
readonly from: PrincipalId | null;
/** Opaque to the polyfill — this layer decodes it (see `deposit.ts`). */
readonly payload: unknown;
/** Deposit timestamp (ms epoch). */
readonly ts: number;
}
/**
* The answer to "what is at this reference?" — TWO states, deliberately.
*
* There is no `absent`. An index never removes anything, so nothing this layer
* does could ever hinge on telling "the object is gone" from "I could not read
* it". Both mean the same thing here: *not added this time*. Collapsing them is
* safe only because of that, and it is the point — the alternative would be a
* label a broker failure could wear, and an index emptied one reference at a time
* by a network problem.
*
* `unresolved` is still a failure, and the curator reports every one of them.
*/
export type ObjectResolution =
/** The read answered, and there is something there. */
| { readonly state: "present"; readonly subjects: readonly UnionSubject[] }
/** Nothing usable came back — gone, unreadable, or the read failed. Not distinguished. */
| { readonly state: "unresolved"; readonly reason: string };
export type { Nuri, NuriLike };
/**
* A port is bound to ONE identity: the polyfill's session is one user's, and no
@@ -52,45 +30,16 @@ export interface NextGraphPort {
*/
createPublicDocument(): Promise<Nuri>;
/**
* Every document this identity has in its PUBLIC store. Backs onto
* `storeRegistry.listMyEntityDocs("public")`.
*
* It is how an owner finds its own indexes again after a page load, and it has to
* be a full listing because NOTHING marks a document as an index — the index's own
* declaration, read from the document, is the only tell. Upstream throws rather
* than hand back a listing whose documents could not be opened, so an answer here
* is a listing, never a shrug.
*/
listPublicDocuments(): Promise<readonly Nuri[]>;
/** Resolves a reference deposited into an index: the two-state answer above. */
resolveObject(doc: NuriLike): Promise<ObjectResolution>;
/**
* Every subject an index document holds. Backs onto `readUnion([doc])`.
*
* CAUTION — an empty result is NOT proof the document is empty, and this
* interface does not pretend otherwise. `readUnion` swallows a failing document
* into `[]` (its `readDoc` is `try {…} catch { return [] }`) and drops documents
* whose cap this user does not hold, so "could not read" and "holds nothing"
* arrive here indistinguishable. It may also reject outright.
*
* So a caller must never read `[]` as "a valid index that happens to be empty".
* Nothing here does: `descriptorOf` refuses a document that declares no field —
* which is exactly what an unreadable one looks like — and that refusal aborts
* curation before a single write.
*/
readDocument(doc: NuriLike): Promise<readonly UnionSubject[]>;
/**
* Adds ONE value for `predicate` on `subject`, inside `doc`. Only the
* document's OWNER may — holding a read key never grants a write.
*
* There is no counterpart that removes, and no way to pass "no value": this
* package must not be able to take anything out of an index, and making that
* structural beats leaving it to whoever edits the curator next. Backs onto
* structural beats leaving it to whoever edits this next. Backs onto
* `docs.sparqlUpdate` with an `INSERT DATA`.
*
* One caller and one use: writing the field an index declares, at creation.
*/
addLiteralProperty(
doc: NuriLike,
@@ -105,7 +54,8 @@ export interface NextGraphPort {
* document has no address. Backs onto `storeRegistry.openDocumentInbox(doc)`.
*
* Returns nothing on purpose — an application names a document or a person,
* never an inbox.
* never an inbox, and keeping that address out of sight is the whole of what
* this package is for.
*/
openInbox(doc: NuriLike): Promise<void>;
@@ -118,31 +68,4 @@ export interface NextGraphPort {
* vanishes silently is worse than a refusal.
*/
depositTo(doc: NuriLike, payload: unknown): Promise<void>;
/**
* The deposits made to this document's inbox, oldest first.
*
* THROWS for anyone but the document's owner — you may deposit into anyone's
* inbox, you may only READ your own. Backs onto `inbox.readForDocument(doc)`.
*/
readDeposits(doc: NuriLike): Promise<readonly IncomingDeposit[]>;
/**
* Calls back for as long as this session lives, every time something is deposited
* into this document's inbox. OWNER only, for the same reason `readDeposits` is:
* being told what landed in an inbox is reading it.
*
* Backs onto `inbox.watch(address, onDeposits)`, whose address comes from
* `storeRegistry.openDocumentInbox(doc)` — the one call that hands one out, and
* which is idempotent within a page. The address never leaves this layer: a
* document is what everything above names, exactly as upstream intends.
*
* `onDeposits` takes nothing: what arrived is not read from the callback but from
* the inbox itself, which is re-read whole. That is not an omission — a deposit is
* never an instruction, so the only thing a notification can say is "look again".
*
* Nothing stops it. This session watches its own indexes for as long as it lasts,
* which is what the polyfill does with the inboxes it watches on its own account.
*/
watchDeposits(doc: NuriLike, onDeposits: () => Promise<void>): Promise<void>;
}
src/resolution.ts
-36
View File
@@ -1,36 +0,0 @@
import type { ObjectResolution, UnionSubject } from "./port";
/**
* Why an empty read is `unresolved` and never `present`.
*
* `readUnion` cannot tell an absent object from one it failed to read, and does
* not claim to: its per-document read is wrapped in `try {…} catch { return [] }`
* (`readDoc`, in `@ng-eventually/polyfill`'s `src/surface/read-model.ts`), and
* documents whose cap this user does not hold are dropped from the batch before
* it reads at all. An empty result means absent OR unreadable OR failed, with
* nothing to separate them.
*
* That is not a gap this package has to close, because nothing it does depends on
* the answer: an index only ever grows, so every reading leads to the same act —
* do not add, and say so.
*
* The rule lives here, apart from the I/O, for one reason: in the adapter it was
* unreachable by any test, and dropping it there left the suite green while a
* FAILED read got reported as `skipped: "no-field"` — a failure filed as a
* property of the object. That is exactly the misclassification this layer exists
* to avoid, so the rule is now a pure function with its own tests.
*/
export const EMPTY_READ =
"the read came back empty — the object is absent, unreadable, or the read failed; " +
"the polyfill does not distinguish them, and this layer does not need it to";
/** Turns what a read returned into the two-state answer the curator acts on. */
export function resolutionFromRead(subjects: readonly UnionSubject[]): ObjectResolution {
if (subjects.length === 0) return { state: "unresolved", reason: EMPTY_READ };
return { state: "present", subjects };
}
/** Turns a read that threw into the same two-state answer. Never `present`. */
export function resolutionFromFailure(error: unknown): ObjectResolution {
return { state: "unresolved", reason: String(error) };
}
src/sparql.ts
+3 -2
View File
@@ -1,5 +1,6 @@
/**
* The SPARQL this layer writes — and it writes only one kind of statement.
* The SPARQL this layer writes — one statement, and one use for it: the field an
* index declares, written on the index document's own subject at creation.
*
* There is NO delete builder, and there must not be one: an index never removes
* an entry, and the surest way to keep that true is for this package to be unable
@@ -61,7 +62,7 @@ export function escapeIri(value: string): string {
/**
* One triple, for the ANCHORED default graph. The only statement this package
* ever writes.
* ever writes, and it writes it once per index.
*
* No `GRAPH <…>` wrapper, and the document is not a parameter at all: it is named
* once, as `docs.sparqlUpdate`'s anchor, which already scopes the write to that
src/vocabulary.ts
+10 -3
View File
@@ -1,5 +1,6 @@
/**
* The IRIs this layer writes into an index document.
* The IRIs an index document is written with — published because a reader needs
* them to make sense of one.
*
* `urn:ng-eventually:` is RESERVED by the polyfill — triples whose subject falls
* under it are dropped on read. This layer stays out of it entirely and uses its
@@ -9,9 +10,15 @@
/**
* On the index document's own subject: the predicate an indexed object must
* carry, declared once by whoever creates the index.
* carry, declared once by whoever creates the index. `create` writes it.
*/
export const INDEX_FIELD = "urn:ng-helpers:index:field";
/** On an entry (subject = the indexed object's NURI): the value of that field. */
/**
* On an entry (subject = the indexed object's NURI): the value of that field.
*
* This package never writes one — an entry is made by the layer that processes the
* index's inbox. It is published because reading an index means recognising this
* predicate on the subjects that come back.
*/
export const ENTRY_VALUE = "urn:ng-helpers:index:value";
test/adapter.test.ts
+62 -138
View File
@@ -1,7 +1,6 @@
import { expect, test } from "bun:test";
import { indexing } from "../src/indexing";
import { curate } from "../src/curator";
import { ENTRY_VALUE, INDEX_FIELD } from "../src/vocabulary";
import { indexingOn } from "../src/indexing";
import { INDEX_FIELD } from "../src/vocabulary";
import type { NextGraphPort, Nuri } from "../src/port";
import { DESTRUCTIVE, blankLiterals, installFakePolyfill } from "./fake-polyfill";
@@ -10,22 +9,18 @@ import { DESTRUCTIVE, blankLiterals, installFakePolyfill } from "./fake-polyfill
*
* ## What was wrong with the gate this replaces
*
* It mocked the polyfill to constants and drove exactly TWO of the adapter's seven
* methods — `addLiteralProperty` and (through `createIndex`) `createPublicDocument`
* and `openInbox`. Everything else was inert. Six methods could be gutted — write
* nothing, return `[]` — with the suite still green, and the gate's own promise
* ("every query it emits is read back") held only for the queries those two methods
* emitted. A destructive statement planted in `readDeposits` was never recorded,
* because `readDeposits` was never called. That matters most exactly there:
* retiring an applied deposit — the README's own open question — lands in
* `readDeposits`.
* It mocked the polyfill to constants and drove only the methods `create` happens to
* use. Everything else was inert: a method could be gutted — write nothing, return
* `[]` — with the suite still green, and the gate's own promise ("every query it
* emits is read back") held only for the queries those methods emitted. A
* destructive statement planted in a method nobody drove was never recorded.
*
* ## What holds now
*
* The adapter runs on `fake-polyfill.ts`, an in-memory polyfill that ANSWERS rather
* than returning constants, so the tests below are ordinary behavioural tests that
* happen to run through the real wiring. Gut any of the seven and something here
* goes red, because each one is now load-bearing for an outcome that is asserted.
* happen to run through the real wiring. Gut any of the four and something here goes
* red, because each one is load-bearing for an outcome that is asserted.
*
* Three properties are checked on top of behaviour, and each closes a hole the
* review walked through:
@@ -83,122 +78,39 @@ async function publish(port: NextGraphPort, field: string, value: string): Promi
return object;
}
// --- the whole loop, through the real adapter -----------------------------
// --- both acts, through the real adapter ----------------------------------
test("the real adapter carries the whole loop: create, publish, refer, curate, read", async () => {
const index = await as("alice", async (alice) => (await indexing(alice)).createIndex(FIELD));
test("the real adapter carries both acts: create, publish, add", async () => {
const index = await as("alice", async (alice) => indexingOn(alice).create(FIELD));
const article = await as("bob", async (bob) => {
const object = await publish(bob, FIELD, "2026-07-08");
await (await indexing(bob)).refer(index, object);
await indexingOn(bob).add(index, object);
return object;
});
const report = await as("alice", (alice) => curate(alice, index));
expect(report.outcomes).toEqual([{ result: "indexed", object: article, value: "2026-07-08" }]);
const entries = await as("alice", async (alice) => (await indexing(alice)).read(index));
expect(entries).toEqual([{ object: article, value: "2026-07-08" }]);
});
test("a deposit made while the owner is connected is curated as it lands, with nobody asking", async () => {
const seen = await as("alice", async (alice) => {
const api = await indexing(alice);
const index = await api.createIndex(FIELD);
// An owner may deposit into her own index: `refer` is open to anyone, and here it
// keeps both sides on one session, which is all this fake models at a time.
const object = await publish(alice, FIELD, "2026-09-09");
await api.refer(index, object);
// NOTHING CALLS CURATION. The session is told a deposit landed on an inbox it
// watches, and processes that inbox itself — through the real adapter, so the
// address resolution and `inbox.watch` are the ones an application would get.
await world.deliverNotifications();
return { object, subjects: await alice.readDocument(index) };
});
expect(seen.subjects.find((s) => s.subject === seen.object)?.props[ENTRY_VALUE]).toEqual([
"2026-09-09",
// The index declares its field, and Bob's bare reference is waiting in its inbox.
// That is the whole of what these two acts do; making an entry of that reference
// is the business of the layer below, and nothing here can do it or ask for it.
expect(world.contentsOf(index)).toEqual([
{ subject: index, predicate: INDEX_FIELD, values: [FIELD] },
]);
expect(world.depositsIn(index)).toEqual([{ from: "bob", payload: article, ts: 1 }]);
});
test("two indexes are two documents, each owned by whoever created it", async () => {
const [first, second] = await as("alice", async (alice) => [
await (await indexing(alice)).createIndex(FIELD),
await (await indexing(alice)).createIndex(FIELD),
await indexingOn(alice).create(FIELD),
await indexingOn(alice).create(FIELD),
]);
expect(first).not.toBe(second);
// A stranger holding the NURI still cannot write it — reaching is not owning.
await expect(
as("bob", (bob) => bob.addLiteralProperty(first, first, ENTRY_VALUE, "2026-01-01")),
as("bob", (bob) => bob.addLiteralProperty(first, first, INDEX_FIELD, "urn:forged")),
).rejects.toThrow(/only a document's owner writes to it/);
});
// --- the two answers a resolve may give, and why they must stay apart -----
test("a reference that could not be READ comes back unresolved", async () => {
const index = await as("alice", async (alice) => (await indexing(alice)).createIndex(FIELD));
const article = await as("bob", async (bob) => {
const object = await publish(bob, FIELD, "2026-07-08");
await (await indexing(bob)).refer(index, object);
return object;
});
// `readDoc` catches and yields `[]`, so a failed read arrives looking exactly
// like an object that holds nothing. Telling them apart is not possible; filing
// the failure as a FACT about the object is what must not happen.
world.breakReadsOf(article, "broker unreachable");
try {
const report = await as("alice", (alice) => curate(alice, index));
expect(report.outcomes).toEqual([
{ result: "unresolved", object: article, reason: expect.stringContaining("absent") },
]);
} finally {
world.healReadsOf(article);
}
});
test("an object that really carries nothing for the field is SKIPPED — a different answer", async () => {
const index = await as("alice", async (alice) => (await indexing(alice)).createIndex(FIELD));
const unrelated = await as("bob", async (bob) => {
const object = await publish(bob, "http://schema.org/name", "Anemone");
await (await indexing(bob)).refer(index, object);
return object;
});
// Paired with the test above ON PURPOSE. `resolveObject` is the only thing
// keeping these two apart: replace `resolutionFromRead(subjects)` with
// `{ state: "present", subjects }` and the unreadable object above is reported
// here's answer instead — a broker failure filed as a property of the object.
const report = await as("alice", (alice) => curate(alice, index));
expect(report.outcomes).toEqual([
{ result: "skipped", object: unrelated, reason: "no-field" },
]);
});
test("an EMPTY read is unresolved, straight off the adapter method", async () => {
// The same rule as the pair above, asserted directly rather than through a
// curation report, so the wiring of `resolutionFromRead` is held by two
// independent tests and not by one.
const blank = await as("alice", (alice) => alice.createPublicDocument());
const resolution = await as("alice", (alice) => alice.resolveObject(blank));
expect(resolution.state).toBe("unresolved");
expect(resolution.state === "unresolved" && resolution.reason).toContain("absent, unreadable");
});
test("a read that REJECTS outright is unresolved too, and names the failure", async () => {
world.breakReadUnion("session lost");
try {
const resolution = await as("alice", (alice) => alice.resolveObject("did:ng:o:whatever"));
expect(resolution.state).toBe("unresolved");
expect(resolution.state === "unresolved" && resolution.reason).toContain("session lost");
} finally {
world.healReadUnion();
}
});
// --- the inbox, from both sides -------------------------------------------
test("a document whose owner never opened an inbox REFUSES the deposit", async () => {
@@ -209,34 +121,33 @@ test("a document whose owner never opened an inbox REFUSES the deposit", async (
);
});
test("anyone may deposit into an index, only its owner may read what was deposited", async () => {
const index = await as("alice", async (alice) => (await indexing(alice)).createIndex(FIELD));
await as("bob", async (bob) => (await indexing(bob)).refer(index, "did:ng:o:some-object"));
test("anyone may hand a reference to an index they do not own", async () => {
const index = await as("alice", async (alice) => indexingOn(alice).create(FIELD));
await as("bob", async (bob) => indexingOn(bob).add(index, "did:ng:o:some-object"));
const own = await as("alice", (alice) => alice.readDeposits(index));
expect(own.map((deposit) => deposit.payload)).toEqual(["did:ng:o:some-object"]);
expect(own[0]?.from).toBe("bob");
await expect(as("bob", (bob) => bob.readDeposits(index))).rejects.toThrow(
/may only READ your own/,
);
// Bob needed no permission and got no write, and the deposit carries who made it.
const waiting = world.depositsIn(index);
expect(waiting?.map((deposit) => deposit.payload)).toEqual(["did:ng:o:some-object"]);
expect(waiting?.[0]?.from).toBe("bob");
expect(world.contentsOf(index)).toEqual([
{ subject: index, predicate: INDEX_FIELD, values: [FIELD] },
]);
});
// --- what the adapter actually wrote --------------------------------------
test("readDocument returns what was written, and refuses a document that is no index", async () => {
const index = await as("alice", async (alice) => (await indexing(alice)).createIndex(FIELD));
const subjects = await as("alice", (alice) => alice.readDocument(index));
expect(subjects).toEqual([{ subject: index, graph: index, props: { [INDEX_FIELD]: [FIELD] } }]);
test("creating an index writes its field declaration, and nothing else", async () => {
const index = await as("alice", async (alice) => indexingOn(alice).create(FIELD));
expect(world.contentsOf(index)).toEqual([
{ subject: index, predicate: INDEX_FIELD, values: [FIELD] },
]);
// …and an ordinary public document is left exactly as it was made.
const ordinary = await as("alice", (alice) => alice.createPublicDocument());
await expect(as("alice", async (alice) => (await indexing(alice)).read(ordinary))).rejects.toThrow(
/declares no index field/,
);
expect(world.contentsOf(ordinary)).toEqual([]);
});
test("the write is the polyfill's canonical anchored form: the document named once, as the anchor", async () => {
const index = await as("alice", async (alice) => (await indexing(alice)).createIndex(FIELD));
const index = await as("alice", async (alice) => indexingOn(alice).create(FIELD));
const writes = world.calls.filter((call) => call.entry === "docs.sparqlUpdate");
const last = writes.at(-1);
expect(last?.args[1]).toBe(`INSERT DATA { <${index}> <${INDEX_FIELD}> "${FIELD}" }`);
@@ -247,17 +158,15 @@ test("the write is the polyfill's canonical anchored form: the document named on
test("a hostile value lands as ONE inert literal, not as a second statement", async () => {
const HOSTILE = '" } ; DROP GRAPH <did:ng:o:doc-1> ; INSERT DATA { <a> <b> "c';
const doc = await as("alice", async (alice) => {
const object = await alice.createPublicDocument();
await alice.addLiteralProperty(object, object, ENTRY_VALUE, HOSTILE);
return object;
});
// A field is caller-supplied and goes straight into the statement, so it is the
// value this package really does have to survive.
const index = await as("alice", async (alice) => indexingOn(alice).create(HOSTILE));
// Semantic, not a quote count: the value round-trips through a parser that would
// have refused the query outright had the literal closed early — and had it
// closed early and still parsed, the extra subject would show up here.
expect(world.contentsOf(doc)).toEqual([
{ subject: doc, predicate: ENTRY_VALUE, values: [HOSTILE] },
expect(world.contentsOf(index)).toEqual([
{ subject: index, predicate: INDEX_FIELD, values: [HOSTILE] },
]);
});
@@ -278,9 +187,24 @@ test("nothing the adapter sent the polyfill, in any method, carries a destructiv
}
});
test("the port the entry point builds has NO operation that reads or removes", async () => {
// Building it must not touch the broker: `indexing(sessionId)` builds one for every
// handle, and only the calls on it talk to anything. This check lived on the
// published surface while `polyfillPort` was published; the port is internal now,
// and the properties it guards are not.
const { polyfillPort } = await import("../src/polyfill-adapter");
const port = polyfillPort({ sessionId: "session:alice" });
expect(typeof port.createPublicDocument).toBe("function");
expect(typeof port.addLiteralProperty).toBe("function");
const forbidden = Object.keys(port).filter((name) =>
/delete|remove|clear|drop|read|list|watch/i.test(name),
);
expect(forbidden).toEqual([]);
});
test("LAST — every method the adapter exposes was driven above", async () => {
// Read off the adapter itself, so there is no list to remember to extend: add an
// eighth method and this fails until something above actually calls it. That is
// Read off the adapter itself, so there is no list to remember to extend: add a
// fifth method and this fails until something above actually calls it. That is
// the whole answer to "the gate proves the methods it drives" — it now names them
// from the code rather than from a test's memory.
const { polyfillPort } = await import("../src/polyfill-adapter");
test/fake-nextgraph.ts
+37 -264
View File
@@ -1,11 +1,4 @@
import type {
IncomingDeposit,
NextGraphPort,
Nuri,
NuriLike,
ObjectResolution,
UnionSubject,
} from "../src/port";
import type { NextGraphPort, Nuri, NuriLike } from "../src/port";
import { asNuri } from "../src/nuri";
/**
@@ -18,60 +11,41 @@ import { asNuri } from "../src/nuri";
*
* - only a document's owner writes to it, and a read key never grants a write
* (`contract_polyfill-surface`, "Guarantees");
* - anyone may DEPOSIT into a document's inbox, but reading one THROWS for anyone
* but its owner (`inbox.read`'s `assertOwnInbox`: "you may DEPOSIT into anyone's
* inbox; you may only READ your own");
* - depositing into a document whose owner never opened an inbox THROWS
* (`inbox.postToDocument`), rather than silently going nowhere;
* - anyone may DEPOSIT into a document's inbox (`inbox.postToDocument`);
* - depositing into a document whose owner never opened an inbox THROWS, rather
* than silently going nowhere;
* - opening an inbox on a document is refused to anyone but its owner
* (`openDocumentInbox`: doing so publishes the document's address);
* - a document in a public store is readable by whoever knows its NURI;
* - a document that cannot be read REJECTS, and a rejection means "unknown",
* never "absent";
* - being TOLD what landed in an inbox is reading it, so watching one is refused to
* anyone but the document's owner, exactly as opening one is.
* - a document nobody created cannot be named: reaching for one REJECTS.
*
* ## Telling a watcher crosses the network, so it is a step of its own
* ## Reading is done from OUTSIDE the port, on purpose
*
* A deposit is stored the moment it is made — that is the durable fact, and it is
* what the owner's next connection finds. Notifying a session that is watching goes
* over the wire, and this fake holds those notifications until a test calls
* {@link FakeNextGraph.deliverNotifications}. A test that never calls it is a test
* in which the owner has not been told yet: a real state, and precisely the one the
* catch-up at connection exists for.
* This package neither reads a document nor reads an inbox — what an index receives
* is made into entries by the layer below. So `contentsOf` and `depositsIn` are
* inspections of this double, not operations of the port: a test asks what the two
* acts LEFT BEHIND, and cannot accidentally hand the package back a way to read.
*/
type Properties = Map<string, string[]>;
/** One session watching one document's inbox. */
interface Watch {
readonly doc: Nuri;
readonly user: string;
readonly onDeposits: () => Promise<void>;
/** One deposit this double is holding, as its inbox holds it. */
export interface StoredDeposit {
/** The depositor, as the polyfill defaults it to the current user. */
readonly from: string;
readonly payload: unknown;
readonly ts: number;
}
type Properties = Map<string, string[]>;
interface StoredDocument {
readonly nuri: Nuri;
readonly owner: string;
readonly subjects: Map<string, Properties>;
/** `undefined` until the owner opens one — the state `postToDocument` refuses. */
deposits: IncomingDeposit[] | undefined;
deposits: StoredDeposit[] | undefined;
}
export class FakeNextGraph {
readonly #documents = new Map<string, StoredDocument>();
/** Documents the broker currently cannot answer about. See `breakReadsOf`. */
readonly #unreachable = new Map<string, string>();
/** Inboxes the broker currently cannot READ. See `breakInboxReadsOf`. */
readonly #inboxUnreadable = new Map<string, string>();
/** Inboxes the broker currently refuses to WATCH. See `breakWatchingOf`. */
readonly #inboxUnwatchable = new Map<string, string>();
/** Every live watch, across every identity — a session watching its own inbox. */
#watches: Watch[] = [];
/** Notifications the broker has not handed over yet. See `deliverNotifications`. */
#undelivered: Watch[] = [];
/** Why a store listing cannot answer, when a test has made it fail. */
#listingFailure: string | undefined;
#documentCount = 0;
#clock = 0;
@@ -83,14 +57,6 @@ export class FakeNextGraph {
return network.#createDocument(user);
},
async resolveObject(doc: NuriLike): Promise<ObjectResolution> {
return network.#resolve(asNuri(doc));
},
async readDocument(doc: NuriLike): Promise<readonly UnionSubject[]> {
return network.#read(asNuri(doc));
},
async addLiteralProperty(
doc: NuriLike,
subject: string,
@@ -107,125 +73,27 @@ export class FakeNextGraph {
async depositTo(doc: NuriLike, payload: unknown): Promise<void> {
network.#deposit(user, asNuri(doc), payload);
},
async readDeposits(doc: NuriLike): Promise<readonly IncomingDeposit[]> {
return network.#readDeposits(user, asNuri(doc));
},
async watchDeposits(doc: NuriLike, onDeposits: () => Promise<void>): Promise<void> {
network.#watchDeposits(user, asNuri(doc), onDeposits);
},
async listPublicDocuments(): Promise<readonly Nuri[]> {
return network.#listDocuments(user);
},
};
}
/**
* Makes the broker unable to answer about a document — a transient failure, the
* state a curation run must survive without damaging the index. Reads of it
* REJECT, which is what the real surface does when it could not find out.
*/
breakReadsOf(doc: NuriLike, reason: string): void {
this.#unreachable.set(asNuri(doc), reason);
}
/** The broker can answer about this document again. */
healReadsOf(doc: NuriLike): void {
this.#unreachable.delete(asNuri(doc));
}
/**
* The broker serves the DOCUMENT but not its INBOX.
*
* Not a contrivance: upstream an inbox is a repo of its own, reached through an
* address `openDocumentInbox` resolves and read with that repo's capability,
* while the document itself is read by `readUnion`. Two repos, two reads — so
* one answering while the other does not is what a partial failure looks like,
* and it is the state that makes a catch-up fail on one index and no other.
*/
breakInboxReadsOf(doc: NuriLike, reason: string): void {
this.#inboxUnreadable.set(asNuri(doc), reason);
}
/** The inbox can be read again. */
healInboxReadsOf(doc: NuriLike): void {
this.#inboxUnreadable.delete(asNuri(doc));
}
/**
* The broker refuses to keep this session posted about that inbox, while
* everything else about it still works.
*
* Watching is a live subscription, set up and held open for as long as the
* session lasts; reading an inbox is one question and one answer. A subscription
* can be refused where a read succeeds, which is the state that leaves an index
* caught up but unwatched — deposits into it going unnoticed until the next
* connection, exactly as the failure this models says.
*/
breakWatchingOf(doc: NuriLike, reason: string): void {
this.#inboxUnwatchable.set(asNuri(doc), reason);
}
/** The inbox can be watched again. */
healWatchingOf(doc: NuriLike): void {
this.#inboxUnwatchable.delete(asNuri(doc));
}
/**
* Hands over every inbox notification the broker was holding, and waits for the
* sessions watching to finish with them — including notifications those very runs
* provoke, so this returns with nothing left in flight.
*/
async deliverNotifications(): Promise<void> {
while (this.#undelivered.length > 0) {
const batch = this.#undelivered;
this.#undelivered = [];
for (const watch of batch) await watch.onDeposits();
}
}
/**
* This identity's page is gone: every watch its sessions had opened stops, and
* anything the broker was about to tell them is dropped. The polyfill's watching
* lasts exactly as long as an identity stays connected, and so does this.
*
* Nothing durable is lost — the deposits are in their inboxes, which is what makes
* the catch-up at the next connection enough on its own.
*/
disconnect(user: string): void {
this.#undelivered = this.#undelivered.filter((watch) => watch.user !== user);
this.#watches = this.#watches.filter((watch) => watch.user !== user);
}
/**
* The store can no longer say which documents an identity has. Upstream throws
* rather than answer a listing it could not establish, so this does too.
*/
breakListing(reason: string): void {
this.#listingFailure = reason;
}
/** A NURI shaped like any other, that no document was ever created for. */
neverCreatedNuri(): Nuri {
return "did:ng:o:doc-never-created" as Nuri;
}
/**
* The document's owner REPLACING a value in their own document, through their
* own application — NOT through this package's port, which deliberately cannot
* delete anything.
*
* This is a real capability and it has to be modelled: an indexed object is an
* ordinary document whose owner keeps editing it, and an index tested only
* against frozen objects would be tested against a world that does not exist.
*/
ownerReplacesValue(doc: NuriLike, subject: string, predicate: string, value: string): void {
/** Every triple a document holds, read from outside the port. */
contentsOf(doc: NuriLike): { subject: string; predicate: string; values: string[] }[] {
const stored = this.#require(asNuri(doc));
const properties = stored.subjects.get(subject);
if (properties === undefined) throw new Error(`${String(doc)} has no subject ${subject}`);
properties.set(predicate, [value]);
const out: { subject: string; predicate: string; values: string[] }[] = [];
for (const [subject, properties] of stored.subjects) {
for (const [predicate, values] of properties) out.push({ subject, predicate, values });
}
return out;
}
/**
* What is waiting in a document's inbox, oldest first — or `null` when its owner
* never opened one, which is a state and not a failure.
*/
depositsIn(doc: NuriLike): readonly StoredDeposit[] | null {
const stored = this.#require(asNuri(doc));
if (stored.deposits === undefined) return null;
return [...stored.deposits].sort((a, b) => a.ts - b.ts);
}
#createDocument(owner: string): Nuri {
@@ -236,53 +104,11 @@ export class FakeNextGraph {
}
#require(doc: Nuri): StoredDocument {
const broken = this.#unreachable.get(doc);
// "A rejection means 'unknown', never 'absent'."
if (broken !== undefined) throw new Error(`cannot reach ${doc}: ${broken}`);
const stored = this.#documents.get(doc);
if (stored === undefined) throw new Error(`cannot open ${doc}`);
return stored;
}
#read(doc: Nuri): UnionSubject[] {
const stored = this.#require(doc);
const out: UnionSubject[] = [];
for (const [subject, properties] of stored.subjects) {
// Built EXACTLY as `readUnion` builds it — a plain object literal filled by
// `(props[p] ??= []).push(o)` (`read-model.ts`). Neither detail is cosmetic:
// the literal inherits from `Object.prototype`, and `??=` does NOT assign
// over an inherited truthy member, so a predicate named `constructor` or
// `toString` leaves `.push` undefined and the read THROWS. Assigning plainly
// here made this double more forgiving than the real thing, and a test
// written against it asserted an outcome production can never produce.
const props: Record<string, string[]> = {};
for (const [predicate, values] of properties) {
for (const value of values) {
(props[predicate] ??= []).push(value);
}
}
out.push({ subject, graph: doc, props });
}
return out;
}
/**
* Two states, like the port: something is there, or nothing usable came back.
* A document that cannot be reached and a document that holds nothing both come
* back `unresolved` — this layer never has to tell them apart, and the reason
* string is the only difference.
*/
#resolve(doc: Nuri): ObjectResolution {
let subjects: UnionSubject[];
try {
subjects = this.#read(doc);
} catch (error) {
return { state: "unresolved", reason: String(error) };
}
if (subjects.length === 0) return { state: "unresolved", reason: `${doc} holds nothing` };
return { state: "present", subjects };
}
#add(user: string, doc: Nuri, subject: string, predicate: string, value: string): void {
const stored = this.#require(doc);
if (stored.owner !== user) {
@@ -323,65 +149,12 @@ export class FakeNextGraph {
}
this.#clock += 1;
stored.deposits.push({ from: user, payload, ts: this.#clock });
// Stored first, told afterwards: the deposit is a fact even if nobody is ever
// told, which is what makes the catch-up at connection sufficient on its own.
for (const watch of this.#watches) {
if (watch.doc === doc) this.#undelivered.push(watch);
}
}
#watchDeposits(user: string, doc: Nuri, onDeposits: () => Promise<void>): void {
const stored = this.#require(doc);
if (stored.owner !== user) {
throw new Error(
`${user} may not watch the inbox of ${doc}: being told what landed in an inbox ` +
"is reading it, and you may only READ your own",
);
}
const unwatchable = this.#inboxUnwatchable.get(doc);
// Refused AFTER the owner check: resolving the address is an owner-only act, so
// a stranger is turned away before any subscription is ever attempted.
if (unwatchable !== undefined) {
throw new Error(`cannot watch the inbox of ${doc}: ${unwatchable}`);
}
// Watching resolves the inbox address, and the call that resolves one opens it
// when there is none — the same idempotent call `openInbox` makes.
stored.deposits ??= [];
this.#watches.push({ doc, user, onDeposits });
}
#listDocuments(user: string): readonly Nuri[] {
if (this.#listingFailure !== undefined) {
throw new Error(`cannot list the public store: ${this.#listingFailure}`);
}
const mine: Nuri[] = [];
for (const stored of this.#documents.values()) {
if (stored.owner === user) mine.push(stored.nuri);
}
return mine;
}
#readDeposits(user: string, doc: Nuri): readonly IncomingDeposit[] {
const stored = this.#require(doc);
// No inbox → there is no address to read, which is a state and not an error.
if (stored.deposits === undefined) return [];
if (stored.owner !== user) {
throw new Error(
`${user} may not read the inbox of ${doc}: you may DEPOSIT into anyone's ` +
"inbox, you may only READ your own",
);
}
const unreadable = this.#inboxUnreadable.get(doc);
if (unreadable !== undefined) {
throw new Error(`cannot read the inbox of ${doc}: ${unreadable}`);
}
return [...stored.deposits].sort((a, b) => a.ts - b.ts);
}
}
/**
* A public document holding one business entity with a value for `field` — what
* an application creates and then refers to an index.
* an application creates and then hands to an index.
*
* Built through the very same port an application has: nothing here reaches
* behind the surface to plant a state a real caller could not produce.
test/fake-polyfill.ts
+35 -179
View File
@@ -1,5 +1,5 @@
import { mock } from "bun:test";
import type { Nuri, UnionSubject } from "../src/port";
import type { Nuri } from "../src/port";
/**
* `@ng-eventually/polyfill` itself, in memory — NOT a mock returning constants.
@@ -9,15 +9,12 @@ import type { Nuri, UnionSubject } from "../src/port";
* one stands BELOW the adapter, so the real adapter runs on top of it and every
* behavioural test becomes a test of the wiring too.
*
* ## Why a real implementation and not a recorder returning `[]`
* ## Why a real implementation and not a recorder returning constants
*
* The previous gate mocked `readUnion` to a constant `[]`. That single constant
* was an escape hatch: a destructive statement guarded by
* `if ((await readUnion([graph])).length > 0)` never ran, so it was never
* recorded, and the gate passed. A mock that always answers the same thing tests
* one state of the world. This answers what was actually written, so a curation
* run that adds an entry to a document already holding a descriptor DOES take the
* non-empty branch.
* A mock that always answers the same thing tests one state of the world, and a
* statement guarded by what a read returned would never run under it. This one
* answers what was actually written, so `contentsOf` and `depositsIn` report what
* the adapter really left behind rather than what a test arranged.
*
* ## Why the SPARQL is EXECUTED and not pattern-matched
*
@@ -33,29 +30,20 @@ import type { Nuri, UnionSubject } from "../src/port";
* ## Every rule below is one the polyfill actually enforces
*
* - only a document's owner writes to it (`docs.sparqlUpdate`'s `assertMayWrite`);
* - anyone may DEPOSIT into a document's inbox, but reading one throws for anyone
* but its owner (`inbox.read`'s `assertOwnInbox`);
* - depositing into a document whose owner never opened an inbox THROWS
* (`inbox.postToDocument`), rather than going nowhere;
* - a document with no inbox READS as `[]` — a state, not an error
* (`depositsForDocument`);
* - anyone may DEPOSIT into a document's inbox (`inbox.postToDocument`);
* - depositing into a document whose owner never opened an inbox THROWS, rather
* than going nowhere;
* - opening an inbox is refused to anyone but the document's owner
* (`openDocumentInbox`), and so is watching one, since being told what landed in an
* inbox is reading it;
* - watching lasts exactly as long as the identity stays connected: signing in as
* somebody else stops every watch the previous identity had opened
* (`contract_polyfill-surface`, "Guarantees");
* - `readUnion` swallows a failing document into `[]` (`readDoc`'s
* `try {…} catch { return [] }`), and may also reject outright;
* - `readUnion` builds each subject's props as a plain object literal filled by
* `(props[p] ??= []).push(o)` — inherited members and all.
* (`openDocumentInbox`: doing so publishes the document's address);
* - a document with no inbox has no address to read — a state, not an error.
*
* ## What is NOT modelled, and what happens then
*
* Anything the adapter reaches for on `docs`, `inbox` or `storeRegistry` that is
* absent here THROWS by name (see {@link namespace}) instead of returning
* `undefined` — so routing a query through `docs.sparqlQuery` is a red test with a
* message that says so. A brand-new TOP-LEVEL import is the one case that degrades:
* message that says so. This package reads nothing at all, so `readUnion` is not
* modelled either: reaching for it is the same red. A brand-new TOP-LEVEL import is the one case that degrades:
* Bun's `mock.module` materialises the module namespace from own keys, so a Proxy
* there is lost and an unmodelled top-level export arrives as `undefined`. Still
* red (`undefined is not a function`), just with a duller message.
@@ -265,25 +253,14 @@ export interface FakePolyfill {
signIn(user: string): void;
/** The session id for the signed-in identity, as `init`'s callback hands it over. */
sessionId(): string;
/**
* The broker can no longer answer about this document. `readDoc` catches and
* yields `[]`, so it arrives indistinguishable from an empty document — which is
* exactly the confusion `resolution.ts` exists to refuse.
*/
breakReadsOf(doc: string, reason: string): void;
healReadsOf(doc: string): void;
/** `readUnion` REJECTS outright — its session-level failure, not a per-document one. */
breakReadUnion(reason: string): void;
healReadUnion(): void;
/**
* Hands over every inbox notification the broker was holding, and waits for the
* watching session to finish with each — the callback is declared `void` upstream,
* so production never waits for it, and this does only because a test needs a point
* at which the work is over.
*/
deliverNotifications(): Promise<void>;
/** Every subject in a document, read from outside the adapter. */
/** Every triple in a document, read from outside the adapter. */
contentsOf(doc: string): { subject: string; predicate: string; values: string[] }[];
/**
* What is waiting in a document's inbox, oldest first — or `null` when its owner
* never opened one. An INSPECTION, not an entry of the polyfill: this package
* never reads an inbox, so nothing it does may depend on being able to.
*/
depositsIn(doc: string): readonly Deposit[] | null;
}
/**
@@ -293,13 +270,7 @@ export interface FakePolyfill {
*/
export function installFakePolyfill(): FakePolyfill {
const documents = new Map<string, StoredDocument>();
const unreachable = new Map<string, string>();
const calls: RecordedCall[] = [];
/** address → the document whose inbox it is. Nothing else resolves one. */
const inboxAddresses = new Map<string, string>();
let watches: { readonly doc: string; readonly onDeposits: (d: Deposit[]) => unknown }[] = [];
let undelivered: { readonly doc: string; readonly onDeposits: (d: Deposit[]) => unknown }[] = [];
let unionFailure: string | undefined;
let currentUser = "nobody";
let documentCount = 0;
let clock = 0;
@@ -310,28 +281,6 @@ export function installFakePolyfill(): FakePolyfill {
return stored;
}
function readOne(doc: string): UnionSubject[] {
const broken = unreachable.get(doc);
if (broken !== undefined) throw new Error(`cannot reach ${doc}: ${broken}`);
const stored = require(doc);
const out: UnionSubject[] = [];
for (const [subject, properties] of stored.subjects) {
// Built EXACTLY as `readUnion` builds it — a plain object literal filled by
// `(props[p] ??= []).push(o)`. Neither detail is cosmetic: the literal
// inherits from `Object.prototype`, and `??=` does NOT assign over an
// inherited truthy member, so a predicate named `constructor` or `toString`
// leaves `.push` undefined and the read THROWS.
const props: Record<string, string[]> = {};
for (const [predicate, values] of properties) {
for (const value of values) {
(props[predicate] ??= []).push(value);
}
}
out.push({ subject, graph: doc as Nuri, props });
}
return out;
}
const docsImpl = {
async sparqlUpdate(sessionId: unknown, query: unknown, anchor?: unknown): Promise<unknown> {
if (typeof query !== "string") throw new Error("[fake-polyfill] the query must be a string");
@@ -371,7 +320,7 @@ export function installFakePolyfill(): FakePolyfill {
async sparqlQuery(_sessionId: unknown, query: unknown): Promise<never> {
// Reached only if this package starts querying, which it does not: it reads
// through `readUnion`. The message splits the two reasons someone lands here,
// nothing at all. The message splits the two reasons someone lands here,
// because one of them is an attempt to write through the read door.
if (typeof query === "string" && DESTRUCTIVE.test(blankLiterals(query))) {
throw new Error(
@@ -381,8 +330,8 @@ export function installFakePolyfill(): FakePolyfill {
);
}
throw new Error(
"[fake-polyfill] docs.sparqlQuery is not modelled — this package reads through " +
"`readUnion`. Model it READ-ONLY here before using it.",
"[fake-polyfill] docs.sparqlQuery is not modelled — this package does not read. " +
"Model it READ-ONLY here before using it.",
);
},
};
@@ -401,46 +350,8 @@ export function installFakePolyfill(): FakePolyfill {
const from = Object.hasOwn(opts, "from") ? (opts.from ?? null) : currentUser;
clock += 1;
stored.deposits.push({ from, payload: opts.payload ?? null, ts: clock });
// Stored first, told afterwards — and told over the wire, which is why the
// notification waits for `deliverNotifications` rather than firing inline.
for (const watch of watches) if (watch.doc === stored.nuri) undelivered.push(watch);
},
watch(targetInbox: unknown, onDeposits: unknown): () => void {
const doc = inboxAddresses.get(String(targetInbox));
if (doc === undefined) {
// "`inbox.post` refuses a target that is not an inbox" — so does watching one,
// and nothing outside `openDocumentInbox` ever hands an address out.
throw new Error(`[fake-polyfill] not an inbox address: ${String(targetInbox)}`);
}
if (require(doc).owner !== currentUser) {
throw new Error(
`${currentUser} may not watch the inbox of ${doc}: you may DEPOSIT into ` +
"anyone's inbox, you may only READ your own",
);
}
if (typeof onDeposits !== "function") {
throw new Error("[fake-polyfill] inbox.watch takes a callback");
}
const watch = { doc, onDeposits: onDeposits as (d: Deposit[]) => unknown };
watches.push(watch);
return () => {
watches = watches.filter((w) => w !== watch);
};
},
async readForDocument(doc: unknown): Promise<Deposit[]> {
const stored = require(String(doc));
// No inbox → there is no address to read, which is a state and not an error.
if (stored.deposits === undefined) return [];
if (stored.owner !== currentUser) {
throw new Error(
`${currentUser} may not read the inbox of ${String(doc)}: you may DEPOSIT into ` +
"anyone's inbox, you may only READ your own",
);
}
return [...stored.deposits].sort((a, b) => a.ts - b.ts);
},
};
const storeRegistryImpl = {
@@ -464,41 +375,13 @@ export function installFakePolyfill(): FakePolyfill {
}
stored.deposits ??= [];
// Idempotent within a page: asking again for a document that already has one
// resolves that same address rather than adding a second inbox.
const address = `${stored.nuri}:inbox`;
inboxAddresses.set(address, stored.nuri);
return address;
},
async listMyEntityDocs(scope: unknown): Promise<Nuri[]> {
if (scope !== "public" && scope !== "mine") {
throw new Error(`[fake-polyfill] unknown scope ${JSON.stringify(scope)}`);
}
// "listMyEntityDocs returns a listing whose documents you can open, or it throws."
const mine: Nuri[] = [];
for (const stored of documents.values()) {
if (stored.owner === currentUser) mine.push(stored.nuri);
}
return mine;
// resolves that same address rather than opening a second inbox. The address
// goes no further — this package drops it, which is the point of `openInbox`
// returning nothing.
return `${stored.nuri}:inbox`;
},
};
async function readUnionImpl(docsLike: unknown): Promise<UnionSubject[]> {
if (unionFailure !== undefined) throw new Error(unionFailure);
const list = Array.isArray(docsLike) ? docsLike : [];
const out: UnionSubject[] = [];
for (const doc of [...new Set(list.filter(Boolean))]) {
try {
out.push(...readOne(String(doc)));
} catch (error) {
// `readDoc` is `try {…} catch { return [] }`: a failing document is skipped
// and never aborts the batch, so failure and emptiness arrive identical.
console.error("[fake-polyfill] read failed for", doc, String(error));
}
}
return out;
}
function userOfSession(sessionId: unknown): string {
const id = String(sessionId);
const user = id.startsWith("session:") ? id.slice("session:".length) : undefined;
@@ -513,8 +396,8 @@ export function installFakePolyfill(): FakePolyfill {
/**
* A namespace whose every member is recorded, and whose UNKNOWN members throw by
* name rather than arriving as `undefined`. `Object.hasOwn` and not `impl[name]`,
* because a plain object literal inherits `toString` and friends — the same trap
* `valuesOf` guards against in `src/index-document.ts`.
* because a plain object literal inherits `toString` and friends, and an
* inherited member is not a modelled entry.
*/
function namespace(name: string, impl: Record<string, (...args: never[]) => unknown>): unknown {
return new Proxy(impl, {
@@ -540,49 +423,22 @@ export function installFakePolyfill(): FakePolyfill {
docs: namespace("docs", docsImpl),
inbox: namespace("inbox", inboxImpl),
storeRegistry: namespace("storeRegistry", storeRegistryImpl),
async readUnion(docsLike: unknown) {
record("readUnion", [docsLike]);
return readUnionImpl(docsLike);
},
}));
return {
calls,
signIn(user: string) {
// "It lasts exactly as long as that identity stays connected — changing identity
// or clearing it stops it." This fake has ONE signed-in identity at a time, as a
// page does, so a watch cannot outlive the identity that opened it. Signing in as
// the same identity again is not a change, and stops nothing.
if (user !== currentUser) {
watches = [];
undelivered = [];
}
// This fake has ONE signed-in identity at a time, as a page does: a polyfill
// session IS one identity, and no call takes an identifier.
currentUser = user;
},
async deliverNotifications() {
while (undelivered.length > 0) {
const batch = undelivered;
undelivered = [];
for (const watch of batch) {
const stored = documents.get(watch.doc);
await watch.onDeposits([...(stored?.deposits ?? [])]);
}
}
},
sessionId() {
return `session:${currentUser}`;
},
breakReadsOf(doc: string, reason: string) {
unreachable.set(doc, reason);
},
healReadsOf(doc: string) {
unreachable.delete(doc);
},
breakReadUnion(reason: string) {
unionFailure = reason;
},
healReadUnion() {
unionFailure = undefined;
depositsIn(doc: string) {
const stored = require(doc);
if (stored.deposits === undefined) return null;
return [...stored.deposits].sort((a, b) => a.ts - b.ts);
},
contentsOf(doc: string) {
const stored = require(doc);
test/inbox-processing.test.ts
-337
View File
@@ -1,337 +0,0 @@
import { expect, mock, test } from "bun:test";
import { indexing, type Indexing } from "../src/indexing";
import { coalescing } from "../src/coalescing";
import type { Nuri } from "../src/port";
import { FakeNextGraph, publishObject } from "./fake-nextgraph";
/**
* WHEN an index is curated — the engagement `createIndex` makes about what becomes
* of what it created: the index is curated at its creator's next connection, and on
* each deposit while the creator is connected.
*
* Nothing below calls curation, because there is nothing to call. Every test here
* drives the two acts an application really has — connecting (obtaining a handle)
* and depositing — and asserts what the index holds afterwards.
*
* The case space is the creator's presence crossed with the deposit's timing:
* away when it was made, connected when it was made, and connected on an index
* that a previous session created. Plus the two that must NOT happen: a stranger
* connecting curates nothing, and a document that is no index is left alone.
*
* And crossing all of it, the three ways connecting can FAIL — it cannot look for
* its indexes, it cannot go through one, it cannot watch one. Each has its own test
* below, because each is a failure wearing the shape of an absence: the session
* carries on, the handle works, and an index quietly holds less than it should. The
* engagement is that none of them denies anything and none of them loses a deposit,
* which is only worth anything if it is exercised rather than asserted.
*/
const PUBLISHED_AT = "http://schema.org/datePublished";
function hardcodedInAppSource(nuri: Nuri): Nuri {
return nuri;
}
/**
* Alice creates an index, and then her page closes. That is the state most of these
* tests start from: an index exists, its creator is away, and nothing is watching
* it — so a deposit made now can only be seen at her next connection.
*/
async function aliceCreatesAnIndexAndLeaves(network: FakeNextGraph): Promise<Nuri> {
const alice = await indexing(network.portFor("alice"));
const index = await alice.createIndex(PUBLISHED_AT);
network.disconnect("alice");
return hardcodedInAppSource(index);
}
/**
* Runs `body` with this package's log stream captured, and reports HOW MANY
* failures it put there alongside whatever the body produced.
*
* The count, never the text: what a log line reads is for a human and nothing
* promises it, so a test that pinned the words would pin the one thing that is
* free to change. What is worth pinning is that a failure was reported AT ALL —
* harmless is not the same as invisible, and the whole risk here is a failure
* passing for an absence.
*/
async function capturingReports<T>(
body: () => Promise<T>,
): Promise<{ result: T; reports: number }> {
const reported = mock((..._args: unknown[]) => {});
const original = console.error;
console.error = reported;
try {
return { result: await body(), reports: reported.mock.calls.length };
} finally {
console.error = original;
}
}
// --- the deposits that piled up while the creator was away ----------------
test("an index is curated at its creator's next connection, with nobody asking", async () => {
const network = new FakeNextGraph();
const index = await aliceCreatesAnIndexAndLeaves(network);
// Bob deposits while Alice is away: her session is never told, and the deposit
// waits in the inbox where only she can see it.
const bobPort = network.portFor("bob");
const article = await publishObject(bobPort, PUBLISHED_AT, "2026-03-04");
await (await indexing(bobPort)).refer(index, article);
const carol = await indexing(network.portFor("carol"));
expect(await carol.read(index)).toEqual([]);
// Alice comes back. This is the whole of it: obtaining her handle IS the trigger.
const alice = await indexing(network.portFor("alice"));
expect(await alice.read(index)).toEqual([{ object: article, value: "2026-03-04" }]);
expect(await carol.read(index)).toEqual([{ object: article, value: "2026-03-04" }]);
});
test("a whole backlog is caught up, across every index the creator owns", async () => {
const network = new FakeNextGraph();
const alicePort = network.portFor("alice");
const bobPort = network.portFor("bob");
const first = await aliceCreatesAnIndexAndLeaves(network);
const second = await aliceCreatesAnIndexAndLeaves(network);
// An ordinary public document of Alice's, which is no index at all.
await publishObject(alicePort, PUBLISHED_AT, "2026-01-01");
const bob = await indexing(bobPort);
const early = await publishObject(bobPort, PUBLISHED_AT, "2026-01-02");
const late = await publishObject(bobPort, PUBLISHED_AT, "2026-05-06");
await bob.refer(first, early);
await bob.refer(first, late);
await bob.refer(second, late);
const alice = await indexing(alicePort);
expect((await alice.read(first)).map((e) => e.value)).toEqual(["2026-01-02", "2026-05-06"]);
expect(await alice.read(second)).toEqual([{ object: late, value: "2026-05-06" }]);
});
// --- the deposits that arrive while the creator is looking ----------------
test("a deposit made while the creator is connected is curated as it lands", async () => {
const network = new FakeNextGraph();
const alice = await indexing(network.portFor("alice"));
const index = hardcodedInAppSource(await alice.createIndex(PUBLISHED_AT));
const bobPort = network.portFor("bob");
const article = await publishObject(bobPort, PUBLISHED_AT, "2026-03-04");
await (await indexing(bobPort)).refer(index, article);
// The index was created in THIS session, so the store search never saw it: what
// brings it under observation is `createIndex` itself.
await network.deliverNotifications();
expect(await alice.read(index)).toEqual([{ object: article, value: "2026-03-04" }]);
});
test("an index from a previous session is watched too, not merely caught up once", async () => {
const network = new FakeNextGraph();
const index = await aliceCreatesAnIndexAndLeaves(network);
const bobPort = network.portFor("bob");
const bob = await indexing(bobPort);
// Alice comes back to an index she created before, with nothing waiting in it.
const alice = await indexing(network.portFor("alice"));
expect(await alice.read(index)).toEqual([]);
// …and only now does Bob deposit. Nothing but the watch can carry this one.
const article = await publishObject(bobPort, PUBLISHED_AT, "2026-07-08");
await bob.refer(index, article);
await network.deliverNotifications();
expect(await alice.read(index)).toEqual([{ object: article, value: "2026-07-08" }]);
});
test("a burst of deposits settles on the same index, whatever order they are told in", async () => {
const network = new FakeNextGraph();
const alice = await indexing(network.portFor("alice"));
const index = hardcodedInAppSource(await alice.createIndex(PUBLISHED_AT));
const bobPort = network.portFor("bob");
const bob = await indexing(bobPort);
for (const date of ["2026-03-04", "2026-01-31", "2025-12-25"]) {
await bob.refer(index, await publishObject(bobPort, PUBLISHED_AT, date));
}
await network.deliverNotifications();
expect((await alice.read(index)).map((e) => e.value)).toEqual([
"2025-12-25",
"2026-01-31",
"2026-03-04",
]);
});
// --- what connecting must NOT do -----------------------------------------
test("connecting curates nothing for anyone but the creator", async () => {
const network = new FakeNextGraph();
const index = await aliceCreatesAnIndexAndLeaves(network);
const bobPort = network.portFor("bob");
const bob = await indexing(bobPort);
const article = await publishObject(bobPort, PUBLISHED_AT, "2026-03-04");
await bob.refer(index, article);
// Bob connects again, and Carol connects: neither owns the index, so neither can
// read its inbox — and connecting must not try, nor fail, nor write anything.
const carol = await indexing(network.portFor("carol"));
await indexing(bobPort);
await network.deliverNotifications();
expect(await carol.read(index)).toEqual([]);
});
test("a public document that is no index is left alone — no inbox, no entry", async () => {
const network = new FakeNextGraph();
const alicePort = network.portFor("alice");
const ordinary = await alicePort.createPublicDocument();
await indexing(alicePort);
// Had connecting treated every public document as an index, it would have opened
// an inbox on this one — which is exactly what makes a deposit possible.
const bob = await indexing(network.portFor("bob"));
await expect(bob.refer(ordinary, "did:ng:o:doc-9")).rejects.toThrow(/has no inbox/);
});
test("a session that could not look for its indexes is still a working handle", async () => {
const network = new FakeNextGraph();
const index = await aliceCreatesAnIndexAndLeaves(network);
const bobPort = network.portFor("bob");
const article = await publishObject(bobPort, PUBLISHED_AT, "2026-03-04");
await (await indexing(bobPort)).refer(index, article);
network.breakListing("broker unreachable");
const reported = mock((..._args: unknown[]) => {});
const original = console.error;
console.error = reported;
let alice: Indexing;
try {
alice = await indexing(network.portFor("alice"));
} finally {
console.error = original;
}
// Reading an index and depositing into one need none of that work, so nothing is
// denied — but the failure is on the log, because a silent one teaches nobody.
expect(await alice.read(index)).toEqual([]);
await alice.refer(index, article);
expect(reported).toHaveBeenCalledTimes(1);
expect(String(reported.mock.calls[0]?.[0])).toContain("public store could not be listed");
});
test("an index whose catch-up failed is still a working handle, and loses no deposit", async () => {
const network = new FakeNextGraph();
const stalled = await aliceCreatesAnIndexAndLeaves(network);
const healthy = await aliceCreatesAnIndexAndLeaves(network);
const bobPort = network.portFor("bob");
const bob = await indexing(bobPort);
const article = await publishObject(bobPort, PUBLISHED_AT, "2026-03-04");
await bob.refer(stalled, article);
await bob.refer(healthy, article);
// The broker answers about the document and not about its inbox. Two repos
// upstream, read with two capabilities, so this is a partial failure and not a
// contrived one — and it is what makes the catch-up fail on THIS index alone.
network.breakInboxReadsOf(stalled, "broker unreachable");
const { result: alice, reports } = await capturingReports(() =>
indexing(network.portFor("alice")),
);
// Obtaining the handle RESOLVED — reaching this line at all is the assertion.
// Reading the index it could not go through still works…
expect(await alice.read(stalled)).toEqual([]);
// …and so does depositing into it: neither ever depended on that work.
await alice.refer(stalled, article);
// The session is not poisoned either: the other index was caught up normally.
expect(await alice.read(healthy)).toEqual([{ object: article, value: "2026-03-04" }]);
expect(reports).toBe(1);
// And nothing was lost. The deposits never left the inbox, so the first
// connection that can read it puts them in — which is the whole reason a failed
// run is allowed to be this quiet.
network.healInboxReadsOf(stalled);
network.disconnect("alice");
const back = await indexing(network.portFor("alice"));
expect(await back.read(stalled)).toEqual([{ object: article, value: "2026-03-04" }]);
});
test("an index that could not be watched is still caught up, and the rest still notices", async () => {
const network = new FakeNextGraph();
const unwatched = await aliceCreatesAnIndexAndLeaves(network);
const watched = await aliceCreatesAnIndexAndLeaves(network);
const bobPort = network.portFor("bob");
const bob = await indexing(bobPort);
const waiting = await publishObject(bobPort, PUBLISHED_AT, "2026-01-01");
await bob.refer(unwatched, waiting);
// The subscription is refused; reading that same inbox still works. A watch is
// held open where a read is one question and one answer, so one can be turned
// down while the other is served.
network.breakWatchingOf(unwatched, "the broker refused the subscription");
const { result: alice, reports } = await capturingReports(() =>
indexing(network.portFor("alice")),
);
// The watch failed and the catch-up ran ANYWAY — the backlog is in. That is the
// order the code goes to some trouble to hold: failing to watch must not cost
// the deposits that were already waiting.
expect(await alice.read(unwatched)).toEqual([{ object: waiting, value: "2026-01-01" }]);
expect(reports).toBe(1);
// What the failure costs, exactly and no more: a deposit made from now on is not
// NOTICED on that index…
const late = await publishObject(bobPort, PUBLISHED_AT, "2026-02-02");
await bob.refer(unwatched, late);
await bob.refer(watched, late);
await network.deliverNotifications();
expect((await alice.read(unwatched)).map((e) => e.value)).toEqual(["2026-01-01"]);
// …while every other index of the very same session goes on noticing its own.
expect(await alice.read(watched)).toEqual([{ object: late, value: "2026-02-02" }]);
// "Until the next connection" is the whole of the damage, and the next
// connection is where it ends.
network.healWatchingOf(unwatched);
network.disconnect("alice");
const back = await indexing(network.portFor("alice"));
expect((await back.read(unwatched)).map((e) => e.value)).toEqual(["2026-01-01", "2026-02-02"]);
});
// --- the primitive that keeps a burst from piling up ----------------------
test("coalescing never runs twice at once, and grants exactly one more run", async () => {
const trace: string[] = [];
const ask = coalescing(async () => {
trace.push("start");
// Yields, so the asks below really do arrive while a run is in flight — which
// is the only situation this primitive exists for.
await Promise.resolve();
trace.push("end");
});
const first = ask();
const during = [ask(), ask(), ask()];
await Promise.all([first, ...during]);
// Three asks during one run earn ONE more run between them, not three — and not
// none, since a deposit that landed after the first run read the inbox would
// otherwise wait for the next connection.
expect(trace).toEqual(["start", "end", "start", "end"]);
// Runs never overlap: no "start" ever follows a "start".
expect(trace.join(" ")).not.toContain("start start");
// And an ask that arrives once everything is quiet is a run of its own.
await ask();
expect(trace.filter((step) => step === "start")).toHaveLength(3);
});
test/indexing.test.ts
+91 -295
View File
@@ -1,11 +1,15 @@
import { expect, test } from "bun:test";
import { indexing, type Indexing } from "../src/indexing";
import { curate } from "../src/curator";
import { indexingOn, type Indexing } from "../src/indexing";
import type { Nuri } from "../src/port";
import { ENTRY_VALUE, INDEX_FIELD } from "../src/vocabulary";
import { INDEX_FIELD } from "../src/vocabulary";
import { FakeNextGraph, publishObject } from "./fake-nextgraph";
/**
* The two acts, and nothing else — this package creates an index and hands one a
* reference. What becomes of that reference is the business of the layer that
* processes the index's inbox, and there is nothing here that could do it or ask
* for it.
*
* Each actor gets their own handle, and they share no variable carrying business
* data. The ONE value that crosses between them is the index's NURI — and that
* crossing is the mechanism this design names: an application references the
@@ -21,336 +25,128 @@ function hardcodedInAppSource(nuri: Nuri): Nuri {
type Port = ReturnType<FakeNextGraph["portFor"]>;
async function world(): Promise<{
function world(): {
network: FakeNextGraph;
alice: Indexing;
bob: Indexing;
carol: Indexing;
ports: { alice: Port; bob: Port; carol: Port };
}> {
ports: { alice: Port; bob: Port };
} {
const network = new FakeNextGraph();
const ports = {
alice: network.portFor("alice"),
bob: network.portFor("bob"),
carol: network.portFor("carol"),
};
// Three connections, none of which owns an index yet: there is nothing to catch up
// on and nothing to watch. What each of them does next is what these tests are about.
return {
network,
alice: await indexing(ports.alice),
bob: await indexing(ports.bob),
carol: await indexing(ports.carol),
ports,
};
const ports = { alice: network.portFor("alice"), bob: network.portFor("bob") };
// A handle is nothing but a port bound to one identity: building one reaches
// nothing, which is why there is nothing to await here.
return { network, alice: indexingOn(ports.alice), bob: indexingOn(ports.bob), ports };
}
/**
* These tests exercise the curation RULES, so they run the curator itself rather
* than wait for an inbox notification: what a run makes of a deposit is what is
* under test, not when the run happens. `inbox-processing.test.ts` covers the when.
*
* The deposits below therefore sit in their inbox, told to nobody, which is exactly
* the state an owner's next connection finds.
*/
// --- creating an index ----------------------------------------------------
test("any user creates an index in their public store, and it declares its field", async () => {
const { alice, ports } = await world();
const { alice, network } = world();
const index = await alice.createIndex(PUBLISHED_AT);
const index = await alice.create(PUBLISHED_AT);
// An ordinary document: what makes it an index is the field it declares, which
// a reader going straight to `readUnion` sees on the index's own subject.
const subjects = await ports.alice.readDocument(index);
const self = subjects.find((s) => s.subject === index);
expect(self?.props[INDEX_FIELD]).toEqual([PUBLISHED_AT]);
expect(await alice.read(index)).toEqual([]);
// An ordinary document: what makes it an index is the field it declares, on the
// index's own subject, which is where a reader of the document finds it.
expect(network.contentsOf(index)).toEqual([
{ subject: index, predicate: INDEX_FIELD, values: [PUBLISHED_AT] },
]);
});
test("reading a document that declares no index field is refused, not answered empty", async () => {
const { alice, ports } = await world();
const ordinary = await ports.alice.createPublicDocument();
await expect(alice.read(ordinary)).rejects.toThrow(/declares no index field/);
test("a new index is ready to receive: its inbox is open the moment create returns", async () => {
const { alice, bob, network } = world();
const index = hardcodedInAppSource(await alice.create(PUBLISHED_AT));
// Nothing to open, register or remember — a stranger deposits straight away.
await bob.add(index, "did:ng:o:doc-9");
expect(network.depositsIn(index)).toHaveLength(1);
});
// --- the whole loop, across three people ----------------------------------
test("a field that could never match an object is refused at creation", async () => {
const { alice } = world();
// It cannot be corrected later — nothing here deletes — so it is refused now.
await expect(alice.create("")).rejects.toThrow(/cannot be changed later/);
await expect(alice.create(" ")).rejects.toThrow(/cannot be changed later/);
});
test("a stranger refers an object, the owner curates, and anyone reads the result", async () => {
const { alice, bob, carol, ports } = await world();
test("two indexes are two documents, each declaring its own field", async () => {
const { alice, network } = world();
const byDate = await alice.create(PUBLISHED_AT);
const byName = await alice.create(NAME);
expect(byDate).not.toBe(byName);
expect(network.contentsOf(byDate)).toEqual([
{ subject: byDate, predicate: INDEX_FIELD, values: [PUBLISHED_AT] },
]);
expect(network.contentsOf(byName)).toEqual([
{ subject: byName, predicate: INDEX_FIELD, values: [NAME] },
]);
});
// --- handing an index a reference -----------------------------------------
test("a stranger hands an index a reference, and it waits in the index's inbox", async () => {
const { alice, bob, ports, network } = world();
// Alice creates the index and its NURI goes into the application's source.
const indexNuri = hardcodedInAppSource(await alice.createIndex(PUBLISHED_AT));
const index = hardcodedInAppSource(await alice.create(PUBLISHED_AT));
// Bob, who owns nothing of Alice's, creates his own public object and hands the
// index a reference to it. He needs no permission and gets no write.
const article = await publishObject(ports.bob, PUBLISHED_AT, "2026-03-04");
await bob.refer(indexNuri, article);
await bob.add(index, article);
// Nothing is in the index until its owner acts.
expect(await carol.read(indexNuri)).toEqual([]);
// THE WHOLE PAYLOAD is the reference: no wrapper, no claim, no copy of the value,
// and no index either — the inbox it landed in is what identifies one. This is the
// one thing this package hands the layer that will make an entry of it.
expect(network.depositsIn(index)).toEqual([
{ from: "bob", payload: article, ts: expect.any(Number) },
]);
const report = await curate(ports.alice, indexNuri);
expect(report.outcomes).toEqual([{ result: "indexed", object: article, value: "2026-03-04" }]);
// Carol knows only the NURI from the application's source, and gets the entry.
const entries = await carol.read(indexNuri);
expect(entries).toEqual([{ object: article, value: "2026-03-04" }]);
// The entry is a usable reference: Carol opens the object straight from it,
// holding nothing but what she read out of the index.
const first = entries[0];
expect(first).toBeDefined();
const opened = await ports.carol.readDocument(first!.object);
expect(opened[0]?.props[PUBLISHED_AT]).toEqual(["2026-03-04"]);
// And it stayed a deposit: nothing wrote it into the index document.
expect(network.contentsOf(index)).toEqual([
{ subject: index, predicate: INDEX_FIELD, values: [PUBLISHED_AT] },
]);
});
test("an entry is a subject keyed by the object's NURI, so reading needs nothing new", async () => {
const { alice, bob, ports } = await world();
const indexNuri = hardcodedInAppSource(await alice.createIndex(PUBLISHED_AT));
test("the same reference handed over twice is two deposits, and nothing is lost", async () => {
const { alice, bob, ports, network } = world();
const index = hardcodedInAppSource(await alice.create(PUBLISHED_AT));
const article = await publishObject(ports.bob, PUBLISHED_AT, "2026-03-04");
await bob.refer(indexNuri, article);
await curate(ports.alice, indexNuri);
// What `readUnion([indexNuri])` hands an application that never loaded this
// package: the index's own subject, plus one subject per indexed object.
const subjects = await ports.bob.readDocument(indexNuri);
const entry = subjects.find((s) => s.subject === article);
expect(entry?.props[ENTRY_VALUE]).toEqual(["2026-03-04"]);
expect(subjects.map((s) => s.subject).sort()).toEqual([article, indexNuri].sort());
});
await bob.add(index, article);
await bob.add(index, article);
// --- only the owner curates ----------------------------------------------
test("nobody but the index's owner can curate it: the inbox is refused to others", async () => {
const { alice, bob, ports } = await world();
const indexNuri = hardcodedInAppSource(await alice.createIndex(PUBLISHED_AT));
const article = await publishObject(ports.bob, PUBLISHED_AT, "2026-03-04");
await bob.refer(indexNuri, article);
await expect(curate(ports.bob, indexNuri)).rejects.toThrow(/may only READ your own/);
expect(await alice.read(indexNuri)).toEqual([]);
});
test("nobody but the owner writes an index, whatever they know about it", async () => {
const { alice, ports } = await world();
const indexNuri = hardcodedInAppSource(await alice.createIndex(PUBLISHED_AT));
await expect(
ports.bob.addLiteralProperty(indexNuri, "did:ng:o:forged", ENTRY_VALUE, "2999-01-01"),
).rejects.toThrow(/only a document's owner writes to it/);
await expect(ports.bob.openInbox(indexNuri)).rejects.toThrow(/may not open an inbox/);
// Nothing here de-duplicates: a deposit is an invitation to look, so repeating one
// is legitimate and cheap, and whoever processes the inbox is the one that settles.
expect(network.depositsIn(index)?.map((d) => d.payload)).toEqual([article, article]);
});
test("an index whose owner never opened an inbox refuses a deposit rather than losing it", async () => {
const { bob, ports } = await world();
const { bob, ports } = world();
// A public document that was never made into an index: no inbox was opened.
const notAnIndex = hardcodedInAppSource(await ports.alice.createPublicDocument());
await expect(bob.refer(notAnIndex, "did:ng:o:doc-9")).rejects.toThrow(/has no inbox/);
await expect(bob.add(notAnIndex, "did:ng:o:doc-9")).rejects.toThrow(/has no inbox/);
});
// --- adding is idempotent -------------------------------------------------
test("the same reference deposited twice produces one entry", async () => {
const { alice, bob, ports } = await world();
const indexNuri = hardcodedInAppSource(await alice.createIndex(PUBLISHED_AT));
const article = await publishObject(ports.bob, PUBLISHED_AT, "2026-03-04");
await bob.refer(indexNuri, article);
await bob.refer(indexNuri, article);
const report = await curate(ports.alice, indexNuri);
expect(report.outcomes).toEqual([
{ result: "indexed", object: article, value: "2026-03-04" },
{ result: "unchanged", object: article },
]);
expect(await alice.read(indexNuri)).toEqual([{ object: article, value: "2026-03-04" }]);
test("a payload that is not a reference is refused here, not deposited for someone else to find", async () => {
const { alice, bob } = world();
const index = hardcodedInAppSource(await alice.create(PUBLISHED_AT));
// Both sides are checked, because both go on to name a document. An index's inbox
// takes anything anyone posts to it; what THIS package puts there is a NURI.
await expect(bob.add(index, "please index my article")).rejects.toThrow(/not a NURI/);
await expect(bob.add("http://example.org/index", "did:ng:o:doc-9")).rejects.toThrow(
/not a NURI/,
);
});
test("curating twice changes nothing the second time — deposits are not consumed", async () => {
const { alice, bob, ports } = await world();
const indexNuri = hardcodedInAppSource(await alice.createIndex(PUBLISHED_AT));
const article = await publishObject(ports.bob, PUBLISHED_AT, "2026-03-04");
await bob.refer(indexNuri, article);
// --- only the owner owns it -----------------------------------------------
await curate(ports.alice, indexNuri);
const before = await alice.read(indexNuri);
test("nobody but the owner writes an index, whatever they know about it", async () => {
const { alice, ports } = world();
const index = hardcodedInAppSource(await alice.create(PUBLISHED_AT));
const second = await curate(ports.alice, indexNuri);
expect(second.outcomes).toEqual([{ result: "unchanged", object: article }]);
expect(await alice.read(indexNuri)).toEqual(before);
});
// --- a read that cannot answer must never cost the index anything ---------
test("a reference the broker cannot resolve is reported, and adds nothing", async () => {
const { network, alice, bob, ports } = await world();
const indexNuri = hardcodedInAppSource(await alice.createIndex(PUBLISHED_AT));
const first = await publishObject(ports.bob, PUBLISHED_AT, "2026-01-01");
await bob.refer(indexNuri, first);
await curate(ports.alice, indexNuri);
const second = await publishObject(ports.bob, PUBLISHED_AT, "2026-02-02");
await bob.refer(indexNuri, second);
network.breakReadsOf(second, "broker unreachable");
const report = await curate(ports.alice, indexNuri);
const unresolved = report.outcomes.filter((o) => o.result === "unresolved");
expect(unresolved).toHaveLength(1);
expect(unresolved[0]).toMatchObject({ object: second });
// THE POINT: the entry that was already there is untouched.
expect(await alice.read(indexNuri)).toEqual([{ object: first, value: "2026-01-01" }]);
});
test("an already-indexed object survives its own reads failing, and is not even re-read", async () => {
const { network, alice, bob, carol, ports } = await world();
const indexNuri = hardcodedInAppSource(await alice.createIndex(PUBLISHED_AT));
const article = await publishObject(ports.bob, PUBLISHED_AT, "2026-01-01");
await bob.refer(indexNuri, article);
await curate(ports.alice, indexNuri);
// A passer-by nudges the index about an entry she found IN IT. Carol obtains
// the reference the only way she could in a real application — by reading the
// index whose NURI her app hardcodes — rather than being handed it by the test.
const seen = await carol.read(indexNuri);
const noticed = seen[0];
expect(noticed).toBeDefined();
// …and only then does the object become unreachable.
network.breakReadsOf(article, "broker unreachable");
await carol.refer(indexNuri, noticed!.object);
const report = await curate(ports.alice, indexNuri);
expect(report.outcomes.every((o) => o.result === "unchanged")).toBe(true);
expect(await alice.read(indexNuri)).toEqual([{ object: article, value: "2026-01-01" }]);
});
test("a failed resolve is self-correcting: the next curation adds what it could not", async () => {
const { network, alice, bob, ports } = await world();
const indexNuri = hardcodedInAppSource(await alice.createIndex(PUBLISHED_AT));
const article = await publishObject(ports.bob, PUBLISHED_AT, "2026-05-06");
await bob.refer(indexNuri, article);
network.breakReadsOf(article, "broker unreachable");
expect((await curate(ports.alice, indexNuri)).outcomes[0]?.result).toBe("unresolved");
expect(await alice.read(indexNuri)).toEqual([]);
// The deposit is still there, so nothing has to be re-deposited.
network.healReadsOf(article);
expect((await curate(ports.alice, indexNuri)).outcomes[0]).toEqual({
result: "indexed",
object: article,
value: "2026-05-06",
});
expect(await alice.read(indexNuri)).toEqual([{ object: article, value: "2026-05-06" }]);
});
test("a reference to something that was never created is reported, not silently dropped", async () => {
const { network, alice, bob, ports } = await world();
const indexNuri = hardcodedInAppSource(await alice.createIndex(PUBLISHED_AT));
await bob.refer(indexNuri, network.neverCreatedNuri());
const report = await curate(ports.alice, indexNuri);
expect(report.outcomes).toHaveLength(1);
expect(report.outcomes[0]?.result).toBe("unresolved");
expect(await alice.read(indexNuri)).toEqual([]);
});
// --- an object that does not fit the index --------------------------------
test("an object carrying nothing for the index's field is not added", async () => {
const { alice, bob, ports } = await world();
const indexNuri = hardcodedInAppSource(await alice.createIndex(PUBLISHED_AT));
// Exists, is public, is readable — but says nothing about the field this index
// is built on. OPEN QUESTION: this is the narrow behaviour, not a settled policy.
const object = await publishObject(ports.bob, NAME, "an object with no date");
await bob.refer(indexNuri, object);
const report = await curate(ports.alice, indexNuri);
expect(report.outcomes).toEqual([{ result: "skipped", object, reason: "no-field" }]);
expect(await alice.read(indexNuri)).toEqual([]);
});
test("an object carrying several values for the field is not added", async () => {
const { alice, bob, ports } = await world();
const indexNuri = hardcodedInAppSource(await alice.createIndex(PUBLISHED_AT));
const object = await publishObject(ports.bob, PUBLISHED_AT, "2026-01-01");
await ports.bob.addLiteralProperty(object, object, PUBLISHED_AT, "2026-09-09");
await bob.refer(indexNuri, object);
const report = await curate(ports.alice, indexNuri);
expect(report.outcomes).toEqual([{ result: "skipped", object, reason: "several-values" }]);
expect(await alice.read(indexNuri)).toEqual([]);
});
test("a payload that is not a reference is reported as foreign and changes nothing", async () => {
const { alice, bob, ports } = await world();
const indexNuri = hardcodedInAppSource(await alice.createIndex(PUBLISHED_AT));
const article = await publishObject(ports.bob, PUBLISHED_AT, "2026-03-04");
await bob.refer(indexNuri, article);
// Anyone may deposit anything into an inbox, so untrusted payloads do arrive.
await ports.bob.depositTo(indexNuri, { drop: "everything" });
const report = await curate(ports.alice, indexNuri);
expect(report.outcomes).toEqual([
{ result: "indexed", object: article, value: "2026-03-04" },
{ result: "foreign", reason: "payload is not a reference" },
]);
expect(await alice.read(indexNuri)).toEqual([{ object: article, value: "2026-03-04" }]);
});
test("an index referred to itself is skipped, so its declaration cannot become an entry", async () => {
const { alice, bob, ports } = await world();
const indexNuri = hardcodedInAppSource(await alice.createIndex(PUBLISHED_AT));
await bob.refer(indexNuri, indexNuri);
const report = await curate(ports.alice, indexNuri);
expect(report.outcomes).toEqual([
{ result: "skipped", object: indexNuri, reason: "self-reference" },
]);
expect(await alice.read(indexNuri)).toEqual([]);
});
// --- indexing by a date is an instance of indexing by a field -------------
test("an index whose field is a date reads back in chronological order", async () => {
const { alice, bob, carol, ports } = await world();
const indexNuri = hardcodedInAppSource(await alice.createIndex(PUBLISHED_AT));
const march = await publishObject(ports.bob, PUBLISHED_AT, "2026-03-04");
const january = await publishObject(ports.bob, PUBLISHED_AT, "2026-01-31");
const december = await publishObject(ports.bob, PUBLISHED_AT, "2025-12-25");
// Referred out of order, on purpose.
await bob.refer(indexNuri, march);
await bob.refer(indexNuri, december);
await bob.refer(indexNuri, january);
await curate(ports.alice, indexNuri);
expect((await carol.read(indexNuri)).map((e) => e.value)).toEqual([
"2025-12-25",
"2026-01-31",
"2026-03-04",
]);
});
test("two indexes over the same objects, on different fields, do not interfere", async () => {
const { alice, bob, ports } = await world();
const byDate = hardcodedInAppSource(await alice.createIndex(PUBLISHED_AT));
const byName = hardcodedInAppSource(await alice.createIndex(NAME));
const object = await publishObject(ports.bob, PUBLISHED_AT, "2026-03-04");
await ports.bob.addLiteralProperty(object, object, NAME, "Anemone");
await bob.refer(byDate, object);
await bob.refer(byName, object);
await curate(ports.alice, byDate);
await curate(ports.alice, byName);
expect(await alice.read(byDate)).toEqual([{ object, value: "2026-03-04" }]);
expect(await alice.read(byName)).toEqual([{ object, value: "Anemone" }]);
await expect(
ports.bob.addLiteralProperty(index, "did:ng:o:forged", INDEX_FIELD, "http://schema.org/aaa"),
).rejects.toThrow(/only a document's owner writes to it/);
await expect(ports.bob.openInbox(index)).rejects.toThrow(/may not open an inbox/);
});
test/only-grows.test.ts
-299
View File
@@ -1,299 +0,0 @@
import { expect, mock, test } from "bun:test";
import { indexing } from "../src/indexing";
import { curate } from "../src/curator";
import { entriesOf, entryValue } from "../src/index-document";
import { resolutionFromFailure, resolutionFromRead } from "../src/resolution";
import type { Nuri, UnionSubject } from "../src/port";
import { ENTRY_VALUE, INDEX_FIELD } from "../src/vocabulary";
import { FakeNextGraph, publishObject } from "./fake-nextgraph";
/**
* The invariant this package is built around — an index only ever grows — and the
* hole that was in it.
*
* `entriesOf` used to require EXACTLY ONE value per entry, so a subject carrying
* two read as absent. An index could therefore SHRINK through nothing but
* additions: no delete involved, the guarantee defeated by the one operation
* meant to uphold it. These tests pin the fix at both levels.
*/
const FIELD = "http://schema.org/datePublished";
function subject(iri: string, values: string[]): UnionSubject {
return { subject: iri, graph: "did:ng:o:index" as Nuri, props: { [ENTRY_VALUE]: values } };
}
test("an entry with several values still reads as one entry, deterministically", () => {
const s = subject("did:ng:o:a", ["2026-02-02", "2026-01-01"]);
expect(entryValue(s)).toBe("2026-01-01");
// Order of arrival must not change the answer: two readers must agree.
expect(entryValue(subject("did:ng:o:a", ["2026-01-01", "2026-02-02"]))).toBe("2026-01-01");
});
test("a subject with no value at all is not an entry", () => {
expect(entryValue({ subject: "did:ng:o:a", graph: "did:ng:o:i" as Nuri, props: {} })).toBeUndefined();
expect(entryValue(subject("did:ng:o:a", []))).toBeUndefined();
});
test("entriesOf keeps a multi-valued entry instead of dropping it", () => {
const index = "did:ng:o:index" as Nuri;
const entries = entriesOf([subject("did:ng:o:a", ["2026-02-02", "2026-01-01"])], index);
expect(entries).toEqual([{ object: "did:ng:o:a" as Nuri, value: "2026-01-01" }]);
});
test("one stray non-NURI subject cannot make every real entry unreadable", () => {
const index = "did:ng:o:index" as Nuri;
// An index document is an ordinary document; its owner may put anything in it.
// This used to THROW out of `entriesOf`, losing the whole index to one triple.
const entries = entriesOf(
[
subject("http://example.org/not-a-nuri", ["2026-02-02"]),
subject("did:ng:o:real", ["2026-01-01"]),
],
index,
);
expect(entries).toEqual([{ object: "did:ng:o:real" as Nuri, value: "2026-01-01" }]);
});
test("an entry whose value is the empty string is still an entry", () => {
const index = "did:ng:o:index" as Nuri;
expect(entriesOf([subject("did:ng:o:a", [""])], index)).toEqual([
{ object: "did:ng:o:a" as Nuri, value: "" },
]);
});
test("adding a second value to an entry cannot make it disappear", async () => {
const network = new FakeNextGraph();
const ownerPort = network.portFor("alice");
const owner = await indexing(ownerPort);
const index = await owner.createIndex(FIELD);
const article = await publishObject(network.portFor("bob"), FIELD, "2026-01-01");
await (await indexing(network.portFor("bob"))).refer(index, article);
await curate(ownerPort, index);
// A pure ADD — the only write this package has. Before the fix this emptied
// `read()` while both triples sat in the document.
await ownerPort.addLiteralProperty(index, article, ENTRY_VALUE, "2026-02-02");
expect(await owner.read(index)).toEqual([{ object: article, value: "2026-01-01" }]);
});
test("a raced double-add settles, and does not make every later run re-add", async () => {
const network = new FakeNextGraph();
const ownerPort = network.portFor("alice");
const owner = await indexing(ownerPort);
const index = await owner.createIndex(FIELD);
const bobPort = network.portFor("bob");
const article = await publishObject(bobPort, FIELD, "2026-01-01");
await (await indexing(bobPort)).refer(index, article);
await curate(ownerPort, index);
// What two curation runs racing each other leave behind: the object's owner
// edited it between their reads, so each added its own value.
network.ownerReplacesValue(article, article, FIELD, "2026-02-02");
await ownerPort.addLiteralProperty(index, article, ENTRY_VALUE, "2026-02-02");
// The entry is still there, and the curator recognises it as already indexed —
// before the fix it was invisible, so every run added yet another value.
const report = await curate(ownerPort, index);
expect(report.outcomes).toEqual([{ result: "unchanged", object: article }]);
expect(await owner.read(index)).toEqual([{ object: article, value: "2026-01-01" }]);
});
// --- the descriptor follows the SAME rule, for the same reason ------------
test("a second declared field stops curation LOUDLY and costs no entry", async () => {
const network = new FakeNextGraph();
const ownerPort = network.portFor("alice");
const owner = await indexing(ownerPort);
const bobPort = network.portFor("bob");
const index = await owner.createIndex(FIELD);
for (const date of ["2026-01-01", "2026-02-02", "2026-03-03"]) {
await (await indexing(bobPort)).refer(index, await publishObject(bobPort, FIELD, date));
}
await curate(ownerPort, index);
expect(await owner.read(index)).toHaveLength(3);
// One add-only write through the published surface — and the SMALLER string, the
// direction in which "smallest wins" would have switched the index onto it.
await ownerPort.addLiteralProperty(index, index, INDEX_FIELD, "http://schema.org/aaa");
// Reading is untouched: an entry already written is a fact, and does not become
// unreadable because the declaration above it turned ambiguous.
expect(await owner.read(index)).toHaveLength(3);
// Curating refuses, and says why instead of quietly picking one.
await expect(curate(ownerPort, index)).rejects.toThrow(/declares 2 index fields/);
});
test("a mixed-field index is never produced: curation refuses before adding anything", async () => {
const network = new FakeNextGraph();
const ownerPort = network.portFor("alice");
const owner = await indexing(ownerPort);
const bobPort = network.portFor("bob");
const NAME = "http://schema.org/name";
const index = await owner.createIndex(NAME);
const first = await publishObject(bobPort, NAME, "Anemone");
await (await indexing(bobPort)).refer(index, first);
await curate(ownerPort, index);
// "…/datePublished" < "…/name", so under "smallest wins" the new field took over
// while `first` kept its old value forever — one list ordered by two properties.
await ownerPort.addLiteralProperty(index, index, INDEX_FIELD, FIELD);
const second = await publishObject(bobPort, FIELD, "2026-02-02");
await (await indexing(bobPort)).refer(index, second);
await expect(curate(ownerPort, index)).rejects.toThrow(/refusing to curate rather than pick one/);
expect(await owner.read(index)).toEqual([{ object: first, value: "Anemone" }]);
});
test("an index declaring no field at all is still refused", async () => {
const network = new FakeNextGraph();
const ownerPort = network.portFor("alice");
const ordinary = await ownerPort.createPublicDocument();
await expect((await indexing(ownerPort)).read(ordinary)).rejects.toThrow(/declares no index field/);
});
test("curating a document that declares no field refuses, and writes nothing", async () => {
const network = new FakeNextGraph();
const ownerPort = network.portFor("alice");
const bobPort = network.portFor("bob");
// A document of Alice's with an inbox open and a reference waiting in it, and no
// field declared. This is also the shape an INDEX arrives in when it could not be
// read — the real `readUnion` turns a failed read into `[]` — so the two are one
// case here, and the refusal has to hold for both.
const noField = await ownerPort.createPublicDocument();
await ownerPort.openInbox(noField);
const article = await publishObject(bobPort, FIELD, "2026-01-01");
await (await indexing(bobPort)).refer(noField, article);
await expect(curate(ownerPort, noField)).rejects.toThrow(/declares no index field/);
// It refused instead of curating on a field it does not have, and it refused
// BEFORE writing: the document is still empty, so no entry was invented for it,
// and the deposit is still in the inbox for a run that knows what to do with it.
expect(await ownerPort.readDocument(noField)).toEqual([]);
expect(await ownerPort.readDeposits(noField)).toHaveLength(1);
});
test("a field that could never match an object is refused at creation", async () => {
const owner = await indexing(new FakeNextGraph().portFor("alice"));
// It cannot be corrected later — nothing here deletes — so it is refused now.
await expect(owner.createIndex("")).rejects.toThrow(/cannot be changed later/);
await expect(owner.createIndex(" ")).rejects.toThrow(/cannot be changed later/);
});
// --- a field named like an Object.prototype member ------------------------
test("a field colliding with Object.prototype neither crashes nor is silently mis-read", async () => {
const network = new FakeNextGraph();
const ownerPort = network.portFor("alice");
const owner = await indexing(ownerPort);
const bobPort = network.portFor("bob");
for (const field of ["constructor", "toString", "valueOf", "hasOwnProperty"]) {
const index = await owner.createIndex(field);
// An object that CARRIES such a predicate cannot be read at all: `readUnion`
// fills `props` with `(props[p] ??= []).push(o)`, and `??=` does not assign
// over the inherited member, so `.push` is undefined and the read throws.
// Upstream's behaviour, mirrored by the double — so this comes back as a
// failure to resolve, NOT as an entry.
const carries = await publishObject(bobPort, field, "a value");
// An object that merely LACKS it must still resolve cleanly: reading the field
// off a plain object literal would otherwise hand back an inherited function.
const lacks = await publishObject(bobPort, "http://schema.org/name", "unrelated");
await (await indexing(bobPort)).refer(index, lacks);
await (await indexing(bobPort)).refer(index, carries);
const report = await curate(ownerPort, index);
expect(report.outcomes[0]).toEqual({ result: "skipped", object: lacks, reason: "no-field" });
expect(report.outcomes[1]?.result).toBe("unresolved");
expect(await owner.read(index)).toEqual([]);
}
});
// --- the resolution rule, which used to be unreachable in the adapter -----
test("an empty read resolves as unresolved — never as an object with no field", () => {
const resolution = resolutionFromRead([]);
expect(resolution.state).toBe("unresolved");
// The distinction that matters: had this said `present`, the curator would have
// filed a FAILED read as `skipped: "no-field"` — a fact about the object.
expect(resolution.state === "unresolved" && resolution.reason).toContain("absent, unreadable");
});
test("a non-empty read resolves as present, carrying the subjects through", () => {
const subjects = [subject("did:ng:o:a", ["v"])];
expect(resolutionFromRead(subjects)).toEqual({ state: "present", subjects });
});
test("a read that threw resolves as unresolved, naming the error", () => {
const resolution = resolutionFromFailure(new Error("broker unreachable"));
expect(resolution).toEqual({ state: "unresolved", reason: "Error: broker unreachable" });
});
// --- a failure must SURFACE, not just be returned -------------------------
test("an unresolved reference is warned about, not only reported", async () => {
const network = new FakeNextGraph();
const ownerPort = network.portFor("alice");
const owner = await indexing(ownerPort);
const index = await owner.createIndex(FIELD);
const article = await publishObject(network.portFor("bob"), FIELD, "2026-01-01");
await (await indexing(network.portFor("bob"))).refer(index, article);
network.breakReadsOf(article, "broker unreachable");
const warn = mock((..._args: unknown[]) => {});
const original = console.warn;
console.warn = warn;
try {
await curate(ownerPort, index);
} finally {
console.warn = original;
}
expect(warn).toHaveBeenCalledTimes(1);
expect(String(warn.mock.calls[0]?.[0])).toContain("broker unreachable");
});
test("a normal run warns about nothing", async () => {
const network = new FakeNextGraph();
const ownerPort = network.portFor("alice");
const owner = await indexing(ownerPort);
const index = await owner.createIndex(FIELD);
const article = await publishObject(network.portFor("bob"), FIELD, "2026-01-01");
await (await indexing(network.portFor("bob"))).refer(index, article);
const warn = mock((..._args: unknown[]) => {});
const original = console.warn;
console.warn = warn;
let report;
try {
report = await curate(ownerPort, index);
} finally {
console.warn = original;
}
// Assert the run actually DID something — otherwise this passes for a curation
// that indexed nothing at all, which would warn about nothing either.
expect(report.outcomes).toEqual([{ result: "indexed", object: article, value: "2026-01-01" }]);
expect(warn).not.toHaveBeenCalled();
});
// --- an unreadable index must not be diagnosed as a malformed one ---------
test("an index that could not be read is refused, and says so without blaming the document", async () => {
const network = new FakeNextGraph();
const ownerPort = network.portFor("alice");
const owner = await indexing(ownerPort);
const index = await owner.createIndex(FIELD);
network.breakReadsOf(index, "broker unreachable");
// The real `readUnion` turns a failed read into `[]`, so the failure arrives
// looking like a blank document. Whatever the shape, nothing may be written.
await expect(curate(ownerPort, index)).rejects.toThrow();
await expect(owner.read(index)).rejects.toThrow();
network.healReadsOf(index);
expect(await owner.read(index)).toEqual([]);
});
test/public-surface.test.ts
+48 -40
View File
@@ -1,62 +1,70 @@
import { expect, test } from "bun:test";
import { expect, mock, test } from "bun:test";
// Deliberately the PACKAGE ENTRY POINT, not the modules behind it: this is the
// surface an application gets, and it must be usable on its own. It pulls in
// `polyfill-adapter.ts`, so this also proves the real `@ng-eventually/polyfill`
// still loads and still exports everything this package compiles against.
import {
indexing,
decodeReference,
polyfillPort,
ENTRY_VALUE,
INDEX_FIELD,
type IndexEntry,
type NextGraphPort,
} from "../src/index";
import * as published from "../src/index";
import { ENTRY_VALUE, INDEX_FIELD, indexing } from "../src/index";
import { indexingOn } from "../src/indexing";
import { FakeNextGraph, publishObject } from "./fake-nextgraph";
const PUBLISHED_AT = "http://schema.org/datePublished";
test("the published surface carries the whole loop, end to end", async () => {
test("the published surface carries both acts, end to end", async () => {
const network = new FakeNextGraph();
const ownerPort: NextGraphPort = network.portFor("alice");
const strangerPort: NextGraphPort = network.portFor("bob");
const ownerPort = network.portFor("alice");
const strangerPort = network.portFor("bob");
const owner = await indexing(ownerPort);
const stranger = await indexing(strangerPort);
// `indexingOn` is what `indexing(sessionId)` calls once it has built the port for
// you: the two acts below are the published ones, reached here over an in-memory
// NextGraph. `adapter.test.ts` drives these same acts through the real adapter.
const owner = indexingOn(ownerPort);
const stranger = indexingOn(strangerPort);
const index = await owner.createIndex(PUBLISHED_AT);
const index = await owner.create(PUBLISHED_AT);
const article = await publishObject(strangerPort, PUBLISHED_AT, "2026-07-08");
await stranger.refer(index, article);
await stranger.add(index, article);
// NOBODY CURATES — there is nothing on this surface to call. Alice is connected,
// so her session is told a deposit landed and processes that inbox itself.
await network.deliverNotifications();
const entries: IndexEntry[] = await owner.read(index);
expect(entries).toEqual([{ object: article, value: "2026-07-08" }]);
// What the two acts leave behind, and the whole of it: a document declaring its
// field, and a bare reference waiting in its inbox. Making an entry of that
// reference is the business of the layer below, and there is nothing on this
// surface that does it or asks for it.
expect(network.contentsOf(index)).toEqual([
{ subject: index, predicate: INDEX_FIELD, values: [PUBLISHED_AT] },
]);
expect(network.depositsIn(index)?.map((deposit) => deposit.payload)).toEqual([article]);
});
test("the published surface offers no way to run, aim or schedule curation", async () => {
const handle = await indexing(new FakeNextGraph().portFor("alice"));
// Read off the handle rather than from a list: an application gets these three
// acts and nothing else, and curation is not one of them.
expect(Object.keys(handle).sort()).toEqual(["createIndex", "read", "refer"]);
test("the published surface offers TWO acts, and neither reads nor processes anything", () => {
const handle = indexingOn(new FakeNextGraph().portFor("alice"));
// Read off the handle rather than from a list: an application gets these two acts
// and nothing else. Reading an index is not one of them, and neither is anything
// that would make its deposits into entries.
expect(Object.keys(handle).sort()).toEqual(["add", "create"]);
});
test("the published surface exposes the deposit decoder and the two IRIs it writes", () => {
expect(decodeReference("did:ng:o:doc-1")).toBe("did:ng:o:doc-1");
expect(decodeReference({ object: "did:ng:o:doc-1" })).toBeNull();
test("the package publishes ONE function and the two IRIs, and nothing else", () => {
// Read off the module rather than from a list of names to remember: publishing a
// helper again — a port builder, a deposit decoder, a read — fails here first.
expect(Object.keys(published).sort()).toEqual(["ENTRY_VALUE", "INDEX_FIELD", "indexing"]);
expect(INDEX_FIELD).toBe("urn:ng-helpers:index:field");
expect(ENTRY_VALUE).toBe("urn:ng-helpers:index:value");
});
test("polyfillPort is published and builds a port without a live session", () => {
// Constructing it must not touch the broker — an application wires it at
// startup, and only the calls on it talk to anything.
const port: NextGraphPort = polyfillPort({ sessionId: "session-under-test" });
expect(typeof port.resolveObject).toBe("function");
expect(typeof port.addLiteralProperty).toBe("function");
// The port has NO operation that could take an entry out of an index.
const removing = Object.keys(port).filter((name) => /delete|remove|clear|drop/i.test(name));
expect(removing).toEqual([]);
test("indexing takes a session id and reaches nothing at all", () => {
// The REAL polyfill, unconfigured — which would fail every call that needs a
// broker. Building the handle does not make one, so this passes without any
// arrangement, and there is nothing on the log stream to report. It also proves
// the entry point builds its own port: an application hands over the session id
// `init(…)` gave it, and never sees a port at all.
const reported = mock((..._args: unknown[]) => {});
const original = console.error;
console.error = reported;
try {
const handle = indexing("session-under-test");
expect(Object.keys(handle).sort()).toEqual(["add", "create"]);
} finally {
console.error = original;
}
expect(reported).not.toHaveBeenCalled();
});
test/units.test.ts
+1 -14
View File
@@ -1,7 +1,6 @@
import { expect, test } from "bun:test";
import { buildInsertTriple, escapeIri, escapeLiteral, isIriSafe } from "../src/sparql";
import { asNuri, isNuri } from "../src/nuri";
import { decodeReference } from "../src/deposit";
// --- what a reference is, and what it is not ------------------------------
@@ -24,18 +23,6 @@ test("asNuri throws on a non-reference rather than passing it on", () => {
expect(() => asNuri("nope")).toThrow(/not a NURI/);
});
test("a deposit is a bare reference — anything else decodes to null", () => {
expect(decodeReference("did:ng:o:doc-7")).toBe("did:ng:o:doc-7");
// The shapes a well-meaning caller might invent, all refused: the payload IS
// the reference, it is not wrapped and it carries nothing else.
expect(decodeReference({ object: "did:ng:o:doc-7" })).toBeNull();
expect(decodeReference({ assert: "published", object: "did:ng:o:doc-7" })).toBeNull();
expect(decodeReference(["did:ng:o:doc-7"])).toBeNull();
expect(decodeReference("please index did:ng:o:doc-7")).toBeNull();
expect(decodeReference(null)).toBeNull();
expect(decodeReference(7)).toBeNull();
});
// --- escaping -------------------------------------------------------------
test("escapeLiteral leaves no raw quote that could close a SPARQL literal", () => {
@@ -147,7 +134,7 @@ test("a tripwire: no destructive SPARQL keyword is written anywhere under src/",
// a keyword hidden behind the comment-stripper, `DELETE{` with no space, and a
// literal split across concatenated lines.
//
// What actually guards the invariant is `test/adapter-write-path.test.ts`,
// What actually guards the invariant is `test/adapter.test.ts`,
// which RUNS the adapter and reads back every query it emits; all four evasions
// fail there. This stays as a cheap tripwire that catches the obvious
// regression early and names the file — it is not the proof.