diff --git a/.gitignore b/.gitignore index 37ee4b1..5d78b48 100644 --- a/.gitignore +++ b/.gitignore @@ -2,8 +2,6 @@ node_modules/ dist/ *.tsbuildinfo .DS_Store -bun.lockb -bun.lock e2e/.dist/ *.ngw diff --git a/.project/concepts/indexing/indexing-layer/contract_indexing-layer.md b/.project/concepts/indexing/indexing-layer/contract_indexing-layer.md index c90fb36..731c729 100644 --- a/.project/concepts/indexing/indexing-layer/contract_indexing-layer.md +++ b/.project/concepts/indexing/indexing-layer/contract_indexing-layer.md @@ -1,6 +1,6 @@ --- type: contract -summary: The API @ng-helpers/indexing exposes to an application — creating an index, depositing references into it, reading it back; curating is not on it: it is what an index's inbox being processed does +summary: The API @ng-helpers/indexing exposes to an application — creating an index, depositing references into it, reading it back; what a reference becomes is not a call: it becomes an entry once the index's creator is connected --- # contract_indexing-layer — `@ng-helpers/indexing` @@ -9,23 +9,25 @@ summary: The API @ng-helpers/indexing exposes to an application — creating an This package builds an **index** on top of NextGraph: an ordinary public document that holds one entry per indexed object, keyed by that object's NURI and carrying its value for a single declared field. -It covers creating an index, handing one a reference to an object (open to anyone), and reading the entries back in order. Resolving those references and adding what can be added is covered too, but never as a call: it is what happens when the index's inbox is processed. +It covers creating an index, depositing a reference to an object into one (open to anyone), and reading the entries back in order. What becomes of them is covered too, but never as a call — see `## Guarantees`. -It does not cover NextGraph itself — documents, identity, sharing, inboxes, transport — all of which reach it through a port you supply. It does not cover search, filtering, pagination, or querying by anything but the index's own field. It **never removes anything**, anywhere — an engagement, not a missing feature. +It does not cover NextGraph itself — documents, identity, sharing, transport — all of which reach it through a port you supply. It does not cover search, filtering, pagination, or querying by anything but the index's field. It **never removes anything**, anywhere — an engagement, not a missing feature. ### Deployment requirements An application using this package must: -- have a NextGraph session already open under the identity it wants to act as, and build the port from it — `polyfillPort({ sessionId })`, where `sessionId` is what `@ng-eventually/polyfill`'s own `init(…)` hands its callback; -- **await `indexing(port)` once at startup and keep what it produces.** That handle is one identity's and is also that identity's connection: awaiting it is what curates, dropping it is what stops; -- reach a broker, since every operation here is a document read, a document write or an inbox deposit; -- **supply `@ng-eventually/polyfill` itself.** This package declares it a *peer*: the application names it among its own dependencies, and that copy must be the one its own code calls — that package requires exactly one instance of itself in an application, for reasons its own contract states; -- **hardcode the index's NURI in its own source, and be the creator's own application if the index is ever to fill.** One requirement, not two: nothing marks a document as an index, so the reference an application carries is the only way anyone reaches it, and its creator's connections are the only thing that curates it. An index nobody hardcodes is unreachable; one whose creator never comes back stays as it was, however many references it is handed. +- have a NextGraph session open under the identity it wants to act as, and build the port from it — `polyfillPort({ sessionId })`, where `sessionId` is what `@ng-eventually/polyfill`'s own `init(…)` hands its callback; +- **await `indexing(port)` once at startup and keep what it produces.** That handle is one identity's and is also that identity's connection: awaiting it is what makes entries appear, dropping it is what stops them; +- reach a broker: every operation here reaches NextGraph, and nothing is answered locally; +- **supply `@ng-eventually/polyfill` itself.** This package declares it a *peer*: the application names it among its own dependencies, and that copy must be the one its own code calls — that package requires exactly one instance of itself, for reasons its own contract states; +- **connect as an index's creator if that index is ever to fill.** Unconditional, whoever holds its reference: entries are made while the creator is connected and by nothing else, so an index whose creator never returns stays as it was, however many references it is handed. One handle is one identity: the port carries a session, no call takes an identifier, and two users mean two handles. -**Obtaining it.** Not published to npm or any other host, and not built output: the entry point is TypeScript source, so whatever builds the application compiles it. `@ng-eventually/polyfill` arrives the same way. What this contract fixes is the version you pin and what you must provide alongside it. +**Holding an index's reference.** An index is reached by its NURI, held however the application holds any other reference — per user, per context, or read out of a document it opens anyway. Nothing marks a document as an index, so that reference is the only way anyone reaches it, and losing it loses the index. **Hardcoding it in the source is what a single GLOBAL index needs, and only that case**: one index serving the whole application has nothing else to be discovered by. Anything narrower is discovered, not hardcoded. + +**Obtaining it.** Not published to npm or any other host, and not built output: the entry point is TypeScript source, so whatever builds the application compiles it, and `@ng-eventually/polyfill` arrives the same way. What this contract fixes is the version you pin and what you must provide alongside it. ## Surface @@ -35,8 +37,7 @@ Full typed shape: the package's `types` entry, `@ng-helpers/indexing`. The load- // ── wiring: one handle, one identity, and that identity's connection ───────── export function polyfillPort(options: PolyfillPortOptions): NextGraphPort; export interface PolyfillPortOptions { readonly sessionId: string | number } -/** Produces this identity's handle — and before resolving, goes through the inbox of - * every index it owns, leaving each watched for as long as the handle lives. */ +/** This identity's handle, and its connection: see `## Guarantees`. */ export function indexing(port: NextGraphPort): Promise; // ── addressing (re-exported so you import them from here) ──────────────────── @@ -46,19 +47,18 @@ export type { PrincipalId, UnionSubject, NextGraphPort, IncomingDeposit, ObjectR // ── the three acts an application performs ─────────────────────────────────── export interface Indexing { - /** Produces a new index in THIS identity's public store, its inbox open, and the NURI - * to hardcode. Any user may. `field` is the predicate an indexed object must carry, - * declared once and for good; an empty or blank one throws. */ + /** A new index in THIS identity's public store, and its NURI. Any user may. `field` + * is the predicate an indexed object must carry; an empty or blank one throws. */ createIndex(field: string): Promise; - /** Deposits a bare reference into the index's inbox. Open to ANYONE. Produces nothing: - * no receipt, and no inbox address is ever handed out. Throws if there is no inbox. */ + /** Deposits a bare reference to an object into the index. Open to ANYONE. Produces + * nothing. Throws when the document cannot take one, rather than losing it. */ refer(index: NuriLike, object: NuriLike): Promise; - /** Produces the entries, ordered by value. Refuses a document that declares no index - * field rather than producing an empty list. Sugar over `readUnion([index])`. */ + /** The entries, ordered by value. Refuses a document that declares no index field + * rather than producing an empty list. Sugar over `readUnion([index])`. */ read(index: NuriLike): Promise; } -// ── what an index holds, and what travels from a depositor to a curator ────── +// ── what an index holds, and what travels from a depositor to an index ─────── export interface IndexEntry { readonly object: Nuri; readonly value: string } export interface IndexDescriptor { readonly field: string } export type IndexDeposit = Nuri; // the reference IS the whole payload @@ -73,66 +73,68 @@ export const ENTRY_VALUE: string; // on an entry: that object's value for the **An index is an ordinary public document, and nothing marks it as one.** It lives in its creator's public store, so any reader opens it from the reference alone; its creator owns it, and any user may create one. -**What becomes of what was created: the index is curated at its creator's next connection, and on each deposit while the creator is connected.** Awaiting `indexing(port)` is that connection — it goes through the inbox of every index the identity owns, backlog and all, and leaves each watched, so a deposit made from then on is applied as it lands. There is nothing to call, schedule or configure, and no way to aim curating at one index. Only the owner could anyway: nobody else reads that inbox, and nobody else writes that document. +**A reference deposited into an index becomes an entry once the index's creator is connected.** Awaiting `indexing(port)` is that connection: across every index that identity owns, what was deposited while it was away becomes an entry then, and what arrives from that moment on becomes one as it lands. There is nothing to call, schedule or configure, and no way to aim it at one index — the creator's session is the only thing that ever adds an entry, and it serves all of them at once. -**The field is declared once, inside the document, and cannot be changed.** `createIndex` refuses an empty or blank one at the door: nothing here deletes, so an index created on a useless field is useless for good. Declaring it in the document rather than in an application's source stops two applications curating one index on two fields. +**The field is declared once, inside the document, and cannot be changed.** `createIndex` refuses an empty or blank one at the door: nothing here deletes, so an index created on a useless field is useless for good. Declaring it in the document, not in an application's source, stops two applications indexing one on two fields. -**`createIndex` opens the index's inbox itself, and brings the new index under observation.** Only the owner can open one, and creation is the one moment the owner is present; and since the search at connection ran before this document existed, what was just created is added to what the session watches. +**A new index is ready the moment `createIndex` produces it.** It can be deposited into straight away, and the session that created it is already the one making its entries — nothing to open, register or reconnect. -**Depositing is open to anyone; writing is the owner's alone.** `refer` is a deposit into the index document's inbox — not a write — so a stranger can contribute to an index they do not own. The deposit is a **bare reference**: no operation, no index reference (the inbox address identifies the index), no copy of the indexed value. What the object itself says is what goes in. +**Depositing is open to anyone; writing is the creator's alone.** `refer` is a deposit, not a write, so a stranger contributes to an index they could not write. What is deposited is a **bare reference**: no operation, no claim, no copy of the value. What the object itself says, when its entry is made, is what goes in. -**A BET, named as one.** That a document can have an inbox is aligned with NextGraph: a repository takes an inbox capability, at most one, driven by a real commit upstream. **What a deposit carries, and what processing one does, are ours.** Upstream's inbox content type declares `Link`, `Patch` and four others as bare names with no payload at all — reserved words, not shapes — and the only two kinds carrying data are unrelated to indexing; that set is closed, with no trait, table of handlers or hook. An index deposit therefore has a shape NextGraph has not defined. When upstream defines those variants, this layer moves with them, and a `major` is how you hear about it. +**An index ONLY EVER GROWS.** No call removes an entry, for anyone including the creator, and none is planned: this package cannot express a removal at all, and it was deliberately never built rather than left for later. Do not design around a future delete — the answer to "this entry must go" is a fresh index. -**An index ONLY EVER GROWS.** No call removes an entry, for anyone including the owner, and none is planned: this package cannot express a removal at all, and it was deliberately never built rather than left for later. Do not design around a future delete — the only answer to "this entry must go" is a fresh index. +**The same references produce the same index, whatever order they arrived in.** One reference deposited a hundred times leaves one entry; an object already indexed is passed over rather than read again; a burst settles exactly where those references would one at a time. The cost: no deposit is ever retired, so the work behind an index is linear in its whole history. -**Curating is convergent and order-independent.** Deposits are never retired, so every run reads every deposit ever made to that index — linear in its history — and re-applying one lands on the same result; an already-indexed object is passed over. Neither the order references arrived in nor the number of notifications a burst produced changes anything: runs on one index never overlap, and an arrival during a run earns exactly one more run after it. +**None of this can deny you anything.** A session that could not find its indexes, catch one up, or stay posted about one still hands you a working handle: reading an index and depositing into one never depended on that work. Every such failure is on this package's log stream, as is every reference that could not be resolved — harmless is not the same as invisible. Nothing is lost either way: a reference already deposited is still waiting, and the next connection makes its entry. -**None of this can deny you anything.** A session that could not read its public store, watch an index, or go through one still hands you a working handle: reading an index and depositing into one never depended on that work. Every such failure is on this package's log stream, and so is every reference that could not be resolved — harmless is not the same as invisible. Nothing is lost either way: the deposits stay in their inbox for the next notification or connection. +**Reading is per-entry tolerant.** `read` returns entries ordered by value, ties broken on the object NURI, so two readers always see the same order. Values are compared **as strings**, so an index whose field holds ISO-8601 dates comes out in chronological order. A subject that is not a NURI is passed over rather than thrown on, and only own properties are read: one stray triple cannot make every real entry unreadable. -**Reading is per-entry tolerant.** `read` returns entries ordered by value, ties broken on the object NURI, so two readers always see the same order. Values are compared **as strings** — an index whose field holds ISO-8601 dates comes out in chronological order. A subject that is not a NURI is passed over rather than thrown on, and only own properties are read: one stray triple cannot make every real entry unreadable. +**An entry carrying several values keeps the smallest, deterministically** — which two sessions racing each other can produce, and which keeps the entry visible with every reader agreeing on it. -**An entry carrying several values keeps the smallest, deterministically** — which two runs racing each other can produce, and which keeps the entry visible with every reader agreeing on it. +**The document's own declaration is read strictly for writing and leniently for reading.** `read` refuses a document that declares no field at all rather than answering "an empty index": an unreadable document and an empty one arrive as the same empty result, so an empty answer would be a failure wearing the shape of a fact — retry before concluding it is malformed. An index declaring SEVERAL fields stops gaining entries, loudly and permanently, and stays readable: picking one would order a single list by two properties, since entries already made are never revisited. That cannot be undone — start a fresh index. -**The document's own declaration is read strictly for curating and leniently for reading.** `read` refuses a document that declares no field at all rather than answering "an empty index": an unreadable document and an empty one arrive as the same empty result, so an empty answer would be a failure wearing the shape of a fact — retry before concluding it is malformed. An index declaring SEVERAL fields stops being curated, loudly and permanently, and stays readable: picking one would leave a single list ordered by two properties, since entries already written are never re-read. That cannot be undone — curate into a fresh index. +**Reading needs nothing from this package.** An application that knows the NURI can call the polyfill's `readUnion([index])` and get one subject per indexed object, plus the index's own subject declaring its field, which `read` drops. `INDEX_FIELD` and `ENTRY_VALUE` are published for that reader. -**Reading needs nothing from this package.** An application that knows the NURI can call the polyfill's `readUnion([index])` and get one subject per indexed object, keyed by its NURI, plus the index's own subject declaring its field, which `read` drops. `INDEX_FIELD` and `ENTRY_VALUE` are published for that reader. +**Everything deposited is untrusted.** Anyone may deposit anything; `decodeReference` returns `null` for whatever is not a reference, and such a payload is passed over rather than stopping the rest. -**Every inbox payload is untrusted.** Anyone may deposit anything; `decodeReference` returns `null` for whatever is not a reference, and such a payload is passed over rather than crashing the run. +**A BET, named as one — this layer's, not yours.** Nothing here is an application's to do or handle; it is what this layer stands on. That a document can receive deposits at all is aligned with NextGraph — a repository takes an inbox capability, at most one, on a real commit upstream. **What a deposit carries, and what receiving one does, are ours**: upstream's own set of deposit kinds is closed, carries no payload this could travel in, and offers no hook to extend it, so an index deposit has a shape NextGraph has not defined. When upstream defines it, this layer moves with it, and a `major` is how you hear about that. ## Non-guarantees -**Nothing reports curating to you.** No report, no outcome list, no callback: an application that cannot ask for it has nowhere to receive the result. A reference that could not be resolved is warned about on the log stream; an object carrying nothing for the field, one carrying several values, a self-reference and a payload that is not a reference are not reported at all. Reading the index is how you find out. +**Nothing tells you what became of a reference.** No report, no outcome list, no callback: an application that cannot ask for it has nowhere to receive the result. A reference that could not be resolved is warned about on the log stream; an object carrying nothing for the field, one carrying several, a self-reference and a payload that is no reference are not reported at all. Reading the index is how you find out. -**No timing.** A deposit is in the index once its creator's session has been through that inbox; nothing says how long that takes or lets you wait, and if the creator is not connected it waits for as long as that lasts. There is no queue depth and no ordering between a deposit and a read. +**No timing.** Nothing says how long a reference takes to become an entry or lets you wait, and while its creator stays away it waits for as long as that lasts. There is no depth to inspect and no ordering between a deposit and a read. -**No refresh.** An already-indexed object is never re-read, so one whose value changes later keeps its original indefinitely. +**No refresh.** An already-indexed object is never read again, so one whose value changes later keeps its original indefinitely. -**No private data.** Only objects the curator can open itself are indexed; one the owner cannot read is not added. +**No private data.** Only objects the index's creator can open are indexed; one it cannot read is not added. -**A handle is one identity for its whole life, and nothing detaches the inboxes it watches.** An application that changes identity within one page must build a new handle and drop the old one, which goes on watching under a session that holds nothing. +**A handle is one identity for its whole life, and nothing releases what it holds.** An application that changes identity within one page must build a new handle and drop the old one, which goes on listening under a session that holds nothing. -**Connecting reads this identity's whole public store** — a store read plus one read per document, every time a handle is built, because nothing marks a document as an index. An index whose read did not answer in that moment is not found, silently, and is curated at the next connection instead. +**Connecting reads this identity's whole public store** — a store read plus one read per document, every time a handle is built, because nothing marks one as an index. An index whose read did not answer then is not found, silently, and its entries are made at the next connection. **The narrow behaviours are open questions, not promises.** An object carrying nothing for the field is not added; one carrying several values is not added; a raced entry keeps the smallest value. Each is implemented in its narrowest form rather than generalised, and each may change. -**No stable error text.** What a throw or a log line reads is for a human. Do not parse it or branch on it. +**No stable error text.** What a throw or a log line reads is for a human. Do not parse or branch on it. **No cross-broker reach.** A NURI resolves for users of one broker. -**No depositor authentication or rate limit.** Anyone may deposit any number of payloads into any inbox. +**No depositor authentication and no rate limit.** Anyone may deposit any number of payloads into any index. ## Change policy **Semver, and majors are the normal case.** This layer sits on a polyfill itself converging on a NextGraph that does not ship yet, several of its behaviours are open questions above, and one part of it is a bet. Settling any of those narrows this surface, so the major number moves often — that frequency is the honest signal about this package, not an apology. -- **major** — an exported symbol is removed or renamed, **or** an existing call narrows: it throws where it returned, or reports a state you did not have to handle before. Settling an open question counts, and so does anything the bet forces. A signature change a caller must react to counts; one that only accepts more than before does not. +- **major** — an exported symbol is removed or renamed, **or** an existing call narrows: it throws where it returned, or reports a state you did not have to handle before. Settling an open question counts, and so does anything the bet forces. A signature change you must react to counts; one that only accepts more does not. - **minor** — a symbol is added and nothing existing moves: a new read helper, a new optional option. - **patch** — a fix that changes neither the exported surface nor anything under `## Guarantees`, throw text included. -**A tag says where it comes from.** A release cut on `main` carries a **full version** (`2.0.0`); work on a branch carries a **pre-release** of the version it heads for (`2.1.0-dev.3`), which sorts below it by construction, and between two pre-releases of the same version nothing is promised. Nothing you pinned is ever withdrawn: a pre-release keeps resolving once the full version appears alongside it. The tag is bare — `v2.0.0` — because this repository publishes exactly one engagement; should a second ever ship here, tags take the package name from then on (`indexing/v…`). +**A tag says where it comes from.** A release cut on `main` carries a **full version** (`2.0.1`); work on a branch carries a **pre-release** of the version it heads for (`2.1.0-dev.3`), which sorts below it by construction, and between two pre-releases of one version nothing is promised. Nothing you pinned is ever withdrawn: a pre-release keeps resolving once the full version appears alongside it. The tag is bare — `v2.0.1` — because this repository publishes exactly one engagement; should a second ever ship here, tags take the package name (`indexing/v…`). -**`2.0.0` took the curating call off this surface, and made obtaining a handle asynchronous.** `Indexing.curate(index)` is gone, and with it `CurationReport`, `CurationOutcome` and `SkipReason`, which nothing published produces any more; `indexing(port)` now returns a promise, because obtaining a handle is what goes through this identity's inboxes and a caller has to be able to await it. Both are removals under the rule above, hence the major. The reason is not tidiness: a published `curate(index)` asked every application to decide who owns an index and when curating runs, and neither is an application's decision — the owner is the only one who can, and "when" is "whenever a deposit arrives, or has been waiting". **Migrating**: delete every call to `curate`, and `await` the `indexing(port)` you already make. If you read `CurationReport` for what happened, read the index instead, and the log for what did not resolve. +**`2.0.1` is prose, and one requirement corrected.** This surface was described throughout as an *inbox* you deposit into and a thing that gets *curated*; neither is yours to know — you deposit a reference and you read entries, and how that travels is this layer's business and free to change under you — so the same facts are stated as effects now. And "hardcode the index's NURI in your source" was stated unconditionally, fused with the creator-must-connect requirement; it is neither. A reference to an index is held however you hold any other, and hardcoding is what a single GLOBAL index needs. Creator-must-connect is unchanged and stays unconditional. **Nothing to migrate** — no symbol moved, no call behaves differently, and the requirement that changed asks less than before. A **patch** under the rule above, worth spelling out because the wording moved so much: no exported symbol changed, nothing under `## Guarantees` promises anything it did not promise in `2.0.0` — only how it is said — and the requirement that did change is a deployment one, asking less. -`1.0.1` and `1.0.0` keep resolving and neither is forced to upgrade, `1.0.0` having been uninstallable from anywhere but one working copy. This engagement is cut on `main`, so `2.0.0` is what you pin, and your `usage_` leaf anchors `against:` on that exact string — `against: @ng-helpers/indexing@2.0.0`. +**`2.0.0` removed `Indexing.curate(index)` — and `CurationReport`, `CurationOutcome`, `SkipReason` with it — and made `indexing(port)` a promise.** Removals, hence the major: that call asked an application to decide who owns an index and when its entries are made, neither of which is an application's decision. **Migrating from `1.x`**: delete every call to `curate`, `await` the `indexing(port)` you already make, and where you read a `CurationReport` read the index instead — the log carries what did not resolve. `1.0.1` and `1.0.0` keep resolving and neither is forced to upgrade. + +Cut on `main`, so `2.0.1` is what you pin, and your `usage_` leaf anchors `against:` on that exact string — `against: @ng-helpers/indexing@2.0.1`. There is no changelog file and no deprecation window: **the sections above are the release note.** Diff this leaf between two pulls, `## Guarantees` and `## Non-guarantees` before `## Surface`, because that is where a narrowing shows up first. diff --git a/README.md b/README.md index 186ca62..65b76df 100644 --- a/README.md +++ b/README.md @@ -88,14 +88,18 @@ Deliberately not settled. Each is implemented in its narrowest form and reported `@ng-eventually/polyfill`, declared as a **peer** dependency: an application using this package supplies it, so exactly one copy of it exists in that application. That is a requirement of the polyfill itself, which keeps its state in the package — two copies mean two subscription registries and two current identities, and nothing detects it. -For this repository's own tests and typecheck it is *also* a `devDependency` by local path (`file:../ng-eventually-js/packages/polyfill`), which expects that repository to sit beside this one. A dev dependency is not installed by a consumer, so this local path never reaches one. `ng-e2e-helpers` is a `devDependency` by local path on the same expectation. +For this repository's own tests and typecheck it is *also* a `devDependency` by local path (`link:../ng-eventually-js/packages/polyfill`), which expects that repository to sit beside this one. A dev dependency is not installed by a consumer, so this local path never reaches one. `ng-e2e-helpers` is a `devDependency` by local path on the same expectation. + +**`link:`, not `file:`, and pnpm makes that a real difference.** pnpm COPIES a `file:` directory into its virtual store, and a copy is cut off from the sibling checkout's own `node_modules` — the polyfill's optional peers (`@ng-org/shex-orm`, `@ng-org/alien-deepsignals`) stop resolving and the typecheck fails on them. `link:` symlinks the sibling package where it lives, so it keeps its own dependencies and an edit made there is the one this repository tests against. ## Running it ```sh -npm install # or: pnpm install +pnpm install bunx tsc --noEmit -p tsconfig.json bun test ``` -**`bun install` does not work in this repository** (checked with bun 1.3.9): bun resolves a mandatory peer dependency against the npm registry whatever local path provides it, and `@ng-eventually/polyfill` is published to no registry, so the install stops on `GET https://registry.npmjs.org/@ng-eventually%2fpolyfill - 404`. `npm install` and `pnpm install` both resolve it from the sibling checkout. `bun test` itself is unaffected — it is only the installer that cannot express this. +**pnpm installs; bun runs.** `pnpm-lock.yaml` is the committed lockfile and `pnpm install` is the only install path — the same package manager the sibling `ng-eventually-js` uses. `bun` stays the test runner and `bunx tsc` the typechecker; neither reads a lockfile, so nothing about that changed. + +**`bun install` does not work in this repository** (checked with bun 1.3.9): bun resolves a mandatory peer dependency against the npm registry whatever local path provides it, and `@ng-eventually/polyfill` is published to no registry, so the install stops on `GET https://registry.npmjs.org/@ng-eventually%2fpolyfill - 404`. That is why no `bun.lock` is kept here: bun cannot regenerate one, so the file that was here could only rot. `bun test` itself is unaffected — it is only the installer that cannot express this. diff --git a/package.json b/package.json index 33f48f2..437e005 100644 --- a/package.json +++ b/package.json @@ -1,9 +1,9 @@ { "name": "@ng-helpers/indexing", - "version": "2.0.0", + "version": "2.0.1", "private": true, "type": "module", - "description": "An indexing layer built on top of NextGraph, via @ng-eventually/polyfill. An index is an ordinary public document; contributions reach it through its inbox; processing that inbox is what curates it, and that happens at its creator's connections.", + "description": "An indexing layer built on top of NextGraph, via @ng-eventually/polyfill. An index is an ordinary public document; anyone may hand it a reference to an object, and that reference becomes an entry once the index's creator is connected.", "main": "./src/index.ts", "types": "./src/index.ts", "exports": { @@ -13,10 +13,10 @@ "@ng-eventually/polyfill": "*" }, "devDependencies": { - "@ng-eventually/polyfill": "file:../ng-eventually-js/packages/polyfill", + "@ng-eventually/polyfill": "link:../ng-eventually-js/packages/polyfill", "@ng-org/web": "0.1.2-alpha.13", "@types/bun": "latest", - "ng-e2e-helpers": "file:../ng-eventually-js/packages/ng-e2e-helpers", + "ng-e2e-helpers": "link:../ng-eventually-js/packages/ng-e2e-helpers", "playwright": "1.61.1", "typescript": "^5.6.0" }, diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml new file mode 100644 index 0000000..43405c4 --- /dev/null +++ b/pnpm-lock.yaml @@ -0,0 +1,110 @@ +lockfileVersion: '9.0' + +settings: + autoInstallPeers: true + excludeLinksFromLockfile: false + +importers: + + .: + devDependencies: + '@ng-eventually/polyfill': + specifier: link:../ng-eventually-js/packages/polyfill + version: link:../ng-eventually-js/packages/polyfill + '@ng-org/web': + specifier: 0.1.2-alpha.13 + version: 0.1.2-alpha.13 + '@types/bun': + specifier: latest + version: 1.3.14 + ng-e2e-helpers: + specifier: link:../ng-eventually-js/packages/ng-e2e-helpers + version: link:../ng-eventually-js/packages/ng-e2e-helpers + playwright: + specifier: 1.61.1 + version: 1.61.1 + typescript: + specifier: ^5.6.0 + version: 5.9.3 + +packages: + + '@ng-org/web@0.1.2-alpha.13': + resolution: {integrity: sha512-/xO0c+3NTphnws5Do2LDqgZWmAf+aNnYdChJKdU0dnp1U1iVSgi/y3yb8AYryf0v9sooj0aYJxt08B6DpirFMQ==} + + '@types/bun@1.3.14': + resolution: {integrity: sha512-h1hFqFVcvAvD9j9K7ZW7vd82aSA+rTdznZa+5bwvCwqSB1jmmfLcbIWhOLx1/+boy/xmjgCs/OMUL8hRJSmnPw==} + + '@types/node@26.2.0': + resolution: {integrity: sha512-5IviulTZeRNp2vAJ514cc/HUlY5nZ9fCbq9DMyC52BrhFZACo3nI0R7qBxhQmo/d27NFe96ur/b7Wwxklda+kg==} + + async-proxy@0.4.1: + resolution: {integrity: sha512-4e+zNtoGL4+cnqib8v169CnKcRfAsAubp2EsjBhAA5jyW7jjI3t36rVvuqLwmhtliwf8JvSnxinE4ecQN+DK4w==} + + bun-types@1.3.14: + resolution: {integrity: sha512-4N0ig0fEomHt5R0KCFWjovxow98rIoRwKolrYdCcknNwMekCXRnWEUvgu5soYV8QXtVsrUD8B95MBOZGPvr6KQ==} + + fsevents@2.3.2: + resolution: {integrity: sha512-xiqMQR4xAeHTuB9uWm+fFRcIOgKBMiOBP+eXiyT7jsgVCq1bkVygt00oASowB7EdtpOHaaPgKt812P9ab+DDKA==} + engines: {node: ^8.16.0 || ^10.6.0 || >=11.0.0} + os: [darwin] + + object-path-operator@3.0.0: + resolution: {integrity: sha512-Z7dlPUeXqRU/lLfGerP24dPC66n7ehyXaTM81k71EFlsaaEjOHkf4/uq1WGicfGfiO7snYShneE1YZZUkyRiLQ==} + + playwright-core@1.61.1: + resolution: {integrity: sha512-h7Qlt6m4REp25qvIdvbDtVmD4LqVXfpRxhORv9L0jzETM05p4fuPJ3dKyuSXQxDSbXnmS79HAgi9589lGSpLkg==} + engines: {node: '>=18'} + hasBin: true + + playwright@1.61.1: + resolution: {integrity: sha512-DWnY5o3YbLWK4GovuAVwpqL+1VwGNdUGrRr++8j8PtQQzvAVZUIMjKQ90fY689sEJZJBbZVw1rXaOKSTitkzPQ==} + engines: {node: '>=18'} + hasBin: true + + typescript@5.9.3: + resolution: {integrity: sha512-jl1vZzPDinLr9eUt3J/t7V6FgNEw9QjvBPdysz9KfQDD41fQrC2Y4vKQdiaUpFT4bXlb1RHhLpp8wtm6M5TgSw==} + engines: {node: '>=14.17'} + hasBin: true + + undici-types@8.3.0: + resolution: {integrity: sha512-j375ScV60dom+YkPFIfTLcOiPxkN/buHz5GobjLhixFuANaNs3C9l4GmrWqejgXWJ7BbJcFYpTEUkS1Ge8bpZQ==} + +snapshots: + + '@ng-org/web@0.1.2-alpha.13': + dependencies: + async-proxy: 0.4.1 + + '@types/bun@1.3.14': + dependencies: + bun-types: 1.3.14 + + '@types/node@26.2.0': + dependencies: + undici-types: 8.3.0 + + async-proxy@0.4.1: + dependencies: + object-path-operator: 3.0.0 + + bun-types@1.3.14: + dependencies: + '@types/node': 26.2.0 + + fsevents@2.3.2: + optional: true + + object-path-operator@3.0.0: {} + + playwright-core@1.61.1: {} + + playwright@1.61.1: + dependencies: + playwright-core: 1.61.1 + optionalDependencies: + fsevents: 2.3.2 + + typescript@5.9.3: {} + + undici-types@8.3.0: {} diff --git a/test/fake-nextgraph.ts b/test/fake-nextgraph.ts index d161ae0..5d8ffa1 100644 --- a/test/fake-nextgraph.ts +++ b/test/fake-nextgraph.ts @@ -62,6 +62,10 @@ export class FakeNextGraph { readonly #documents = new Map(); /** Documents the broker currently cannot answer about. See `breakReadsOf`. */ readonly #unreachable = new Map(); + /** Inboxes the broker currently cannot READ. See `breakInboxReadsOf`. */ + readonly #inboxUnreadable = new Map(); + /** Inboxes the broker currently refuses to WATCH. See `breakWatchingOf`. */ + readonly #inboxUnwatchable = new Map(); /** Every live watch, across every identity — a session watching its own inbox. */ #watches: Watch[] = []; /** Notifications the broker has not handed over yet. See `deliverNotifications`. */ @@ -132,6 +136,43 @@ export class FakeNextGraph { this.#unreachable.delete(asNuri(doc)); } + /** + * The broker serves the DOCUMENT but not its INBOX. + * + * Not a contrivance: upstream an inbox is a repo of its own, reached through an + * address `openDocumentInbox` resolves and read with that repo's capability, + * while the document itself is read by `readUnion`. Two repos, two reads — so + * one answering while the other does not is what a partial failure looks like, + * and it is the state that makes a catch-up fail on one index and no other. + */ + breakInboxReadsOf(doc: NuriLike, reason: string): void { + this.#inboxUnreadable.set(asNuri(doc), reason); + } + + /** The inbox can be read again. */ + healInboxReadsOf(doc: NuriLike): void { + this.#inboxUnreadable.delete(asNuri(doc)); + } + + /** + * The broker refuses to keep this session posted about that inbox, while + * everything else about it still works. + * + * Watching is a live subscription, set up and held open for as long as the + * session lasts; reading an inbox is one question and one answer. A subscription + * can be refused where a read succeeds, which is the state that leaves an index + * caught up but unwatched — deposits into it going unnoticed until the next + * connection, exactly as the failure this models says. + */ + breakWatchingOf(doc: NuriLike, reason: string): void { + this.#inboxUnwatchable.set(asNuri(doc), reason); + } + + /** The inbox can be watched again. */ + healWatchingOf(doc: NuriLike): void { + this.#inboxUnwatchable.delete(asNuri(doc)); + } + /** * Hands over every inbox notification the broker was holding, and waits for the * sessions watching to finish with them — including notifications those very runs @@ -297,6 +338,12 @@ export class FakeNextGraph { "is reading it, and you may only READ your own", ); } + const unwatchable = this.#inboxUnwatchable.get(doc); + // Refused AFTER the owner check: resolving the address is an owner-only act, so + // a stranger is turned away before any subscription is ever attempted. + if (unwatchable !== undefined) { + throw new Error(`cannot watch the inbox of ${doc}: ${unwatchable}`); + } // Watching resolves the inbox address, and the call that resolves one opens it // when there is none — the same idempotent call `openInbox` makes. stored.deposits ??= []; @@ -324,6 +371,10 @@ export class FakeNextGraph { "inbox, you may only READ your own", ); } + const unreadable = this.#inboxUnreadable.get(doc); + if (unreadable !== undefined) { + throw new Error(`cannot read the inbox of ${doc}: ${unreadable}`); + } return [...stored.deposits].sort((a, b) => a.ts - b.ts); } } diff --git a/test/inbox-processing.test.ts b/test/inbox-processing.test.ts index b9afc53..00249f4 100644 --- a/test/inbox-processing.test.ts +++ b/test/inbox-processing.test.ts @@ -15,9 +15,15 @@ import { FakeNextGraph, publishObject } from "./fake-nextgraph"; * * The case space is the creator's presence crossed with the deposit's timing: * away when it was made, connected when it was made, and connected on an index - * that a previous session created. Plus the three that must NOT happen: a stranger - * connecting curates nothing, a document that is no index is left alone, and a - * session that could not look for its indexes is still a working handle. + * that a previous session created. Plus the two that must NOT happen: a stranger + * connecting curates nothing, and a document that is no index is left alone. + * + * And crossing all of it, the three ways connecting can FAIL — it cannot look for + * its indexes, it cannot go through one, it cannot watch one. Each has its own test + * below, because each is a failure wearing the shape of an absence: the session + * carries on, the handle works, and an index quietly holds less than it should. The + * engagement is that none of them denies anything and none of them loses a deposit, + * which is only worth anything if it is exercised rather than asserted. */ const PUBLISHED_AT = "http://schema.org/datePublished"; @@ -38,6 +44,29 @@ async function aliceCreatesAnIndexAndLeaves(network: FakeNextGraph): Promise( + body: () => Promise, +): Promise<{ result: T; reports: number }> { + const reported = mock((..._args: unknown[]) => {}); + const original = console.error; + console.error = reported; + try { + return { result: await body(), reports: reported.mock.calls.length }; + } finally { + console.error = original; + } +} + // --- the deposits that piled up while the creator was away ---------------- test("an index is curated at its creator's next connection, with nobody asking", async () => { @@ -198,6 +227,87 @@ test("a session that could not look for its indexes is still a working handle", expect(String(reported.mock.calls[0]?.[0])).toContain("public store could not be listed"); }); +test("an index whose catch-up failed is still a working handle, and loses no deposit", async () => { + const network = new FakeNextGraph(); + const stalled = await aliceCreatesAnIndexAndLeaves(network); + const healthy = await aliceCreatesAnIndexAndLeaves(network); + + const bobPort = network.portFor("bob"); + const bob = await indexing(bobPort); + const article = await publishObject(bobPort, PUBLISHED_AT, "2026-03-04"); + await bob.refer(stalled, article); + await bob.refer(healthy, article); + + // The broker answers about the document and not about its inbox. Two repos + // upstream, read with two capabilities, so this is a partial failure and not a + // contrived one — and it is what makes the catch-up fail on THIS index alone. + network.breakInboxReadsOf(stalled, "broker unreachable"); + + const { result: alice, reports } = await capturingReports(() => + indexing(network.portFor("alice")), + ); + + // Obtaining the handle RESOLVED — reaching this line at all is the assertion. + // Reading the index it could not go through still works… + expect(await alice.read(stalled)).toEqual([]); + // …and so does depositing into it: neither ever depended on that work. + await alice.refer(stalled, article); + // The session is not poisoned either: the other index was caught up normally. + expect(await alice.read(healthy)).toEqual([{ object: article, value: "2026-03-04" }]); + expect(reports).toBe(1); + + // And nothing was lost. The deposits never left the inbox, so the first + // connection that can read it puts them in — which is the whole reason a failed + // run is allowed to be this quiet. + network.healInboxReadsOf(stalled); + network.disconnect("alice"); + const back = await indexing(network.portFor("alice")); + expect(await back.read(stalled)).toEqual([{ object: article, value: "2026-03-04" }]); +}); + +test("an index that could not be watched is still caught up, and the rest still notices", async () => { + const network = new FakeNextGraph(); + const unwatched = await aliceCreatesAnIndexAndLeaves(network); + const watched = await aliceCreatesAnIndexAndLeaves(network); + + const bobPort = network.portFor("bob"); + const bob = await indexing(bobPort); + const waiting = await publishObject(bobPort, PUBLISHED_AT, "2026-01-01"); + await bob.refer(unwatched, waiting); + + // The subscription is refused; reading that same inbox still works. A watch is + // held open where a read is one question and one answer, so one can be turned + // down while the other is served. + network.breakWatchingOf(unwatched, "the broker refused the subscription"); + + const { result: alice, reports } = await capturingReports(() => + indexing(network.portFor("alice")), + ); + + // The watch failed and the catch-up ran ANYWAY — the backlog is in. That is the + // order the code goes to some trouble to hold: failing to watch must not cost + // the deposits that were already waiting. + expect(await alice.read(unwatched)).toEqual([{ object: waiting, value: "2026-01-01" }]); + expect(reports).toBe(1); + + // What the failure costs, exactly and no more: a deposit made from now on is not + // NOTICED on that index… + const late = await publishObject(bobPort, PUBLISHED_AT, "2026-02-02"); + await bob.refer(unwatched, late); + await bob.refer(watched, late); + await network.deliverNotifications(); + expect((await alice.read(unwatched)).map((e) => e.value)).toEqual(["2026-01-01"]); + // …while every other index of the very same session goes on noticing its own. + expect(await alice.read(watched)).toEqual([{ object: late, value: "2026-02-02" }]); + + // "Until the next connection" is the whole of the damage, and the next + // connection is where it ends. + network.healWatchingOf(unwatched); + network.disconnect("alice"); + const back = await indexing(network.portFor("alice")); + expect((await back.read(unwatched)).map((e) => e.value)).toEqual(["2026-01-01", "2026-02-02"]); +}); + // --- the primitive that keeps a burst from piling up ---------------------- test("coalescing never runs twice at once, and grants exactly one more run", async () => { diff --git a/test/only-grows.test.ts b/test/only-grows.test.ts index 4858fc2..69b961b 100644 --- a/test/only-grows.test.ts +++ b/test/only-grows.test.ts @@ -154,6 +154,29 @@ test("an index declaring no field at all is still refused", async () => { await expect((await indexing(ownerPort)).read(ordinary)).rejects.toThrow(/declares no index field/); }); +test("curating a document that declares no field refuses, and writes nothing", async () => { + const network = new FakeNextGraph(); + const ownerPort = network.portFor("alice"); + const bobPort = network.portFor("bob"); + + // A document of Alice's with an inbox open and a reference waiting in it, and no + // field declared. This is also the shape an INDEX arrives in when it could not be + // read — the real `readUnion` turns a failed read into `[]` — so the two are one + // case here, and the refusal has to hold for both. + const noField = await ownerPort.createPublicDocument(); + await ownerPort.openInbox(noField); + const article = await publishObject(bobPort, FIELD, "2026-01-01"); + await (await indexing(bobPort)).refer(noField, article); + + await expect(curate(ownerPort, noField)).rejects.toThrow(/declares no index field/); + + // It refused instead of curating on a field it does not have, and it refused + // BEFORE writing: the document is still empty, so no entry was invented for it, + // and the deposit is still in the inbox for a run that knows what to do with it. + expect(await ownerPort.readDocument(noField)).toEqual([]); + expect(await ownerPort.readDeposits(noField)).toHaveLength(1); +}); + test("a field that could never match an object is refused at creation", async () => { const owner = await indexing(new FakeNextGraph().portFor("alice")); // It cannot be corrected later — nothing here deletes — so it is refused now.