ng-helpers
Helpers built on top of NextGraph. The first — and for now the only — one is @ng-helpers/indexing: an index that an application can point at and read.
Why this is not in the polyfill
NextGraph has no indexing concept, at any level, and will not grow one. This is not scaffolding standing in for something that will arrive upstream; it is a construction above NextGraph, and it stays in its own repository so that the two are never confused.
So the dependency runs one way and only one way: this repo depends on @ng-eventually/polyfill, and the polyfill must never learn anything about indexing. No index vocabulary, no index type, no inbox variant that knows what a deposit means. The polyfill's inbox stays generic and carries opaque deposits; what a deposit means is decided here.
That boundary is held by one file. src/polyfill-adapter.ts is the only place that imports the polyfill at runtime; everything else is written against NextGraphPort (src/port.ts), a small interface describing what this layer needs. Only entries listed in that package's published contract_polyfill-surface are used — nothing is reached for inside it.
What an index is
An ordinary document. Nothing marks it as an index. What makes it one is that an application references its NURI in its own source.
- Any user may create one, and its creator owns it.
- It lives in the creator's public store, so any reader can open it from the reference alone.
- It declares, on its own subject, the field it indexes by — a predicate, since the objects are RDF. An index "by a date" is simply an index whose field is a date predicate: there is no separate kind of index. Entries come out in chronological order because ISO-8601 sorts as a string.
- Each entry is a subject keyed by the indexed object's NURI, carrying that object's value for the field.
Contributing is a deposit, not a write. Creating an object that belongs in an index means depositing into the index document's inbox — inbox.postToDocument, which is exactly "reach this document's owner" and is open to anyone. Nobody but the owner ever writes the index.
A deposit is a bare reference. Nothing else. It states no claim and gives no instruction: no operation, no index reference (the inbox address already identifies the index), no copy of the indexed value. When the owner curates, it resolves the reference and opens the object itself — which it can, because indexing is limited to public data for now. What the object says is what goes in. This is the shape NextGraph already uses upstream, where a SocialQueryRequest carries a reference to an RDF definition and the recipient composes its own update; a payload carrying an operation would be a licence for anyone to rewrite someone else's document.
Curating is not an act an application performs. There is no curate to call, and there never will be: it would ask an application to decide who owns an index and when curation runs, and neither is its decision. Curation is what happens when the index's inbox is processed — at its creator's next connection, and on each deposit while the creator is connected. await indexing(port) IS that connection: it goes through the inbox of every index that identity owns, and leaves each one watched. src/observation.ts is where that lives.
Reading needs nothing new. An application that knows the NURI calls readUnion([indexNuri]) and gets the entries as subjects. Indexing.read is sugar over exactly that, dropping the index's own declaration subject.
An index only ever grows
Nothing is ever removed from an index — by anyone, including its owner. An index only ever grows. There is no removal function, and there never was one: removal was deliberately never built, not built and then withdrawn, and nothing is planned. This is written down here rather than left to be deduced from a missing function, because someone who needs an entry gone should learn that it was never possible instead of hunting for the call that does it. Today the only answer to that need is a fresh index.
It is what makes the failure story trivial. Since the only write is an addition, a reference that does not resolve — the object is gone, or unreadable, or the broker simply did not answer — can only ever mean not added this time. It cannot damage what is already there, and a later deposit adds it. Nothing has to tell an absence from a failure, so nothing can get that wrong.
That distinction is worth stating, because getting it wrong is a defect this family of code has already produced: in ng-eventually-js, resolveAccount returned null on a failed lookup, so a read that failed came back as "this account does not exist" — a failure disguised as an absence, which silently broke document sharing (fixed in 8c8ade7 and e32b6d0).
NextGraphPort has no operation that removes, and nothing in src/ can express a deletion — but the write path was never where this went wrong. Three rounds of adversarial review broke the guarantee five times, and not one of the breaks involved deleting anything. Every one was on the read path:
- reading an entry required exactly one value, so a subject carrying two read as absent — and a second
addLiteralProperty, the only write this package has, made an entry vanish while both triples sat in the document. Two curation runs racing each other produce exactly that; - the same rule on the declared field was worse: one add-only write of a second
INDEX_FIELDmade the descriptor unreadable and took every entry in the document with it, permanently; - one subject that is not a NURI threw out of
entriesOfand made every real entry unreadable at once; - a field named
constructorortoStringread back an inherited function from the plain objectreadUnionbuilds, crashing curation for every remaining deposit; - and the fix for the second of these silently corrupted the index instead: "smallest wins" switched the field while entries already written kept their old one, so
read()returned a single list "ordered by value" whose values came from two different properties.
What holds now: entries take at least one value, smallest wins, deterministically. Reading is per-entry tolerant — a stray subject is skipped, never thrown on — and reads own properties only. Reading an index asks only is this an index?; the field is required only to curate, and an ambiguous one refuses loudly rather than picking, because a quiet wrong answer is worse than a loud stop. That refusal is permanent, which is the honest price of having no delete, and the message says so instead of suggesting a retry.
The lesson is worth keeping: "nothing removes" is a claim about the write path, and an invariant about what a reader can see has to be checked on the read path too.
What enforces the write half is no longer a pattern over source. test/adapter.test.ts runs the real adapter on test/fake-polyfill.ts, an in-memory polyfill whose SPARQL is executed by an engine that understands one statement — an anchored INSERT DATA of literal triples — and refuses everything else. A removal is therefore not detected, it is unrunnable: DELETE WHERE … fails on the first keyword, a second statement smuggled after the closing brace fails on the trailing text, a keyword hidden inside a literal stays inside the literal because a parser tokenises where a regex only matches, and splitting the keyword across concatenated strings buys nothing, since it is one string by the time it arrives. The regex over src/ in test/units.test.ts stays as a cheap tripwire that names the file early; it is not the proof.
A failed resolve is still a failure, and still surfaces. Harmless is not the same as invisible. Nobody asked for the run, so there is nobody to hand a report to: every reference that could not be resolved is warned about on this package's log stream, and so is a run, a watch or a store read that could not happen at all. None of them denies the application anything — reading an index and depositing into one never depended on that work, and the deposits stay in their inbox for the next connection.
Open questions
Deliberately not settled. Each is implemented in its narrowest form and reported rather than generalised.
-
An object that carries nothing for the index's field. Narrow behaviour: it is not added, and nothing says so. There is no key to index it by, and inventing one — a placeholder, the deposit's timestamp — would put something in the index that the object does not say. Whether it should instead be indexed under an absent key, or refused louder, is open.
-
An object that carries several values for the field. Not added either. Which of them the entry would hold has not been decided.
-
An already-indexed object is never re-read. Curation skips it outright, so an object whose field value changes later keeps its original value in the index. Refreshing it would be a write nobody asked for, and it is the same question as removal.
-
Which value a raced entry should keep. Two curation runs racing each other can leave an entry with two values; the smallest is chosen so that readers agree and the entry stays visible. That the entry must survive is settled; which of the two it should hold is not.
Note what this means, since the two points above are easy to read as one: an entry's value can change after it is made. Not by re-reading the object — that never happens — but because a second value arriving for the same entry can be the smaller one, and
read()returns the smallest. Index an object at2026-05-05, let a raced run add2026-01-01, andread()answers2026-01-01. What never changes is the set of entries and the fact that each stays visible; the value one of them reports is settled by "smallest wins", not by arrival order. -
How an index recovers from an ambiguous declaration. Today it does not: curation refuses for good and the only way forward is a fresh index. Since nothing here removes anything, giving it a way back needs a mechanism that does not exist yet.
-
Deposits are never retired. Every curation run sees every deposit ever made. That is affordable because re-applying one is a no-op, but it is linear in the history. How a curator retires an applied deposit is open —
inbox.processInboxmay be the answer, but its semantics are not published. -
How an owner finds its own indexes. Nothing marks a document as an index, so connecting reads that identity's whole public store and looks at each document. It is the only question the surface can ask, and it does not scale with a large store. A marker on the document, or a listing narrowed by shape, would both settle it — and both are decisions about what an index is.
-
Nothing stops a handle.
inbox.watch's unsubscribe is dropped, so a session watches its own indexes until the page goes. An application that changes identity in one page has to drop the old handle and know that it goes on watching. -
What an entry holds besides the object reference and the field value, and how several index kinds would coexist, are both untouched.
Layout
| Path | What it is |
|---|---|
src/port.ts |
What this layer needs from NextGraph, and nothing more |
src/polyfill-adapter.ts |
The only runtime import of @ng-eventually/polyfill |
src/deposit.ts |
The deposit's shape: a bare reference |
src/index-document.ts |
An index's declaration and its entries |
src/curator.ts |
Resolving references and adding what is there |
src/observation.ts |
When that happens: this identity's indexes, caught up and watched |
src/coalescing.ts |
Runs a job, never twice at once, and grants exactly one more run |
src/indexing.ts |
The public surface, bound to one identity |
src/sparql.ts |
The one statement this package writes — no deletion exists |
test/fake-nextgraph.ts |
An in-memory NextGraph behind NextGraphPort, enforcing the polyfill's published guarantees |
test/fake-polyfill.ts |
An in-memory @ng-eventually/polyfill that executes the SPARQL — an engine that can only add |
test/adapter.test.ts |
Runs the real adapter on it: behaviour, every query emitted, and every method driven |
Depends on
@ng-eventually/polyfill, declared as a peer dependency: an application using this package supplies it, so exactly one copy of it exists in that application. That is a requirement of the polyfill itself, which keeps its state in the package — two copies mean two subscription registries and two current identities, and nothing detects it.
For this repository's own tests and typecheck it is also a devDependency by local path (link:../ng-eventually-js/packages/polyfill), which expects that repository to sit beside this one. A dev dependency is not installed by a consumer, so this local path never reaches one. ng-e2e-helpers is a devDependency by local path on the same expectation.
link:, not file:, and pnpm makes that a real difference. pnpm COPIES a file: directory into its virtual store, and a copy is cut off from the sibling checkout's own node_modules — the polyfill's optional peers (@ng-org/shex-orm, @ng-org/alien-deepsignals) stop resolving and the typecheck fails on them. link: symlinks the sibling package where it lives, so it keeps its own dependencies and an edit made there is the one this repository tests against.
Running it
pnpm install
bunx tsc --noEmit -p tsconfig.json
bun test
pnpm installs; bun runs. pnpm-lock.yaml is the committed lockfile and pnpm install is the only install path — the same package manager the sibling ng-eventually-js uses. bun stays the test runner and bunx tsc the typechecker; neither reads a lockfile, so nothing about that changed.
bun install does not work in this repository (checked with bun 1.3.9): bun resolves a mandatory peer dependency against the npm registry whatever local path provides it, and @ng-eventually/polyfill is published to no registry, so the install stops on GET https://registry.npmjs.org/@ng-eventually%2fpolyfill - 404. That is why no bun.lock is kept here: bun cannot regenerate one, so the file that was here could only rot. bun test itself is unaffected — it is only the installer that cannot express this.