ng-helpers/README.md

ng-helpers

Helpers built on top of NextGraph. The first — and for now the only — one is @ng-helpers/indexing: an index that an application can point at and read.

Why this is not in the polyfill

NextGraph has no indexing concept, at any level, and will not grow one. This is not scaffolding standing in for something that will arrive upstream; it is a construction above NextGraph, and it stays in its own repository so that the two are never confused.

So the dependency runs one way and only one way: this repo depends on @ng-eventually/polyfill, and the polyfill must never learn anything about indexing. No index vocabulary, no index type, no inbox variant that knows what a deposit means. The polyfill's inbox stays generic and carries opaque deposits; what a deposit means is decided here.

That boundary is held by one file. src/polyfill-adapter.ts is the only place that imports the polyfill at runtime; everything else is written against NextGraphPort (src/port.ts), a small interface describing what this layer needs. Only entries listed in that package's published contract_polyfill-surface are used — nothing is reached for inside it.

What an index is

An ordinary document. Nothing marks it as an index. What makes it one is that an application references its NURI in its own source.

• Any user may create one, and its creator owns it.
• It lives in the creator's public store, so any reader can open it from the reference alone.
• It declares, on its own subject, the field it indexes by — a predicate, since the objects are RDF. An index "by a date" is simply an index whose field is a date predicate: there is no separate kind of index. Entries come out in chronological order because ISO-8601 sorts as a string.
• Each entry is a subject keyed by the indexed object's NURI, carrying that object's value for the field.

Contributing is a deposit, not a write. Creating an object that belongs in an index means depositing into the index document's inbox — inbox.postToDocument, which is exactly "reach this document's owner" and is open to anyone. Nobody but the owner ever writes the index.

A deposit is a bare reference. Nothing else. It states no claim and gives no instruction: no operation, no index reference (the inbox address already identifies the index), no copy of the indexed value. When the owner curates, it resolves the reference and opens the object itself — which it can, because indexing is limited to public data for now. What the object says is what goes in. This is the shape NextGraph already uses upstream, where a SocialQueryRequest carries a reference to an RDF definition and the recipient composes its own update; a payload carrying an operation would be a licence for anyone to rewrite someone else's document.

Reading needs nothing new. An application that knows the NURI calls readUnion([indexNuri]) and gets the entries as subjects. Indexing.read is sugar over exactly that, dropping the index's own declaration subject.

An index only ever grows

Nothing is ever removed from an index — by anyone, including its owner. This is a deliberate limitation, written down here rather than left to be discovered.

It is what makes the failure story trivial. Since the only write is an addition, a reference that does not resolve — the object is gone, or unreadable, or the broker simply did not answer — can only ever mean not added this time. It cannot damage what is already there, and a later deposit adds it. Nothing has to tell an absence from a failure, so nothing can get that wrong.

That distinction is worth stating, because getting it wrong is a defect this family of code has already produced: in ng-eventually-js, resolveAccount returned null on a failed lookup, so a read that failed came back as "this account does not exist" — a failure disguised as an absence, which silently broke document sharing (fixed in 8c8ade7 and e32b6d0).

NextGraphPort has no operation that removes, and nothing in src/ can express a deletion — but the write path was never where this went wrong. Three rounds of adversarial review broke the guarantee five times, and not one of the breaks involved deleting anything. Every one was on the read path:

• reading an entry required exactly one value, so a subject carrying two read as absent — and a second addLiteralProperty, the only write this package has, made an entry vanish while both triples sat in the document. Two curation runs racing each other produce exactly that;
• the same rule on the declared field was worse: one add-only write of a second INDEX_FIELD made the descriptor unreadable and took every entry in the document with it, permanently;
• one subject that is not a NURI threw out of entriesOf and made every real entry unreadable at once;
• a field named constructor or toString read back an inherited function from the plain object readUnion builds, crashing curation for every remaining deposit;
• and the fix for the second of these silently corrupted the index instead: "smallest wins" switched the field while entries already written kept their old one, so read() returned a single list "ordered by value" whose values came from two different properties.

What holds now: entries take at least one value, smallest wins, deterministically. Reading is per-entry tolerant — a stray subject is skipped, never thrown on — and reads own properties only. Reading an index asks only is this an index?; the field is required only to curate, and an ambiguous one refuses loudly rather than picking, because a quiet wrong answer is worse than a loud stop. That refusal is permanent, which is the honest price of having no delete, and the message says so instead of suggesting a retry.

The lesson is worth keeping: "nothing removes" is a claim about the write path, and an invariant about what a reader can see has to be checked on the read path too.

A failed resolve is still a failure, and still surfaces. Harmless is not the same as invisible. Every reference that could not be resolved comes back as an unresolved outcome in the curation report and is warned about — a failure that looks exactly like a normal outcome teaches nobody anything.

Open questions

Deliberately not settled. Each is implemented in its narrowest form and reported rather than generalised.

• An object that carries nothing for the index's field. Narrow behaviour: it is not added, and reported as skipped: "no-field". There is no key to index it by, and inventing one — a placeholder, the deposit's timestamp — would put something in the index that the object does not say. Whether it should instead be indexed under an absent key, or refused louder, is open.
• An object that carries several values for the field. Not added, reported as skipped: "several-values". Which of them the entry would hold has not been decided.
• An entry never changes after it is made. An already-indexed object is not even re-read, so an object whose field value changes later keeps its original value in the index. Refreshing it would be a write nobody asked for, and it is the same question as removal.
• Which value a raced entry should keep. Two curation runs racing each other can leave an entry with two values; the smallest is chosen so that readers agree and the entry stays visible. That the entry must survive is settled; which of the two it should hold is not.
• How an index recovers from an ambiguous declaration. Today it does not: curation refuses for good and the only way forward is a fresh index. Since nothing here removes anything, giving it a way back needs a mechanism that does not exist yet.
• Deposits are never retired. Every curation run sees every deposit ever made. That is affordable because re-applying one is a no-op, but it is linear in the history. How a curator retires an applied deposit is open — inbox.processInbox may be the answer, but its semantics are not published.
• What an entry holds besides the object reference and the field value, and how several index kinds would coexist, are both untouched.

Layout

Path	What it is
src/port.ts	What this layer needs from NextGraph, and nothing more
src/polyfill-adapter.ts	The only runtime import of @ng-eventually/polyfill
src/deposit.ts	The deposit's shape: a bare reference
src/index-document.ts	An index's declaration and its entries
src/curator.ts	Resolving references and adding what is there
src/indexing.ts	The public surface, bound to one identity
src/sparql.ts	The one statement this package writes — no deletion exists
test/fake-nextgraph.ts	An in-memory NextGraph enforcing the polyfill's published guarantees
test/adapter-write-path.test.ts	Runs the real adapter against a recorder and reads back every query it emits

Depends on

@ng-eventually/polyfill, by local path (file:../ng-eventually-js/packages/polyfill), which expects that repository to sit beside this one.

Running it

bun install
bunx tsc --noEmit -p tsconfig.json
bun test