chore: T01.f est close - readUnion retire et la lecture par document documentee
This commit is contained in:
Sylvain Duchesne
@@ -0,0 +1,141 @@
|
||||
---
|
||||
id: T01.f
|
||||
title: Remove readUnion, and the documents that cite it
|
||||
parent: T01
|
||||
depends_on: [T01.e]
|
||||
created: 2026-09-16
|
||||
claimed_by: T01.f-worker
|
||||
claimed_at: 2026-09-21
|
||||
blocked: false
|
||||
---
|
||||
|
||||
## Description
|
||||
|
||||
Remove `readUnion` — and `UnionSubject` with it, if no published signature still uses it — from the published surface entirely, once nothing internal calls it. The remaining internal single-document reads become **anchored SPARQL queries**.
|
||||
|
||||
**Why it goes.** It has no counterpart upstream: it is N calls plus a batch policy. Its head annotation reads `kind=aligned level=2 ref=sdk/js/lib-wasm/src/lib.rs:sparql_query` — a citation that is exact, attached to a claim that is false, which is the worst failure mode of the annotation mechanism because a reader checks it, finds it correct, and moves on (`.project/rules/nextgraph/surface-mirrors-the-sdk.md`). A **composition cannot be aligned on a single function**.
|
||||
|
||||
**Call sites at planning time — verify, they may have moved.** In `src`: `packages/polyfill/src/emulated-verifier/index-deposit.ts:253` and `:304`, both single-document reads; `packages/polyfill/src/surface/watch-shape.ts:351`, which T01.d should have removed — confirm it is gone rather than assuming. Outside `src`: `examples/notebook/app.ts:108` and `:130`; `packages/polyfill/e2e/polyfill-entry.ts:402, 425, 427, 675`. Tests that exercise it: `test/read-model.test.ts`, `test/open-repo.test.ts`, `test/cross-user-access.test.ts`, `test/reload-own-document.test.ts`, `test/watch-shape.test.ts`, `test/inbox-dispatch.test.ts`, `test/listing-surfaces-failure.test.ts`, `test/deferred-inbox-processing.test.ts`. The anti-cheat applies in full: a test that proved behaviour **through** `readUnion` is re-pointed at whatever replaces it, never deleted and never weakened. `test/open-repo.test.ts` in particular uses it as the only integration path onto `ensureRepoOpen` — that coverage has to survive by another route.
|
||||
|
||||
**Anchored, not anchorless.** The mechanics are level-1 VERIFIED and recorded in `docs/api-contract.md` § 6: an anchor restricts the query to that repo's graph as default graph (`resolve_target_for_sparql`, `engine/verifier/src/request_processor.rs`), while an **anchorless** query unions every named graph in the session store. Replacing several anchored reads with one anchorless query would be a silent widening of what a read can reach — do not do it.
|
||||
|
||||
**Two documents cite it, and both must be revised.**
|
||||
|
||||
1. `.project/concepts/app-contract/polyfill-surface/contract_polyfill-surface.md` — `## Surface` (the signature and the `UnionSubject` interface) and `## Non-guarantees`, where the entry for the `useShape` broker bug currently tells applications *"Use `watchShape` for a reactive read and `readUnion` for a one-shot one — neither goes through that path"*. With `watchShape` renamed (T01.c), possibly re-pathed (T01.d), and `readUnion` gone, that published workaround **for an open MAJOR bug** no longer exists. Say what replaces it, or say plainly that nothing does. Do not leave the sentence standing.
|
||||
2. The parent task states that `ng-helpers`' contract cites `readUnion` and must be revised. At planning time a grep of `.project/concepts/e2e-harness/ng-e2e-helpers/` found **no** such citation — neither `contract_ng-e2e-helpers.md` nor `usage_polyfill-e2e.md` names it — while the e2e **code** under `packages/polyfill/e2e/` uses it heavily. Find where the citation actually lives before revising anything; if it turns out to be only the code, record that in `## Result
|
||||
|
||||
Status: done. `1.0.0-dev.8`, major. Two commits: `0c21ade` (the removal) and `0a2b8b1` (the documents).
|
||||
|
||||
### The opening barrier in `useShapeQuery` was NOT added
|
||||
|
||||
The orchestrator first framed the blocker as "no shape read serves a public index", then withdrew it: an index is read BY ANCHOR, not by shape, and `docs.sparqlQuery` already fetches the public store's key at its door (`packages/polyfill/src/surface/docs.ts:155`). The barrier had been written and verified against a new test; both were reverted, uncommitted, on instruction. `use-shape-query.ts` carries one comment edit in this task and nothing else.
|
||||
|
||||
### What unblocks the removal, and how it was verified
|
||||
|
||||
VERIFIED: `packages/polyfill/test/cross-user-access.test.ts` — *"a bare reference is enough for a PUBLIC document, and not for a protected one"*. It now asserts, under Bob, that he holds no capability for either document, reads the public one through the anchored `docs.sparqlQuery`, and holds its key afterwards — obtained from the document, not handed across by the test. That is the property `readUnion` was protecting, and it belongs to the primitive underneath it.
|
||||
|
||||
### What was removed, and what each call site became
|
||||
|
||||
`readUnion` and `UnionSubject` are gone from the entry. `packages/polyfill/src/surface/read-model.ts` is deleted; `packages/polyfill/src/surface/read-document.ts` replaces it with `readDocumentSubjects(doc)` — ONE anchored query, INTERNAL, `kind=invention` (the anchored query itself is `docs.sparqlQuery`, annotated there; what is ours is the cold-start open and the row grouping).
|
||||
|
||||
- `index-deposit.ts` (two sites) — the `onUnreadable` callback became a `catch`; both still return `unreadable`, and `noRowsMeansAbsent` still consults `getSyncState`, which is why the `ensureRepoOpen` in front of the read cannot move to the caller.
|
||||
- `examples/notebook/app.ts`, `packages/polyfill/e2e/polyfill-entry.ts` — each composes its own read over the published `docs.sparqlQuery`. The two multi-document e2e sites carry a VISIBLE `catch` and report what they skipped.
|
||||
- Tests re-pointed, none deleted or weakened. `open-repo.test.ts` keeps its integration path onto `ensureRepoOpen` by the same route. `read-model.test.ts` → `read-document.test.ts`: three cases that tested the batch POLICY were re-pointed at what replaced it (a failed read throws; an empty document answers `[]`; two reads never merge), and the file header says which and why.
|
||||
|
||||
### The per-document tolerance
|
||||
|
||||
RELOCATED to the caller, not lost — and shedding it is an alignment gain, since upstream an anchored query on a repo the verifier does not hold rejects (`resolve_target_for_sparql` → `NgError::RepoNotFound`). No tolerant batch helper was introduced anywhere.
|
||||
|
||||
### Two behaviour changes, both recorded in the contract
|
||||
|
||||
1. A document the holder has no capability for is REFUSED out loud instead of dropped from a batch.
|
||||
2. Subjects under the reserved `urn:ng-eventually:` prefix are no longer filtered for an application. No rights move: the only registers an ordinary document carries are its inbox address and, for a public document, the key its store already serves to whoever asks; the cap-bearing `inboxCap` register lives in its owner's private store.
|
||||
|
||||
### Does our emulation reject, or answer empty?
|
||||
|
||||
VERIFIED: it REJECTS, and it does not swallow. `docs.sparqlQuery` has no `catch` — it forwards `ng.sparql_query`'s rejection unchanged — and `readDocumentSubjects` has none either. The swallow that existed was `readDoc` inside the deleted module. One nuance, pre-existing: for a document whose public-store cap fetch failed transiently, `fetchReadCap` returns `false` and the rejection that reaches the caller is this package's "you hold no cap", not the broker's outage. That conflation is documented in `public-store.ts` and is not introduced here.
|
||||
|
||||
### Where `ng-helpers` cites `readUnion`
|
||||
|
||||
In its OWN published `usage_ng-helpers.md` against `polyfill-surface`, under `## Consumed surface` and `## Needs` — not in any leaf of this repo, and not in `contract_ng-e2e-helpers.md` / `usage_polyfill-e2e.md`. The planner's grep was right about where it is NOT; the parent's claim that "`ng-helpers`' contract cites `readUnion`" was wrong about which document. Read through the declaration only, never their code.
|
||||
|
||||
### Follow-ups (not fixed here)
|
||||
|
||||
- `useShapeQuery` never opens the documents named in `scope.graphs`, so an application naming an unopened or public-store document gets an empty result with no error. Real, out of scope, probably deserves a `bug_` leaf in `app-contract`.
|
||||
- `ensureReposOpen` now has no production caller (only `open-repo.test.ts`). Internal dead code.
|
||||
- `examples/notebook/tsconfig.json` does not typecheck — PRE-EXISTING, verified against the branch tip before any edit: the hand-written `NoteShape` does not satisfy `ShapeType<BaseType>`.
|
||||
- The `ng-helpers` declaration still reports `1 passed / 29 failed`-era breakage and three Needs this removal does not answer; their `against:` pin is `1.0.0-dev.6` and the surface is now `1.0.0-dev.8`.
|
||||
- `.project/concepts/app-contract/_debt.md` carries markers for `orm.ts` and `use-shape-query.ts` raised before this task; left alone.
|
||||
- The dated `docs/briefs/` were deliberately not rewritten: they describe a July state that the documents themselves date.
|
||||
- The e2e suite was NOT run (it needs a live broker); it is not part of this task's gate.
|
||||
` so the parent's claim is corrected rather than silently dropped.
|
||||
|
||||
**This child owns the contract edit and the version bump for its own change** — a published symbol removed is a **major** by the contract's own `## Change policy`. No other child edits the contract for this change.
|
||||
|
||||
**Also in scope:** `packages/polyfill/src/surface/read-model.ts` (the module goes if nothing else lives in it), `packages/polyfill/src/index.ts`, `test/vocabulary.test.ts` and `test/app-surface.test.ts`, `docs/api-contract.md`, `packages/polyfill/README.md`, `packages/polyfill/docs/polyfill-reference.md`, and the `docs/` briefs only where they state something that is now false.
|
||||
|
||||
**Hard gate:** `bun test` → 0 fail; `bunx tsc --noEmit -p packages/polyfill/tsconfig.json` → exit 0; `bunx tsc --noEmit -p packages/polyfill/e2e/tsconfig.json` → exit 0; `python3 ~/.claude/skills/concept/lint.py` → 0 ERROR. Anti-cheat: no `any`, no `@ts-ignore`, no deleting or skipping a test, no widening an assertion.
|
||||
|
||||
**Resolved since planning (2026-09-16).** Do NOT read `ng-helpers` — not its code, not its tests. The isolation discipline says a counterpart is reached through its engagement and its usage declaration only. The planner's finding (that no contract leaf cites `readUnion`) was correct but obtained by a forbidden method. If this task needs to know who depends on `readUnion`, the answer belongs in a `usage_` declaration; `ng-helpers` has never authored one, and producing that is the fix, not a workaround by reading.
|
||||
|
||||
**Resolved since planning (2026-09-21) - `readUnion` HAS a declared consumer.** `ng-helpers` published its usage declaration (`polyfill-surface--ng-helpers`) and it says YES: `e2e/indexing-app.ts` calls `readUnion([doc])` twice, and `src/index.ts` REPUBLISHES it to ng-helpers' own consumers as the documented way to read an index. No ORM is wired there (`useShape: () => undefined`), so `useShapeQuery` is unreachable for them; their only fallback is an anchored `docs.sparqlQuery`. They declared this as a Need at full strength.
|
||||
|
||||
So the removal is not free. Before removing, decide with the owner what a consumer with no ORM reads with. Their declaration is the channel: read it at `ng-helpers`' repo through its published `usage_` leaf, never by reading their code.
|
||||
|
||||
**Also relevant, and worse than it looks.** That declaration reports `ng-helpers` still calling `createEntityDoc("public")` and `listMyEntityDocs("public")`, both invalid since `1.0.0-dev.5`, while its own test suite stays green because it runs against a fake that still models the superseded surface. A consumer can therefore be broken against us with nothing red on either side. Do not treat their green suite as evidence.
|
||||
|
||||
**Update (2026-09-21) - wiring the ORM does NOT unblock the removal, and the reason is ours.** `ng-helpers` wired the ORM as the contract prescribes and re-measured. Its verdict, recorded in its published usage declaration: neither shape read fetches a PUBLIC STORE's key, so a stranger reading a public index **from its reference alone** - the central case for an index - is served by no shape query at all. `readUnion` is the only thing that serves it today.
|
||||
|
||||
So the removal is blocked by a capability gap on OUR side, not by a consumer's reluctance. Before removing, either that gap closes or the removal does not happen. Read their declaration for the three secondary costs they list; do not read their code.
|
||||
|
||||
Their e2e also went from a measured `1 passed / 29 failed` (they were already broken against us, with both suites green) to `28 passed / 2 failed`, the two remaining being a provider defect: `useShapeQuery` never starts because `orm_start_graph is not a function`, swallowed, `isPending` for ever - measured on three consecutive runs, with the SDK's `ng` answering that member and `initNg` discarding its first argument.
|
||||
|
||||
## Progress
|
||||
|
||||
## Result
|
||||
|
||||
Status: done. `1.0.0-dev.8`, major. Two commits: `0c21ade` (the removal) and `0a2b8b1` (the documents).
|
||||
|
||||
### The opening barrier in `useShapeQuery` was NOT added
|
||||
|
||||
The orchestrator first framed the blocker as "no shape read serves a public index", then withdrew it: an index is read BY ANCHOR, not by shape, and `docs.sparqlQuery` already fetches the public store's key at its door (`packages/polyfill/src/surface/docs.ts:155`). The barrier had been written and verified against a new test; both were reverted, uncommitted, on instruction. `use-shape-query.ts` carries one comment edit in this task and nothing else.
|
||||
|
||||
### What unblocks the removal, and how it was verified
|
||||
|
||||
VERIFIED: `packages/polyfill/test/cross-user-access.test.ts` — *"a bare reference is enough for a PUBLIC document, and not for a protected one"*. It now asserts, under Bob, that he holds no capability for either document, reads the public one through the anchored `docs.sparqlQuery`, and holds its key afterwards — obtained from the document, not handed across by the test. That is the property `readUnion` was protecting, and it belongs to the primitive underneath it.
|
||||
|
||||
### What was removed, and what each call site became
|
||||
|
||||
`readUnion` and `UnionSubject` are gone from the entry. `packages/polyfill/src/surface/read-model.ts` is deleted; `packages/polyfill/src/surface/read-document.ts` replaces it with `readDocumentSubjects(doc)` — ONE anchored query, INTERNAL, `kind=invention` (the anchored query itself is `docs.sparqlQuery`, annotated there; what is ours is the cold-start open and the row grouping).
|
||||
|
||||
- `index-deposit.ts` (two sites) — the `onUnreadable` callback became a `catch`; both still return `unreadable`, and `noRowsMeansAbsent` still consults `getSyncState`, which is why the `ensureRepoOpen` in front of the read cannot move to the caller.
|
||||
- `examples/notebook/app.ts`, `packages/polyfill/e2e/polyfill-entry.ts` — each composes its own read over the published `docs.sparqlQuery`. The two multi-document e2e sites carry a VISIBLE `catch` and report what they skipped.
|
||||
- Tests re-pointed, none deleted or weakened. `open-repo.test.ts` keeps its integration path onto `ensureRepoOpen` by the same route. `read-model.test.ts` → `read-document.test.ts`: three cases that tested the batch POLICY were re-pointed at what replaced it (a failed read throws; an empty document answers `[]`; two reads never merge), and the file header says which and why.
|
||||
|
||||
### The per-document tolerance
|
||||
|
||||
RELOCATED to the caller, not lost — and shedding it is an alignment gain, since upstream an anchored query on a repo the verifier does not hold rejects (`resolve_target_for_sparql` → `NgError::RepoNotFound`). No tolerant batch helper was introduced anywhere.
|
||||
|
||||
### Two behaviour changes, both recorded in the contract
|
||||
|
||||
1. A document the holder has no capability for is REFUSED out loud instead of dropped from a batch.
|
||||
2. Subjects under the reserved `urn:ng-eventually:` prefix are no longer filtered for an application. No rights move: the only registers an ordinary document carries are its inbox address and, for a public document, the key its store already serves to whoever asks; the cap-bearing `inboxCap` register lives in its owner's private store.
|
||||
|
||||
### Does our emulation reject, or answer empty?
|
||||
|
||||
VERIFIED: it REJECTS, and it does not swallow. `docs.sparqlQuery` has no `catch` — it forwards `ng.sparql_query`'s rejection unchanged — and `readDocumentSubjects` has none either. The swallow that existed was `readDoc` inside the deleted module. One nuance, pre-existing: for a document whose public-store cap fetch failed transiently, `fetchReadCap` returns `false` and the rejection that reaches the caller is this package's "you hold no cap", not the broker's outage. That conflation is documented in `public-store.ts` and is not introduced here.
|
||||
|
||||
### Where `ng-helpers` cites `readUnion`
|
||||
|
||||
In its OWN published `usage_ng-helpers.md` against `polyfill-surface`, under `## Consumed surface` and `## Needs` — not in any leaf of this repo, and not in `contract_ng-e2e-helpers.md` / `usage_polyfill-e2e.md`. The planner's grep was right about where it is NOT; the parent's claim that "`ng-helpers`' contract cites `readUnion`" was wrong about which document. Read through the declaration only, never their code.
|
||||
|
||||
### Follow-ups (not fixed here)
|
||||
|
||||
- `useShapeQuery` never opens the documents named in `scope.graphs`, so an application naming an unopened or public-store document gets an empty result with no error. Real, out of scope, probably deserves a `bug_` leaf in `app-contract`.
|
||||
- `ensureReposOpen` now has no production caller (only `open-repo.test.ts`). Internal dead code.
|
||||
- `examples/notebook/tsconfig.json` does not typecheck — PRE-EXISTING, verified against the branch tip before any edit: the hand-written `NoteShape` does not satisfy `ShapeType<BaseType>`.
|
||||
- The `ng-helpers` declaration still reports `1 passed / 29 failed`-era breakage and three Needs this removal does not answer; their `against:` pin is `1.0.0-dev.6` and the surface is now `1.0.0-dev.8`.
|
||||
- `.project/concepts/app-contract/_debt.md` carries markers for `orm.ts` and `use-shape-query.ts` raised before this task; left alone.
|
||||
- The dated `docs/briefs/` were deliberately not rewritten: they describe a July state that the documents themselves date.
|
||||
- The e2e suite was NOT run (it needs a live broker); it is not part of this task's gate.
|
||||
|
||||
Reference in New Issue
Block a user