diff --git a/.project/concepts/app-contract/knowledge_what-a-read-scope-is-upstream.md b/.project/concepts/app-contract/knowledge_what-a-read-scope-is-upstream.md index ba8b8dc..22330cb 100644 --- a/.project/concepts/app-contract/knowledge_what-a-read-scope-is-upstream.md +++ b/.project/concepts/app-contract/knowledge_what-a-read-scope-is-upstream.md @@ -1,7 +1,7 @@ --- type: knowledge summary: What the target offers for reading many objects — a graph is one document, a store's graph holds only its containment triples, and scoping a read by store is not expressible at all -last_checked: 2026-09-16 +last_checked: 2026-09-21 --- # What a read scope is upstream @@ -10,7 +10,16 @@ Read at the source in `nextgraph-rs` (`213338f6`). Every claim points at a symbo ## The type, and the two axes it filters on -`Scope = { graphs?: string[] | string; subjects?: string[] }` (`sdk/js/orm/src/types.ts`), normalised by `normalizeScope` into `{ graphs: string[]; subjects: string[] }`. Its own documentation states the edges: `["did:ng:i"]` or `[""]` means the whole dataset, while `[]` or `undefined` returns **no objects at all**. Two axes, no third. +`Scope = { graphs?: string[] | string; subjects?: string[] }` (`sdk/js/orm/src/types.ts` at `213338f6`), normalised by `normalizeScope` into `{ graphs: string[]; subjects: string[] }`. **In the installed `@ng-org/orm@0.1.2-alpha.21`, `graphs` is REQUIRED** (`graphs: string[] | string`, `dist/types.d.ts`); `subjects?` stays optional. A caller that omitted `graphs` no longer compiles. + +**The two axes read `[]` in OPPOSITE directions, and this is the trap.** Upstream documents each field separately, and they do not agree: + +| field | `["did:ng:i"]` or `[""]` | `[]` or `undefined` | +|---|---|---| +| `graphs` | the whole dataset | **no objects at all** - the EMPTY set | +| `subjects` | n/a | **no filtering** - every subject | + +So `{ graphs: [], subjects: [] }` returns nothing, and it is `graphs` alone that decides it. Reading the pair as symmetrical is how a read that should have been unscoped becomes a read that matches nothing, silently, since a shape query matching nothing is indistinguishable from a scope holding nothing. Two axes, no third. ## A `graphs` entry is one document's graph, never a set of them @@ -48,7 +57,7 @@ That is `declared-not-wired` in the taxonomy of [[no-divergence-from-nextgraph]] ## "Not ready" and "ready and empty" are distinguishable — one level down -`OrmSubscription` (`sdk/js/orm/src/connector/GraphOrmSubscription.ts`) is exported from the package entry — `core.ts`, which `@ng-org/orm` resolves `.` to — and carries a public `readyPromise` getter. So at that level the two states **are** distinguishable. The React `useShape` adapter drops the distinction: it returns a `DeepSignalSet` and documents it as *"the orm objects or an empty set, if still loading"*. The gap is between level 3's class and level 3's hook, not between us and the target. +`OrmSubscription` (`sdk/js/orm/src/connector/GraphOrmSubscription.ts`) is exported from the package entry — `core.ts`, which `@ng-org/orm` resolves `.` to — and carries a public `readyPromise` getter. In the installed alpha.21 both the symbol and its file are renamed, to `RdfOrmSubscription` and `connector/RdfOrmSubscription.ts`, and it gains an `isReady` getter that answers the same question without awaiting. So at that level the two states **are** distinguishable. The React `useShape` adapter drops the distinction: it returns a `DeepSignalSet` and documents it as *"the orm objects or an empty set, if still loading"*. The gap is between level 3's class and level 3's hook, not between us and the target. ## What this package does instead diff --git a/.project/concepts/app-contract/knowledge_what-the-orm-writes-and-deletes-upstream.md b/.project/concepts/app-contract/knowledge_what-the-orm-writes-and-deletes-upstream.md index 80275cd..c740249 100644 --- a/.project/concepts/app-contract/knowledge_what-the-orm-writes-and-deletes-upstream.md +++ b/.project/concepts/app-contract/knowledge_what-the-orm-writes-and-deletes-upstream.md @@ -1,23 +1,25 @@ --- type: knowledge summary: What the target offers for writing and removing — deletion is a SET MUTATION wired end to end, it removes the shape's predicates rather than the subject, and the ORM's writes never pass through `sparql_update` -last_checked: 2026-09-16 +last_checked: 2026-09-21 --- # What the ORM writes and deletes upstream -Read at the source in `nextgraph-rs` (`213338f697df15afc086c042f4bab1bffd01aa04`), with `@ng-org/orm@0.1.2-alpha.19`, `@ng-org/web@0.1.2-alpha.13` and `@ng-org/alien-deepsignals@0.1.2-alpha.11` installed in `packages/polyfill`. Every claim points at a symbol rather than a line, so `git grep` settles any doubt after the code moves. Companion to [[knowledge_what-a-read-scope-is-upstream]], which answers the reading half. +Read at the source in `nextgraph-rs` (`213338f697df15afc086c042f4bab1bffd01aa04`), which declares `@ng-org/orm@0.1.2-alpha.18`, `@ng-org/web@0.1.2-alpha.13`, `@ng-org/shex-orm@0.1.2-alpha.8` and `@ng-org/alien-deepsignals@0.1.2-alpha.11`. `packages/polyfill` installs `@ng-org/orm@0.1.2-alpha.21`, `@ng-org/web@0.1.2-alpha.14`, `@ng-org/shex-orm@0.1.2-alpha.9` and `@ng-org/alien-deepsignals@0.1.2-alpha.13` - the published packages are AHEAD of the source this repo cites, so where the two disagree both are named below. Every claim points at a symbol rather than a line, so `git grep` settles any doubt after the code moves. Companion to [[knowledge_what-a-read-scope-is-upstream]], which answers the reading half. ## Removing an object is a set mutation, and LEVEL 3 answers -`sdk/js/orm/README.md` teaches it as one line — `dogs.delete(aDog)` — on the `DeepSignalSet` a subscription hands back. There is no `deleteObject` symbol and no deletion file: `sdk/js/orm/src/core.ts` exports exactly `initNg`, `ngSession`, `OrmSubscription`, `DiscreteOrmSubscription`, `insertObject`, `getObjects`, plus `getRaw`/`watch`/`effect` re-exported from `@ng-org/alien-deepsignals`. That absence is **not** the finding, and per [[no-divergence-from-nextgraph]] it would prove nothing on its own — the finding is that the set mutation is fully implemented, so the cascade stops at level 3 and there is nothing to invent. +`sdk/js/orm/README.md` teaches it as one line — `dogs.delete(aDog)` — on the `DeepSignalSet` a subscription hands back. There is no `deleteObject` symbol at any version: `sdk/js/orm/src/core.ts` at the cited commit exports exactly `initNg`, `ngSession`, `OrmSubscription`, `DiscreteOrmSubscription`, `insertObject`, `getObjects`, plus `getRaw`/`watch`/`effect` re-exported from `@ng-org/alien-deepsignals`. The installed `@ng-org/orm@0.1.2-alpha.21` renames `OrmSubscription` to `RdfOrmSubscription` and adds one export the source does not carry, `removeObject(graphNuri, subjectIri): Promise` - see the note at the end of this section. That absence is **not** the finding, and per [[no-divergence-from-nextgraph]] it would prove nothing on its own — the finding is that the set mutation is fully implemented, so the cascade stops at level 3 and there is nothing to invent. + +**`removeObject` exists in the package and not in the source.** Its own documentation declares a wider effect than the set mutation: `DELETE WHERE { GRAPH { ?p ?o } }`, every predicate of the subject rather than the shape's own, while claiming in the same comment to have "the same effect" as a `delete()` on a subscription's object. The two cannot both hold, and nothing in the cited source settles it. It stays unpublished here for the reason `contract_polyfill-surface` records: a symbol whose only evidence is an npm build is not something to build a guarantee on. ## The path, hop by hop 1. **The set proxy.** `deepSignal`'s set handler (`sdk/js/alien-deepsignals/src/deepSignal.ts`, the `delete` branch) emits, for an OBJECT member, `{ path: [...containerPath, syntheticId], op: "remove" }` — a `DeepPatch` with no `type`. For a LITERAL inside a multi-valued property it emits `{ path: containerPath, op: "remove", type: "set", value }` instead. -2. **The synthetic id is `"|"`**, minted by `OrmSubscription.signalObjectPropGenerator`, which also generates a subject IRI (`:q:<33 random bytes base64url>`) when `@id` is `""`. +2. **The synthetic id is `"|"`**, minted by `OrmSubscription.signalObjectPropGenerator` (`RdfOrmSubscription` in alpha.21), which also generates a subject IRI (`:q:<33 random bytes base64url>`) when `@id` is `""`. 3. **`deepPatchesToWasm`** (`sdk/js/orm/src/connector/utils.ts`) joins the path into the string `/|`. -4. **`OrmSubscription.onSignalObjectUpdate`** batches in a microtask (or into `pendingPatches` inside a transaction) and calls `ng.graph_orm_update(subscriptionId, patches, session_id)`. +4. **`OrmSubscription.onSignalObjectUpdate`** (`RdfOrmSubscription` in alpha.21) batches in a microtask (or into `pendingPatches` inside a transaction) and calls `ng.graph_orm_update(subscriptionId, patches, session_id)`. 5. **The binding.** `graph_orm_update` (`sdk/js/lib-wasm/src/lib.rs`) deserialises `OrmPatches` — `OrmPatch { op, valType, path, value }` with `OrmPatchOp::{add, remove}` and `OrmPatchType::set` (`engine/net/src/orm.rs`) — into `AppRequest::new_orm_update`. 6. **The engine.** `Verifier::orm_frontend_update` → `create_sparql_update_query_for_patches` (`engine/verifier/src/orm/graph/handle_frontend_update.rs`): a `remove` whose path resolves to a ROOT object (one segment, no predicate) calls `SparqlBuilder::remove_object` and marks the tracked object `TrackedOrmObjectValidity::Invalid`. The queries are then run through `process_sparql_update`, anchored on the graph taken from the first patch's path. @@ -43,7 +45,7 @@ Both return types are **inferred**, not annotated, and both functions are thin w ## The three remaining symbols -**`OrmSubscription`** — a class, and the machinery every other read and write goes through. Private constructor; `static getOrCreate(shapeType, scope)` pooled on `"||"` with refcounting and a 500 ms close delay. Instance surface: `signalObject: DeepSignalSet`, `readyPromise`, `close()`, `beginTransaction()`, `commitTransaction()`, `inTransaction`, `shapeType`, `scope`. It hands a caller nothing the target will not hand them — a shape, a scope and a set — and `useShape`, `getObjects` and `insertObject` are all built on it. +**`OrmSubscription`**, `RdfOrmSubscription` in alpha.21 — a class, and the machinery every other read and write goes through. Private constructor; `static getOrCreate(shapeType, scope)` pooled on `"||"` with refcounting and a 500 ms close delay. Instance surface: `signalObject: DeepSignalSet`, `readyPromise`, `close()`, `beginTransaction()`, `commitTransaction()`, `inTransaction`, `shapeType`, `scope`. In alpha.21 the second argument is no longer a bare `Scope` but an `RdfOrmConfig` (`Scope & { where?, orderBy?, pageSize?, maxActivePages? }`), `readyPromise` resolves to the subscription data rather than to nothing, and the instance also carries `mode`, `isReady`, `addChangeListener`, `removeChangeListener`, `nextPage` and `previousPage`. Whether any of that reaches the engine is NOT settled by the cited source, where `Verifier::start_orm` still builds its config from `json!({})` behind a `// TODO` (`engine/verifier/src/orm/graph/initialize.rs`): as of `213338f6` the richness is expressible at level 3 and discarded at level 1. Measuring it is not done here. It hands a caller nothing the target will not hand them — a shape, a scope and a set — and `useShape`, `getObjects` and `insertObject` are all built on it. **`DiscreteOrmSubscription`** — the same class shape for JSON-CRDT documents. `static getOrCreate(documentId: string)`; `signalObject` is `DeepSignal | undefined` (a getter, undefined before the initial payload); it reaches the engine through `orm_start_discrete` and `discrete_orm_update`. Its unit of subscription is **one document id**, never a `Scope`. @@ -57,4 +59,6 @@ That is a fact about where a guard can sit, not a proposal for one. ## Live behaviour against the deployed broker -Verified by measurement, not by reading: a live ORM read still does not start against the deployment, and the reason is **not** the one previously recorded. See [[bug_useshape-live-read-needs-a-newer-broker]], and `packages/polyfill/e2e/orm-live-read.ts`, which is the measurement. +Verified by measurement, not by reading: a live ORM read STARTS against the deployment, and the delete reaches the document. `packages/polyfill/e2e/orm-live-read.ts` is the measurement - `5 passed, 0 failed` on 2026-09-21, the read ready in 13ms, the seeded subject's triples gone after `.delete()`. + +The read was blocked until 2026-09-21, and the recorded reason was wrong twice over. It was not that the broker needed to catch up: the DEPLOYMENT was ahead and the pinned client packages were behind, `orm_start_graph` having gained a `config` parameter at index 4, before the callback, which the five-argument client of the day filled with its callback. Updating the packages answered it. The bug leaf that carried the wrong mechanism was deleted with the question it described. diff --git a/.project/concepts/e2e-harness/knowledge_what-each-suite-judges.md b/.project/concepts/e2e-harness/knowledge_what-each-suite-judges.md index 20c273d..6cbd2a4 100644 --- a/.project/concepts/e2e-harness/knowledge_what-each-suite-judges.md +++ b/.project/concepts/e2e-harness/knowledge_what-each-suite-judges.md @@ -11,7 +11,7 @@ summary: Which suite answers which question, what a batch costs, and why two run **The reactivity suite** isolates document subscription. -**The ORM live-read measurement** asks one question of the deployment rather than of this package: does a live `useShape` read actually start. It is the only runner that imports `@ng-org/orm` — an optional peer `bun install` does not fetch — which is why it stands apart from the polyfill suite instead of inside it: a checkout without the ORM must still be able to run the other three. It is EXPECTED to exit non-zero while the deployed broker and the published client packages disagree on `orm_start_graph` (see `app-contract/bug_useshape-live-read-needs-a-newer-broker`); green means they agree. +**The ORM live-read measurement** asks one question of the deployment rather than of this package: does a live `useShape` read actually start. It is the only runner that imports `@ng-org/orm` — an optional peer `bun install` does not fetch — which is why it stands apart from the polyfill suite instead of inside it: a checkout without the ORM must still be able to run the other three. It exited non-zero for as long as the deployed broker and the published client packages disagreed on `orm_start_graph`; since the packages caught up on 2026-09-21 it is GREEN, and a red run now means they have drifted apart again. A unit suite cannot replace any of the first three, and none of them replaces the unit suite: they are slow, they depend on live external services, and they cannot enumerate a case space.