docs: ce que l'ORM offre pour ecrire et supprimer, et la vraie cause du useShape mort
This commit is contained in:
Sylvain Duchesne
@@ -2,7 +2,7 @@
|
||||
type: overview
|
||||
summary: What an application may rely on from @ng-eventually/polyfill, and what it will have to delete
|
||||
triggers:
|
||||
keywords: [polyfill, sdk, surface, contract, publish, published, entry, export, migration, unlearn, consumer, app-facing, inbox, deposit, IRI, urn, namespace, vocabulary, ontology, predicate]
|
||||
keywords: [polyfill, sdk, surface, contract, publish, published, entry, export, migration, unlearn, consumer, app-facing, inbox, deposit, IRI, urn, namespace, vocabulary, ontology, predicate, orm, useShape, getObjects, insertObject, OrmSubscription, ngSession]
|
||||
paths:
|
||||
- "packages/polyfill/src/index.ts"
|
||||
- "packages/polyfill/src/surface/**"
|
||||
@@ -40,5 +40,6 @@ This repo is the **provider** of `polyfill-surface`; consuming applications live
|
||||
- `knowledge_what-an-app-deletes-at-migration` — the two fates a published symbol can have.
|
||||
- `knowledge_what-an-iri-says-upstream` — the same two fates for a written *triple*: NextGraph's own vocabulary, ours, and which of our IRIs must outlive the migration.
|
||||
- `knowledge_what-a-read-scope-is-upstream` — what the target offers for reading many objects at once, and the one question it cannot express.
|
||||
- `knowledge_what-the-orm-writes-and-deletes-upstream` — the writing half: removal is a set mutation wired end to end, and an ORM write never passes through `sparql_update`.
|
||||
- `knowledge_what-an-inbox-is-upstream` — what the target actually offers around inboxes, and where a choice of ours begins.
|
||||
- `knowledge_who-processes-an-inbox` — the division of labour along a deposit's path: what this package applies, what it merely delivers, and what a layer above owes itself.
|
||||
|
||||
@@ -1,52 +1,46 @@
|
||||
---
|
||||
type: bug
|
||||
summary: A live read through the published `useShape` never starts against a broker deployed before `orm_start` was renamed `orm_start_graph` — it stays empty and pending forever, with one console error as the only trace
|
||||
summary: A live read through the published `useShape` never starts against the deployed broker — its `orm_start_graph` expects a `config` argument the published `@ng-org/orm` does not send, so the subscription stays empty and pending for ever with one console error as the only trace
|
||||
severity: major
|
||||
opened: 2026-08-17
|
||||
last_checked: 2026-08-17
|
||||
last_checked: 2026-09-16
|
||||
---
|
||||
|
||||
# `useShape` opens no live read against the deployed broker
|
||||
|
||||
The published `useShape` is a passthrough to `@ng-org/orm`'s hook (`packages/polyfill/src/surface/use-shape.ts`) — this package adds the read filter and forwards. Against the broker deployed today, the subscription that hook opens **never starts**: its set stays empty, its readiness never settles, no engine update ever arrives, and the only trace is one `console.error`. An application sees "this scope holds nothing", indefinitely, which is the exact confusion the rest of the read surface is built to prevent.
|
||||
|
||||
The cause is upstream, in NextGraph. **Nothing in this package can work around it** — the failure is a name lookup two hops below our surface.
|
||||
The cause is upstream, in the disagreement between the deployed broker and the published client packages. **Nothing in this package can work around it.**
|
||||
|
||||
## The mechanism — one method name, kept in three places by hand
|
||||
## The mechanism — an ARGUMENT, not a name
|
||||
|
||||
VERIFIED 2026-08-17 by reading `/home/sylvain/projects/nextgraph/nextgraph-rs` at `213338f6`.
|
||||
MEASURED 2026-09-16 against the real broker by `packages/polyfill/e2e/orm-live-read.ts`, which drives the published `useShape` over the real `@ng-org/orm`. The engine answers:
|
||||
|
||||
A live ORM read is one call, and it is streamed: the engine pushes into a callback rather than answering once. Nothing in the transport infers that — each hop looks the method up in a **hand-maintained table** mapping method name to the index of its callback argument, and a method missing from the table takes the plain request/response branch instead.
|
||||
```
|
||||
Deserialization error of config Error: invalid type: JsValue(Function), expected any valid JSON value
|
||||
```
|
||||
|
||||
1. `sdk/js/orm/src/connector/GraphOrmSubscription.ts` — the `OrmSubscription` constructor is the only place a live graph read begins: `ng.orm_start_graph(scope.graphs, scope.subjects, shapeType, session.session_id, this.onBackendMessage)`. Five arguments, callback last. `useShape` reaches it through `OrmSubscription.getOrCreate` (`sdk/js/orm/src/frontendAdapters/react/useShape.ts`).
|
||||
2. `sdk/js/web/src/index.ts` — the application-side `ng` proxy. `streamed_api` must contain the method, or `rpc` posts the arguments verbatim to the parent window, callback included. A function is not structured-cloneable, so that post throws.
|
||||
3. `sdk/js/api-web/main.ts` — the broker-side proxy, an independent copy of the same table. Same fallback into `myWorker.postMessage`, same throw.
|
||||
A live ORM read is one call and it is streamed: `OrmSubscription`'s constructor issues `ng.orm_start_graph(graphs, subjects, shapeType, session_id, onBackendMessage)` — **five** arguments, callback last — and the engine pushes into that callback rather than answering once. What comes back instead names a parameter, `config`, that the client never sent.
|
||||
|
||||
The hops between the two tables carry no table of their own and cannot repair a mismatch: `infra/ngnet/auth/src/main.ts` forwards `{method, args, streamed, port}` unchanged, and `engine/broker/auth/src/App.svelte` re-appends a callback **only** when `streamed` is true.
|
||||
The same measurement establishes where that parameter sits, by calling the SDK member directly with `{}` in position 4: **six arguments are ACCEPTED**, and the engine then streams back. So the deployed signature is `(graph_scope, subject_scope, shapeType, session_id, config, callback)`, and the published client's five-argument call lands its **callback** exactly where `config` is expected — which is precisely what the error describes.
|
||||
|
||||
**The name changed.** Both tables keyed the method as `orm_start` until commit `d5ecd0fd` (2026-01-16, "refactor discrete orm for yjs") renamed it to `orm_start_graph` in both files at once — while the ORM had been calling `orm_start_graph`. A build made before that commit therefore does not recognise the method a live read needs.
|
||||
**The direction is the opposite of what this leaf said until 2026-09-16.** The earlier reading — a broker built before `orm_start` was renamed `orm_start_graph` (upstream `d5ecd0fd`, 2026-01-16) — is **refuted**: the name resolves, and the call fails *inside* the function, on an argument. That arity exists in **no** commit of `nextgraph-rs` at `213338f6`: the one window that carried a `config` at all (`c38aa371` 2026-04-12 → `7c8abd0a` 2026-04-21, *"rm unused config param"*) put it **last**, after the callback, where a five-argument call would leave it merely absent. The deployment is therefore built from a source **ahead of** that checkout, and it is the published `@ng-org/orm@0.1.2-alpha.19` / `@ng-org/web@0.1.2-alpha.13` this package pins that are behind it.
|
||||
|
||||
**The application side is already correct**, and that is what isolates the defect. This package pins `@ng-org/web@0.1.2-alpha.13` (`packages/polyfill/package.json`), which is the version at `213338f6` — the rename is in it. `@ng-org/api-web` is `private: true` and has never been version-bumped past `0.1.2`: it is **not published**, it ships only inside the broker build. Its table is whatever the operator last deployed, and no consumer can pin it.
|
||||
|
||||
**Why it fails in silence:** `orm_start_graph` is awaited inside a `try` whose `catch` is `console.error(e)`, and `resolveReady()` is called on one path only — the arrival of initial data. A rejected start leaves the readiness promise pending forever and the set permanently empty. This is the same upstream property already recorded in `docs/api-contract.md` (a failed read *is* an eternal pending upstream), reached here through a different door.
|
||||
**Why it fails in silence:** `orm_start_graph` is awaited inside a `try` whose `catch` is `console.error(e)`, and `resolveReady()` is called on one path only — the arrival of initial data (`GraphOrmSubscription.ts:handleInitialResponse`). A rejected start leaves the readiness promise pending for ever and the set permanently empty. This is the same upstream property already recorded in `docs/api-contract.md` (a failed read *is* an eternal pending upstream), reached here through a different door.
|
||||
|
||||
## What is affected, and what is not
|
||||
|
||||
- **Affected:** the published `useShape`, and only it.
|
||||
- **Not affected:** `watchShape`, `subscribeDoc`, `subscribeDocs`, `readUnion`. These open one `doc_subscribe` per document, and `doc_subscribe: 2` has been in both tables continuously under a stable name. `watchShape` is built on that path, not on the ORM subscription — it is a working reactive read today.
|
||||
- **Affected:** the published `useShape`, and only it. By the same argument, `insertObject` and `getObjects` — both built on `OrmSubscription` — cannot start either, so **no ORM write or deletion can be exercised against this deployment**.
|
||||
- **Not affected:** `watchShape`, `subscribeDoc`, `subscribeDocs`, `readUnion`. These open one `doc_subscribe` per document, and `doc_subscribe: 2` has been in the method tables continuously under a stable name and arity. `watchShape` is built on that path, not on the ORM subscription — it is a working reactive read today.
|
||||
|
||||
That is why the library's own suites and the reference application do not show it: nothing in this package calls `useShape` internally.
|
||||
|
||||
## Not verified
|
||||
|
||||
The deployed broker's table was **not read**. `https://nextgraph.net/redir/` returns a page containing none of the protocol markers, so it is not the bundle that holds it, and probing further was out of mandate. What is established is the mechanism and its exact precondition — a broker built before `d5ecd0fd` — not that the running deployment sits before it. The symptom was reported by the consuming application, not reproduced here.
|
||||
|
||||
## What to do
|
||||
|
||||
- **Do not build on live `useShape`.** Use `watchShape` for a reactive read, `readUnion` for a one-shot one. This is stated in the contract's `## Non-guarantees` so a consumer meets it without reading this leaf.
|
||||
- **The fix is not ours.** It is a NextGraph deployment: rebuild and redeploy the broker from a source at or after `d5ecd0fd`. Nothing to change in this repository, and nothing to open upstream from here — report it, do not patch around it.
|
||||
- **To check whether it is closed:** open a live `useShape` against the deployment and watch for an initial payload. A permanently empty set with one `console.error` naming `orm_start_graph` means the table is still stale.
|
||||
- **The fix is not ours, and it is an ALIGNMENT, not an upgrade in one direction.** Either the deployment rolls to a build whose `orm_start_graph` matches the published packages, or `@ng-org/orm` / `@ng-org/web` are republished against the deployed signature. Report it; do not patch around it, and do not pass a `config` argument from here — a client that guesses an unpublished arity breaks the day the two ends agree.
|
||||
- **To check whether it is closed:** `bun run e2e/orm-live-read.ts` from `packages/polyfill` (it needs `@ng-org/orm`, an optional peer `bun install` does not fetch). It exits non-zero while this is open and names what the engine expected. Green means the two ends agree and this leaf can go.
|
||||
|
||||
## The shape worth remembering
|
||||
|
||||
A method name duplicated across three independently-deployed artifacts, with a **silent** fallback when they disagree, is a defect that cannot be caught by any one repository's tests. Renaming it in the two tables in one commit fixed the source; it did not fix anything already running. Anything this package routes through a streamed method inherits the same exposure — that is the reason `subscribeDocs` composing per-document `doc_subscribe` is worth more than it looks.
|
||||
A method **signature** — name and arity together — duplicated across three independently-deployed artifacts, with a **silent** failure when they disagree, is a defect no single repository's tests can catch. Pinning the client packages does not help: the broker-side half (`@ng-org/api-web`, `private: true`, shipped only inside the broker build) is pinnable by nobody, so the two ends drift in either direction and the symptom is identical both ways. Anything this package routes through a streamed method inherits the same exposure — which is the reason `subscribeDocs` composing per-document `doc_subscribe` is worth more than it looks.
|
||||
|
||||
@@ -0,0 +1,60 @@
|
||||
---
|
||||
type: knowledge
|
||||
summary: What the target offers for writing and removing — deletion is a SET MUTATION wired end to end, it removes the shape's predicates rather than the subject, and the ORM's writes never pass through `sparql_update`
|
||||
last_checked: 2026-09-16
|
||||
---
|
||||
|
||||
# What the ORM writes and deletes upstream
|
||||
|
||||
Read at the source in `nextgraph-rs` (`213338f697df15afc086c042f4bab1bffd01aa04`), with `@ng-org/orm@0.1.2-alpha.19`, `@ng-org/web@0.1.2-alpha.13` and `@ng-org/alien-deepsignals@0.1.2-alpha.11` installed in `packages/polyfill`. Every claim points at a symbol rather than a line, so `git grep` settles any doubt after the code moves. Companion to [[knowledge_what-a-read-scope-is-upstream]], which answers the reading half.
|
||||
|
||||
## Removing an object is a set mutation, and LEVEL 3 answers
|
||||
|
||||
`sdk/js/orm/README.md` teaches it as one line — `dogs.delete(aDog)` — on the `DeepSignalSet` a subscription hands back. There is no `deleteObject` symbol and no deletion file: `sdk/js/orm/src/core.ts` exports exactly `initNg`, `ngSession`, `OrmSubscription`, `DiscreteOrmSubscription`, `insertObject`, `getObjects`, plus `getRaw`/`watch`/`effect` re-exported from `@ng-org/alien-deepsignals`. That absence is **not** the finding, and per [[no-divergence-from-nextgraph]] it would prove nothing on its own — the finding is that the set mutation is fully implemented, so the cascade stops at level 3 and there is nothing to invent.
|
||||
|
||||
## The path, hop by hop
|
||||
|
||||
1. **The set proxy.** `deepSignal`'s set handler (`sdk/js/alien-deepsignals/src/deepSignal.ts`, the `delete` branch) emits, for an OBJECT member, `{ path: [...containerPath, syntheticId], op: "remove" }` — a `DeepPatch` with no `type`. For a LITERAL inside a multi-valued property it emits `{ path: containerPath, op: "remove", type: "set", value }` instead.
|
||||
2. **The synthetic id is `"<graph>|<subject>"`**, minted by `OrmSubscription.signalObjectPropGenerator`, which also generates a subject IRI (`<first 53 chars of the graph NURI>:q:<33 random bytes base64url>`) when `@id` is `""`.
|
||||
3. **`deepPatchesToWasm`** (`sdk/js/orm/src/connector/utils.ts`) joins the path into the string `/<graph>|<subject>`.
|
||||
4. **`OrmSubscription.onSignalObjectUpdate`** batches in a microtask (or into `pendingPatches` inside a transaction) and calls `ng.graph_orm_update(subscriptionId, patches, session_id)`.
|
||||
5. **The binding.** `graph_orm_update` (`sdk/js/lib-wasm/src/lib.rs`) deserialises `OrmPatches` — `OrmPatch { op, valType, path, value }` with `OrmPatchOp::{add, remove}` and `OrmPatchType::set` (`engine/net/src/orm.rs`) — into `AppRequest::new_orm_update`.
|
||||
6. **The engine.** `Verifier::orm_frontend_update` → `create_sparql_update_query_for_patches` (`engine/verifier/src/orm/graph/handle_frontend_update.rs`): a `remove` whose path resolves to a ROOT object (one segment, no predicate) calls `SparqlBuilder::remove_object` and marks the tracked object `TrackedOrmObjectValidity::Invalid`. The queries are then run through `process_sparql_update`, anchored on the graph taken from the first patch's path.
|
||||
|
||||
**The trap that makes a delete a no-op.** The patch is emitted only when `target.delete(rawValue)` returned true **and** a synthetic id is known for that object. So the object passed must be the one the set is holding — reached by iterating it, by `.first()` or by `.getBy(graph, subject)`. `getObjects` returns a `deepClone`, so deleting an item of ITS result removes nothing and reports nothing.
|
||||
|
||||
## `remove_object` deletes the SHAPE's predicates, not the subject and not the document
|
||||
|
||||
`SparqlBuilder::remove_object` loops the shape's own `predicates` and emits one `DELETE { GRAPH <g> { <s> <p> ?o } } WHERE { … }` per predicate. Triples on the same subject under a predicate the shape does not declare **survive**, and so does the document: nothing on this path deletes a document, and the ORM offers no call that does.
|
||||
|
||||
Removing a child from a parent's multi-valued object property is a different arm — `remove_link`, a `DELETE DATA` of the one triple that ties parent to child. The README states the consequence plainly: only that tie goes, the child's own quads stay.
|
||||
|
||||
## `getObjects` and `insertObject` — the exact surface
|
||||
|
||||
```ts
|
||||
getObjects<T extends BaseType>(shapeType: ShapeType<T>, scope: Scope | string): Promise<DeepSignalSet<T>>
|
||||
insertObject<T extends BaseType>(shapeType: ShapeType<T>, object: T): Promise<void>
|
||||
```
|
||||
|
||||
Both return types are **inferred**, not annotated, and both functions are thin wrappers over `OrmSubscription`.
|
||||
|
||||
- **`getObjects`** (`sdk/js/orm/src/connector/getObjects.ts`) — `getOrCreate(shapeType, normalizeScope(scope))`, awaits `readyPromise`, schedules `close()` one second later, returns `deepClone(signalObject)`. The result is **detached**: a plain `Set` of plain objects, no two-way binding, no reactivity. `docs/api-contract.md` § 6 records this accurately — checked against the source, not re-derived.
|
||||
- **`insertObject`** (`sdk/js/orm/src/connector/insertObject.ts`) — opens a subscription scoped to `{ graphs: [] }` (deliberately subscribing to no document), awaits ready, `signalObject.add(object)`, `close()`. The object **must** carry `@graph`: a root object without one makes `signalObjectPropGenerator` throw *"When adding new root orm objects, you must specify the @graph"*. `@id: ""` means "generate one".
|
||||
|
||||
## The three remaining symbols
|
||||
|
||||
**`OrmSubscription`** — a class, and the machinery every other read and write goes through. Private constructor; `static getOrCreate(shapeType, scope)` pooled on `"<shape>|<graphs>|<subjects>"` with refcounting and a 500 ms close delay. Instance surface: `signalObject: DeepSignalSet<T>`, `readyPromise`, `close()`, `beginTransaction()`, `commitTransaction()`, `inTransaction`, `shapeType`, `scope`. It hands a caller nothing the target will not hand them — a shape, a scope and a set — and `useShape`, `getObjects` and `insertObject` are all built on it.
|
||||
|
||||
**`DiscreteOrmSubscription`** — the same class shape for JSON-CRDT documents. `static getOrCreate(documentId: string)`; `signalObject` is `DeepSignal<DiscreteArray | DiscreteObject> | undefined` (a getter, undefined before the initial payload); it reaches the engine through `orm_start_discrete` and `discrete_orm_update`. Its unit of subscription is **one document id**, never a `Scope`.
|
||||
|
||||
**`ngSession`** — not a function and not a class: a module-level `Promise<{ ng: NG; session: Session }>` (`sdk/js/orm/src/connector/initNg.ts`), resolved exactly once by `initNg`. Every subscription awaits it before touching the engine. Its value is the SDK handle plus the session object.
|
||||
|
||||
## A write never passes through `sparql_update`
|
||||
|
||||
`graph_orm_update` is its own SDK member, distinct from `sparql_update`. So a guard implemented by overriding `ng.sparql_update` — which is what `packages/polyfill/src/surface/ng-proxy.ts` does — **does not see an ORM write at all**, neither an add nor a delete. The read-filtered view (`packages/polyfill/src/emulated-verifier/read-filter.ts`) is symmetrical by design: it forwards `add`, `delete` and `clear` to the underlying set untouched, because those yield no items to leak.
|
||||
|
||||
That is a fact about where a guard can sit, not a proposal for one.
|
||||
|
||||
## Live behaviour against the deployed broker
|
||||
|
||||
Verified by measurement, not by reading: a live ORM read still does not start against the deployment, and the reason is **not** the one previously recorded. See [[bug_useshape-live-read-needs-a-newer-broker]], and `packages/polyfill/e2e/orm-live-read.ts`, which is the measurement.
|
||||
Reference in New Issue
Block a user