Reconnexion/festipod
1
0
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity

fix(@data): round-trip the seed/read path against the real broker

Browse Source 
Multiple compounding defects kept the connected @data read at 0 entities:
- writeEntity/updateEntityField and registration helpers wrote into an explicit
  GRAPH <plainNuri> named graph, invisible to the anchored default-graph read
  (read-model.readDoc) after the read switched to per-doc anchored. Drop the
  wrapper so writes land in the repo's default graph (matches the read).
- Seed entities are now owned by the CURRENT account, so protected seed docs
  (user profiles) pass the per-document ReadCap gate and round-trip.
- Suppress the double seed (explicit loadTestData + 3s dev auto-seed) and add a
  re-list signal so freshly-seeded protected docs enter the read set.
- @data step awaits the seed result and waits for events AND users > 0.

Documents the anchored-default-graph write pitfall in rule_document-per-entity.

Validated: connexion-nextgraph.feature @data = 4 scenarios / 13 steps green.
NB: the shared test wallet's private store bloats across runs and makes anchored
queries hang (>15s); a fresh .playwright-profile restores ~1.5s — durable wallet
hygiene is a follow-up.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Sylvain Duchesne
2026-07-06 12:46:39 +02:00
parent 02cda056b8
commit 0911b1f9de
8 changed files with 163 additions and 84 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.project/concepts/data-layer/_debt.md
-9
View File
@@ -1,9 +0,0 @@
# Doc-debt — data-layer
> Presence of a block = doc to update. Processed → delete the block; no blocks left → delete this file.
> One block = one "big change": `why` + `files` + `verify` (leaves to review).
## Raw markers (consolidate into blocks, then delete)
- TOUCHED src/shared/data/readEntities.ts @2026-07-05 (session 0b064e8b-1717-421f-a20e-a4318ad217b1)
- TOUCHED src/shared/context/FestipodDataContext.tsx @2026-07-05 (session 0b064e8b-1717-421f-a20e-a4318ad217b1)
- TOUCHED src/shared/utils/ngBootstrap.ts @2026-07-06 (session 0b064e8b-1717-421f-a20e-a4318ad217b1)
.project/concepts/data-layer/rule_document-per-entity.md
+13 -1
View File
@@ -73,7 +73,19 @@ synchrone (boucle de seed, première création). Contre le vrai broker, un `add`
lève « Set is readonly because scope is empty » (les tests unitaires fake-ng ne l'attrapent pas).
Donc : **écriture = SPARQL direct dans le doc de l'entité** (immédiat, par-document) ;
**lecture = union + re-query** (ci-dessus). Idem pour la **mutation d'un champ** existant (p. ex. `participantCount`) : muter une valeur
**lecture = union + re-query** (ci-dessus).
**Piège de graphe (INSERT/DELETE sans wrapper `GRAPH`).** L'écriture doit viser le **graphe par
défaut** du document — on passe le NURI du document comme **ancre** de `docs.sparqlUpdate` et on
écrit le corps SPARQL **sans** clause `GRAPH <…>` explicite. La lecture union interroge elle aussi
le graphe par défaut ancré (`readEntities`/`readUnion`) ; un corps enveloppé dans un
`GRAPH <nuriDuDoc>` explicite écrit dans un graphe **nommé distinct** que cette lecture ne voit
pas → l'entité ne fait jamais l'aller-retour (elle « disparaît » silencieusement). Vaut pour
`writeEntity`, `updateEntityField` et les écritures de `registration.ts`. (Le *pourquoi* côté SDK
— comment l'ancre restreint la requête au graphe du repo — appartient au SDK `@ng-eventually/client`,
pas ici.)
Idem pour la **mutation d'un champ** existant (p. ex. `participantCount`) : muter une valeur
en mémoire ne tient pas — la re-query union relit la valeur **persistée** depuis le broker
(retour à l'ancienne valeur) → persister via SPARQL (`updateEntityField` : DELETE puis
INSERT du triplet) pour que le changement tienne et que la relecture concorde. Chaque champ est écrit avec le **bon terme RDF** selon la shape SHEX (xsd:integer /
src/modules/auth/steps/data/connexion.steps.ts
+27 -13
View File
@@ -48,22 +48,36 @@ When('je charge les données de test', async function (this: FestipodWorld) {
});
(this as any)._eventCountBefore = countBefore;
await this.appFrame!.evaluate(() => {
// AWAIT the seed's own promise (loadTestData returns a BootstrapResult promise)
// and record whether it actually seeded — so the propagation wait below can tell
// a genuinely-populated wallet (nothing to appear) from an empty one that must
// seed. Fire-and-forget here would let the assertions race the async seed.
const seededResult = await this.appFrame!.evaluate(async () => {
const td = (window as any).__testData;
td.loadTestData();
const r = await td.loadTestData();
return { seeded: r?.seeded ?? false };
});
(this as any)._loadSeeded = seededResult.seeded;
// Wait for data to propagate (if wallet was empty, data should appear)
await this.appFrame!.waitForFunction(
() => {
const td = (window as any).__testData;
// Either data was already there, or it should appear after loading
return td.events.size > 0 || td._loadResult?.seeded === false;
},
{ timeout: 75000 },
).catch(() => {
// Timeout is OK if wallet was already populated (idempotent case)
});
// Wait for data to propagate. Events reach the read via the discovery index (a
// fast, independent path); the seeded PROTECTED user docs reach it only through
// the by-need re-list, which can lag the public read under load. So wait for BOTH
// events AND users to settle (not just events) — otherwise `contient des
// utilisateurs` asserts before the protected read lands and flakes to users:0.
// On a wallet that already had data (seeded === false) there is nothing to wait
// for. The assertions still verify the real counts; this only synchronizes.
if (seededResult.seeded) {
await this.appFrame!.waitForFunction(
() => {
const td = (window as any).__testData;
return td.events.size > 0 && td.users.size > 0;
},
{ timeout: 75000 },
).catch(() => {
// Timeout tolerated — the assertions below surface the real failure with a
// clearer message than a raw waitForFunction timeout.
});
}
});
// --- Assertions ---
src/shared/context/FestipodDataContext.tsx
+37 -4
View File
@@ -232,6 +232,16 @@ function useNgData(): FestipodDataContextValue {
// read re-runs and picks up the change (there is no reactive union query).
const [readTick, setReadTick] = useState(0);
const bumpRead = useCallback(() => setReadTick(t => t + 1), []);
// RE-LIST signal: bumped after a SEED so the by-need listing effect re-runs and
// re-reads the now-populated scope INDEX documents. `registerDoc` alone is not
// enough for PROTECTED user docs: events also reach the read via the discovery
// index (a second, reliable path), but protected docs have no such fallback, so
// if the listing effect ran BEFORE the seed wrote the protected index (the
// common race — the effect fires on session-ready, the seed lands later) the
// seeded protected docs never enter `allReadDocs`. Bumping this makes the effect
// re-read `listMyEntityDocs(owner, 'protected')` once the index is populated.
const [listTick, setListTick] = useState(0);
const relist = useCallback(() => setListTick(t => t + 1), []);
/** Add a freshly-created entity document to its scope's read set AND trigger a
* re-query (reactivity: the new doc joins the union read immediately). */
@@ -282,8 +292,10 @@ function useNgData(): FestipodDataContextValue {
}
})();
return () => { cancelled = true; };
// `listTick` re-runs the listing after a seed so the freshly-written scope
// index (esp. PROTECTED user docs) is re-read into the read set.
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [ready, username]);
}, [ready, username, listTick]);
// --- The BY-NEED READ (replaces the reactive ORM fan-out) -----------------
// Read the bounded by-need docs via the SDK (per-document, independent of wallet
@@ -346,15 +358,24 @@ function useNgData(): FestipodDataContextValue {
if (hasTriedAutoSeed.current) return;
if (!ready) return;
const t = setTimeout(() => {
// RE-CHECK inside the timer: an explicit `loadTestData` sets this ref at its
// START, but a timer scheduled BEFORE that call is already pending and would
// otherwise fire a SECOND, racing seed (observed: events double to 10, and
// the two seeds' registerDoc/relist interleave, losing the protected docs).
// Bail if a seed has already been initiated by any path.
if (hasTriedAutoSeed.current) return;
hasTriedAutoSeed.current = true;
const walletHasData = events.length > 0 || users.length > 0;
if (!walletHasData) {
console.log('[FestipodData] Dev auto-seed: wallet empty, bootstrapping…');
bootstrapWallet(walletHasData, createEntityDoc)
bootstrapWallet(walletHasData, createEntityDoc, username || undefined)
.then(({ createdDocs }) => {
// Register the seeded per-entity docs into the read set (+ re-query).
createdDocs.public.forEach(d => registerDoc('public', d));
createdDocs.protected.forEach(d => registerDoc('protected', d));
// Re-list so the seeded PROTECTED index docs re-enter the read set even
// if a racing render dropped the direct registrations (see loadTestData).
relist();
})
.catch(err => console.error('[FestipodData] Auto-seed failed:', err));
} else {
@@ -689,12 +710,24 @@ function useNgData(): FestipodDataContextValue {
const loadTestData = useCallback(async (): Promise<BootstrapResult> => {
console.log('[FestipodData] loadTestData (NG)');
// An EXPLICIT load is authoritative — SUPPRESS the dev auto-seed so only ONE
// seed runs. Without this the two paths race: the auto-seed's 3s-timer effect
// captured a render where events/users were still 0, so it ALSO fires a second
// `bootstrapWallet`, doubling every write (events:10 = 5×2) and interleaving
// the two seeds' registerDoc calls. Marking the auto-seed as already-tried at
// the START (before the awaited seed) closes that window: the timer either
// already fired the guard, or its callback bails on `hasTriedAutoSeed.current`.
hasTriedAutoSeed.current = true;
const walletHasData = events.length > 0 || users.length > 0;
const result = await bootstrapWallet(walletHasData, createEntityDoc);
const result = await bootstrapWallet(walletHasData, createEntityDoc, username || undefined);
result.createdDocs.public.forEach(d => registerDoc('public', d));
result.createdDocs.protected.forEach(d => registerDoc('protected', d));
// Re-list AFTER the seed: the seed just wrote the protected scope index, so a
// re-run of the listing effect re-reads those user docs into the read set even
// if the direct `registerDoc` state updates were lost to a racing render.
relist();
return result;
}, [events.length, users.length, registerDoc]);
}, [events.length, users.length, registerDoc, relist, username]);
return {
currentUserId, currentUser,
src/shared/data/entityWrites.ts
+18 -7
View File
@@ -91,14 +91,21 @@ export async function updateEntityField(
term: EntityTerm,
): Promise<void> {
const sid = (await sessionPromise).session_id;
const g = assertNuri(graphNuri);
const s = assertNuri(subject);
const pred = `${FP}${field}`;
const obj = renderTerm(term);
const del = `DELETE WHERE { GRAPH <${g}> { <${s}> <${pred}> ?o } }`;
// NO explicit `GRAPH <${graphNuri}>` wrapper: anchored to `graphNuri`, both the
// DELETE and the INSERT target that repo's DEFAULT graph — the exact graph the
// anchored default-graph READ queries (read-model.ts readDoc). An explicit
// `GRAPH <plainNuri>` body writes into a NAMED graph the anchored read never
// sees, so the mutation would not round-trip (same fix as writeEntity / the
// lib's inbox.post). `assertNuri(graphNuri)` is still done implicitly by
// `docs.sparqlUpdate`'s anchor handling — validate `subject` here as it lands
// in an IRI position.
const del = `DELETE WHERE { <${s}> <${pred}> ?o }`;
await docs.sparqlUpdate(sid, del, graphNuri);
if (obj !== null) {
const ins = `INSERT DATA { GRAPH <${g}> { <${s}> <${pred}> ${obj} } }`;
const ins = `INSERT DATA { <${s}> <${pred}> ${obj} }`;
await docs.sparqlUpdate(sid, ins, graphNuri);
}
}
@@ -117,7 +124,6 @@ export async function writeEntity(
fields: Record<string, EntityTerm>,
): Promise<string> {
const sid = (await sessionPromise).session_id;
const g = assertNuri(graphNuri);
// The entity IS its own document (one document per entity), so its subject IRI
// is the DOCUMENT NURI itself (a `did:ng:…`). This gives the ORM a `did:ng:`
// `@id` (what the @data assertions expect) and makes the entity self-addressing.
@@ -128,11 +134,16 @@ export async function writeEntity(
if (obj === null) continue;
triples.push(`<${FP}${field}> ${obj}`);
}
// NO explicit `GRAPH <${g}>` wrapper: anchored to `graphNuri`, the write lands in
// that repo's DEFAULT graph — the exact graph the anchored default-graph READ
// queries (read-model.ts readDoc). An explicit `GRAPH <plainNuri>` body instead
// writes into a NAMED graph distinct from the repo's default graph, which the
// anchored default-graph read never sees (the old anchorless `GRAPH ?g` scan did,
// which is why it worked before the read switched to per-doc anchored). Same shape
// as the lib's inbox.post / entity writes: anchor scopes the write, no GRAPH clause.
const update = `
INSERT DATA {
GRAPH <${g}> {
<${assertNuri(subject)}> ${triples.join(' ;\n ')} .
}
<${assertNuri(subject)}> ${triples.join(' ;\n ')} .
}`;
await docs.sparqlUpdate(sid, update, graphNuri);
return subject;
src/shared/data/registration.ts
+40 -37
View File
@@ -203,17 +203,20 @@ async function countParticipations(
eventId: string,
userId: string,
): Promise<number> {
const g = assertNuri(graphNuri);
const evL = escapeLiteral(eventId);
const usL = escapeLiteral(userId);
// NO explicit `GRAPH <${graphNuri}>` wrapper: participations are written by
// `writeEntity` into the anchored DEFAULT graph (one doc per entity), so this
// count MUST read that same default graph — anchored to `graphNuri`, with no
// `GRAPH` clause. An explicit `GRAPH <plainNuri>` body reads a NAMED graph the
// writes never land in → always 0 (the graph-mismatch bug — same fix as
// writeEntity/updateEntityField and the lib's read-model/inbox).
const query = `
SELECT (COUNT(DISTINCT ?s) AS ?n) WHERE {
GRAPH <${g}> {
?s a <${P.partType}> ;
<${P.partEvent}> ?event ;
<${P.partUser}> ?user .
FILTER( STR(?event) = "${evL}" && STR(?user) = "${usL}" )
}
?s a <${P.partType}> ;
<${P.partEvent}> ?event ;
<${P.partUser}> ?user .
FILTER( STR(?event) = "${evL}" && STR(?user) = "${usL}" )
}`;
const result = await docs.sparqlQuery(sid, query, undefined, graphNuri);
// Tolerant binding extraction (mirrors the lib's readBindings shape).
@@ -261,7 +264,6 @@ export async function deleteParticipation(
subjectIri?: string,
): Promise<DeleteParticipationResult> {
const sid = (await sessionPromise).session_id;
const g = assertNuri(graphNuri);
// The ORM batches `ngSet.add` into a microtask + the broker needs a moment to
// land it in the SPARQL-queryable graph. A leave that follows a join tightly
// (tests; a fast user) can reach here BEFORE the join's write is queryable —
@@ -292,21 +294,23 @@ export async function deleteParticipation(
const usIri = escapeIri(userId);
const evL = escapeLiteral(eventId);
const usL = escapeLiteral(userId);
// NO explicit `GRAPH <${graphNuri}>` wrapper: participations live in the
// anchored DEFAULT graph (writeEntity), so the sweep must DELETE from that same
// default graph — anchored to `graphNuri`, no `GRAPH` clause. Deleting from an
// explicit `GRAPH <plainNuri>` named graph would no-op (the triples aren't
// there), silently leaving the participation → the F2 resurrection. Same fix as
// countParticipations / writeEntity.
const sweep = `
DELETE {
GRAPH <${g}> { ?s ?p ?o }
}
DELETE { ?s ?p ?o }
WHERE {
GRAPH <${g}> {
?s a <${P.partType}> ;
<${P.partEvent}> ?event ;
<${P.partUser}> ?user ;
?p ?o .
FILTER(
( sameTerm(?event, <${evIri}>) || STR(?event) = "${evL}" ) &&
( sameTerm(?user, <${usIri}>) || STR(?user) = "${usL}" )
)
}
?s a <${P.partType}> ;
<${P.partEvent}> ?event ;
<${P.partUser}> ?user ;
?p ?o .
FILTER(
( sameTerm(?event, <${evIri}>) || STR(?event) = "${evL}" ) &&
( sameTerm(?user, <${usIri}>) || STR(?user) = "${usL}" )
)
}`;
await docs.sparqlUpdate(sid, sweep, graphNuri);
@@ -316,13 +320,10 @@ export async function deleteParticipation(
// bound as an IRI, cannot no-op on drift.
if (hasSubject) {
const s = assertNuri(subjectIri!);
// Anchored default-graph (no `GRAPH` clause), like the sweep above.
const bySubject = `
DELETE {
GRAPH <${g}> { <${s}> ?p ?o }
}
WHERE {
GRAPH <${g}> { <${s}> ?p ?o }
}`;
DELETE { <${s}> ?p ?o }
WHERE { <${s}> ?p ?o }`;
await docs.sparqlUpdate(sid, bySubject, graphNuri);
}
@@ -342,25 +343,27 @@ export async function insertNotification(
notif: Omit<FpNotificationData, 'id'>,
): Promise<string> {
const sid = (await sessionPromise).session_id;
const g = assertNuri(graphNuri);
const subject = `urn:festipod:notif:${Date.now()}:${Math.random().toString(36).slice(2)}`;
// recipient/ref are bare domain ids ("user-1", "event-1"), not absolute IRIs;
// store them as string literals to keep the INSERT valid (the raw shape read
// is not the primary surfacing path — the inbox read is). Every literal is
// escaped via the lib's escapeLiteral (guards \ " \n \r \t — SPARQL injection).
const refTriple = notif.ref ? `\n <${P.ref}> "${escapeLiteral(notif.ref)}" ;` : '';
const refTriple = notif.ref ? `\n <${P.ref}> "${escapeLiteral(notif.ref)}" ;` : '';
const payloadTriple = notif.payload
? `\n <${P.payload}> "${escapeLiteral(notif.payload)}" ;`
? `\n <${P.payload}> "${escapeLiteral(notif.payload)}" ;`
: '';
// NO explicit `GRAPH <${graphNuri}>` wrapper: anchored to `graphNuri`, the
// INSERT lands in that repo's DEFAULT graph — consistent with every other
// per-entity write (writeEntity / updateEntity / the lib's inbox.post). An
// explicit `GRAPH <plainNuri>` body targets a phantom named graph that no
// anchored default-graph read ever sees (graph-mismatch bug).
const update = `
INSERT DATA {
GRAPH <${g}> {
<${assertNuri(subject)}> a <${NOTIF_TYPE_IRI}> ;
<${P.recipient}> "${escapeLiteral(notif.recipientId)}" ;
<${P.type}> "${escapeLiteral(notif.type)}" ;${refTriple}${payloadTriple}
<${P.timestamp}> "${escapeLiteral(notif.timestamp)}" ;
<${P.isRead}> "${notif.isRead}" .
}
<${assertNuri(subject)}> a <${NOTIF_TYPE_IRI}> ;
<${P.recipient}> "${escapeLiteral(notif.recipientId)}" ;
<${P.type}> "${escapeLiteral(notif.type)}" ;${refTriple}${payloadTriple}
<${P.timestamp}> "${escapeLiteral(notif.timestamp)}" ;
<${P.isRead}> "${notif.isRead}" .
}`;
await docs.sparqlUpdate(sid, update, graphNuri);
return subject;
src/shared/test-harness/harness-ng.tsx
+13 -6
View File
@@ -310,14 +310,16 @@ function ConnectedHarness() {
const protectedDocs = await reg.listEntityDocs('protected');
let total = 0;
for (const g of protectedDocs) {
// Anchored default-graph (no `GRAPH` clause): participations are
// written by writeEntity into each doc's DEFAULT graph, so the
// authoritative count must read that same graph (matches the app's
// registration.ts countParticipations after the graph-mismatch fix).
const query = `
SELECT (COUNT(DISTINCT ?s) AS ?n) WHERE {
GRAPH <${g}> {
?s a <http://festipod.org/Participation> ;
<http://festipod.org/event> ?event ;
<http://festipod.org/user> ?user .
FILTER( STR(?event) = "${esc(eventId)}" && STR(?user) = "${esc(userId)}" )
}
?s a <http://festipod.org/Participation> ;
<http://festipod.org/event> ?event ;
<http://festipod.org/user> ?user .
FILTER( STR(?event) = "${esc(eventId)}" && STR(?user) = "${esc(userId)}" )
}`;
const result: any = await docs.sparqlQuery(session.session_id, query, undefined, g);
const rows = Array.isArray(result) ? result : result?.results?.bindings ?? [];
@@ -569,15 +571,20 @@ function ConnectedHarness() {
const reg = await import('../utils/storeRegistry');
const disc = await import('../data/discovery');
reg.resetRegistryCache();
console.error('[PROBE] publishPublicEventAs: ensureAccount(publisher)…');
await reg.ensureAccount(publisher);
console.error('[PROBE] publishPublicEventAs: createEntityDoc(publisher,public)…');
const doc = await reg.createEntityDoc(publisher, 'public');
console.error('[PROBE] publishPublicEventAs: publisher doc=' + doc);
// Deposit AS the current identity: the inbox guard binds `from` to the
// CURRENT user and rejects a spoofed `from`. So make the publisher the
// current identity (its normalized-username key = the cap-owner key),
// then submit WITHOUT a spoofed explicit `from` — the SDK stamps the
// current identity itself (anonymous submission also allowed).
setCurrentUser(normalizeUsername(publisher));
console.error('[PROBE] publishPublicEventAs: submitEventToIndex…');
await disc.submitEventToIndex({ doc, id: doc, title }, getCurrentUser());
console.error('[PROBE] publishPublicEventAs: submitted OK');
return { doc };
},
async discoverPublicEventsAs(discoverer: string) {
src/shared/utils/ngBootstrap.ts
+15 -7
View File
@@ -44,6 +44,7 @@ export interface BootstrapResult {
export async function bootstrapWallet(
walletHasData: boolean,
createEntityDoc: CreateEntityDoc,
owner?: string,
): Promise<BootstrapResult> {
const createdDocs = { public: [] as string[], protected: [] as string[] };
// Already has data → returning user, nothing to seed
@@ -54,13 +55,20 @@ export async function bootstrapWallet(
console.log('[Bootstrap] First time for this wallet — seeding per-entity docs...');
// OWNER: all seed entities are owned by the SINGLE seed owner account (the
// perceived-login user). The seed users are FIXTURES, not real login accounts —
// minting a full owner account per seed user would be dozens of
// broker round-trips (unusably slow against the real broker) with no product
// meaning. One account owns them; each entity is still ITS OWN document (the
// model's per-document isolation is unchanged — only the cap OWNER is shared).
const seedOwner = seedUsers[0] ? normalizeUsername(seedUsers[0].username) : 'seed';
// OWNER: all seed entities are owned by the SINGLE seed owner account — the
// CURRENT logged-in account (`owner`), so "load test data into MY wallet" makes
// the current user the owner. This matters for PROTECTED entities (seed user
// profiles, participations): per-document isolation grants a protected doc's
// ReadCap to its OWNER (+ connections), so if the seed owned them as someone
// ELSE (e.g. the fixture's `mariedupont`) they'd be correctly HIDDEN from the
// current fresh-scenario user and never round-trip. Owning them as the current
// user makes them readable. PUBLIC events are world-readable regardless of owner.
// The seed users are FIXTURES, not real login accounts — minting a full owner
// account per seed user would be dozens of broker round-trips (unusably slow)
// with no product meaning. One account (the current user) owns them; each entity
// is still ITS OWN document (per-document isolation unchanged — only the cap
// OWNER is shared). Falls back to the fixture username when no login is present.
const seedOwner = owner ?? (seedUsers[0] ? normalizeUsername(seedUsers[0].username) : 'seed');
// SEED FOOTPRINT (perf). Each entity is its OWN document, and each `docCreate`
// is a SERIAL ~2s broker round-trip (the verifier serializes creations — they