festipod/.project/handoff/refine-provider-abstraction.md

subject, created, status

subject	created	status
Refine as Festipod's provider-swap abstraction layer	2026-10-08	open

Refine as the provider-swap abstraction layer

Where the work lives

Branch refine-backends, worktree /home/sylvain/projects/festipod/festipod/.claude/worktrees/refine-backends, pushed to origin; the tip records the T04 tree close plus the adversarial-review round — ## Current gate state below names the states by date and tip.

It is based on ng-eventually at c7cda38 — the most advanced branch, and the reason this branch exists. An earlier attempt, festipod-refine (tip 352d1f1, worktree .claude/worktrees/festipod-64), was built on main, which turned out to be far behind; the Refine work was transferred here and that branch is now only an archive. It holds the only worked implementation of a @ng-org/refine-providers backend (src/shared/refine/providers/nextgraph.ts, in its history up to the commit that withdrew it) — the reference for the shape of the next backend.

The main checkout /home/sylvain/projects/festipod/festipod is itself on ng-eventually. Do not modify it. Note that the concept hook reads it, so it announces leaves that may not exist in whichever worktree you are in.

The goal

Put all of Festipod's persistence behind Refine so that several provider sets can serve the same app. ActivityPods is the default and the sole priority — directive of 2026-10-09 (recorded in data-layer/brief_2026-10-09_activitypods-first-backend): everything that can goes through @activitypods/refine-providers, and a backend for actions on behalf of a user is to be added. A Refine provider over NextGraph plus its polyfill remains the declared second target, but its development is suspended, temporarily — no work goes into it while ActivityPods is wired.

Directives of 2026-10-09 (resumption) — and first reconnaissance

Recorded in data-layer/brief_2026-10-09_activitypods-first-backend; the essentials and what the reconnaissance (three read-only passes over the user-designated reference repos) established:

• The on-behalf mechanism is mapped. An app backend IS an ActivityPub actor (@activitypods/app AppService, Moleculer). It signs pod requests with HTTP signatures as the app actor; the pod provider checks the app's registration and grants, then impersonates the user (pod-provider/backend/middlewares/app-control.js). Services call pod-outbox.post, pod-resources.*, pod-collections.*, pod-wac-groups.*, pod-permissions.* with an actorUri. Frontend calls use the Solid-OIDC dual-identity token instead (webId = app client_id, impersonatedUser = user). Reference implementations: welcometomyplace/backend/services/registration.service.js, attendees-matcher.service.js; both example apps run the same stack (@activitypods/app 2.3 + @semapps/* 1.2, Moleculer backend + Vite/React/Refine frontend, Docker-deployed).
• The refine-providers package is NOT in the platform monorepo — it lives at /home/sylvain/projects/activitypods/refine-providers (a /home/sylvain/projects/nextgraph/activitypodes-refine-providers checkout also exists; the pinned reconstruction remains the work-time specification).
• Shapes: 15 shape trees published at https://shapes.activitypods.org from /home/sylvain/projects/activitypods/shapes (repo evolvable by us). The 2026-10-09 directives amended the first mapping proposal — minimise new shapes, interoperability first: a MeetingPoint is a plain as/Event carrying an as:context link to its parent event (event within an event; the as/Event shape is open — no sh:closed — so nothing breaks and no new shape is needed). For Participation the existing vocabulary was surveyed: see the registration precedent in the next bullet.
• Registration/Participation — production precedent found. welcometomyplace persists no registration resource: join = as:Join activity in the user's outbox, handled by the backend service which adds the actor to the event's apods:attendees collection (PodCollectionsHandlerMixin, attachPredicate); leave = as:Leave; reads via pod-collections.getItems. The one open design question this raises — collection creator-side (this pattern) vs participant-owned participation documents (the 2026-10-03 structure) — is recorded in the brief, to arbitrate before implementation.
• No unattended per-user backend token exists on the platform: server-side per-user actions ride the app-actor signature + grants; the platform-idiomatic channel for user-facing reactions is the app outbox → user inbox (notifications style).

What exists, and what deliberately does not

In src/shared/refine/:

• variant.ts + variant.test.ts — pure resolveVariant(pathname, known): which provider set the URL selects. 8 tests covering the case space, including the one true today (empty registry → every URL is the default).
• providers/types.ts — the contract a backend must satisfy (RefineBackend, RecordBinding), each field derived from a measured encounter rather than anticipated.
• providers/index.ts — the registry. EMPTY on purpose: no backend is implemented yet.
• RefineProvider.tsx — mounts <Refine> over the backend the URL named.

None of it is wired. Nothing imports RefineProvider; FestipodDataContext on this base is untouched and still talks to the polyfill directly. Wiring it means rewriting that context against this base's much-evolved version — that is the step everything else waits on.

In .project/:

• concepts/data-layer/activitypods-providers/contract_activitypods-providers.md — a reconstruction, not a pull: that package publishes no engagement, so this is our own observation, pinned (built_version: 66616484b9…). It is the specification to code against.
• concepts/data-layer/activitypods-providers/usage_festipod.md — our declaration toward them. Its Frictions/Needs are the channel to the provider.
• concepts/data-layer/brief_2026-10-03_structure-for-two-rights-models.md — the data-shape design, below.
• concepts/data-layer/brief_2026-10-09_activitypods-first-backend.md — the 2026-10-09 directives (ActivityPods priority, on-behalf backend, NextGraph suspended) and the reference material; the brief this hand-off's resumption added.
• handoff/refine-provider-abstraction.md — this document.
• contracts.yaml — publish: (both declarations, keys qualified by interface) and three consume: entries (polyfill-surface, indexing-layer, activitypods-providers).
• contracts.local.yaml — gitignored, per-worktree. Must exist for contracts.py check to resolve the ActivityPods entry.

Decisions, and why

One app variant per provider set, selected by a URL path prefix. The default at the root, another behind a prefix naming it; switching is a plain link and costs a full page reload. The reload is the mechanism, not a cost: it discards the previous backend's Refine cache and session with no code to undo them, and losing access to the left backend's data is accepted. An earlier plan for in-session hot switching was dropped. The deep reason: the sign-in experience is not abstractable — the NextGraph SDK mounts its own full-screen barrier and owns the whole wallet experience (src/app/AuthGate.tsx renders no access screen), while a Solid backend hands the app an OIDC flow it must render itself (Pod-provider picker, callback route). One confiscates the screen, the other delegates a flow.

The prefix is the backend's name, so a backend must never be named after a top-level route (events, profile, home, settings, users).

The router does not yet strip the prefix, deliberately. parsePath/navigate in src/app/router.tsx are a shared primitive every screen depends on; the suites that would prove a change there are not run in this worktree; and while one backend is registered no URL carries a prefix, so nothing is broken. That half lands with the second backend, when it becomes testable.

The data shape: references, not embedded content. Three levels — Event (own document, public, written by its creator, carrying an attached list of references); Participation (own document, owned by the participant, carrying event, active, participant); Profile (own document, protected). Three facts force it: the participation must be writable by its participant and nobody else, and NextGraph grants write per document; per-reader variation is impossible inside one document where encryption is per document, so it is obtained by indirection (the reference is always present, the ability to follow it varies); and rule_document-per-entity already says this for that reason.

The non-visibility mechanism differs and both land on the same application state — a reference one may or may not resolve. On NextGraph the target is undecipherable for want of a key (nothing is absent). On ActivityPods it is refused by WAC, or omitted by server-side projection. So the domain type models the NextGraph shape, where everything is present: active and participant are not optional; only resolution fails. (Proposing them optional was an error — it lifted an ActivityPods adaptation into the common model.)

No stored counter, ever. The participant count is participations.length. knownParticipants is the participations whose participant resolves — a presence test, not an access check. Withdrawal is active: false on one's own document.

NextGraph is the common denominator. Its model is the strict subset: no WAC groups, no revocation, sharing wholesale per document, but writes work offline. Structure for it and ActivityPods serves the structure, more cheaply even. The converse fails: anything needing a server projection or a revocation is untenable on NextGraph.

The Refine boundary rule. Refine's DataProvider is flat CRUD (getList, getMany?, getOne, create, createMany?, update, updateMany?, deleteOne, deleteMany?, getApiUrl, custom?) — no relation or join concept at all, and no getManyReference. Everything that makes Festipod social (append to someone else's collection, grant/revoke read, discovery, offline queueing) is outside its vocabulary. Since cache, live subscription and optimistic update are bought per resource, the rule is: anything the app displays and that changes must be a Refine resource; anything one-shot and invisible may live in the seam. A write passed beside Refine does not invalidate its cache, so the list would not refresh.

Refine does support several data providers at once (Resource.dataProviderName, useList({dataProviderName})). Deliberately unused for switching, but it is the tool for reading one entity from another backend (discovery from a shared index) or for migration.

The seam should shrink. types.ts was written from measurements against one raw provider that could not be changed. Since we can contribute to the providers, several of its fields are symptoms rather than requirements: updateReplaces + toPatch should vanish (a provider exposes one semantic), and recordKey vs domainId should collapse to id (ActivityPods already does). toDomain/toValues shrink to vocabulary mapping; visibility and resource stay — visibility is a product declaration, not a transport fact. The measure of success is how small RecordBinding gets. Capability differences belong inside the provider, never in the app.

What is known about @activitypods/refine-providers

The reconstruction is the specification. Its source was read once, deliberately, at adoption time (that is what a reconstruction is); it is not to be read again at work time — a gap goes into the declaration's Frictions/Needs. Checkouts, if ever needed again: the package at /home/sylvain/projects/nextgraph/activitypodes-refine-providers, the platform at /home/sylvain/projects/activitypods/activitypods.

• A record's id is the LDP resource URI, and it is the only identifier — getOne/update/deleteOne use it directly as the URL. The seam's two identifiers collapse into one here.
• update fetches the current resource, strips id/@context, and PUTs {...current, ...variables} — so a partial write is safe, the merge being shallow.
• getList resolves the resource's container from the session's own WebID (public Solid TypeIndex, best-effort private), so it returns only what the current user holds. getOne/getMany fetch a URI directly and so do cross Pods — they just need the URI.
• No server-side filtering, sorting or pagination: the whole container is fetched, then filtered in memory.
• getMany silently drops failed fetches (allSettled then filter) — it erases exactly the "unreadable reference" state the model needs.
• Module-level caches (shape trees, type registrations per WebID, subscription endpoints) are never invalidated.
• authProvider installs a setInterval and a visibilitychange listener at construction with no teardown; the only opt-out is appStatusCheckInterval: false.
• Live updates are real push (notify:WebSocketChannel2023), but a list subscription reports membership only (Add/Remove), never a member's field change.
• types takes precedence over shapeTreeUri when both are set — the opposite of what the configuration type documents.
• custom is not implemented, so there is no escape hatch through Refine for non-CRUD operations.
• Collections are POSTable LDP resources (/as/collection) with semapps:dereferenceItems — "all items will be dereferenced (if permissions allow)". That is how event.participations can come back as full objects for a reader who is allowed, and as URIs otherwise, in one read. Items are added/removed by SparqlPatch on as:items.
• acl:Append exists among the WAC modes and is really used in app access needs. Whether it lets a third party insert an item without being able to remove one is NOT VERIFIED — enforcement lives in @semapps/webacl, absent from the checkout. The structure does not depend on the answer: if it works that way the participant appends directly; otherwise the route is apods:PostOutbox to the owner's inbox, which is exactly the NextGraph shape, and only latency changes.
• An app declares accessNeeds (shapeTreeUri + accessMode) plus special needs: apods:CreateCollection, apods:CreateWacGroup, apods:PostOutbox, apods:QuerySparqlEndpoint, apods:UpdateWebId, inbox/outbox reads.
• Backend-less apps are supported: onlineBackend is false only when the app description document at the clientId URI is not fetchable. A static file suffices.
• A custom ontology prefix must be registered at prefix.cc (the Pod provider requires every ontology to have a known prefix) — this concerns Festipod's fp:. Shape trees must sit at a public URL, one per entity, and cannot reference other shapes (no sharing cascade).

Four contributions this points to (we may contribute to these providers): getMany must signal the unreadable instead of swallowing it; an append path into a collection one does not own; collection support as a resource with dereferenceItems; and a teardown for the auth provider's timer.

The deduplication question — answered, and still open

The attendance model (brief_2026-07-20_attendance-set-model, PO 2026-07-27) requires the creator to tell two references from the same person apart from two from different people without knowing who, or the count is not a count of people and a participant can inflate it.

Checked on this base: not solved. A different problem is, and well. materializeAttendance (src/shared/data/registration.ts) keys its active set by a per-deposit uid minted from time plus randomness, so a broker re-sync replaying the same deposits converges — that is replay idempotence, not person identity. The count counts deposits: the same person joining twice counts twice. Further: the plaintext userId travels in the deposit and the cancellation fallback depends on it (a leave without a known regUid cancels the earliest still-active join for that userId), so the creator does know who and the anonymity the model rests on does not hold. Participations are in the protected scope rather than public, and withdrawal is still a deletion. And the gap was never raised through the channel meant for it.

So both platforms count deposits rather than people, and the gap sits in neither Refine provider — polyfill on one side, server on the other. ActivityPods could arbitrate server-side because the server sees everything.

Problems hit, and how they turned out

• Two false claims were written into durable records and corrected. First, that the participant count needs an inbox and that a CRUD-only backend therefore could not carry the model — wrong: every participation is readable by everyone, only the link to a person is protected, so counting is ordinary CRUD. Second, a declared "need" for a WebID filter on getList — Festipod does not need it: getOne/getMany already cross Pods, the participations arrive with the event, and discovery is an index problem (already decided on this base: decision_2026-08-17_discovery-through-a-shared-index), not a provider one. Both corrections are in the files.
• __FESTIPOD_BACKEND__ was a lie in a comment claiming build.ts defined it. Nothing did, and nothing could: this project serves from src/ in production too (bun run start), so build.ts's define block never reaches what runs (tech-stack, knowledge_build-pipeline). The runtime mechanism this base already uses instead is a config route — /festipod-config.json served by src/index.ts, read by src/app/frontend.tsx before it imports App. Replaced by the URL read.
• Three separate times the answer was already on ng-eventually (the runtime config route, the delegation of wallet access to the SDK, the dedup implementation). That is what triggered rebasing onto it.
• The migration was delegated to a subagent and verified independently afterwards: the five transferred files are byte-identical, the manifest merge preserved both pre-existing entries, and the gates were re-run by hand. Its one worry — that a missing local node_modules had made the baseline incomparable — was unfounded (pnpm had installed the full tree locally; 22 entries against the main checkout's 21). Its debt block was better than the specification it was given: it caught the __FESTIPOD_BACKEND__ consequence and two branch-relative phrasings.
• Moving base falsified three claims in the transferred code, all corrected: the inbox the count comment points at is local here; the worked provider implementation is not in this branch's ancestry; and the MeetingPoint SHEX shape does exist here, so its binding stopped being optional (a backend that cannot carry one of the four entities cannot serve the app).

Current gate state (2026-10-10, adversarial-review round closed, tip = the commit recording this state)

Verified by the orchestrator's own runs after the round's fixes: bunx tsc --noEmit exit 0 ; bun test 96 pass / 0 fail ; the UI cucumber layer 125 scenarios / 0 failed (54 fully implemented + 71 Scénario non implémenté placeholders that carry a skipped step by design) ; smoke @e2e-activitypods 3/3 against the live local stack (pod provider :3000, festipod backend :3003, app served on an ephemeral port) ; bun run build.ts ok ; contracts.py check --integrity-only clean.

The round: three adversarial passes (auth & security, data seam & backend, gates & ops); every confirmed finding was fixed in the commit recording this state. Fix themes — auth: redirect param validated (same-origin leading-slash only), callback re-entry guarded against StrictMode double-exchange, stale ?code= URLs degrade to the picker instead of the fatal screen, register_app without a session falls back to the picker (was a permanent blank screen), ActivityPods logout wired through the provider, deep links survive login, clientId trailing slash normalized. Data fidelity: the meeting point's host is read from the Pod-stamped dc:creator (the organizer term is absent from the platform context and never round-tripped, so the Join lost its addressee), location/city are written as inline named nodes (as:location is @type: @id; a free-text string resolved into a garbage IRI), record ids are URL-encoded (absolute IRIs broke the router's /events/:id parse), participant resolution is keyed by actor URI (profiles were stored under their document URI and never matched), one delayed re-invalidation reconciles the count after join/leave, a Join/Leave without addressee throws instead of posting to no one, anonymous placeholder rows render on the event detail screen. Backend & ops: the attendees collection gets a public acl:Read grant in after.createAndAttach (the count is readable by everyone; anonymity lives in the profiles' own WAC), Join/Leave handlers retry while the collection is still attaching, tsx promoted to dependencies so the --prod image starts, compose env_file made optional with fuseki/redis services added and the frontend published on :4000 (the SEMAPPS_FRONT_URL default — fixes the shipped redirect_uri mismatch; :5000 belongs to the pod provider frontend and is deliberately avoided), test:data removed (zero matching scenarios since the suspension — a vacuous green), strict: true on both cucumber configs.

contracts.py check also reports, pre-existing: polyfill-surface: provider moved: a33fb8a214 → a8d53010c2 — the pulled copy is behind the pinned ref.

Open points

• Adding ## Needs to the polyfill declaration is where the per-person discriminator belongs. It is outward-facing (the provider reads it on its next pull), so it awaits the user's go.
• acl:Append behaviour, as above.
• The router's prefix stripping, with the second backend.
• The four provider contributions.
• Wiring the seam: DONE 2026-10-09 (T04 tree, tip f8c2e3b) — FestipodDataContext runs over Refine with the ActivityPods backend as the registered default; the join→collection loop is built but @suspendu until the app is installed on a pod (consent screen). Current state: ## Current gate state above. The first half of the app-install blocker is CLEARED 2026-10-12 (tip 3aa9131): the pod provider's OIDC adapter fetches the client_id URI and requires a Solid-OIDC Client ID Document (@context with solid/oidc-context.jsonld, client_id equal to the URI, plain redirect_uris); GET /app now serves exactly that when the request carries the JsonLdContext header, and keeps serving the app actor otherwise (the SAI install flow fetches the same URL without it). What remains on follow-up #1: a second, deeper drift surfaced on the first real consent (tip 6b48b24) — @activitypods/app 2.3.2's app-registrations.verify expects interop:satisfiesAccessNeed on the AccessGrant, while the local pod provider HEAD writes it on the DataGrants (hasDataGrant), so every AppRegistration notification was rejected with "One or more required access needs have not been granted" and the consent looped; a backend middleware shim (backend/services/apods-verify-compat.service.js) reimplements verify resolving through DataGrants, the drift itself is platform-side and reported. After a user-side reinstall (the pod notifies once per consent and the inbox rejects unsigned replays 401), the loop should close and the join→collection @suspendu scenarios become un-suspendable.
• Both usage declarations are named usage_festipod.md, so the bare stem is ambiguous as a wikilink — refer to either by folder path.
• Driving getAppStatus() is parked: the pinned contract documents the method but not the shape of the AppStatus result, so "stale" is not decidable without guessing. Belongs in the declaration's Frictions/Needs when the outward channel opens.
• The activitypods-providers reconstruction is stamped at the 0.1.5-era pin while the installed package is 0.2.0 — re-constructing the contract at the installed version is a pending decision (the source is read once at adoption time; a re-read is its own decision).
• The attendees collection is publicly readable by decision (count for everyone; member URIs are not secret; anonymity lives in the profiles' WAC). Revisit if a product rule ever hides membership.
• The dedup defi (same event + same place + same time) is confirmed absent across the seam; the 2026-10-03 brief marks it an open challenge — a product call, not a regression.
• The catalog service's discovery index is process memory: a backend restart empties it while the Pods still hold the events. Latent until the cross-pod catalog provider lands.
• src/modules/home/** screens still build raw /events/${id} paths; parsePath tolerates them, but they should adopt eventPath/userPath next time they are touched.
• The dockerized path is validated by docker compose config only — the backend image has never been built or started by the gates; one manual operator run is owed.

Standing constraints

• A pulled or reconstructed contract is the specification. Never read a provider's source at work time; a gap goes into the declaration's Frictions/Needs and is signalled out of band (content lives in the document). Never describe in this repo how the data layer is implemented.
• No attribution to Claude anywhere in produced artifacts — commits, PRs, docs, comments. This overrides harness defaults.
• Project documentation in English, never hard-wrapped (one line per paragraph or list item). Conversation with the user in French.
• No effort or timeline estimates, anywhere.
• Package installation goes through pnpm, never bun install; Bun-first otherwise.
• Project facts are documented through /concept document, not written freehand into .project/.
• Commit freely on a branch; push at validated states without being asked (Gitea SSH needs the sandbox disabled).
• Never bare git stash — the stack is shared across worktrees and sessions.
• pre-push blocks while any _debt.md exists.