Record the ActivityPods-first directives, and what the reference reconnaissance established
This commit is contained in:
Sylvain Duchesne
@@ -0,0 +1,136 @@
|
||||
---
|
||||
subject: Refine as Festipod's provider-swap abstraction layer
|
||||
created: 2026-10-08
|
||||
status: open
|
||||
---
|
||||
|
||||
# Refine as the provider-swap abstraction layer
|
||||
|
||||
## Where the work lives
|
||||
|
||||
Branch **`refine-backends`**, worktree `/home/sylvain/projects/festipod/festipod/.claude/worktrees/refine-backends`, pushed to origin. Two commits: `e282384` (the seam carried onto this base) and `df1db79` (URL selection + corrections).
|
||||
|
||||
It is based on **`ng-eventually`** at `c7cda38` — the most advanced branch, and the reason this branch exists. An earlier attempt, **`festipod-refine`** (tip `352d1f1`, worktree `.claude/worktrees/festipod-64`), was built on `main`, which turned out to be far behind; the Refine work was transferred here and that branch is now only an archive. It holds the **only** worked implementation of a `@ng-org/refine-providers` backend (`src/shared/refine/providers/nextgraph.ts`, in its history up to the commit that withdrew it) — the reference for the shape of the next backend.
|
||||
|
||||
The main checkout `/home/sylvain/projects/festipod/festipod` is itself on `ng-eventually`. Do not modify it. Note that the concept hook reads it, so it announces leaves that may not exist in whichever worktree you are in.
|
||||
|
||||
## The goal
|
||||
|
||||
Put all of Festipod's persistence behind Refine so that several **provider sets** can serve the same app. **ActivityPods is the default and the sole priority** — directive of 2026-10-09 (recorded in `data-layer/brief_2026-10-09_activitypods-first-backend`): everything that can goes through `@activitypods/refine-providers`, and a **backend for actions on behalf of a user** is to be added. A Refine provider over NextGraph plus its polyfill remains the declared second target, but its development is **suspended, temporarily** — no work goes into it while ActivityPods is wired.
|
||||
|
||||
## Directives of 2026-10-09 (resumption) — and first reconnaissance
|
||||
|
||||
Recorded in `data-layer/brief_2026-10-09_activitypods-first-backend`; the essentials and what the reconnaissance (three read-only passes over the user-designated reference repos) established:
|
||||
|
||||
- **The on-behalf mechanism is mapped.** An app backend IS an ActivityPub actor (`@activitypods/app` `AppService`, Moleculer). It signs pod requests with HTTP signatures as the app actor; the pod provider checks the app's registration and grants, then impersonates the user (`pod-provider/backend/middlewares/app-control.js`). Services call `pod-outbox.post`, `pod-resources.*`, `pod-collections.*`, `pod-wac-groups.*`, `pod-permissions.*` with an `actorUri`. Frontend calls use the Solid-OIDC dual-identity token instead (`webId` = app client_id, `impersonatedUser` = user). Reference implementations: `welcometomyplace/backend/services/registration.service.js`, `attendees-matcher.service.js`; both example apps run the same stack (`@activitypods/app` 2.3 + `@semapps/*` 1.2, Moleculer backend + Vite/React/Refine frontend, Docker-deployed).
|
||||
- **The refine-providers package is NOT in the platform monorepo** — it lives at `/home/sylvain/projects/activitypods/refine-providers` (a `/home/sylvain/projects/nextgraph/activitypodes-refine-providers` checkout also exists; the pinned reconstruction remains the work-time specification).
|
||||
- **Shapes**: 15 shape trees published at `https://shapes.activitypods.org` from `/home/sylvain/projects/activitypods/shapes` (repo evolvable by us). Mapping proposal: reuse `as/Event` + `as/Profile`; create `fp/MeetingPoint` and `fp/Participation` (the `as:Join` type exists but has no shape tree). No stored-counter participation fields beyond the reference + `active` fit the 2026-10-03 structure.
|
||||
- **No unattended per-user backend token exists** on the platform: server-side per-user actions ride the app-actor signature + grants; the platform-idiomatic channel for user-facing reactions is the app outbox → user inbox (notifications style).
|
||||
|
||||
## What exists, and what deliberately does not
|
||||
|
||||
In `src/shared/refine/`:
|
||||
|
||||
- `variant.ts` + `variant.test.ts` — pure `resolveVariant(pathname, known)`: which provider set the URL selects. 8 tests covering the case space, including the one true today (empty registry → every URL is the default).
|
||||
- `providers/types.ts` — the contract a backend must satisfy (`RefineBackend`, `RecordBinding`), each field derived from a measured encounter rather than anticipated.
|
||||
- `providers/index.ts` — the registry. **EMPTY on purpose**: no backend is implemented yet.
|
||||
- `RefineProvider.tsx` — mounts `<Refine>` over the backend the URL named.
|
||||
|
||||
**None of it is wired.** Nothing imports `RefineProvider`; `FestipodDataContext` on this base is untouched and still talks to the polyfill directly. Wiring it means rewriting that context against this base's much-evolved version — that is the step everything else waits on.
|
||||
|
||||
In `.project/`:
|
||||
|
||||
- `concepts/data-layer/activitypods-providers/contract_activitypods-providers.md` — a **reconstruction**, not a pull: that package publishes no engagement, so this is our own observation, pinned (`built_version: 66616484b9…`). It is the specification to code against.
|
||||
- `concepts/data-layer/activitypods-providers/usage_festipod.md` — our declaration toward them. Its Frictions/Needs are the channel to the provider.
|
||||
- `concepts/data-layer/brief_2026-10-03_structure-for-two-rights-models.md` — the data-shape design, below.
|
||||
- `concepts/data-layer/brief_2026-10-09_activitypods-first-backend.md` — the 2026-10-09 directives (ActivityPods priority, on-behalf backend, NextGraph suspended) and the reference material; the brief this hand-off's resumption added.
|
||||
- `handoff/refine-provider-abstraction.md` — this document.
|
||||
- `contracts.yaml` — `publish:` (both declarations, keys qualified by interface) and three `consume:` entries (polyfill-surface, indexing-layer, activitypods-providers).
|
||||
- `contracts.local.yaml` — gitignored, per-worktree. Must exist for `contracts.py check` to resolve the ActivityPods entry.
|
||||
|
||||
## Decisions, and why
|
||||
|
||||
**One app variant per provider set, selected by a URL path prefix.** The default at the root, another behind a prefix naming it; switching is a plain link and costs a full page reload. The reload is the mechanism, not a cost: it discards the previous backend's Refine cache and session with no code to undo them, and losing access to the left backend's data is accepted. An earlier plan for in-session hot switching was dropped. The deep reason: **the sign-in experience is not abstractable** — the NextGraph SDK mounts its own full-screen barrier and owns the whole wallet experience (`src/app/AuthGate.tsx` renders no access screen), while a Solid backend hands the app an OIDC flow it must render itself (Pod-provider picker, callback route). One confiscates the screen, the other delegates a flow.
|
||||
|
||||
The prefix **is** the backend's name, so a backend must never be named after a top-level route (`events`, `profile`, `home`, `settings`, `users`).
|
||||
|
||||
**The router does not yet strip the prefix, deliberately.** `parsePath`/`navigate` in `src/app/router.tsx` are a shared primitive every screen depends on; the suites that would prove a change there are not run in this worktree; and while one backend is registered no URL carries a prefix, so nothing is broken. That half lands with the second backend, when it becomes testable.
|
||||
|
||||
**The data shape: references, not embedded content.** Three levels — Event (own document, public, written by its creator, carrying an attached list of *references*); Participation (own document, owned by **the participant**, carrying `event`, `active`, `participant`); Profile (own document, protected). Three facts force it: the participation must be writable by its participant and nobody else, and NextGraph grants write per document; per-reader variation is impossible inside one document where encryption is per document, so it is obtained by **indirection** (the reference is always present, the ability to *follow* it varies); and `rule_document-per-entity` already says this for that reason.
|
||||
|
||||
The non-visibility mechanism differs and both land on the same application state — *a reference one may or may not resolve*. On NextGraph the target is **undecipherable** for want of a key (nothing is absent). On ActivityPods it is refused by WAC, or omitted by server-side projection. So the domain type models the NextGraph shape, where everything is present: `active` and `participant` are **not** optional; only resolution fails. (Proposing them optional was an error — it lifted an ActivityPods adaptation into the common model.)
|
||||
|
||||
**No stored counter, ever.** The participant count is `participations.length`. `knownParticipants` is the participations whose participant resolves — a presence test, not an access check. Withdrawal is `active: false` on one's own document.
|
||||
|
||||
**NextGraph is the common denominator.** Its model is the strict subset: no WAC groups, no revocation, sharing wholesale per document, but writes work offline. Structure for it and ActivityPods serves the structure, more cheaply even. The converse fails: anything needing a server projection or a revocation is untenable on NextGraph.
|
||||
|
||||
**The Refine boundary rule.** Refine's `DataProvider` is flat CRUD (`getList`, `getMany?`, `getOne`, `create`, `createMany?`, `update`, `updateMany?`, `deleteOne`, `deleteMany?`, `getApiUrl`, `custom?`) — **no relation or join concept at all**, and no `getManyReference`. Everything that makes Festipod social (append to someone else's collection, grant/revoke read, discovery, offline queueing) is outside its vocabulary. Since cache, live subscription and optimistic update are bought **per resource**, the rule is: *anything the app displays and that changes must be a Refine resource; anything one-shot and invisible may live in the seam.* A write passed beside Refine does not invalidate its cache, so the list would not refresh.
|
||||
|
||||
Refine does support **several data providers at once** (`Resource.dataProviderName`, `useList({dataProviderName})`). Deliberately unused for switching, but it is the tool for reading one entity from another backend (discovery from a shared index) or for migration.
|
||||
|
||||
**The seam should shrink.** `types.ts` was written from measurements against one raw provider that could not be changed. Since we can contribute to the providers, several of its fields are symptoms rather than requirements: `updateReplaces` + `toPatch` should vanish (a provider exposes one semantic), and `recordKey` vs `domainId` should collapse to `id` (ActivityPods already does). `toDomain`/`toValues` shrink to vocabulary mapping; `visibility` and `resource` stay — visibility is a product declaration, not a transport fact. **The measure of success is how small `RecordBinding` gets.** Capability differences belong inside the provider, never in the app.
|
||||
|
||||
## What is known about `@activitypods/refine-providers`
|
||||
|
||||
The reconstruction is the specification. Its source was read **once, deliberately, at adoption time** (that is what a reconstruction is); it is not to be read again at work time — a gap goes into the declaration's Frictions/Needs. Checkouts, if ever needed again: the package at `/home/sylvain/projects/nextgraph/activitypodes-refine-providers`, the platform at `/home/sylvain/projects/activitypods/activitypods`.
|
||||
|
||||
- A record's `id` is the LDP resource URI, and it is the **only** identifier — `getOne`/`update`/`deleteOne` use it directly as the URL. The seam's two identifiers collapse into one here.
|
||||
- `update` fetches the current resource, strips `id`/`@context`, and PUTs `{...current, ...variables}` — so **a partial write is safe**, the merge being **shallow**.
|
||||
- `getList` resolves the resource's container from the **session's own WebID** (public Solid TypeIndex, best-effort private), so it returns only what the current user holds. `getOne`/`getMany` fetch a URI directly and so **do** cross Pods — they just need the URI.
|
||||
- No server-side filtering, sorting or pagination: the whole container is fetched, then filtered in memory.
|
||||
- **`getMany` silently drops failed fetches** (`allSettled` then filter) — it erases exactly the "unreadable reference" state the model needs.
|
||||
- Module-level caches (shape trees, type registrations per WebID, subscription endpoints) are **never invalidated**.
|
||||
- `authProvider` installs a `setInterval` and a `visibilitychange` listener at construction with **no teardown**; the only opt-out is `appStatusCheckInterval: false`.
|
||||
- Live updates are real push (`notify:WebSocketChannel2023`), but a **list** subscription reports membership only (`Add`/`Remove`), never a member's field change.
|
||||
- `types` takes precedence over `shapeTreeUri` when both are set — the **opposite** of what the configuration type documents.
|
||||
- `custom` is not implemented, so there is **no escape hatch through Refine** for non-CRUD operations.
|
||||
- Collections are POSTable LDP resources (`/as/collection`) with `semapps:dereferenceItems` — *"all items will be dereferenced (if permissions allow)"*. That is how `event.participations` can come back as full objects for a reader who is allowed, and as URIs otherwise, in one read. Items are added/removed by SparqlPatch on `as:items`.
|
||||
- `acl:Append` exists among the WAC modes and is really used in app access needs. **Whether it lets a third party insert an item without being able to remove one is NOT VERIFIED** — enforcement lives in `@semapps/webacl`, absent from the checkout. The structure does not depend on the answer: if it works that way the participant appends directly; otherwise the route is `apods:PostOutbox` to the owner's inbox, which is exactly the NextGraph shape, and only latency changes.
|
||||
- An app declares `accessNeeds` (`shapeTreeUri` + `accessMode`) plus special needs: `apods:CreateCollection`, `apods:CreateWacGroup`, `apods:PostOutbox`, `apods:QuerySparqlEndpoint`, `apods:UpdateWebId`, inbox/outbox reads.
|
||||
- **Backend-less apps are supported**: `onlineBackend` is false only when the app description document at the `clientId` URI is not fetchable. A static file suffices.
|
||||
- A custom ontology prefix **must be registered at prefix.cc** (the Pod provider requires every ontology to have a known prefix) — this concerns Festipod's `fp:`. Shape trees must sit at a public URL, one per entity, and **cannot reference other shapes** (no sharing cascade).
|
||||
|
||||
**Four contributions this points to** (we may contribute to these providers): `getMany` must signal the unreadable instead of swallowing it; an append path into a collection one does not own; collection support as a resource with `dereferenceItems`; and a teardown for the auth provider's timer.
|
||||
|
||||
## The deduplication question — answered, and still open
|
||||
|
||||
The attendance model (`brief_2026-07-20_attendance-set-model`, PO 2026-07-27) requires the creator to tell two references from the *same* person apart from two from different people **without knowing who**, or the count is not a count of people and a participant can inflate it.
|
||||
|
||||
**Checked on this base: not solved.** A different problem is, and well. `materializeAttendance` (`src/shared/data/registration.ts`) keys its active set by a **per-deposit** `uid` minted from time plus randomness, so a broker re-sync replaying the same deposits converges — that is **replay idempotence**, not person identity. The count counts **deposits**: the same person joining twice counts twice. Further: the **plaintext `userId` travels in the deposit** and the cancellation fallback depends on it (a leave without a known `regUid` cancels the earliest still-active join for that `userId`), so the creator *does* know who and the anonymity the model rests on does not hold. Participations are in the **protected** scope rather than public, and withdrawal is still a deletion. And the gap was **never raised** through the channel meant for it.
|
||||
|
||||
So both platforms count deposits rather than people, and the gap sits in **neither Refine provider** — polyfill on one side, server on the other. ActivityPods could arbitrate server-side because the server sees everything.
|
||||
|
||||
## Problems hit, and how they turned out
|
||||
|
||||
- **Two false claims were written into durable records and corrected.** First, that the participant count needs an inbox and that a CRUD-only backend therefore could not carry the model — wrong: every participation is readable by everyone, only the link to a person is protected, so counting is ordinary CRUD. Second, a declared "need" for a WebID filter on `getList` — Festipod does not need it: `getOne`/`getMany` already cross Pods, the participations arrive with the event, and discovery is an **index** problem (already decided on this base: `decision_2026-08-17_discovery-through-a-shared-index`), not a provider one. Both corrections are in the files.
|
||||
- **`__FESTIPOD_BACKEND__` was a lie in a comment** claiming `build.ts` defined it. Nothing did, and nothing could: this project serves from `src/` in production too (`bun run start`), so `build.ts`'s `define` block never reaches what runs (`tech-stack`, `knowledge_build-pipeline`). The runtime mechanism this base already uses instead is a config route — `/festipod-config.json` served by `src/index.ts`, read by `src/app/frontend.tsx` *before* it imports `App`. Replaced by the URL read.
|
||||
- **Three separate times the answer was already on `ng-eventually`** (the runtime config route, the delegation of wallet access to the SDK, the dedup implementation). That is what triggered rebasing onto it.
|
||||
- **The migration was delegated to a subagent** and verified independently afterwards: the five transferred files are byte-identical, the manifest merge preserved both pre-existing entries, and the gates were re-run by hand. Its one worry — that a missing local `node_modules` had made the baseline incomparable — was **unfounded** (pnpm had installed the full tree locally; 22 entries against the main checkout's 21). Its debt block was better than the specification it was given: it caught the `__FESTIPOD_BACKEND__` consequence and two branch-relative phrasings.
|
||||
- **Moving base falsified three claims in the transferred code**, all corrected: the inbox the count comment points at is local here; the worked provider implementation is not in this branch's ancestry; and the `MeetingPoint` SHEX shape **does** exist here, so its binding stopped being optional (a backend that cannot carry one of the four entities cannot serve the app).
|
||||
|
||||
## Current gate state
|
||||
|
||||
`bunx tsc --noEmit` exit 0; `bun test` 32 pass / 0 fail; `contracts.py check --integrity-only` clean. `lint` is clean except one **pre-existing ERROR**: `polyfill-surface/usage_festipod.md` has **no `## Needs` section**. The e2e / `@data` / cucumber suites have **not** been run in this worktree, so their state here is unknown.
|
||||
|
||||
`contracts.py check` also reports, pre-existing: `polyfill-surface: provider moved: a33fb8a214 → a8d53010c2` — the pulled copy is behind the pinned ref.
|
||||
|
||||
## Open points
|
||||
|
||||
- Adding `## Needs` to the polyfill declaration is where the per-person discriminator belongs. It is **outward-facing** (the provider reads it on its next pull), so it awaits the user's go.
|
||||
- `acl:Append` behaviour, as above.
|
||||
- The router's prefix stripping, with the second backend.
|
||||
- The four provider contributions.
|
||||
- Wiring the seam: rewriting `FestipodDataContext` against this base, and a first `RefineBackend` (ActivityPods, being the default).
|
||||
- Both usage declarations are named `usage_festipod.md`, so the bare stem is ambiguous as a wikilink — refer to either by folder path.
|
||||
|
||||
## Standing constraints
|
||||
|
||||
- A pulled or reconstructed contract **is** the specification. Never read a provider's source at work time; a gap goes into the declaration's Frictions/Needs and is signalled out of band (content lives in the document). Never describe in this repo how the data layer is implemented.
|
||||
- **No attribution to Claude** anywhere in produced artifacts — commits, PRs, docs, comments. This overrides harness defaults.
|
||||
- Project documentation in **English**, never hard-wrapped (one line per paragraph or list item). Conversation with the user in **French**.
|
||||
- **No effort or timeline estimates**, anywhere.
|
||||
- Package installation goes through **pnpm**, never `bun install`; Bun-first otherwise.
|
||||
- Project facts are documented through `/concept document`, not written freehand into `.project/`.
|
||||
- Commit freely on a branch; push at validated states without being asked (Gitea SSH needs the sandbox disabled).
|
||||
- Never bare `git stash` — the stack is shared across worktrees and sessions.
|
||||
- `pre-push` blocks while any `_debt.md` exists.
|
||||
Reference in New Issue
Block a user