Record the ActivityPods-first directives, and what the reference reconnaissance established
This commit is contained in:
Sylvain Duchesne
@@ -0,0 +1,36 @@
|
||||
---
|
||||
type: brief
|
||||
summary: ActivityPods is the sole priority provider — everything that can goes through the Refine providers, a backend for actions on behalf of a user is to be added, the NextGraph+polyfill backend is suspended (temporary); reference material and open questions recorded 2026-10-09
|
||||
---
|
||||
|
||||
# Brief (2026-10-09) — ActivityPods first, and a backend for on-behalf actions
|
||||
|
||||
Directives recorded at the resumption of the hand-off (`.project/handoff/refine-provider-abstraction.md`), to anchor the ActivityPods integration workstream.
|
||||
|
||||
## Directives
|
||||
|
||||
1. **ActivityPods is the priority.** The Refine backend over NextGraph plus its polyfill is **suspended, temporarily** — it is not abandoned (the two-rights-models structure still targets both), but no work goes into it while ActivityPods is being wired.
|
||||
2. **Everything that can must go through `@activitypods/refine-providers`.** The Refine boundary rule from the hand-off applies: what Refine can express is a Refine resource; what it cannot (collection creation, WAC, groups, outbox posting) happens beside Refine, in the seam or in the new backend. Capability differences stay inside the provider layer, never in the app.
|
||||
3. **A backend must be added for actions "on behalf of a user".** Festipod needs server-side actions executed for the account of a user (the pattern used by the reference apps below). Its mechanism, packages and deployment shape are the first design question of this workstream.
|
||||
|
||||
## Reference material (user-designated 2026-10-09)
|
||||
|
||||
Reading these at this stage is adoption-time reading, explicitly directed — the standing constraint against reading provider sources at work time is unaffected (`@activitypods/refine-providers` remains specified by its pinned reconstruction, `activitypods-providers/contract_activitypods-providers.md`).
|
||||
|
||||
- **Example apps** — `/home/sylvain/projects/reconnexion/welcometomyplace` and `/home/sylvain/projects/reconnexion/mutual-aid.app`: working integrations to mine for provider configuration, resources, and the on-behalf backend pattern.
|
||||
- **ActivityPods platform source** — `/home/sylvain/projects/activitypods/activitypods`; Semapps lives in a sibling folder.
|
||||
- **Shapes** — `/home/sylvain/projects/activitypods/shapes`: readily usable, and **evolvable by us** if Festipod's entities require it (that repo is ours to adapt).
|
||||
|
||||
## Constraints already on record (see the 2026-10-03 brief and the reconstruction)
|
||||
|
||||
- `fp:` must be registered at prefix.cc; shape trees must sit at public URLs, one per entity, no sharing cascade.
|
||||
- The Refine providers cover CRUD only; `custom` is not implemented — no escape hatch through Refine.
|
||||
- `getMany` silently drops failed fetches — it erases the "unreadable reference" state the model needs (a candidate contribution).
|
||||
- `acl:Append` third-party insert-without-remove behaviour is NOT VERIFIED; the structure does not depend on the answer.
|
||||
|
||||
## Open questions
|
||||
|
||||
- The on-behalf mechanism: how a Festipod backend authenticates and acts for a user (tokens? app actor? delegated outbox posting?) — to be mapped from the platform source and the example apps.
|
||||
- Which shapes from the shapes repo map to Festipod's entities (Event, MeetingPoint, Profile, Participation) as-is, and which need to be created or evolved.
|
||||
- The backend's deployment shape (service alongside the app? serverless? a platform app?).
|
||||
- Whether the four pointed-to provider contributions (from the hand-off) are still the right set once the backend exists — some may move from "contribution to the providers" to "our backend's job".
|
||||
@@ -0,0 +1,136 @@
|
||||
---
|
||||
subject: Refine as Festipod's provider-swap abstraction layer
|
||||
created: 2026-10-08
|
||||
status: open
|
||||
---
|
||||
|
||||
# Refine as the provider-swap abstraction layer
|
||||
|
||||
## Where the work lives
|
||||
|
||||
Branch **`refine-backends`**, worktree `/home/sylvain/projects/festipod/festipod/.claude/worktrees/refine-backends`, pushed to origin. Two commits: `e282384` (the seam carried onto this base) and `df1db79` (URL selection + corrections).
|
||||
|
||||
It is based on **`ng-eventually`** at `c7cda38` — the most advanced branch, and the reason this branch exists. An earlier attempt, **`festipod-refine`** (tip `352d1f1`, worktree `.claude/worktrees/festipod-64`), was built on `main`, which turned out to be far behind; the Refine work was transferred here and that branch is now only an archive. It holds the **only** worked implementation of a `@ng-org/refine-providers` backend (`src/shared/refine/providers/nextgraph.ts`, in its history up to the commit that withdrew it) — the reference for the shape of the next backend.
|
||||
|
||||
The main checkout `/home/sylvain/projects/festipod/festipod` is itself on `ng-eventually`. Do not modify it. Note that the concept hook reads it, so it announces leaves that may not exist in whichever worktree you are in.
|
||||
|
||||
## The goal
|
||||
|
||||
Put all of Festipod's persistence behind Refine so that several **provider sets** can serve the same app. **ActivityPods is the default and the sole priority** — directive of 2026-10-09 (recorded in `data-layer/brief_2026-10-09_activitypods-first-backend`): everything that can goes through `@activitypods/refine-providers`, and a **backend for actions on behalf of a user** is to be added. A Refine provider over NextGraph plus its polyfill remains the declared second target, but its development is **suspended, temporarily** — no work goes into it while ActivityPods is wired.
|
||||
|
||||
## Directives of 2026-10-09 (resumption) — and first reconnaissance
|
||||
|
||||
Recorded in `data-layer/brief_2026-10-09_activitypods-first-backend`; the essentials and what the reconnaissance (three read-only passes over the user-designated reference repos) established:
|
||||
|
||||
- **The on-behalf mechanism is mapped.** An app backend IS an ActivityPub actor (`@activitypods/app` `AppService`, Moleculer). It signs pod requests with HTTP signatures as the app actor; the pod provider checks the app's registration and grants, then impersonates the user (`pod-provider/backend/middlewares/app-control.js`). Services call `pod-outbox.post`, `pod-resources.*`, `pod-collections.*`, `pod-wac-groups.*`, `pod-permissions.*` with an `actorUri`. Frontend calls use the Solid-OIDC dual-identity token instead (`webId` = app client_id, `impersonatedUser` = user). Reference implementations: `welcometomyplace/backend/services/registration.service.js`, `attendees-matcher.service.js`; both example apps run the same stack (`@activitypods/app` 2.3 + `@semapps/*` 1.2, Moleculer backend + Vite/React/Refine frontend, Docker-deployed).
|
||||
- **The refine-providers package is NOT in the platform monorepo** — it lives at `/home/sylvain/projects/activitypods/refine-providers` (a `/home/sylvain/projects/nextgraph/activitypodes-refine-providers` checkout also exists; the pinned reconstruction remains the work-time specification).
|
||||
- **Shapes**: 15 shape trees published at `https://shapes.activitypods.org` from `/home/sylvain/projects/activitypods/shapes` (repo evolvable by us). Mapping proposal: reuse `as/Event` + `as/Profile`; create `fp/MeetingPoint` and `fp/Participation` (the `as:Join` type exists but has no shape tree). No stored-counter participation fields beyond the reference + `active` fit the 2026-10-03 structure.
|
||||
- **No unattended per-user backend token exists** on the platform: server-side per-user actions ride the app-actor signature + grants; the platform-idiomatic channel for user-facing reactions is the app outbox → user inbox (notifications style).
|
||||
|
||||
## What exists, and what deliberately does not
|
||||
|
||||
In `src/shared/refine/`:
|
||||
|
||||
- `variant.ts` + `variant.test.ts` — pure `resolveVariant(pathname, known)`: which provider set the URL selects. 8 tests covering the case space, including the one true today (empty registry → every URL is the default).
|
||||
- `providers/types.ts` — the contract a backend must satisfy (`RefineBackend`, `RecordBinding`), each field derived from a measured encounter rather than anticipated.
|
||||
- `providers/index.ts` — the registry. **EMPTY on purpose**: no backend is implemented yet.
|
||||
- `RefineProvider.tsx` — mounts `<Refine>` over the backend the URL named.
|
||||
|
||||
**None of it is wired.** Nothing imports `RefineProvider`; `FestipodDataContext` on this base is untouched and still talks to the polyfill directly. Wiring it means rewriting that context against this base's much-evolved version — that is the step everything else waits on.
|
||||
|
||||
In `.project/`:
|
||||
|
||||
- `concepts/data-layer/activitypods-providers/contract_activitypods-providers.md` — a **reconstruction**, not a pull: that package publishes no engagement, so this is our own observation, pinned (`built_version: 66616484b9…`). It is the specification to code against.
|
||||
- `concepts/data-layer/activitypods-providers/usage_festipod.md` — our declaration toward them. Its Frictions/Needs are the channel to the provider.
|
||||
- `concepts/data-layer/brief_2026-10-03_structure-for-two-rights-models.md` — the data-shape design, below.
|
||||
- `concepts/data-layer/brief_2026-10-09_activitypods-first-backend.md` — the 2026-10-09 directives (ActivityPods priority, on-behalf backend, NextGraph suspended) and the reference material; the brief this hand-off's resumption added.
|
||||
- `handoff/refine-provider-abstraction.md` — this document.
|
||||
- `contracts.yaml` — `publish:` (both declarations, keys qualified by interface) and three `consume:` entries (polyfill-surface, indexing-layer, activitypods-providers).
|
||||
- `contracts.local.yaml` — gitignored, per-worktree. Must exist for `contracts.py check` to resolve the ActivityPods entry.
|
||||
|
||||
## Decisions, and why
|
||||
|
||||
**One app variant per provider set, selected by a URL path prefix.** The default at the root, another behind a prefix naming it; switching is a plain link and costs a full page reload. The reload is the mechanism, not a cost: it discards the previous backend's Refine cache and session with no code to undo them, and losing access to the left backend's data is accepted. An earlier plan for in-session hot switching was dropped. The deep reason: **the sign-in experience is not abstractable** — the NextGraph SDK mounts its own full-screen barrier and owns the whole wallet experience (`src/app/AuthGate.tsx` renders no access screen), while a Solid backend hands the app an OIDC flow it must render itself (Pod-provider picker, callback route). One confiscates the screen, the other delegates a flow.
|
||||
|
||||
The prefix **is** the backend's name, so a backend must never be named after a top-level route (`events`, `profile`, `home`, `settings`, `users`).
|
||||
|
||||
**The router does not yet strip the prefix, deliberately.** `parsePath`/`navigate` in `src/app/router.tsx` are a shared primitive every screen depends on; the suites that would prove a change there are not run in this worktree; and while one backend is registered no URL carries a prefix, so nothing is broken. That half lands with the second backend, when it becomes testable.
|
||||
|
||||
**The data shape: references, not embedded content.** Three levels — Event (own document, public, written by its creator, carrying an attached list of *references*); Participation (own document, owned by **the participant**, carrying `event`, `active`, `participant`); Profile (own document, protected). Three facts force it: the participation must be writable by its participant and nobody else, and NextGraph grants write per document; per-reader variation is impossible inside one document where encryption is per document, so it is obtained by **indirection** (the reference is always present, the ability to *follow* it varies); and `rule_document-per-entity` already says this for that reason.
|
||||
|
||||
The non-visibility mechanism differs and both land on the same application state — *a reference one may or may not resolve*. On NextGraph the target is **undecipherable** for want of a key (nothing is absent). On ActivityPods it is refused by WAC, or omitted by server-side projection. So the domain type models the NextGraph shape, where everything is present: `active` and `participant` are **not** optional; only resolution fails. (Proposing them optional was an error — it lifted an ActivityPods adaptation into the common model.)
|
||||
|
||||
**No stored counter, ever.** The participant count is `participations.length`. `knownParticipants` is the participations whose participant resolves — a presence test, not an access check. Withdrawal is `active: false` on one's own document.
|
||||
|
||||
**NextGraph is the common denominator.** Its model is the strict subset: no WAC groups, no revocation, sharing wholesale per document, but writes work offline. Structure for it and ActivityPods serves the structure, more cheaply even. The converse fails: anything needing a server projection or a revocation is untenable on NextGraph.
|
||||
|
||||
**The Refine boundary rule.** Refine's `DataProvider` is flat CRUD (`getList`, `getMany?`, `getOne`, `create`, `createMany?`, `update`, `updateMany?`, `deleteOne`, `deleteMany?`, `getApiUrl`, `custom?`) — **no relation or join concept at all**, and no `getManyReference`. Everything that makes Festipod social (append to someone else's collection, grant/revoke read, discovery, offline queueing) is outside its vocabulary. Since cache, live subscription and optimistic update are bought **per resource**, the rule is: *anything the app displays and that changes must be a Refine resource; anything one-shot and invisible may live in the seam.* A write passed beside Refine does not invalidate its cache, so the list would not refresh.
|
||||
|
||||
Refine does support **several data providers at once** (`Resource.dataProviderName`, `useList({dataProviderName})`). Deliberately unused for switching, but it is the tool for reading one entity from another backend (discovery from a shared index) or for migration.
|
||||
|
||||
**The seam should shrink.** `types.ts` was written from measurements against one raw provider that could not be changed. Since we can contribute to the providers, several of its fields are symptoms rather than requirements: `updateReplaces` + `toPatch` should vanish (a provider exposes one semantic), and `recordKey` vs `domainId` should collapse to `id` (ActivityPods already does). `toDomain`/`toValues` shrink to vocabulary mapping; `visibility` and `resource` stay — visibility is a product declaration, not a transport fact. **The measure of success is how small `RecordBinding` gets.** Capability differences belong inside the provider, never in the app.
|
||||
|
||||
## What is known about `@activitypods/refine-providers`
|
||||
|
||||
The reconstruction is the specification. Its source was read **once, deliberately, at adoption time** (that is what a reconstruction is); it is not to be read again at work time — a gap goes into the declaration's Frictions/Needs. Checkouts, if ever needed again: the package at `/home/sylvain/projects/nextgraph/activitypodes-refine-providers`, the platform at `/home/sylvain/projects/activitypods/activitypods`.
|
||||
|
||||
- A record's `id` is the LDP resource URI, and it is the **only** identifier — `getOne`/`update`/`deleteOne` use it directly as the URL. The seam's two identifiers collapse into one here.
|
||||
- `update` fetches the current resource, strips `id`/`@context`, and PUTs `{...current, ...variables}` — so **a partial write is safe**, the merge being **shallow**.
|
||||
- `getList` resolves the resource's container from the **session's own WebID** (public Solid TypeIndex, best-effort private), so it returns only what the current user holds. `getOne`/`getMany` fetch a URI directly and so **do** cross Pods — they just need the URI.
|
||||
- No server-side filtering, sorting or pagination: the whole container is fetched, then filtered in memory.
|
||||
- **`getMany` silently drops failed fetches** (`allSettled` then filter) — it erases exactly the "unreadable reference" state the model needs.
|
||||
- Module-level caches (shape trees, type registrations per WebID, subscription endpoints) are **never invalidated**.
|
||||
- `authProvider` installs a `setInterval` and a `visibilitychange` listener at construction with **no teardown**; the only opt-out is `appStatusCheckInterval: false`.
|
||||
- Live updates are real push (`notify:WebSocketChannel2023`), but a **list** subscription reports membership only (`Add`/`Remove`), never a member's field change.
|
||||
- `types` takes precedence over `shapeTreeUri` when both are set — the **opposite** of what the configuration type documents.
|
||||
- `custom` is not implemented, so there is **no escape hatch through Refine** for non-CRUD operations.
|
||||
- Collections are POSTable LDP resources (`/as/collection`) with `semapps:dereferenceItems` — *"all items will be dereferenced (if permissions allow)"*. That is how `event.participations` can come back as full objects for a reader who is allowed, and as URIs otherwise, in one read. Items are added/removed by SparqlPatch on `as:items`.
|
||||
- `acl:Append` exists among the WAC modes and is really used in app access needs. **Whether it lets a third party insert an item without being able to remove one is NOT VERIFIED** — enforcement lives in `@semapps/webacl`, absent from the checkout. The structure does not depend on the answer: if it works that way the participant appends directly; otherwise the route is `apods:PostOutbox` to the owner's inbox, which is exactly the NextGraph shape, and only latency changes.
|
||||
- An app declares `accessNeeds` (`shapeTreeUri` + `accessMode`) plus special needs: `apods:CreateCollection`, `apods:CreateWacGroup`, `apods:PostOutbox`, `apods:QuerySparqlEndpoint`, `apods:UpdateWebId`, inbox/outbox reads.
|
||||
- **Backend-less apps are supported**: `onlineBackend` is false only when the app description document at the `clientId` URI is not fetchable. A static file suffices.
|
||||
- A custom ontology prefix **must be registered at prefix.cc** (the Pod provider requires every ontology to have a known prefix) — this concerns Festipod's `fp:`. Shape trees must sit at a public URL, one per entity, and **cannot reference other shapes** (no sharing cascade).
|
||||
|
||||
**Four contributions this points to** (we may contribute to these providers): `getMany` must signal the unreadable instead of swallowing it; an append path into a collection one does not own; collection support as a resource with `dereferenceItems`; and a teardown for the auth provider's timer.
|
||||
|
||||
## The deduplication question — answered, and still open
|
||||
|
||||
The attendance model (`brief_2026-07-20_attendance-set-model`, PO 2026-07-27) requires the creator to tell two references from the *same* person apart from two from different people **without knowing who**, or the count is not a count of people and a participant can inflate it.
|
||||
|
||||
**Checked on this base: not solved.** A different problem is, and well. `materializeAttendance` (`src/shared/data/registration.ts`) keys its active set by a **per-deposit** `uid` minted from time plus randomness, so a broker re-sync replaying the same deposits converges — that is **replay idempotence**, not person identity. The count counts **deposits**: the same person joining twice counts twice. Further: the **plaintext `userId` travels in the deposit** and the cancellation fallback depends on it (a leave without a known `regUid` cancels the earliest still-active join for that `userId`), so the creator *does* know who and the anonymity the model rests on does not hold. Participations are in the **protected** scope rather than public, and withdrawal is still a deletion. And the gap was **never raised** through the channel meant for it.
|
||||
|
||||
So both platforms count deposits rather than people, and the gap sits in **neither Refine provider** — polyfill on one side, server on the other. ActivityPods could arbitrate server-side because the server sees everything.
|
||||
|
||||
## Problems hit, and how they turned out
|
||||
|
||||
- **Two false claims were written into durable records and corrected.** First, that the participant count needs an inbox and that a CRUD-only backend therefore could not carry the model — wrong: every participation is readable by everyone, only the link to a person is protected, so counting is ordinary CRUD. Second, a declared "need" for a WebID filter on `getList` — Festipod does not need it: `getOne`/`getMany` already cross Pods, the participations arrive with the event, and discovery is an **index** problem (already decided on this base: `decision_2026-08-17_discovery-through-a-shared-index`), not a provider one. Both corrections are in the files.
|
||||
- **`__FESTIPOD_BACKEND__` was a lie in a comment** claiming `build.ts` defined it. Nothing did, and nothing could: this project serves from `src/` in production too (`bun run start`), so `build.ts`'s `define` block never reaches what runs (`tech-stack`, `knowledge_build-pipeline`). The runtime mechanism this base already uses instead is a config route — `/festipod-config.json` served by `src/index.ts`, read by `src/app/frontend.tsx` *before* it imports `App`. Replaced by the URL read.
|
||||
- **Three separate times the answer was already on `ng-eventually`** (the runtime config route, the delegation of wallet access to the SDK, the dedup implementation). That is what triggered rebasing onto it.
|
||||
- **The migration was delegated to a subagent** and verified independently afterwards: the five transferred files are byte-identical, the manifest merge preserved both pre-existing entries, and the gates were re-run by hand. Its one worry — that a missing local `node_modules` had made the baseline incomparable — was **unfounded** (pnpm had installed the full tree locally; 22 entries against the main checkout's 21). Its debt block was better than the specification it was given: it caught the `__FESTIPOD_BACKEND__` consequence and two branch-relative phrasings.
|
||||
- **Moving base falsified three claims in the transferred code**, all corrected: the inbox the count comment points at is local here; the worked provider implementation is not in this branch's ancestry; and the `MeetingPoint` SHEX shape **does** exist here, so its binding stopped being optional (a backend that cannot carry one of the four entities cannot serve the app).
|
||||
|
||||
## Current gate state
|
||||
|
||||
`bunx tsc --noEmit` exit 0; `bun test` 32 pass / 0 fail; `contracts.py check --integrity-only` clean. `lint` is clean except one **pre-existing ERROR**: `polyfill-surface/usage_festipod.md` has **no `## Needs` section**. The e2e / `@data` / cucumber suites have **not** been run in this worktree, so their state here is unknown.
|
||||
|
||||
`contracts.py check` also reports, pre-existing: `polyfill-surface: provider moved: a33fb8a214 → a8d53010c2` — the pulled copy is behind the pinned ref.
|
||||
|
||||
## Open points
|
||||
|
||||
- Adding `## Needs` to the polyfill declaration is where the per-person discriminator belongs. It is **outward-facing** (the provider reads it on its next pull), so it awaits the user's go.
|
||||
- `acl:Append` behaviour, as above.
|
||||
- The router's prefix stripping, with the second backend.
|
||||
- The four provider contributions.
|
||||
- Wiring the seam: rewriting `FestipodDataContext` against this base, and a first `RefineBackend` (ActivityPods, being the default).
|
||||
- Both usage declarations are named `usage_festipod.md`, so the bare stem is ambiguous as a wikilink — refer to either by folder path.
|
||||
|
||||
## Standing constraints
|
||||
|
||||
- A pulled or reconstructed contract **is** the specification. Never read a provider's source at work time; a gap goes into the declaration's Frictions/Needs and is signalled out of band (content lives in the document). Never describe in this repo how the data layer is implemented.
|
||||
- **No attribution to Claude** anywhere in produced artifacts — commits, PRs, docs, comments. This overrides harness defaults.
|
||||
- Project documentation in **English**, never hard-wrapped (one line per paragraph or list item). Conversation with the user in **French**.
|
||||
- **No effort or timeline estimates**, anywhere.
|
||||
- Package installation goes through **pnpm**, never `bun install`; Bun-first otherwise.
|
||||
- Project facts are documented through `/concept document`, not written freehand into `.project/`.
|
||||
- Commit freely on a branch; push at validated states without being asked (Gitea SSH needs the sandbox disabled).
|
||||
- Never bare `git stash` — the stack is shared across worktrees and sessions.
|
||||
- `pre-push` blocks while any `_debt.md` exists.
|
||||
Reference in New Issue
Block a user