Integrate metadata-cleaner service into the file upload flow #47
Labels
No labels
bug
discussion
enhancement
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
edufeed/edufeed-app#47
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Motivation
A user requested being able to use our metadata-cleaner service from inside the app. Uploaded files (PDFs from Canva & co., photos with EXIF data) carry tool provenance and potentially sensitive metadata. The service (deployed at
cleaner.edufeed.org) can inspect, strip, and — for PDFs — recompress embedded images.Design (approved)
UX: optional review step. After picking a file in the upload components, a "Check metadata" button appears (only when the feature is configured and the file type is supported: PDF, JPG, PNG, TIFF, WebP). It opens a modal that:
Skipping the button = today's behavior. Service unreachable → friendly error, uploading the original still works.
Scope: content uploads.
LicensedFileInput(resource/publication attachments — the only PDF surface, so the only one with compression) andLicensedImageInput(cover images: resource wizard, article, calendar event). Avatars, banners, and MarkdownEditor inline images are out of scope for now.Integration: SvelteKit server proxy
/api/metaclean/[...path](the service sends no CORS headers, and this matches the/api/oer//api/enrichpattern):POST /api/metaclean/filesPOST /api/files(upload → session + inspected fields)GET /api/metaclean/files/{id}/ops/stripPOST /api/metaclean/files/{id}/applyGET /api/metaclean/files/{id}/downloadStrict allowlist; anything else → 404.
Configuration: new env var
METADATA_CLEANER_URL(server-side only, e.g.https://cleaner.edufeed.org)./api/configexposes onlymetadataCleaner: { enabled: boolean }. Unset → proxy returns 503, UI hidden entirely.i18n: all strings via Paraglide (DE/EN).
Testing: unit tests for config exposure, proxy allowlist/503/forwarding, client helper; component test for the modal with mocked API. No E2E (external service).
Out of scope (possible follow-ups)
POST /api/oer-ops+ sidecar) hooked into the resource wizardImplemented and merged into
dev(e4960690).LicensedFileInput+LicensedImageInput: shows all metadata grouped by store with sensitive flags, strip-provenance preset with reviewable field list, PDF compression (off/balanced/strong), verified result with leak scan./api/metaclean/[...path](strict allowlist, 503 whenMETADATA_CLEANER_URLunset, body cap viaMETADATA_CLEANER_MAX_UPLOAD_MB, default 200).BLOSSOM_MAX_FILE_SIZElimit (live-verified: 15.5 MB → 906 KB).Deployment note: adapter-node needs
BODY_SIZE_LIMITraised (default 512K) for/api/metacleanuploads.Out of scope / follow-ups: avatars/banners/inline markdown images, OER provenance writing (
oer-ops+ sidecar), field-level editing.UX revision in progress (user feedback): the auto-opening review modal is too prominent/confusing for non-technical users.
Revised design:
Branch:
worktree-metaclean-quiet-flow.Quiet flow shipped and merged into
dev(9677aebe):Verified live against cleaner.edufeed.org (15.6 MB → 906 KB rescue, inspect view, quiet checkbox flow).
Closing — implemented, revised after user feedback, and both rounds are on
dev.Verified at
dev@09842a48rather than relying on the merge notes above:src/routes/api/metaclean/[...path]/+server.js— the proxysrc/lib/helpers/metaclean.js— client helpersrc/lib/components/shared/MetadataCleanerModal.svelte, wired into bothLicensedFileInput.svelteandLicensedImageInput.svelte(the approved scope: content uploads, not avatars/banners/inline images)metadataCleaner: { enabled }in/api/config(+server.js:411),enabled: Boolean(env.METADATA_CLEANER_URL)— inert when unset, as designedmetaclean_*)api-metaclean-route,api-config-metaclean,metaclean-helper,MetadataCleanerModal, plusLicensedFileInput.metacleanandLicensedImageInput.metacleane1257495,f907e426,638f79f7,e4960690and9677aebeare all ancestors ofdev.Deployment note preserved from the thread, because it is easy to lose:
METADATA_CLEANER_URLis set on the dev instance only, so the feature is currently absent from edufeed.org — and adapter-node needsBODY_SIZE_LIMITraised above its 512K default for the uploads. Both are tracked in thedev -> mainrelease checklist (PR #56), so closing this does not drop them.The follow-ups listed in the last comment (avatars/banners/inline markdown images, OER provenance writing) were explicitly out of scope; if they are wanted they deserve their own issue rather than keeping this one open.