Published events not offered to the EventStore/IDB cache — stale reads and phantom cache entries (siblings of #62) #64

New issue

Closed

opened 2026-07-30 12:23:14 +00:00 by laoc · 4 comments

laoc commented 2026-07-30 12:23:14 +00:00

Owner

Copy link

Found while fixing #62 (PR #63). Same class, different call sites. All verified by grep at dev @ 3a4b9e8c, none of them reproduced in a browser — so treat the severity ordering as reasoned, not measured.

Why this class matters at all

The local IndexedDB cache is the first step of applesauce's addressPointerLoadingSequence, and a cache hit ends the sequence — relay hints, additional relays and lookup relays are never tried. So for any cacheable kind, whatever is in IDB is what the app shows, and a relay cannot correct it.

The cache is fed exclusively from eventStore.insert$ (event-cache.svelte.js:70-78). publishEvent does not touch the EventStore. So await publishEvent(...) without a following eventStore.add(signed) means the new version never reaches the cache.

1. relay-settings-service.js:66 — kind 10002, the worst of the set

grep -c eventStore src/lib/services/relay-settings-service.js returns 0. Kind 10002 is cacheable (event-cache.svelte.js:21), and it is the relay list itself — a stale read there mis-routes every subsequent query, so the blast radius is not one screen.

Fix: eventStore.add(signed) after a successful publish.

2. calendar-actions.svelte.js:279 — kind 31924 (calendars)

Cacheable (event-cache.svelte.js:29), no eventStore.add. Milder than #62 because it is a create: there is no stale prior version to be served, only "missing until a relay round-trip". Same one-line shape.

3. publishEventOptimistic's failure path cannot reach IDB — phantom cache entries

publish-service.js:333 calls eventStore.remove(signedEvent) when no relay accepted the publish. But the cache pipeline is insert-only: grep -rn 'nostrIDB.remove|nostrIDB.delete|deleteEvents' src/lib/ returns nothing, and the only IDB delete in the app is nostrIDB.deleteAllEvents() (event-cache.svelte.js:187, the manual "clear cache" action).

Failure is detected only after Promise.allSettled against a 5000ms default timeout, while persistEventsToCache batches at 1000ms — so unless every relay fails inside 1s, the batch has already flushed and the event is in IDB permanently.

For an addressable kind that is worse than a leak. nostr-idb keys by kind:pubkey:d, newest-wins, so the phantom overwrites the last good version: eventStore.remove leaves memory empty at that address, IDB holds an event that exists on no relay, and the cache-hit short-circuit means no relay is ever asked to correct it. Failed edit → the app shows the edit forever, and the real version is unreachable from cache.

createEvent (calendar-actions.svelte.js:105) takes this path live today for kinds 31922/31923.

The discriminating experiment is dead-relay → save → reload. Not run.

Fix options: a single-event IDB delete wired to eventStore.removed$, or stop adding optimistically for cacheable kinds and add only on success (the EditProfileModal / pin-list-service / helpers/comments.js shape, which is what #62 adopted).

4. calendarActions.deleteEvent (calendar-actions.svelte.js:234) — dead code encoding the wrong pattern

Publishes a kind 5 with no eventStore.add and no cacheDeletion — exactly the failure the cacheDeletion doc comment (event-cache.svelte.js:106-113) warns about, deleted content reappearing on reload.

It is not a live bug: grep -rn '\.deleteEvent(' src/ returns nothing. The calendar UI goes through deleteCalendarEvent in $lib/helpers/eventDeletion.js, which does both (eventDeletion.js:69-70), imported at CalendarEventDetailView.svelte:30 and EventManagementActions.svelte:8.

Worth deleting so it does not get copied.

Checked and NOT affected

• pin-list-service.js:102,135 — eventStore.add(signed) at 103 and 136. Correct already, and kind 10001 is not cacheable.
• EditProfileModal:283, EditCommunityModal:299, helpers/comments.js:41-49 — all await publishEvent(...) then eventStore.add on success. This is the pattern the others should follow.

Credit: sites 1–4 were surfaced and corrected by TestOER during the #62 review; the not-affected list is theirs too. I re-ran every grep above myself.

Found while fixing #62 (PR #63). Same class, different call sites. All verified by grep at `dev` @ `3a4b9e8c`, none of them reproduced in a browser — so treat the severity ordering as reasoned, not measured. ## Why this class matters at all The local IndexedDB cache is the **first** step of applesauce's `addressPointerLoadingSequence`, and a cache hit **ends** the sequence — relay hints, additional relays and lookup relays are never tried. So for any cacheable kind, whatever is in IDB is what the app shows, and a relay cannot correct it. The cache is fed exclusively from `eventStore.insert$` (`event-cache.svelte.js:70-78`). `publishEvent` does not touch the EventStore. So `await publishEvent(...)` without a following `eventStore.add(signed)` means the new version never reaches the cache. ## 1. `relay-settings-service.js:66` — kind 10002, the worst of the set `grep -c eventStore src/lib/services/relay-settings-service.js` returns **0**. Kind 10002 is cacheable (`event-cache.svelte.js:21`), and it is the **relay list itself** — a stale read there mis-routes every subsequent query, so the blast radius is not one screen. Fix: `eventStore.add(signed)` after a successful publish. ## 2. `calendar-actions.svelte.js:279` — kind 31924 (calendars) Cacheable (`event-cache.svelte.js:29`), no `eventStore.add`. Milder than #62 because it is a *create*: there is no stale prior version to be served, only "missing until a relay round-trip". Same one-line shape. ## 3. `publishEventOptimistic`'s failure path cannot reach IDB — phantom cache entries `publish-service.js:333` calls `eventStore.remove(signedEvent)` when no relay accepted the publish. But the cache pipeline is **insert-only**: `grep -rn 'nostrIDB.remove|nostrIDB.delete|deleteEvents' src/lib/` returns nothing, and the only IDB delete in the app is `nostrIDB.deleteAllEvents()` (`event-cache.svelte.js:187`, the manual "clear cache" action). Failure is detected only after `Promise.allSettled` against a 5000ms default timeout, while `persistEventsToCache` batches at 1000ms — so unless every relay fails inside 1s, the batch has already flushed and the event is in IDB permanently. **For an addressable kind that is worse than a leak.** nostr-idb keys by `kind:pubkey:d`, newest-wins, so the phantom **overwrites the last good version**: `eventStore.remove` leaves memory empty at that address, IDB holds an event that exists on no relay, and the cache-hit short-circuit means no relay is ever asked to correct it. Failed edit → the app shows the edit forever, and the real version is unreachable from cache. `createEvent` (`calendar-actions.svelte.js:105`) takes this path live today for kinds 31922/31923. The discriminating experiment is dead-relay → save → reload. Not run. Fix options: a single-event IDB delete wired to `eventStore.removed$`, or stop adding optimistically for cacheable kinds and add only on success (the `EditProfileModal` / `pin-list-service` / `helpers/comments.js` shape, which is what #62 adopted). ## 4. `calendarActions.deleteEvent` (`calendar-actions.svelte.js:234`) — dead code encoding the wrong pattern Publishes a kind 5 with **no** `eventStore.add` and **no** `cacheDeletion` — exactly the failure the `cacheDeletion` doc comment (`event-cache.svelte.js:106-113`) warns about, deleted content reappearing on reload. It is **not** a live bug: `grep -rn '\.deleteEvent(' src/` returns nothing. The calendar UI goes through `deleteCalendarEvent` in `$lib/helpers/eventDeletion.js`, which does both (`eventDeletion.js:69-70`), imported at `CalendarEventDetailView.svelte:30` and `EventManagementActions.svelte:8`. Worth deleting so it does not get copied. ## Checked and NOT affected - `pin-list-service.js:102,135` — `eventStore.add(signed)` at 103 and 136. Correct already, and kind 10001 is not cacheable. - `EditProfileModal:283`, `EditCommunityModal:299`, `helpers/comments.js:41-49` — all `await publishEvent(...)` then `eventStore.add` on success. This is the pattern the others should follow. Credit: sites 1–4 were surfaced and corrected by TestOER during the #62 review; the not-affected list is theirs too. I re-ran every grep above myself.

laoc commented 2026-07-30 13:08:50 +00:00

Author

Owner

Copy link

A fifth item for this list: the same-second replacement tie

Found while closing out #62 (PR #63), and it is not calendar-specific — it applies to every replaceable/addressable update in the app.

unixNow() in applesauce-core/helpers/time.js is Math.round(Date.now() / 1000), not floor. So an event created at …x.5–…x.999 is stamped in the next second, and an edit made shortly after lands in that same second. A created_at tie is then resolved three different ways, none of them in the user's favour:

layer	tie-break	outcome
relay (NIP-01)	lower id wins	coin flip
applesauce EventStore.add	lower id wins (event-store.js, incomingBeatsWinner)	coin flip
nostr-idb addEvents	strict event.created_at > existing (database/insert.js:23)	always keeps the old

The cache is deterministically stale, and because the IDB cache is the first step of applesauce's address-loader sequence and a cache hit ends that sequence, no relay ever corrects it. Measured on the e2e suite: the tie predicted pass/fail for 6 of 6 attempts.

updateEvent now stamps Math.max(unixNow(), existing.created_at + 1) (edufeed-app 485cab0d). Every other replaceable update in the app still has this, including the relay-settings-service.js kind-10002 path listed above — so that file needs both fixes, not just the eventStore.add.

Worth pairing with a check on any code that assumes unixNow() floors; it can also stamp up to 500ms into the future.

## A fifth item for this list: the same-second replacement tie Found while closing out #62 (PR #63), and it is **not** calendar-specific — it applies to every replaceable/addressable update in the app. `unixNow()` in `applesauce-core/helpers/time.js` is `Math.round(Date.now() / 1000)`, **not** floor. So an event created at `…x.5`–`…x.999` is stamped in the *next* second, and an edit made shortly after lands in that same second. A `created_at` tie is then resolved three different ways, none of them in the user's favour: | layer | tie-break | outcome | |---|---|---| | relay (NIP-01) | lower id wins | coin flip | | applesauce `EventStore.add` | lower id wins (`event-store.js`, `incomingBeatsWinner`) | coin flip | | `nostr-idb addEvents` | strict `event.created_at > existing` (`database/insert.js:23`) | **always keeps the old** | The cache is deterministically stale, and because the IDB cache is the first step of applesauce's address-loader sequence and a cache hit ends that sequence, no relay ever corrects it. Measured on the e2e suite: the tie predicted pass/fail for **6 of 6** attempts. `updateEvent` now stamps `Math.max(unixNow(), existing.created_at + 1)` (edufeed-app `485cab0d`). **Every other replaceable update in the app still has this**, including the `relay-settings-service.js` kind-10002 path listed above — so that file needs both fixes, not just the `eventStore.add`. Worth pairing with a check on any code that assumes `unixNow()` floors; it can also stamp up to 500ms into the future.

laoc referenced this issue 2026-07-30 13:21:45 +00:00

Toggling all-day on an existing calendar event changes the kind, so nothing is replaced (split from #62) #65

laoc commented 2026-07-30 13:22:15 +00:00

Author

Owner

Copy link

Adding the second half of #62's root cause here, because it is not calendar-specific and this issue is where the sibling audit lives.

unixNow() is Math.round, not Math.floor:

export function unixNow() { return Math.round(Date.now() / 1000); }

applesauce-core/dist/helpers/time.js

So an event created at .5–.999 of a second is stamped in the next second with zero elapsed time, and a quick edit lands in that same second. A created_at tie is then resolved differently at three layers:

layer	tie-break on equal created_at	outcome
relay (NIP-01)	lower id wins	coin flip
applesauce EventStore.add — incomingBeatsWinner, event-store.js:168-185	lower id wins	coin flip
nostr-idb addEvents — event.created_at > existing, database/insert.js:23	strictly greater	always keeps the old

The cache layer is the dangerous one: it is not a race, it is deterministic. On a tie the local cache keeps the pre-edit event, and because a cache hit ends applesauce's addressPointerLoadingSequence (address-loader.js:57-63), no relay is ever asked to correct it.

updateEvent now guards with created_at: Math.max(unixNow(), existing.created_at + 1) (485cab0d), but that fix is local to the calendar update path. Every other replaceable-event update in the app has the same exposure — including relay-settings-service.js (kind 10002), already the worst entry on this issue for the missing eventStore.add. A same-second edit of the relay list would be dropped from cache deterministically, and a stale relay list mis-routes every query after it.

Two tabs on one coordinate, or an edit straight after a create, both reach this outside the tests.

Whoever picks this up: the guard belongs in a shared helper rather than copied per call site, otherwise it goes the same way as the eventStore.add omissions this issue is already about.

Adding the second half of #62's root cause here, because it is **not** calendar-specific and this issue is where the sibling audit lives. **`unixNow()` is `Math.round`, not `Math.floor`:** ```js export function unixNow() { return Math.round(Date.now() / 1000); } ``` `applesauce-core/dist/helpers/time.js` So an event created at `.5`–`.999` of a second is stamped in the *next* second with zero elapsed time, and a quick edit lands in that same second. A `created_at` tie is then resolved differently at three layers: | layer | tie-break on equal `created_at` | outcome | |---|---|---| | relay (NIP-01) | lower id wins | coin flip | | applesauce `EventStore.add` — `incomingBeatsWinner`, `event-store.js:168-185` | lower id wins | coin flip | | `nostr-idb` `addEvents` — `event.created_at > existing`, `database/insert.js:23` | strictly greater | **always keeps the old** | The cache layer is the dangerous one: it is not a race, it is deterministic. On a tie the local cache keeps the pre-edit event, and because a cache hit ends applesauce's `addressPointerLoadingSequence` (`address-loader.js:57-63`), no relay is ever asked to correct it. `updateEvent` now guards with `created_at: Math.max(unixNow(), existing.created_at + 1)` (`485cab0d`), **but that fix is local to the calendar update path.** Every other replaceable-event update in the app has the same exposure — including `relay-settings-service.js` (kind 10002), already the worst entry on this issue for the missing `eventStore.add`. A same-second edit of the relay list would be dropped from cache deterministically, and a stale relay list mis-routes every query after it. Two tabs on one coordinate, or an edit straight after a create, both reach this outside the tests. Whoever picks this up: the guard belongs in a shared helper rather than copied per call site, otherwise it goes the same way as the `eventStore.add` omissions this issue is already about.

laoc referenced this issue from a commit 2026-07-30 19:55:51 +00:00

fix(cache): offer published events to the EventStore, and stop caching phantoms (#64)

laoc referenced this issue from a pull request that will close it, 2026-07-30 19:56:19 +00:00

fix(cache): offer published events to the EventStore, and stop caching phantoms (#64) #69

laoc commented 2026-07-30 19:57:12 +00:00

Author

Owner

Copy link

PR #69 opened — #69 (branch fix/issue-64-eventstore-add-omissions, commit 13ac018d).

All four sites in this issue are addressed, plus a shared helper the owner signed off on.

Sites 1, 2, 4 are the one-line shape the issue describes. Site 3 (the phantom) needed two guards rather than one, and the obvious delete turned out to be a no-op: nostr-idb keys replaceable events by kind:pubkey:d, not by id, so deleteEvent(event.id) silently matches nothing for exactly the kinds that matter. uncacheEvent keys by getEventUID and refuses to delete when a newer version holds the address; the write callback separately re-checks eventStore.hasEvent at flush time for the case where the failure lands inside the batch window.

Beyond the issue: nextCreatedAt + cachePublishedEvent moved into helpers/replaceableUpdates.js, and adopting nextCreatedAt surfaced three further update paths carrying the #62 bug with no guard — updateWiki (30818), updateArticle (30023), updateResource (30142).

Verification: 473 files / 5213 tests pass (+20) at 13ac018d, pnpm check 0 errors, lint clean. Each new cache test confirmed load-bearing by reverting the guard it covers.

Still not established, as the issue said: the discriminating experiment for site 3 is dead-relay → save → reload in a browser. Not run — assigned to TestOER. Leaving this issue open until that lands and the PR merges.

**PR #69 opened** — https://git.edufeed.org/edufeed/edufeed-app/pulls/69 (branch `fix/issue-64-eventstore-add-omissions`, commit `13ac018d`). All four sites in this issue are addressed, plus a shared helper the owner signed off on. **Sites 1, 2, 4** are the one-line shape the issue describes. **Site 3** (the phantom) needed two guards rather than one, and the obvious delete turned out to be a no-op: nostr-idb keys replaceable events by `kind:pubkey:d`, not by id, so `deleteEvent(event.id)` silently matches nothing for exactly the kinds that matter. `uncacheEvent` keys by `getEventUID` and refuses to delete when a newer version holds the address; the write callback separately re-checks `eventStore.hasEvent` at flush time for the case where the failure lands inside the batch window. **Beyond the issue:** `nextCreatedAt` + `cachePublishedEvent` moved into `helpers/replaceableUpdates.js`, and adopting `nextCreatedAt` surfaced three further update paths carrying the #62 bug with no guard — `updateWiki` (30818), `updateArticle` (30023), `updateResource` (30142). **Verification:** 473 files / 5213 tests pass (+20) at `13ac018d`, `pnpm check` 0 errors, lint clean. Each new cache test confirmed load-bearing by reverting the guard it covers. **Still not established, as the issue said:** the discriminating experiment for site 3 is dead-relay → save → reload in a browser. Not run — assigned to TestOER. Leaving this issue open until that lands and the PR merges.

laoc referenced this issue from a commit 2026-07-30 21:09:16 +00:00

fix(cache): restore the replaced version when a publish fails outright (#64)

laoc referenced this issue from a commit 2026-07-30 21:09:54 +00:00

Merge pull request 'fix(cache): offer published events to the EventStore, and stop caching phantoms (#64)' (#69) from fix/issue-64-eventstore-add-omissions into dev

laoc closed this issue 2026-07-30 21:09:55 +00:00

laoc commented 2026-07-30 21:10:10 +00:00

Author

Owner

Copy link

Fixed and merged to dev via PR #69.

All four sites in this issue, plus a shared helper (helpers/replaceableUpdates.js) whose adoption surfaced three further update paths carrying the #62 bug — updateWiki (30818), updateArticle (30023), updateResource (30142).

Browser-verified by TestOER across four dead-relay modes, IndexedDB read directly, with negative controls. Two of my original claims did not survive that measurement and were corrected before merge:

• uncacheEvent alone deletes rather than recovers — the phantom overwrites the good version at its address before anything knows the publish failed, so the address ended EMPTY 9/9 (user-visibly a 404 on a resource that exists). publishEventOptimistic now captures the replaced version before the optimistic add and re-adds it after the un-cache.
• The hasEvent flush-time guard does not fire on this path — the fastest possible failure reports at 1006-1015ms against a 1000ms batch, and deleting the guard left outcomes identical 5/5. Kept for the invariant it does cover (superseded versions, NIP-09 deletions), with the comment and test name corrected.

uncacheEvent is the whole of the fix: disabled, the phantom leaks 4/4.

Closing.

**Fixed and merged to `dev` via PR #69.** All four sites in this issue, plus a shared helper (`helpers/replaceableUpdates.js`) whose adoption surfaced three further update paths carrying the #62 bug — `updateWiki` (30818), `updateArticle` (30023), `updateResource` (30142). Browser-verified by TestOER across four dead-relay modes, IndexedDB read directly, with negative controls. Two of my original claims did not survive that measurement and were corrected before merge: - `uncacheEvent` alone **deletes rather than recovers** — the phantom overwrites the good version at its address before anything knows the publish failed, so the address ended EMPTY 9/9 (user-visibly a 404 on a resource that exists). `publishEventOptimistic` now captures the replaced version before the optimistic add and re-adds it after the un-cache. - The `hasEvent` flush-time guard **does not fire on this path** — the fastest possible failure reports at 1006-1015ms against a 1000ms batch, and deleting the guard left outcomes identical 5/5. Kept for the invariant it does cover (superseded versions, NIP-09 deletions), with the comment and test name corrected. `uncacheEvent` is the whole of the fix: disabled, the phantom leaks 4/4. Closing.

laoc referenced this issue 2026-07-30 21:20:33 +00:00

Five DM/inbox/gift-wrap test files fail intermittently under full-suite load (pre-existing on dev) #70

laoc referenced this issue from a commit 2026-07-30 22:08:00 +00:00

fix(cache): make the failed-publish restore survive the from-cache filter (#64)

laoc referenced this issue 2026-07-30 22:08:13 +00:00

fix(cache): make the failed-publish restore survive the from-cache filter (#64) #71

laoc referenced this issue from a commit 2026-07-30 22:20:24 +00:00

Merge pull request 'fix(cache): make the failed-publish restore survive the from-cache filter (#64)' (#71) from fix/issue-64-restore-from-cache into dev

laoc referenced this issue 2026-07-30 22:31:41 +00:00

TemplateResourceForm publishes kind 30142 without the #62 created_at guard or the #64 failed-publish cleanup #72

laoc referenced this issue from a commit 2026-07-30 22:32:29 +00:00

fix(forms): guard TemplateResourceForm's kind-30142 publish (#72)

laoc referenced this issue 2026-07-31 09:26:21 +00:00

TemplateResourceForm publishes kind 30142 without the #62 created_at guard or the #64 failed-publish cleanup #72

Sign in to join this conversation.

No Branch/Tag specified

Branches Tags

main

feat/community-group-pointer

dev

feat/concord-render-gaps

feat/option-commit-on-blur

feat/doi-autofetch

feat/ekkw-metadata-form

feat/field-descriptions

feat/issue-77-form-preview

fix/issue-57-linked-materials-hover

fix/template-resource-form-publish-guards

fix/issue-64-restore-from-cache

fix/issue-64-eventstore-add-omissions

fix/issue-65-block-allday-toggle

fix/issue-62-stale-cache-after-update

fix/issue-39-e2e-selectors

fix/issue-31-ssrf-ip-ranges

fix/issue-54-ext-filter-shape

fix/membership-partial-delivery-visible

fix/membership-latest-application

feature/multi-admin-membership

worktree-cordn-groups

feature/nostr-metadata-forms

worktree-concord-private-channels

feat/feed-category-split

feat/applesauce-v6

feat/feed-issues-34-37

feat/membership-approve-notify

feat/add-nip05-to-profile

feat/nip05-verification

feat/membership-application

feat/mobile-fab-fix

feat/konfi-zeitstruktur-freetext

fix/pin-pnpm-version

No results found.

Labels

Clear labels   

bug

Something is broken

  

discussion

Needs a decision or concept first

  

enhancement

New feature or improvement

No labels

bug

discussion

enhancement

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

edufeed/edufeed-app#64

No description provided.