fix(query): keep field-filtered searches off the chunk-rerank path #23

Merged
laoc merged 1 commit from fix/rerank-field-filters into main 2026-08-19 12:18:48 +00:00
Owner

Fixes #22.

A NIP-50 search combining free text with a field filter ("forschung publisher.name:e-teaching.org", or "<topic> community:<pubkey>") was routed to chunk-rerank, which sends the raw search string to the chunk index — the filter tokens are read as text, and the filter.Matches post-filter ignores the Search field, so the field filter was silently dropped and the query answered wider than asked. Verified live on wss://amb-relay.edufeed.org: "LADEN publisher.name:zzz-bogus-control-xyz" returned free-text hits instead of 0.

Pure field-filter searches were already protected by the searchHasFreeText guard; the combined case leaked.

Change: new registry.rerankOwns(filter) encodes the full ownership rule — chunked target + free text + no field filters — with a table test (TestRerankOwnsSearch) covering the leaking case; QueryStored uses it. Field-filtered searches now take the plain Typesense path, which honours the filter and loses only passage ordering and snippets: fail closed on ranking, never on filtering.

Trade-off / follow-up: community- or publisher-scoped topic searches no longer get chunk-level snippets. Restoring rerank for filtered searches would mean stripping field filters from the chunk query and intersecting with the filtered lexical set (option 2 in #22) — left for a follow-up.

Note: TestSharesBackendFilterExpressions fails identically on clean main (pre-existing, unrelated); everything else passes with this change.

🤖 Generated with Claude Code

Fixes #22. A NIP-50 search combining free text with a field filter (`"forschung publisher.name:e-teaching.org"`, or `"<topic> community:<pubkey>"`) was routed to chunk-rerank, which sends the raw search string to the chunk index — the filter tokens are read as text, and the `filter.Matches` post-filter ignores the `Search` field, so the field filter was **silently dropped** and the query answered wider than asked. Verified live on wss://amb-relay.edufeed.org: `"LADEN publisher.name:zzz-bogus-control-xyz"` returned free-text hits instead of 0. Pure field-filter searches were already protected by the `searchHasFreeText` guard; the combined case leaked. **Change:** new `registry.rerankOwns(filter)` encodes the full ownership rule — chunked target + free text + **no field filters** — with a table test (`TestRerankOwnsSearch`) covering the leaking case; `QueryStored` uses it. Field-filtered searches now take the plain Typesense path, which honours the filter and loses only passage ordering and snippets: fail closed on ranking, never on filtering. **Trade-off / follow-up:** community- or publisher-scoped topic searches no longer get chunk-level snippets. Restoring rerank for filtered searches would mean stripping field filters from the chunk query and intersecting with the filtered lexical set (option 2 in #22) — left for a follow-up. Note: `TestSharesBackendFilterExpressions` fails identically on clean `main` (pre-existing, unrelated); everything else passes with this change. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
fix(query): keep field-filtered searches off the chunk-rerank path
All checks were successful
Build and Push Docker Image / build (pull_request) Successful in 1m38s
4b7dc48629
A NIP-50 search combining free text with a field filter ("forschung
publisher.name:e-teaching.org") was routed to chunk-rerank, which sends
the raw search string to the chunk index — the filter tokens are read as
text and filter.Matches never re-applies NIP-50 field filters, so the
field filter was silently dropped and the query answered wider than
asked. Verified live: "LADEN publisher.name:zzz-bogus" returned free-text
hits instead of zero. Pure field-filter searches were already protected;
the combined case leaked.

New registry.rerankOwns encodes the full ownership rule (chunked target +
free text + NO field filters); QueryStored uses it. Filtered searches now
take the plain Typesense path, which honours the filter and loses only
passage ordering and snippets — fail closed on ranking, never on
filtering. Carrying filters through the rerank (intersecting with the
filtered lexical set) can restore snippets for these searches later.

Closes #22

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
laoc merged commit cc1305170f into main 2026-08-19 12:18:48 +00:00
Sign in to join this conversation.
No reviewers
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
edufeed/amb-relay!23
No description provided.