edufeed/relay-sync
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
No description
18 commits 1 branch 0 tags 105 KiB
  • Go 98.7%
  • Dockerfile 1.3%
Find a file
Exact
Steffen Rörtgen aeb254699b fix(reconcile): stop re-offering permanently-rejected events 
forward() previously marked an event seen only on publish success, so a
valid-signature event the target will never accept — e.g. a community-share
kind (16/30222) pulled from a general source that lacks an h/p community
target — was re-published on every reconcile cycle. Classify the relay's
OK=false reason (NIP-01 blocked:/invalid:/pow:/restricted:/duplicate:) as
permanent and mark such events seen; transient errors (rate-limited:/error:,
transport) still retry. Also skip the connection reset on a permanent
rejection, since the socket is healthy.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-24 13:19:14 +02:00
docs/superpowers docs(plan): relay-sync implementation plan (7 TDD tasks) 2026-06-23 16:00:01 +02:00
.dockerignore build: Dockerfile + env template for the relay-sync mirror service 2026-06-23 17:21:47 +02:00
.env.example build: Dockerfile + env template for the relay-sync mirror service 2026-06-23 17:21:47 +02:00
.gitignore chore: gitignore worktrees, SDD ledger, env, build output 2026-06-23 16:03:09 +02:00
backfill_test.go fix(reconcile): stop re-offering permanently-rejected events 2026-06-24 13:19:14 +02:00
CLAUDE.md fix(reconcile): stop re-offering permanently-rejected events 2026-06-24 13:19:14 +02:00
config.go harden: reject invalid SYNC_KINDS, cover both required-field validation paths 2026-06-23 16:14:11 +02:00
config_test.go harden: reject invalid SYNC_KINDS, cover both required-field validation paths 2026-06-23 16:14:11 +02:00
Dockerfile build: Dockerfile + env template for the relay-sync mirror service 2026-06-23 17:21:47 +02:00
go.mod feat(target): reconnecting QuerierPublisher to the mirror target relay 2026-06-23 16:18:21 +02:00
go.sum build: Dockerfile + env template for the relay-sync mirror service 2026-06-23 17:21:47 +02:00
main.go feat(main): supervisor spawning one worker per source with signal shutdown 2026-06-23 17:19:23 +02:00
README.md feat(reconcile): paginated REQ backfill + bad-signature skip 2026-06-24 13:03:18 +02:00
target.go fix(reconcile): stop re-offering permanently-rejected events 2026-06-24 13:19:14 +02:00
target_test.go fix(reconcile): stop re-offering permanently-rejected events 2026-06-24 13:19:14 +02:00
testutil_test.go fix(reconcile): stop re-offering permanently-rejected events 2026-06-24 13:19:14 +02:00
worker.go fix(reconcile): stop re-offering permanently-rejected events 2026-06-24 13:19:14 +02:00
worker_test.go feat(reconcile): paginated REQ backfill + bad-signature skip 2026-06-24 13:03:18 +02:00

README.md

relay-sync

Mirrors a static set of upstream ("prod") Nostr relays into a target AMB relay in near-real-time. Read-only on the sources — it only ever reads from upstream (live REQ tail + paginated REQ backfill) and writes to the single configured target. The first use is pulling production edufeed relay content into the dev AMB relay so dev has a realistic, fresh corpus.

It is a standalone service, not in-process in amb-relay. The relay stays a pure relay; relay-sync talks to it as an ordinary WebSocket client, so every mirrored event flows through the relay's normal write path (validation → BoltDB → Typesense projection → community stamping) and is indistinguishable from a natively-published event.

How it works

One worker per source relay, each running two cooperating loops that both converge on "publish this event to the target" (the target dedups by event id, so overlap is harmless):

  1. Live tail — a long-lived REQ subscription (since: now). New upstream events reach the target within seconds.
  2. Periodic reconcile — every RECONCILE_INTERVAL, a paginated REQ backfill walks the source's whole history newest-first (bounded pages, until cursor stepping backwards) and republishes it. A per-process id set dedups, so only events the live tail missed (e.g. while disconnected) and the not-yet-mirrored backlog actually reach the target. Plain REQ works uniformly across relay implementations, including large strfry sources where NIP-77 negentropy transferred nothing. Both loops also drop any event whose signature does not verify against its recomputed canonical id, since the target would reject it.

Stateless: on restart, the per-process dedup set is empty, so the first reconcile per source re-walks and re-publishes all history (the target dedups the writes it already holds) and live tails resume from now. The source relays are the durable record — there is no local event store or cursor.

Configuration

Env vars (copy .env.example):

Var Required Default Meaning
TARGET_RELAY yes — WS URL of the relay to mirror into (e.g. ws://amb-relay:3334).
SOURCE_RELAYS yes — Comma-separated upstream WS URLs to mirror from (read-only).
SYNC_KINDS no 30142,30023,30818,31922,31923,31924,31925,16,30222,5 Kinds to mirror. kind-0 is always excluded (see below), even if listed.
RECONCILE_INTERVAL no 10m Go duration for the paginated REQ backfill pass.
LIVE_TAIL no true Enable the live REQ subscription.
RECONCILE_SINCE no 0 Unix-time floor for the backfill filter (0 = full history). Bounds the oldest page.

kind-0 is never mirrored. The AMB relay rejects externally-pushed profiles and self-manages them via its own ProfileManager, so there is nothing for relay-sync to forward. kind-5 deletions DO propagate — they are author-signed, so replaying them through the write path applies correctly.

Run

# local
GOWORK=off go run .

# docker
docker build -t relay-sync . && docker run --env-file .env relay-sync

Deployed to the homelab dev stack via the Ansible relay-sync role (playbooks/deploy_relay_sync.yml in the homelab repo).

Invariants

  • Read-only on sources — no Publish path to a source exists; prod is never mutated. This is what makes "pull from prod into dev" safe.
  • Unidirectional (sources → target, never the reverse) — no sync loop can form.
  • DEV ONLY for now — SOURCE_RELAYS = prod, TARGET_RELAY = dev.

Design docs

docs/superpowers/specs/2026-06-23-relay-sync-design.md (design) and docs/superpowers/plans/2026-06-23-relay-sync.md (implementation plan).