Skip to main content

ADR-0007: Recipient Fan-out V2 Ships Behind a Staged Rollout Flag

Status​

Accepted

Context​

Push recipients were resolved as one endpoint per user (core/device_fanout.py pre-migration behavior). Moving to a per-install recipient set (so a member with two phones is reached on both) changes who receives azaan for every broadcast on the platform — a change that, if wrong, silently under- or over-delivers a live production notification with no easy way to detect the mistake after the fact.

Decision​

Ship the new resolution logic behind a three-state environment variable, FANOUT_V2 (off / shadow / enforce, default off), read on every call so a flip takes effect on the next broadcast with no restart or redeploy:

  • off — pre-migration behavior (one endpoint per user).
  • shadow — delivers exactly what off delivers, but additionally resolves the per-install set and records the difference on the broadcast's audit row, to gather evidence before flipping.
  • enforce — delivers to the per-install set.

An unrecognised value logs a warning and is treated as off — a configuration typo must not silently change who receives azaan. A second flag, FANOUT_LEGACY_BRIDGE, keeps a member with an approved membership but no usable per-install token resolvable from legacy users.*_token columns during the transition.

Alternatives Considered​

Not recorded in source material beyond the shadow-mode pattern itself, which ADR-0011 notes was deliberately not reused for the device registry unification because that cutover had no live production traffic at risk — implying the team's default posture is to reserve staged/shadow rollouts specifically for changes where production traffic is at risk, and skip them otherwise.

Consequences​

  • The eventual enforce state is gated on a measurable, queryable condition (broadcast_audit_logs.fanout_lost_users across a 7-day window showing zero losses over sufficient measured traffic) rather than a subjective "looks fine" judgment — see Backend Deployment for the exact gate query.
  • Rollback is a single environment variable change with no restart, no migration, and no data restore — rehearsed in fastapi-backend/test/test_rollback_rehearsal.py across all seven send paths.
  • Rollback does not undo everything: a push token retired while enforce was active stays retired, because the provider reported it dead — a fact independent of which recipient-set mode was active.
  • As of this writing, the flag remains at its default (off) in production — the decision recorded here is the rollout mechanism itself, not a claim that the fan-out has been fully enforced.