Skip to Content

Every API email failed for about 3.5 hours after the Node 22 upgrade

Shipped 2026-10-19

The fix for Node 22’s removal of import assertions (#983) missed six call sites in shared/email/service and shared/passcode/service. Those calls wrap their options over several lines, and the sweep matched only the single-line form.

Every email the API sends renders its template through setTemplateLocals(), which loads extras/locals.json. From the moment iba-api came up on Node 22 (#984, about 19:00 UTC), that load threw ERR_IMPORT_ATTRIBUTE_MISSING, so every email failed. That included login passcodes and Convergence, Fuse Metrix and Tunn3l booking confirmations.

Members whose only verified channel is email couldn’t log in. The login panel reported “your email provider rejected the message”, because the passcode flow treats every send failure as a delivery failure, a transport error included.

Why nothing caught it

  • The tests cover pure domain/ functions and never render an email.
  • The boot check used for #983 (GET /api, then the translations endpoint) never sends mail.
  • The health check (/api) passed, so the deploy took traffic normally.

Side effects

Each failed passcode called markEmailBounce, which opens the security-answer fallback for that member for the bounce window. Failed booking emails aren’t retried, so those members never got a confirmation.

Recovery

api/src/scripts/resend-outage-emails.js (TUN-899) resends the lost booking confirmations and sign-up verification emails. It identifies bookings by their connector_history timestamp, because connector_booking has none.

EmailResentSkipped
Booking confirmationsOnly where the update link still works: the entry unedited and at the 1-minute placeholder. Inactive only for the 3 bookings whose failed send logged that templateBookings the member had already edited, deleted or set the time on
Sign-up verification7, each with a fresh 24-hour link—

The script doesn’t cover the other lost emails: passcodes (members retried by logging in again), receipts, approvals and other notifications. Run it without --send for a dry run. It’s safe to re-run: anything with a mandrill_logs row for the member’s own address is skipped. Test sends redirected to EMAIL_ALERT log under the same booking ids, so the address has to match.

Last updated on