Every API email failed for about 3.5 hours after the Node 22 upgrade
Shipped 2026-10-19
The fix for Node 22’s removal of import assertions (#983) missed six call sites
in shared/email/service and shared/passcode/service. Those calls wrap their
options over several lines, and the sweep matched only the single-line form.
Every email the API sends renders its template through
setTemplateLocals(), which loads extras/locals.json. From the moment
iba-api came up on Node 22 (#984, about 19:00 UTC), that load threw
ERR_IMPORT_ATTRIBUTE_MISSING, so every email failed. That included login
passcodes and Convergence, Fuse Metrix and Tunn3l booking confirmations.
Members whose only verified channel is email couldn’t log in. The login panel reported “your email provider rejected the message”, because the passcode flow treats every send failure as a delivery failure, a transport error included.
Why nothing caught it
- The tests cover pure
domain/functions and never render an email. - The boot check used for #983 (
GET /api, then the translations endpoint) never sends mail. - The health check (
/api) passed, so the deploy took traffic normally.
Side effects
Each failed passcode called markEmailBounce, which opens the
security-answer fallback for that member for the bounce window. Failed
booking emails aren’t retried, so those members never got a confirmation.
Recovery
api/src/scripts/resend-outage-emails.js (TUN-899) resends the lost booking
confirmations and sign-up verification emails. It identifies bookings by their
connector_history timestamp, because connector_booking has none.
| Resent | Skipped | |
|---|---|---|
| Booking confirmations | Only where the update link still works: the entry unedited and at the 1-minute placeholder. Inactive only for the 3 bookings whose failed send logged that template | Bookings the member had already edited, deleted or set the time on |
| Sign-up verification | 7, each with a fresh 24-hour link | — |
The script doesn’t cover the other lost emails: passcodes (members retried by
logging in again), receipts, approvals and other notifications. Run it without
--send for a dry run. It’s safe to re-run: anything with a mandrill_logs
row for the member’s own address is skipped. Test sends redirected to
EMAIL_ALERT log under the same booking ids, so the address has to match.