Logging
The API and www write logs the same way, and the Datadog monitors depend on that shape. This page is the reference for both.
| App | Logger | Datadog service | source |
|---|---|---|---|
| API (prod) | api/src/shared/utils/logger/ | tunnelflight-api | iba-api |
| API (staging) | same | tunnelflight-dev-api | iba-dev-api |
| www (prod) | www/src/lib/logger/ (a copy of the API’s format) | tunnelflight-www | iba-www |
Logs go to stdout. DigitalOcean’s log forwarding sends them to Datadog
(us5), with the component name as service and the app name as source.
Admin and the assistant forward too, but as unstructured text.
One JSON line per event
Each logger call writes exactly one line. A second line would reach Datadog as a separate, unrelated event, which is how stack traces used to arrive one frame at a time.
Log an object that names the event, and pass the error itself:
logger.error({ event: 'redis.get_json_failed', key, error });
logger.warn({ event: 'auth.token_failed', error });
logger.info({ event: 'app_data.loaded' });| Field | Notes |
|---|---|
timestamp, level | level is what Datadog shows as the log status. |
message | The error’s message, a given message, or the event name. Only error lines are free-text searchable in Datadog, so search the rest by @event. |
event | Dotted name, area.what_happened. Monitors and searches key on it. |
error | { kind, message, code, stack }, Datadog’s standard error attributes, so Error Tracking groups by kind and stack. Axios errors keep status, method and url without the query string, and never config, which holds the request headers. On www, a wrapper error built with new Error('…', { cause }) also carries cause, serialised the same way, up to three levels; the API’s copy of the format doesn’t yet. |
status_value | Any field called status is renamed. Datadog reads a top-level status as the log level, so status: "failure" would turn the line into an emergency. |
A logging call never throws: circular references become [Circular] and
BigInts become strings.
Levels
| Level | Use for | Example |
|---|---|---|
error | Our failure; someone should look at it. | stripe_webhook.failed, redis.connect_failed |
warn | An expected rejection caused by the caller or member. | connector.signature_rejected, an expired session cookie |
info | Normal events worth keeping. | http.request, cron_finished, app.started |
debug | Per-request detail. Not kept in production. | redis.get_json |
The API logger is fixed at info. www reads LOG_LEVEL (default info), so
LOG_LEVEL=debug shows the detail locally.
The request log
Both apps write one http.request line per request at info, whatever the
status. The 5xx monitor filters on @response_status, not on level.
Neither logs DigitalOcean’s /health probe, which runs every 10 seconds.
| Field | API | www |
|---|---|---|
endpoint, method, response_status, execution_time, member_id, ip | ✓ | ✓ |
body (redacted), sp, traceData, useragent | ✓ | — |
www drops the query string from endpoint because its URLs carry reset tokens
and redirect targets. The API keeps it.
On www, member_id is filled on every page a signed-in member loads, because
page routes look the member up for the nav (setNav). Calls through the
/api proxy log member_id: null, because the proxy passes the cookie on
without reading it; the API’s own http.request line for the same call has the
member.
The API’s line can be large, so fitRequestLog trims it to 15,000 characters
to stay under Docker’s 16 KiB line split. Span data goes first, then extra
spans, then stored-procedure detail, then the body. A trimmed field lists
which steps ran.
Redaction
The API redacts request bodies, stored-procedure parameters and trace data
before logging (redact.js):
- Secrets (passwords, PINs, tokens, keys) become
[redacted: N-char string]. - Emails and phone numbers are masked.
- Personal fields (names, addresses, free text) become their length.
Keys are matched on whole words, so member_pin is caught and tunnel_name is
not. Connector bodies stored in connector_history keep member details,
because staff need them; nothing that leaves our systems does.
www logs no bodies, so it has no redaction step. Add one before logging anything a member typed.
Finding a line in Datadog
service:tunnelflight-api @event:connector.signature_rejected
service:tunnelflight-www @event:http.request @response_status:>=500
service:tunnelflight-api status:error- DigitalOcean sometimes delivers a line 2–5 times. Thresholds above zero allow for that.
- Cron scripts run as child processes. The runner forwards their output, so their lines arrive when the job finishes.
- The
[IBA]monitors are defined ininfra/datadog/monitors/and post to#iba-alerts. They matchservice:tunnelflight-api, so staging never triggers them.