Google Translate

CMS content — news articles, flight skills, reference materials — is machine-translated into French and Spanish when an admin saves it. This page covers where that happens, and how to attach the IBA translation glossary so the translator stops rendering currency as money and flyer as a leaflet.

Where translation happens

Two separate code paths, on two different versions of the Google API:

Callers: api/src/features/admin/cms/{news,skills,reference-materials}/service/index.js.

What actually gets translated

Translation fires only when an admin saves CMS content — on create and on update — never on page render. Target languages come from langArr in api/src/shared/utils/constant/app.constant.js, currently ['en', 'fr', 'es']; en is skipped as the source.

Text fields, by entity

In news, the translated copy also gets a /en/ to /{lang}/ URL rewrite applied afterwards, so internal links point at the right locale.

Documents

Only PDFs. The guard is extantion === 'pdf' — any other uploaded file type (images, Word documents, spreadsheets) is pushed to S3 untranslated, silently and by design.

The v3 request also sets isTranslateNativePdfOnly: true, which means native PDFs only. A scanned or image-based PDF will not be OCR’d and comes back untranslated.

The flow is: upload the English original to GCS → translateDocument per language → download the translated PDF from translated/<folder>/ to /tmp → upload to S3 under that locale.

Translations are written once, at save time

Each language is stored as its own database row (insertAllLang), then the Redis base-data cache is reloaded. Nothing re-translates existing content.

This is the single most important consequence for anyone maintaining translations: improving the glossary does not repair content already in the database. Attaching the glossary fixes everything published from that point on. Existing rows have to be re-saved or corrected by hand — which is exactly why TUN-655  needed a manual sweep of roughly 420 strings.

Failure behaviour

Failures degrade to English and are reported, per language. The rules:

• A locale that fails to translate keeps the English text, and the document for that locale is the English original rather than missing. One locale failing never blocks the others.
• The save still succeeds. Content already committed to the database is never rolled back or turned into a 500 because a translation or an S3 upload failed afterwards.
• The response carries a warnings array alongside the usual message, e.g. "Document translation failed for 'es' — English used instead". The field is only present when something degraded, so existing clients reading message are unaffected.
• Every failure is logged with a translation: { entity, lang, stage } object for Datadog.

The three google-bucket helpers (googleUpload, googleTranslate, googleDownload) throw on failure. They previously resolved true/false and swallowed the error — googleUpload reported success even when the upload never happened, which meant every subsequent translation failed against a source file that was not there.

Historical bug worth knowing about. Every language used to download its translated PDF to the same /tmp/<name> path. If one locale’s download failed, the previous locale’s PDF was still sitting there and got uploaded under the wrong language — Spanish readers could be served the French document. Each language now downloads to /tmp/<lang>-<name>, and the file is only repointed after the download actually succeeds.

Version detail

The document path already authenticates against v3 — project tunnelflight-493021, region us-central1, service-account credentials from GOOGLE_TRASLATE_PRIVATEKEY and GOOGLE_TRASLATE_CLIENT_EMAIL, bucket translation-document-one. So this is not a new integration; it is a glossary resource plus a config object, and one small migration of the text path.

What a Google glossary is

A unidirectional glossary is a two-column TSV (source term, target term) held in Cloud Storage and registered as a long-lived resource in the Translation API. On each request you name the glossary and Google does exact-match term replacement on top of the normal output. One glossary per language pair.

The two files live in the repo at api/src/i18n/glossary/:

• iba-glossary-en-fr.tsv
• iba-glossary-en-es.tsv

They mirror the term tables in Convention: Translation glossary. Keep the two in step by hand — nothing syncs them.

Step 1 — upload the TSVs and register the glossaries

One-off per language pair. The bucket already exists.

gsutil cp api/src/i18n/glossary/iba-glossary-en-fr.tsv gs://translation-document-one/glossary/
gsutil cp api/src/i18n/glossary/iba-glossary-en-es.tsv gs://translation-document-one/glossary/

Then register each one. A small script is the tidiest home for this, so it can be re-run after a glossary edit:

import { TranslationServiceClient } from '@google-cloud/translate';
 
const projectId = 'tunnelflight-493021';
const location  = 'us-central1';
const bucket    = 'translation-document-one';
 
const client = new TranslationServiceClient({
  credentials: {
    private_key: process.env.GOOGLE_TRASLATE_PRIVATEKEY.replace(/\n/g, '\n'),
    client_email: process.env.GOOGLE_TRASLATE_CLIENT_EMAIL,
  },
});
 
const createGlossary = async (lang) => {
  const name = client.glossaryPath(projectId, location, `iba-en-${lang}`);
  // Glossaries are immutable — delete before recreating.
  await client.deleteGlossary({ name }).catch(() => {});
  const [operation] = await client.createGlossary({
    parent: client.locationPath(projectId, location),
    glossary: {
      name,
      languagePair: { sourceLanguageCode: 'en', targetLanguageCode: lang },
      inputConfig: {
        gcsSource: { inputUri: `gs://${bucket}/glossary/iba-glossary-en-${lang}.tsv` },
      },
    },
  });
  await operation.promise();   // takes about a minute
  return name;
};
 
for (const lang of ['fr', 'es']) {
  console.log('created', await createGlossary(lang));
}

Region matters. Glossaries are regional resources. A glossary created in us-central1 can only be used by requests whose parent is us-central1 — not global. The document path already uses us-central1, so the text path must move there too rather than staying on global.

Step 2 — attach the glossary to document translation

In googleTranslate, the request object gains one field:

const request = {
  parent: translationClient.locationPath(projectId, location),
  documentInputConfig,
  documentOutputConfig,
  sourceLanguageCode: 'en',
  targetLanguageCode: lang,
  isTranslateNativePdfOnly: true,
  omitApiKey: true,
  glossaryConfig: {
    glossary: translationClient.glossaryPath(projectId, location, `iba-en-${lang}`),
    ignoreCase: true,
  },
};

That is the entire change on this path.

Step 3 — migrate the text path from v2 to v3

api/src/shared/utils/translate/index.js is the code that produced the original bug. v2 cannot take a glossary at all, so it has to move to v3:

import { TranslationServiceClient } from '@google-cloud/translate';
 
const projectId = 'tunnelflight-493021';
const location  = 'us-central1';
 
const translationClient = new TranslationServiceClient({
  credentials: {
    private_key: process.env.GOOGLE_TRASLATE_PRIVATEKEY.replace(/\n/g, '\n'),
    client_email: process.env.GOOGLE_TRASLATE_CLIENT_EMAIL,
  },
});
 
const convertUsingGoogle = async (text, lang) => {
  if (!text) return '';
 
  const [response] = await translationClient.translateText({
    parent: translationClient.locationPath(projectId, location),
    contents: [text],
    mimeType: 'text/html',              // CMS copy is HTML — v2 was guessing
    sourceLanguageCode: 'en',
    targetLanguageCode: lang,
    glossaryConfig: {
      glossary: translationClient.glossaryPath(projectId, location, `iba-en-${lang}`),
      ignoreCase: true,
    },
  });
 
  // With a glossary applied, the result is in glossaryTranslations.
  const translated = (response.glossaryTranslations?.[0] ?? response.translations[0]).translatedText;
  return translated.replace(/\/en\//g, `/${lang}/`);
};

Three things to watch:

1. The response shape changes. v3 returns translations[], and when a glossary is applied it also returns glossaryTranslations[]. Read the glossary one, falling back to the plain one — otherwise the glossary silently does nothing and everything still looks fine.
2. Set mimeType: 'text/html' explicitly. CMS copy contains markup; v2 was auto-detecting, and the existing /en/ to /{lang}/ URL rewrite downstream suggests it has been mangling links.
3. Callers are unchanged. convertUsingGoogle(content, lang) keeps its signature, so the news, skills and reference-materials services need no edit.

Caveat: glossaries are context-blind

Google applies glossary terms by exact match, with no understanding of sense. The glossary maps currency to validité / vigencia. That is right nearly all the time in IBA content, but it would also fire on a genuinely monetary sentence such as “We currently support the currencies below”.

That particular string lives in a static i18n file, not in CMS content, so it is unaffected. But it is the shape of the risk:

• CMS content is about flying, not billing. The glossary is safe there, which is exactly where it is applied.
• Do not apply the glossary to payment, fees or checkout copy if that ever becomes machine-translated. Those need devise / moneda — see the money exception in Convention: Translation glossary.

Cost and rollout

• Glossary-enabled requests bill at the Translation Advanced (v3) rate, higher than the Basic (v2) rate the text path uses today. Volume is low, so the absolute cost is small — but confirm current per-character pricing before rollout rather than assuming.
• Creating and storing glossaries is free; only translation requests are charged.
• Suggested order: register the glossaries → attach to documents (lowest risk, v3 already in use) → migrate the text path → re-save one news article in staging and check the FR/ES output reads validité / vigencia.

The LLM alternative

The original estimate on TUN-655  was 4 hours for the glossary route and 6–8 for moving to an LLM. The glossary route is the right first move: it is bounded, it reuses an integration that already works, and the glossary itself has been signed off by native reviewers.

If quality is still short afterwards, an LLM becomes attractive for a reason a glossary cannot match — it takes the glossary as context rather than as find-and-replace, so it can tell the money sense from the flying sense, get gender and agreement right, and hold register consistent. Those three things produced the worst of the original errors (une balle volador, Choisi un Soufflerie). The assistant/ project already has that plumbing.