Internationalization (i18n)

lakutata/com/i18n (package @lakutata/i18n) manages the messages of an application by keys: the code uses a key, and each language gives the text of that key in its own file. The messages are written in ICU MessageFormat; their numbers, currencies, dates and plurals are formatted for the language by the built-in Intl of Node.js, with no third-party library.

Message files

One JSON file per language, named after its language tag (BCP 47: en, zh-CN, zh-Hant-TW):

locales/
  en.json
  zh-CN.json

{
  "order": {
    "count": "{count, plural, =0 {No orders yet} one {# order} other {# orders}}",
    "created": "Order created for {customer}"
  }
}

The nesting of the JSON gives the keys: the keys above are order.count and order.created. No part of a key may contain ..

A language can also be a directory, in which the names of the files (and of the subdirectories) prefix their keys:

locales/
  zh-TW/
    order.json         → order.*
    admin/user.json    → admin.user.*

Both layouts can be mixed in one directory.

Configuration

import {Application} from 'lakutata'
import {buildI18nOptions} from 'lakutata/com/i18n'

Application
    .alias({'@locales': '/path/to/locales'})
    .run(() => ({
        id: 'shop.app',
        name: 'Shop',
        components: {
            i18n: buildI18nOptions({
                directory: '@locales',               //an alias, or several directories
                defaultLocale: 'en',                 //the default language, the last fallback (en by default)
                fallback: {'zh-TW': 'zh-CN'},        //the messages missing in zh-TW are taken from zh-CN
                currency: 'CNY',                     //the currency of {price, number, currency}
                timeZone: 'Asia/Shanghai'            //the time zone of dates and times (that of the process by default)
            })
        }
    }))

Name the component i18n: the middleware and commanderTranslator use that name.

The messages can also be given in the code instead of a directory: the messages option takes them by language, as object literals or imported JSON. They are then bundled with the code (lakutata package bundle), and no file is read at run time:

import {buildI18nOptions} from 'lakutata/com/i18n'
import en from './locales/en.json' with {type: 'json'}
import zhCN from './locales/zh-CN.json' with {type: 'json'}

const i18n = buildI18nOptions({
    messages: {
        'en': en,
        'zh-CN': zhCN,
        'ja': {order: {created: '{customer} の注文を作成しました'}}
    }
})

Option	Description
directory	The directories of the message files; aliases are accepted
locales	The supported languages, which negotiate() chooses from; by default, the default language followed by the languages found in the messages
defaultLocale	The default language, en by default
fallback	Where to look next when a language lacks a message ({'zh-TW': 'zh-CN'}, or a list)
messages	Messages by language given in the code, loaded after the directories (replacing the same keys); with them, directory is not needed
missing	What happens to a missing message or argument: 'throw' throws; 'key' returns the key (and leaves a missing argument as {name}), with a warning the first time. 'throw' by default, 'key' when NODE_ENV=production
currency	The currency when neither the message nor its arguments give one (ISO 4217)
timeZone	The time zone of dates and times (IANA)
validationMessages	Whether to register the built-in validation messages of the supported languages, true by default (see "Validation errors")

Usage

import {Provider} from 'lakutata'
import {Inject} from 'lakutata/decorator/di'
import {type I18n} from 'lakutata/com/i18n'

class OrderService extends Provider {
    @Inject('i18n')
    protected readonly i18n: I18n

    public summary(count: number, locale: string): string {
        return this.i18n.t('order.count', {count: count}, locale)
    }
}

Method	Description
t(key, params?, locale?)	The message of a key in a language, formatted with the arguments; in the default language when none is given
has(key, locale?)	Whether a language (or the ones it falls back to) has a key
format(message, params?, locale?)	Formats an ICU message that is not in the files
translator(locale)	A translator bound to a language (ITranslator), so that the language is not passed each time
negotiate(requested)	The supported language that best matches the requested ones (preferred first); the default language when none is supported
addMessages(locale, messages)	Adds messages to a language at run time, replacing the same keys
getLocales(), getDefaultLocale(), getKeys(locale), getTimeZone()	The supported languages, the default language, the keys a language defines itself, the time zone

t(), has(), format() and translator() do not negotiate: a language that is not supported is looked up as given, then falls back. addMessages() returns a promise that resolves once the validation messages of a new language are registered; t() gives the new messages as soon as addMessages() returns.

Answering a request in its language

With the I18nMiddleware on the entrypoint, each request gets a language; the objects of the request inject the Translator, which gives the messages in that language:

import {Application} from 'lakutata'
import {buildEntrypoints, Controller} from 'lakutata/com/entrypoint'
import {expressHTTPEntrypoint} from 'lakutata/adapter/express'
import {CLIAction, GET} from 'lakutata/decorator/ctrl'
import {Inject} from 'lakutata/decorator/di'
import {buildI18nOptions, I18nMiddleware, Translator} from 'lakutata/com/i18n'

class OrderController extends Controller {
    @Inject(Translator)
    protected readonly translator: Translator

    @GET('/orders/summary')
    @CLIAction('order-summary')
    public async summary(): Promise<string> {
        const count: string = this.translator.t('order.count', {count: 3})
        const total: string = this.translator.number(1234.5, {style: 'currency', currency: 'EUR'})
        const today: string = this.translator.date(new Date(), {dateStyle: 'long'})
        return `${this.translator.locale}: ${count}, ${total}, ${today}`
    }
}

Application.run(() => ({
    id: 'shop.app',
    name: 'Shop',
    components: {
        i18n: buildI18nOptions({directory: '@locales'}),
        entrypoint: buildEntrypoints({
            controllers: [OrderController],
            middlewares: [I18nMiddleware],
            http: expressHTTPEntrypoint({port: 3000})
        })
    },
    bootstrap: ['entrypoint']
}))

The languages a request asks for are read from the sources below, in order, then negotiated with the supported languages (see "Negotiation and fallbacks"); when none is supported, the request gets the default language:

Entrypoint	Sources (preferred first)
HTTP (Express, Fastify)	The query parameter ?lang=, the cookie lang, then the Accept-Language header (by quality)
Socket.IO	The query parameter lang, the cookie lang and Accept-Language of the connection's handshake
HTTP service adapter, JSON-RPC (HTTP, WebSocket)	The query parameter lang, the cookie lang and Accept-Language of the HTTP request (for WebSocket, of the request opening the connection)
Command line, Electron IPC, JSON-RPC over stdio	The variable named by environmentVariable (none by default), LANGUAGE, LC_ALL, LC_MESSAGES, LANG, then the language of the process
Schedules	None (the default language)

A query or cookie value that is not a language tag is skipped.

• The objects of a request (the controller, the @Scoped providers, the access control rules) inject the same Translator; translator.locale and context.locale are the language of the request.

• The middleware uses the component named i18n. Extend I18nMiddleware to use another component (@Inject('...') protected readonly i18n), to rename the query parameter or the cookie (queryParameter, cookie), to read an environment variable (environmentVariable), or to choose the language another way (for example, the language the signed-in user saved):

  import {type BaseContext} from 'lakutata/com/entrypoint'
  import {I18nMiddleware} from 'lakutata/com/i18n'
  
  export class LanguageMiddleware extends I18nMiddleware {
      protected readonly queryParameter: string = 'locale'
  
      protected readonly environmentVariable: string = 'SHOP_LANG'
  
      //The language saved by the signed-in user first
      protected requested(context: BaseContext): string[] {
          const saved: unknown = context.data.userLanguage
          return [...(typeof saved === 'string' ? [saved] : []), ...super.requested(context)]
      }
  }

• Without the I18nMiddleware, the methods of the Translator throw an I18nTranslatorException.

• A translator (the Translator, and the ITranslator of translator(locale)) has locale, t(key, params), has(key), format(message, params), number(value, options) and date(value, options).

• Outside a request (in the schedules or the singleton components), use the t(key, params, locale) and translator(locale) of the component.

• To choose the language yourself, parseAcceptLanguage(header) reads an Accept-Language header and environmentLocales() reads the languages of the environment (zh_CN.UTF-8 becomes zh-CN); pass their result to negotiate().

Help and prompts of a command line tool

The translate option of the Commander adapter (lakutata/adapter/commander) shows the help and the prompts of a command line tool in the language of the user: the descriptions of the program, of its commands and of their arguments and options, the hints about defaults and choices, the headings of the help (Usage:, Options:), the descriptions of --help and --version, and the questions asked for missing required options. The names of commands and options are not translated.

commanderTranslator() translates them with the messages of an i18n component:

import {Application} from 'lakutata'
import {buildEntrypoints, Controller} from 'lakutata/com/entrypoint'
import {commanderCLIEntrypoint} from 'lakutata/adapter/commander'
import {CLIAction} from 'lakutata/decorator/ctrl'
import {buildI18nOptions, commanderTranslator, I18nMiddleware} from 'lakutata/com/i18n'
import zhCN from '../locales/zh-CN.json' with {type: 'json'}

class GreetController extends Controller {
    @CLIAction('greet', {description: 'Greet someone'})
    public async greet(): Promise<string> {
        return 'Hello'
    }
}

Application.run(() => ({
    id: 'tool.app',
    name: 'Tool',
    components: {
        i18n: buildI18nOptions({messages: {'zh-CN': zhCN}}),
        entrypoint: buildEntrypoints({
            controllers: [GreetController],
            //The controllers that run the commands get a Translator in the same language
            middlewares: [I18nMiddleware],
            //TOOL_LANG=zh-CN tool --help, or LANG=zh_CN.UTF-8
            cli: commanderCLIEntrypoint({bin: 'tool', translate: commanderTranslator({environment: 'TOOL_LANG'})})
        })
    },
    bootstrap: ['entrypoint']
}))

{
  "cli": {
    "description": "示例工具",
    "commands": {
      "greet": {
        "description": "问候某人",
        "arguments": {"name": "被问候的人"},
        "options": {"times": "次数"}
      }
    },
    "help": {"usage": "用法：", "options": "选项：", "default": "默认：{value}", "hints": "（{hints}）", "separator": "，"}
  }
}

• The keys (with the prefix cli. by default, which the prefix option changes):
  • the program: description; the commands: commands.<command>.description, commands.<command>.arguments.<name>, commands.<command>.options.<property>;
  • the help: help.usage, help.arguments, help.options, help.commands, help.helpOption, help.helpCommand, help.versionOption;
  • the hints: help.default (default: {value}), help.choices (choices: {choices}), help.hints ( ({hints}), how the hints follow a description), help.separator (, ), help.negate (Turn off {flag});
  • the questions: prompt.json (Not a JSON object).
• A text without a message stays in the English of the code, so only the other languages need messages.
• The language is chosen once, when the adapter starts: from the variable named by the environment option, then LANGUAGE, LC_ALL, LC_MESSAGES, LANG, then the language of the process, and negotiated with the supported languages.
• Wide characters (Chinese, Japanese, Korean) count as two columns; the long lines of the help can break between them, and no line starts with punctuation.
• translate can be given a function too: translate: () => (text) => messages[text.key] ?? text.text.

Validation errors

With the I18nMiddleware, the validation errors of a request are in its language: the errors of its input (the DTO of @HTTPAction, @CLIAction, @ServiceAction), of the arguments and results of the methods it calls (@Accept, @Return), and of the validations its code runs (DTO.validate, DTO.validateAsync).

Validation messages for 20 languages are built in; when it starts, the component registers those of its supported languages (only the languages in use are loaded):

en, zh-CN, zh-TW, ja, ko, fr, de, es, pt-BR, it, nl, pl, ru, uk, tr, vi, th, id, hi, ar

Each supported language uses the closest built-in one: zh, zh-Hans, zh-SG use zh-CN; zh-Hant, zh-HK, zh-MO use zh-TW; other tags match by their language part (pt-PT uses pt-BR, fr-CA uses fr). A language without built-in messages uses those of the languages it falls back to, then English.

An application can replace or add validation messages in its own messages: under validation, keyed by error code, and written in the template syntax of the validation ({{#label}}, {{#limit}}) rather than ICU:

{
  "validation": {
    "any": {"required": "Please enter {{#label}}"},
    "string": {"min": "{{#label}} needs {{#limit}} characters at least"}
  }
}

• The validation messages of a language are layered: first the built-in and application messages of the languages it falls back to, then its own built-in and application messages; later layers replace earlier ones.
• validationMessages: false registers no built-in messages, only the validation messages of the application.
• The input of a request is validated as usual; only invalid input is validated a second time, in the language of the request, to get the messages. Valid input is not affected at all.
• A few messages contain English text from the validation library itself: the sort order (ascending), the reason for a reference error, the message of a custom assertion. These parts stay in English in every language.
• Without the i18n component, use the functions of lakutata: loadValidationMessages(language) loads the built-in messages of a language and registerValidationMessages(language, messages) registers them. A validation then chooses a language with its {errors: {language}} option, or runInValidationLanguage(language, fn) sets the language of every validation that fn runs, its asynchronous calls included.

Exception messages

With the I18nMiddleware, when a request throws an exception that has a message among the messages of the application, the exception gets that message in the language of the request. The key is errors.<errno of the exception>, and the message is formatted with the parameters of the template of the exception:

import {Exception} from 'lakutata'

class OrderNotFoundException extends Exception {
    public errno: string = 'E_ORDER_NOT_FOUND'
}

throw new OrderNotFoundException('Order {id} not found', {id: 7})

{"errors": {"E_ORDER_NOT_FOUND": "订单 {id} 不存在"}}

• A request in Chinese gets 订单 7 不存在, a request in English the original Order 7 not found; the errno and the other properties stay unchanged.
• Parameters passed one by one (new XxxException('{0} of {1}', a, b)) are referred to as {0}, {1} in the message.
• An exception that has no message in the application stays as it is; so does one whose message cannot be formatted (because of a missing parameter, for example), with a warning.
• Outside a request, exceptionParams(exception) (from lakutata) gives the parameters of an exception, so that you can translate it yourself.

Negotiation and fallbacks

Negotiation (negotiate) tries each requested language in turn: the same tag, then its language alone (en-GB → en), then another region of the same language (pt-PT → pt-BR, in the order of the supported languages). When none matches, it returns the default language.

Fallbacks: when a language lacks a message, the message is looked up in the languages of its fallback, then in its shorter tags (zh-Hant-TW → zh-Hant → zh), then in the default language. A message is always formatted in the requested language: an English message used as the fallback for a German request still writes 1.000.

Message syntax

Syntax	Example
Argument	Hello, {name}: a number is formatted for the language, a Date as a date, anything else as a string
Number	{n, number}, {n, number, integer}, {n, number, percent}
Currency	{price, number, currency} (the currency of the currency argument or of the configuration), {price, number, currency/EUR}
Date, time	{at, date, short}, {at, time, long}, styles short, medium (the default), long, full; a Date, a timestamp, a date string or an object with toDate()
Plural	{count, plural, =0 {none} one {# item} other {# items}}: the =value cases first, then the plural rules of the language; # is the formatted number; offset:1 is supported
Ordinal	{place, selectordinal, one {#st} two {#nd} few {#rd} other {#th}}
Select	{role, select, admin {Administrator} other {User}}
Quoting	'' is an apostrophe; '{literal}' writes the braces ('#' writes # in a plural)

plural, selectordinal and select need an other option. A message is parsed once, when it is first used; a syntax error throws an I18nMessageSyntaxException that gives the language, the key and the position, e.g. zh-CN: order.count: the "other" option is required at 1.

Typed keys

An application can declare its keys and their arguments; a wrong key or a missing argument then fails the compilation. lakutata i18n types generates the declaration from the messages of the default language (see below), or you can write it by hand:

//src/i18n.d.ts
import 'lakutata/com/i18n'

declare module 'lakutata/com/i18n' {
    interface I18nMessages {
        'order.count': { count: number }
        'order.created': { customer: string }
    }
}

Without them, t() accepts any key and any arguments.

Command line tools

The lakutata command line (@lakutata/cli) works on the message directories (--dir, locales by default; separate several with commas):

Command	Description
lakutata i18n check	Checks the messages of each language against the default language (--default, en by default). Errors: an ICU syntax error, a key the default language does not have, an argument the default message does not have, an unknown validation code, or a variable that belongs to another code. Warnings: a missing message (which falls back), an unused argument, a message identical to the default one (probably not translated yet). It fails on errors, and also on warnings with --strict, for CI
lakutata i18n types	Writes the type declaration of the keys of the default language and their arguments (--out, src/i18n.d.ts by default); add --module @lakutata/i18n for applications that import @lakutata/i18n
lakutata i18n add <locale>	Adds a language as a copy of the messages of the default language, ready to translate: a file or a directory, like the default language, without the validation part (the new language uses its built-in validation messages)

The checks are also available as functions: loadCatalogs(directories), checkCatalogs(catalogs, defaultLocale), generateMessageTypes(catalogs, defaultLocale).

Exceptions

Exception	When
I18nCatalogException	An invalid file or configuration: invalid JSON, a file name which is not a language tag, a missing directory, an alias not resolved
I18nMessageSyntaxException	A message has a syntax error
I18nMissingKeyException	A key is missing (missing: 'throw')
I18nMissingParameterException	An argument is missing (missing: 'throw'), or a currency message has no currency
I18nTranslatorException	The Translator is used without the I18nMiddleware, or outside a request