A Clojure library and leiningen plugin to make i18n easier. Provides
convenience functions to access the JVM's localization facilities and
automates managing messages and resource bundles. The tooling for
translators uses GNU gettext, so
that translators can work with .po
files which are widely used and for
which a huge amount of tooling exists.
The main.clj
and example/program
in this repo contain some simple code
that demonstrates how to use the translation functions. Before you can use
it, you need to run make
to generate the necessary
ResourceBundles
. After that, you can use lein run -m puppetlabs.i18n.main
or LANG=de_DE lein run -m puppetlabs.i18n.main
to look at English and German output.
Any Clojure code that needs to generate human-readable text must use the
functions puppetlabs.i18n.core/trs
and puppetlabs.i18n.core/tru
to do
so. Use trs
for messages that should be formatted in the system's locale,
for example log messages, and tru
for messages that will be shown to the
current user, for example an error that happened processing a web request.
When you require puppetlabs.i18n.core
into your namespace, you must
call it either trs
/tru
/trun
/trsn
or
i18n/trs
/i18n/tru
/i18n/trun
/i18n/trsn
(these are the names that
xgettext
will look for when it extracts strings) Typically, you
would have this in your namespace declaration
(ns puppetlabs.myproject
(:require [puppetlabs.i18n.core :as i18n :refer [trs trsn tru trun]]))
You use trs
/tru
very similar to how you use format
, except that the
format string must be a valid
java.text.MessageFormat
pattern. Note that these patterns offer support for localized formatting;
see the Javadocs for details. For example, you would write
(println (trs "It takes {0} software engineers {1} hours to change a light bulb" 3 9))
trsn
/trun
are similar to trs
/tru
except that they support pluralization
of strings. The first argument is the singular version of the string, the second
argument must be the plural form of the string. The third argument is the count value
to determine the level of pluralization. Any additional arguments will be used for additional formatting
(println (trsn "We found one cute puppy" "We found {0} cute puppies" 5))
Use format
to keep whitespace out of translations:
Replace the following string
(str "Verification of client "
client
" provided by HTTP header "
header
"\nSee documentation for details\n")
with something like:
(format "%s\n%s\n"
(trs "Verification of client {0} provided by HTTP header {1}" client header)
(trs "See documentation for details"))
Note that trs
and tru
work exactly the same way as format
in clojure, and
that it does not concatenate the way str
does. If used this way, trs
and
tru
will only recognize the first string given and no exception will be thrown.
Here is a crappy Ruby script that you can point at a Clojure source tree to find most of the strings that will need to be translated:
https://github.com/cprice404/stringtracker/blob/master/getstrings.rb
It is sometimes useful to tell the translator something about the message;
you can do that by preceding the message string in thetrs
/tru
invocation with a comment; in the above example you might want to say
;; This is really just a silly example message. It gets the following
;; arguments:
;; 0 : number of software engineers (an integer)
;; 1 : number of hours (also an integer)
(println (trs "It takes {0} software engineers {1} hours to change a light bulb" 3 9))
The comment will be copied to <project-name>.pot
together with the actual
message so that translators have some context on what they are working
on. Note that such comments must be immediately preceding the string that
is the message. When you write
;; No translator will see this
(trs
"A message on another line")
the comments do not get extracted into <project-name>.pot
.
Single quotes have a special meaning in
java.text.MessageFormat
patterns and need to be escaped with another single quote:
;; The following will produce "Hes going to the store"
(trs "He's going to the store")
;; You may want to supply a comment for devs and
;; translators to make sure the quoting is preserved.
;; The following will produce "He's going to the store"
(trs "He''s going to the store")
In some cases, messages need to be generated separately from when they're
translated; this is common in specialized def
forms or when defining a
constant for reuse. In that case, use the mark
macro to mark strings for
xgettext extraction, and the standard trs
/tru
at the translation site.
Extracting messages and building ResourceBundles requires the command line tools from GNU gettext which you will have to install manually.
If you are using Homebrew on OSX, run brew install gettext
. OSX provides the
BSD gettext library by default and because of that the Homebrew formula for
gettext
is keg-only. keg-only formulas are not symlinked. This can be remedied
by running brew link gettext --force
.
On Red Hat-based operating systems, including Fedora, install gettext via
yum install gettext
- In your
project.clj
, add[puppetlabs/i18n "0.9.0"]
to your project's :plugins and :dependencies vectors (without the version number in :dependencies if your project uses clj-parent). Also addto merge in the translation locales.clj from upstream projects and make sure the plugin is invoked.:uberjar-merge-with {"locales.clj" [(comp read-string slurp) (fn [new prev] (if (map? prev) [new prev] (conj prev new))) #(spit %1 (pr-str %2))]} :prep-tasks [["i18n" "make"] "javac" "compile"]
- Run
lein i18n init
. This will- put a
Makefile.i18n
intodev-resources/
in your project and include it into an existing toplevelMakefile
resp. create a new one that does that. You should check these files into you source control system. - put scripts for comparing and updating PO & POT files in
dev-resources/i18n/bin
. (These scripts and the Makefile.i18n are updated to include your project name, so that the POT file will be named after your project.) These are used by the clj-i18n CI job and can be ignored (they are added to the project's .gitignore file).
- put a
- If there are namespaces/packages in your project with names which do not
start with a prefix derived from the project name: you'll need to list all
of your namespaces/package name prefixes in the
PACKAGES
variable in the top levelMakefile
before the inclusion of thedev-resources/Makefile.i18n
- Add a job using CI job configs' i18n-clj template to your project's CI pipelines. This job will automatically update the POT file when externalized strings are added or changed in the project.
This setup will ensure that compiling your project will also regenerate the Java
ResourceBundle
classes that your code needs to do translations.
You can manually regenerate these files by running make i18n
. Additional
information about the Make targets is available through running make help
.
Note: make i18n
will fail if you don't have at least one string wrapped with a translation function, i.e. trs or tru.
The i18n tools maintain files in three directories:
- message catalogs in
locales/
- compiled translations in
resources/
- temporary files in the project root
/
, for example/mp-e
You should check the files in locales/
into source control, but not the ones
in resources/
or the mp-*
files. A sample .gitignore
for a project might
look something like:
# Ignore these files for clj-i18n
/resources/example/*.class
/resources/locales.clj
/mp-*
If you are working on an HTTP service, you will also need to make sure that
we properly handle the locale that the user requests via the
Accept-Language
header. The library contains the function
locale-negotiator
that you should use as a Ring middleware. It stores the
negotiated locale in the *locale*
binding - ultimately, that's the locale
that the tru
macro will use.
For testing, it is often useful to introduce translations that are
maintained separately from the generally used locales, and whose change is
controlled by developers rather than translators. The i18n
library uses
the file resources/locales.clj
, which is generated and maintained by the
make
targets, to track for which locales translations are
available. Additional locales can be made available by putting one or more
locales.clj
files on the class path whose :package
entry is the same as
the one in resources/locales.clj
but that mentions additional
:locales
.
That makes it possible to introduce additional locales for testing by doing the following:
- Create a file
test/locales.clj
by copyingresources/locales.clj
and edit the copy by changing the:locales
entry to the languages that should be used for testing - For each of the additional locales, create a message catalog. It will
generally be easiest to base that message catalog on properties files
rather than on
.po
files. If you added theeo
locale, you need to create a filetest/<package path>/Messages_eo.properties
. Note that pluralization is not currently supported in properties files. - Use those additional locales in your tests. The
test/
directory of this library has an example of that in thetest-tru
test incore_test.clj
.
The macro with-user-locale
can be used to change the locale under which a
certain test should run, for example, with
(let [eo (string-as-locale "eo")]
(with-user-locale eo
(testing "user-locale is Esperanto"
(is (= eo (user-locale))))))
Translators for Puppet, don't use this workflow. In the Puppet workflow POs are generated in our translation tool, from an up to date POT. We don't, as developers, update or commit POs. So this may only be relevant should a developer want to test or generate a test language.
Prior to generating a po file, make sure the POT is up to date by running make i18n
.
This will put new msgids from the app, into the POT.
To create a .po
file for the language eo:
make locales/eo.po
Note this will just take the contents of the current POT and write the PO from it. Subsequent runs will not keep that file up to date.
Prior to updating a po file, make sure the POT is up to date by running make i18n
.
This will put new msgids from the app, into the POT.
To update the po: msgmerge -U locales/eo.po locales/.pot
This uses the contents of the current POT to update msgids in the target po (eo.po).
When it comes time to make a release, or if you want to use your code in a
different locale before then, you need to generate Java ResourceBundle
classes that contain the localized messages. This is done by running make msgfmt
on your project.
The code is set up as an ordinary leiningen project, with the one exception
that you need to run make
before running lein test
or lein run
, as
there are messages that need to be turned into a message bundle.
Maintainers: David Lutterkort [email protected] and Libby Molina [email protected]
Tickets: File bug tickets at https://tickets.puppetlabs.com/browse/INTL, and add the clj
component to the ticket.