From 885361c33a0889165ea28415d0f3064010fa594b Mon Sep 17 00:00:00 2001 From: Omar Al-Ithawi Date: Sat, 23 Mar 2024 08:52:34 +0300 Subject: [PATCH] docs: design for atlas pull and translations management --- Makefile | 18 +++ docs/0002-atlas-translations-management.rst | 136 ++++++++++++++++++++ 2 files changed, 154 insertions(+) create mode 100644 Makefile create mode 100644 docs/0002-atlas-translations-management.rst diff --git a/Makefile b/Makefile new file mode 100644 index 000000000..2edabbae8 --- /dev/null +++ b/Makefile @@ -0,0 +1,18 @@ +clean_translations_temp_directory: + rm -rf I18N/I18N/ + + +pull_translations: + make clean_translations_temp_directory + atlas pull $(ATLAS_OPTIONS) translations/openedx-app-ios/I18N/I18N/:I18N/I18N/ + python scripts/split_translation_files.py + make clean_translations_temp_directory + + +combine_translations: + make clean_translations_temp_directory + python scripts/combine_translations_files.py + + +validate_translation_files: + python scripts/validate_translation_files.py diff --git a/docs/0002-atlas-translations-management.rst b/docs/0002-atlas-translations-management.rst new file mode 100644 index 000000000..e2872c096 --- /dev/null +++ b/docs/0002-atlas-translations-management.rst @@ -0,0 +1,136 @@ +Atlas Translations Management Design +#################################### + +Date: 25 March 2024 + +Status +****** +Pending + +Context +******* + +Open edX microservices and micro-frontends is using `OEP-58`_ to pull the latest +translations from the `openedx-translations repository`_. + +The gist of `OEP-58`_ is two processes: + +- Extract the English translation source entries and commit them into the `openedx-translations repository`_ +- Provide a command to pull translations in both development and web server image builds + +Microservices, micro-frontends, XBlocks and plugins uses one or two files which makes it easier for Translators +to locate and translate relevant resources. + +The `edx-platform`_ used to have 7 different resources, but this has been deprecated in favor of ``edx-platform.po`` +and ``edx-platform-js.po`` files. The details in the `edx-platform resources combination decision`_. + +The ``openedx-app-ios`` uses a clean architecture. Therefore, each app has its own i18n strings files. This is +useful for Developer Experience, but we believe it's not good for Translator Experience. + +Decision +******** + +This decision will require few changes to the translation workflow in comparison with the old ``edx-app-ios``: + +- Translation files are no longer comitted directly in the mobie app ``openedx-app-ios/android`` repositories +- Keep the i18n string files split into each clean architecture app without hurting Developer Experience. +- Identical string IDs across the application will be treated as the same string/translation. + For example, ``BUTTON_CLOSE="Close"`` in dashboard is the same ``BUTTON_CLOSE="Close"`` in profile app. + Otherwise, a validation script will throw an error and prevent merging the pull request. +- String files are combined into a single file before preparing it to Transifex + (through the `openedx-translations repository`_) +- Translators can translate the entire app as a single resource +- Before releasing to App Store, developers need to run a new ``make pull_translations`` command +- After translations is pulled the translation file is split again into each clean architecture application + as if it has been individual resource (a new script) + + +String files combination and splitting process +============================================== +This is a new process that's being introduced to have the best combination +of Developer Experience and Translator Experience. + +Translators need to have less resources as possible with strings de-duplicated, +while Engineers need to have clean architecture with strings split into each app to aid maintainability. + +Combining the files during ``combine_translations`` +--------------------------------------------------- + +However, combining the string files to aid Translator Experience in the following manner: + +Suppose we have two apps with one string source file in each:: + + Course/Course/en.lproj/Localizable.strings + Dashboard/Dashboard/en.lproj/Localizable.strings + +This script will collect the strings, remove uneeded comments and combine them in a temporary file:: + + I18N/I18N/en.lproj/Localizable.strings + + +This file will be committed into the `openedx-translations repository`_ as described in `OEP-58`_. + +This process happens entirely on the CI server (GitHub in this case) after each pull request merge without developer +intervention. + +GitHub Actions CI Script to avoid duplicate string IDs +------------------------------------------------------ + +A new script will be created to ensure we don't have duplicate IDs with different content. + +For example: + +If the Dashboard has ``BUTTON_CLOSE="Close"`` while Course app has ``BUTTON_CLOSE="CLOSE TAB"``, this +script should fail and require either making the string to match or the ID to be different. + +Spitting the files after ``pull_translations`` +---------------------------------------------- + +After pulling the translations from the `openedx-translations repository`_ via ``atlas pull``, there will be a single +strings file for each language: + +.. code:: + + I18N/I18N/uk.lproj/Localizable.strings + I18N/I18N/ar.lproj/Localizable.strings + I18N/I18N/fr.lproj/Localizable.strings + I18N/I18N/es-419.lproj/Localizable.strings + +- The string will run through each i18n strings file in the clean architecture. +- Identify which entries in the app strings file. +- Create strings file in the each app and put the strings that exists in the : + + +.. code:: + + Course/Course/uk.lproj/Localizable.strings + Course/Course/ar.lproj/Localizable.strings + Course/Course/fr.lproj/Localizable.strings + Course/Course/es-419.lproj/Localizable.strings + ... + Dashboard/Dashboard/uk.lproj/Localizable.strings + Dashboard/Dashboard/ar.lproj/Localizable.strings + Dashboard/Dashboard/fr.lproj/Localizable.strings + Dashboard/Dashboard/es-419.lproj/Localizable.strings + +This script should ensure that every entry in the source English file, should have an entry in the +translated files even if it has no translations. This will ensure app builds don't fail. + + +Python language for scripting +============================= +Python will be used in scripting the pull/push when needed. +This is in-line with the Theming tooling which is has been written in Python. + + +Alternatives +************ +- Writing scripts in native languages such as Kotlin and Swift has been dismissed as per core team request. +- Pushing multiple strings file resources for each mobile app to the `openedx-translations repository`_ is + dismissed to avoid having too many resources per mobile app in the Transifex project. + + +.. _OEP-58: https://docs.openedx.org/en/latest/developers/concepts/oep58.html +.. _openedx-translations repository: https://github.com/openedx/openedx-translations +.. _edx-platform: https://github.com/openedx/edx-platform +.. _edx-platform resources combination decision: https://github.com/openedx/edx-platform/blob/master/docs/decisions/0018-standarize-django-po-files.rst \ No newline at end of file