From 51d73beb0bb52e88ec11c96fc1fba71a639227c5 Mon Sep 17 00:00:00 2001 From: Omar Al-Ithawi Date: Wed, 27 Mar 2024 15:36:23 +0300 Subject: [PATCH] docs: design for atlas pull and translations management --- Makefile | 4 - docs/0002-atlas-translations-management.rst | 118 ++++++++++++-------- 2 files changed, 72 insertions(+), 50 deletions(-) diff --git a/Makefile b/Makefile index 2edabbae8..ab837c3cf 100644 --- a/Makefile +++ b/Makefile @@ -12,7 +12,3 @@ pull_translations: combine_translations: make clean_translations_temp_directory python scripts/combine_translations_files.py - - -validate_translation_files: - python scripts/validate_translation_files.py diff --git a/docs/0002-atlas-translations-management.rst b/docs/0002-atlas-translations-management.rst index e2872c096..4d6fd4dd0 100644 --- a/docs/0002-atlas-translations-management.rst +++ b/docs/0002-atlas-translations-management.rst @@ -5,88 +5,108 @@ Date: 25 March 2024 Status ****** -Pending +Accepted Context ******* -Open edX microservices and micro-frontends is using `OEP-58`_ to pull the latest +Open edX microservices and micro-frontends use the processes outlined in `OEP-58`_ to pull the latest translations from the `openedx-translations repository`_. -The gist of `OEP-58`_ is two processes: +The main changes that `OEP-58`_ introduced to the Open edX project are the following: - Extract the English translation source entries and commit them into the `openedx-translations repository`_ -- Provide a command to pull translations in both development and web server image builds +- The `GitHub Transifex app`_ will automatically commit the translations to the `openedx-translations repository`_ +- Provide a command that utilizes the `atlas`_ tool to pull translations in both development and web server image builds -Microservices, micro-frontends, XBlocks and plugins uses one or two files which makes it easier for Translators +Microservices, micro-frontends, XBlocks and plugins use as few translation files as possible to makes it easier for Translators to locate and translate relevant resources. -The `edx-platform`_ used to have 7 different resources, but this has been deprecated in favor of ``edx-platform.po`` -and ``edx-platform-js.po`` files. The details in the `edx-platform resources combination decision`_. +`edx-platform`_ translations strings used to be split between 7 files, but this has been deprecated in favor of +``edx-platform.po`` and ``edx-platform-js.po`` files. +More details are available in the `edx-platform resources combination decision document`_. -The ``openedx-app-ios`` uses a clean architecture. Therefore, each app has its own i18n strings files. This is -useful for Developer Experience, but we believe it's not good for Translator Experience. +The `openedx-app-ios`_ uses a clean architecture. Therefore, each clean architecture module has its own i18n +localizable strings file. This is useful for Developer Experience. +However, it's not good for Translator Experience because translators +often are unaware of the application engineering architecture and would like to be able to quickly search for strings +regardless of their location in the code. Decision ******** -This decision will require few changes to the translation workflow in comparison with the old ``edx-app-ios``: +This decision requires a few changes to the mobile app translation workflow: -- Translation files are no longer comitted directly in the mobie app ``openedx-app-ios/android`` repositories -- Keep the i18n string files split into each clean architecture app without hurting Developer Experience. -- Identical string IDs across the application will be treated as the same string/translation. - For example, ``BUTTON_CLOSE="Close"`` in dashboard is the same ``BUTTON_CLOSE="Close"`` in profile app. - Otherwise, a validation script will throw an error and prevent merging the pull request. -- String files are combined into a single file before preparing it to Transifex +- Translation files are no longer comitted directly to the mobie app repositories (`openedx-app-ios`_ / `openedx-app-android`_) +- Keep the i18n string files split into each module without hurting Developer Experience. +- String files are combined into a single file before uploading it to Transifex (through the `openedx-translations repository`_) -- Translators can translate the entire app as a single resource +- Translators can translate the entire app within a single transifex resource - Before releasing to App Store, developers need to run a new ``make pull_translations`` command -- After translations is pulled the translation file is split again into each clean architecture application +- After translations are pulled the translation files are split into each module as if it has been individual resource (a new script) -String files combination and splitting process -============================================== +String file combination and splitting process +============================================= This is a new process that's being introduced to have the best combination of Developer Experience and Translator Experience. -Translators need to have less resources as possible with strings de-duplicated, +Translators need to have less resources as possible, while Engineers need to have clean architecture with strings split into each app to aid maintainability. Combining the files during ``combine_translations`` --------------------------------------------------- -However, combining the string files to aid Translator Experience in the following manner: +Combining the string files to aid Translator Experience in the following manner: -Suppose we have two apps with one string source file in each:: +Suppose we have two modules with one Localizable.strings file in each:: Course/Course/en.lproj/Localizable.strings - Dashboard/Dashboard/en.lproj/Localizable.strings + "COURSE_CONTAINER.VIDEOS" = "Videos"; + "COURSE_CONTAINER.DATES" = "Dates"; + "COURSE_CONTAINER.DISCUSSIONS" = "Discussions"; + "COURSE_CONTAINER.HANDOUTS" = "Handouts"; + "COURSE_CONTAINER.HANDOUTS_IN_DEVELOPING" = "Handouts In developing"; -This script will collect the strings, remove uneeded comments and combine them in a temporary file:: + "HANDOUTS_CELL_HANDOUTS.TITLE" = "Handouts"; + "HANDOUTS_CELL_ANNOUNCEMENTS.TITLE" = "Announcements"; - I18N/I18N/en.lproj/Localizable.strings + Dashboard/Dashboard/en.lproj/Localizable.strings: + "TITLE" = "Dashboard"; + "HEADER.COURSES" = "Courses"; + "HEADER.WELCOME_BACK" = "Welcome back. Let's keep learning."; + + "EMPTY.TITLE" = "It's empty"; + "EMPTY.SUBTITLE" = "You are not enrolled in any courses yet."; +This combine translations script will collect the strings, remove unneeded comments and combine them in a temporary +file while prefixing each entry with its module name. This will make splitting clear and +avoid collision in string IDs:: -This file will be committed into the `openedx-translations repository`_ as described in `OEP-58`_. + I18N/I18N/en.lproj/Localizable.strings + "COURSE.COURSE_CONTAINER.VIDEOS" = "Videos"; + "COURSE.COURSE_CONTAINER.DATES" = "Dates"; + "COURSE.COURSE_CONTAINER.DISCUSSIONS" = "Discussions"; + "COURSE.COURSE_CONTAINER.HANDOUTS" = "Handouts"; + "COURSE.COURSE_CONTAINER.HANDOUTS_IN_DEVELOPING" = "Handouts In developing"; + "COURSE.HANDOUTS_CELL_HANDOUTS.TITLE" = "Handouts"; + "COURSE.HANDOUTS_CELL_ANNOUNCEMENTS.TITLE" = "Announcements"; + "DASHBOARD.TITLE" = "Dashboard"; + "DASHBOARD.HEADER.COURSES" = "Courses"; + "DASHBOARD.HEADER.WELCOME_BACK" = "Welcome back. Let's keep learning."; + "DASHBOARD.EMPTY.TITLE" = "It's empty"; + "DASHBOARD.EMPTY.SUBTITLE" = "You are not enrolled in any courses yet."; + +This combined file will be pushed to the `openedx-translations repository`_ as described in `OEP-58`_. This process happens entirely on the CI server (GitHub in this case) after each pull request merge without developer intervention. -GitHub Actions CI Script to avoid duplicate string IDs ------------------------------------------------------- - -A new script will be created to ensure we don't have duplicate IDs with different content. - -For example: +Splitting the files after ``pull_translations`` +----------------------------------------------- -If the Dashboard has ``BUTTON_CLOSE="Close"`` while Course app has ``BUTTON_CLOSE="CLOSE TAB"``, this -script should fail and require either making the string to match or the ID to be different. - -Spitting the files after ``pull_translations`` ----------------------------------------------- - -After pulling the translations from the `openedx-translations repository`_ via ``atlas pull``, there will be a single +After pulling the translations from the `openedx-translations repository`_ via `atlas pull`_, there will be a single strings file for each language: .. code:: @@ -96,10 +116,10 @@ strings file for each language: I18N/I18N/fr.lproj/Localizable.strings I18N/I18N/es-419.lproj/Localizable.strings -- The string will run through each i18n strings file in the clean architecture. -- Identify which entries in the app strings file. -- Create strings file in the each app and put the strings that exists in the : - +- The script will run through each module's ``en.lproj/Localizable.strings`` +- Identify which entries in the app strings file are translated in the e.g. ``I18N/I18N/ar.lproj/Localizable.strings`` file. +- Create module strings file in the each module and put the strings that exists in the ``en.lproj/Localizable.strings`` file +- The automatic module name prefix that ``combine_translations`` script has added is removed .. code:: @@ -128,9 +148,15 @@ Alternatives - Writing scripts in native languages such as Kotlin and Swift has been dismissed as per core team request. - Pushing multiple strings file resources for each mobile app to the `openedx-translations repository`_ is dismissed to avoid having too many resources per mobile app in the Transifex project. - +- Combining the strings files without a prefix is dismissed because it needs a dedicated validation script which + could confuse the community contributors by adding a new rule to ensure no duplicate strings are found. .. _OEP-58: https://docs.openedx.org/en/latest/developers/concepts/oep58.html .. _openedx-translations repository: https://github.com/openedx/openedx-translations .. _edx-platform: https://github.com/openedx/edx-platform -.. _edx-platform resources combination decision: https://github.com/openedx/edx-platform/blob/master/docs/decisions/0018-standarize-django-po-files.rst \ No newline at end of file +.. _atlas: https://github.com/openedx/openedx-atlas +.. _atlas pull: https://github.com/openedx/openedx-atlas?tab=readme-ov-file#usage +.. _edx-platform resources combination decision document: https://github.com/openedx/edx-platform/blob/master/docs/decisions/0018-standarize-django-po-files.rst +.. _GitHub Transifex app: https://github.com/apps/transifex-integration +.. _openedx-app-android: https://github.com/openedx/openedx-app-android +.. _openedx-app-ios: https://github.com/openedx/openedx-app-ios