-
Notifications
You must be signed in to change notification settings - Fork 38
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Add the ability to translate submissions (redux) (#4219)
This builds on the #4134 PR that initially introduced machine translations into Hypha. This isolates the translation behavior; putting pip dependencies in a separate `requirements-translate.txt` and will not attempt any translate imports unless the setting for it is true. Other small changes are also a full docs page explaining how to install language packages & changing the setting once again from `SUBMISSION_TRANSLATIONS_ENABLED` to `APPLICATION_TRANSLATIONS_ENABLED` to reflect the system wide shift away from submission terminology.
- Loading branch information
Showing
33 changed files
with
1,697 additions
and
9 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,29 @@ | ||
| # Machine translations | ||
|
|
||
| Hypha has the ability to utilize [argostranslate](https://github.com/argosopentech/argos-translate) for machine translations of submitted application content. This is disabled by default and the dependencies are not installed to prevent unneeded bloat due to [PyTorch](https://pytorch.org/)'s large language models. | ||
|
|
||
|
|
||
| ## Installing dependencies | ||
|
|
||
| As referenced in the [production deployment guide](../deployment/production/stand-alone.md), it is required to install the dependencies needed for machine translation dependencies via | ||
|
|
||
| ```bash | ||
| python3 -m pip install -r requirements-translate.txt | ||
| ``` | ||
|
|
||
| This requirements file will specifically attempt to install the CPU version of [PyTorch](https://pytorch.org/) if available on the detected platform to play better with heroku (doesn't support GPU processing) and to minimize package bloat (CPU package is ~300MB less than the normal GPU). Depending on your use case, you may want to adjust this. | ||
|
|
||
|
|
||
| ## Installing languages | ||
|
|
||
| Argostranslate handles translations via it's own packages - ie. Arabic -> English translation would be one package, while English -> Arabic would be another. | ||
|
|
||
| Installing/uninstalling these packages can be done with the management commands `install_languages`/`uninstall_languages` respectively, utilizing the format of <from language code>_<to language code>. For example, installing the Arabic -> English & French -> English packages would look like: | ||
|
|
||
| ```bash | ||
| python3 manage.py install_languages ar_en fr_en | ||
| ``` | ||
|
|
||
| ## Enabling on the system | ||
|
|
||
| To enable machine translations on an instance, the proper configuration variables need to be set. These can be found in the [configuration options](configuration.md#hypha-custom-settings) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
125 changes: 125 additions & 0 deletions
125
hypha/apply/funds/templates/funds/includes/translate_application_form.html
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,125 @@ | ||
| {% load i18n static heroicons translate_tags %} | ||
| {% modal_title %}{% trans "Translate" %}{% endmodal_title %} | ||
| <form | ||
| class="px-2 pb-4 form" | ||
| id="translate_form" | ||
| method="POST" | ||
| action="{{ request.path }}" | ||
| hx-post="{{ request.path }}" | ||
| > | ||
| {% csrf_token %} | ||
| {{ form.media }} | ||
| {% for hidden in form.hidden_fields %} | ||
| {{ hidden }} | ||
| {% endfor %} | ||
|
|
||
| <div> | ||
| {% if form.errors %} | ||
| {% for field in form %} | ||
| {% for error in field.errors %} | ||
| <div class="alert alert-danger"> | ||
| <strong>{{ error|escape }}</strong> | ||
| </div> | ||
| {% endfor %} | ||
| {% endfor %} | ||
| {% for error in form.non_field_errors %} | ||
| <div class="alert alert-danger"> | ||
| <strong>{{ error|escape }}</strong> | ||
| </div> | ||
| {% endfor %} | ||
| {% endif %} | ||
| <div class="flex mt-3 justify-center space-x-2"> | ||
| <fieldset class="w-2/5"> | ||
| <div> | ||
| {{ form.from_lang }} | ||
| </div> | ||
| </fieldset> | ||
| <div class="flex flex-col justify-center"> | ||
| {% heroicon_outline "arrow-right" aria_hidden="true" size=15 stroke_width=2 class="inline align-baseline me-1" %} | ||
| </div> | ||
| <fieldset class="w-2/5"> | ||
| <div> | ||
| {{ form.to_lang }} | ||
| </div> | ||
| </fieldset> | ||
| </div> | ||
| </div> | ||
|
|
||
| <div class="mt-5 sm:gap-4 sm:mt-4 sm:flex sm:flex-row-reverse"> | ||
|
|
||
| {# Button text inserted below to prevent redundant translations #} | ||
| <button id="translate-btn" class="w-full button button--primary sm:w-auto" type="submit"></button> | ||
|
|
||
| <button | ||
| type="button" | ||
| class="inline-flex items-center justify-center w-full px-3 py-2 mt-3 text-sm font-semibold text-gray-900 bg-white rounded-sm shadow-sm ring-1 ring-inset ring-gray-300 hover:bg-gray-50 sm:mt-0 sm:w-auto" | ||
| @click="show = false" | ||
| >{% trans "Cancel" %}</button> | ||
| <span class="inline-block" data-tooltip="{% trans "Translations are an experimental feature and may be inaccurate" %}">{% heroicon_outline "information-circle" aria_hidden="true" size=15 stroke_width=2 class="inline align-baseline me-1" %}</span> | ||
| </div> | ||
| </form> | ||
|
|
||
| <script type="module"> | ||
| import Choices from "{% static 'js/esm/choices.js-10-2-0.js' %}"; | ||
|
|
||
| const choices = JSON.parse('{{ json_choices|safe }}') | ||
|
|
||
| {# Define translations for the button text #} | ||
| const CLEAR_TEXT = "{% trans "Clear" %}" | ||
| const TRANSLATE_TEXT = "{% trans "Translate" %}" | ||
|
|
||
| function getToChoices(from_lang) { | ||
| const selected = choices.find((choice) => choice.value === from_lang) | ||
| return selected ? selected.to : [] | ||
| } | ||
|
|
||
| // Check if a given from/to lang combo is the active translation based on the JSON provided from the server | ||
| function isTranslationActive(newFromLang, newToLang) { | ||
| const active = choices.find((choice) => choice.selected === true); | ||
| if (!active) return false | ||
|
|
||
| const activeFrom = active.value; | ||
| const activeTo = active.to.find((to) => to.selected === true); | ||
|
|
||
| return (newFromLang === activeFrom && newToLang == activeTo) | ||
| } | ||
|
|
||
| // Change the button text to indicate the ability to clear the translation | ||
| function showClearBtn(show) { | ||
| translateBtn.textContent = show ? CLEAR_TEXT : TRANSLATE_TEXT | ||
| } | ||
|
|
||
| const selectFromLang = new Choices(document.getElementById('id_from_lang'), { allowHTML: true }).setChoices(choices); | ||
| const selectToLang = new Choices(document.getElementById('id_to_lang'), { allowHTML: true }); | ||
| const translateBtn = document.getElementById('translate-btn'); | ||
|
|
||
| // Initial setting of "to language" choices/disabling of field depending on starting "from language" values | ||
| if(selectFromLang.getValue()?.value) { | ||
| selectToLang.setChoices(getToChoices(selectFromLang.getValue().value)) | ||
| showClearBtn(true) | ||
| } else { | ||
| showClearBtn(false) | ||
| selectToLang.disable(); | ||
| } | ||
|
|
||
| // Event handler for when the "from language" selection is updated | ||
| selectFromLang.passedElement.element.addEventListener('change', (event) => { | ||
| const toLangChoices = getToChoices(event.detail.value) | ||
| if (toLangChoices.length > 0) { | ||
| selectToLang.setChoices(toLangChoices, 'value', 'label', true) | ||
| selectToLang.enable(); | ||
| showClearBtn(isTranslationActive(event.detail.value, selectToLang.getValue().value)); | ||
| } else { | ||
| selectToLang.disable(); | ||
| showClearBtn(false); | ||
| translateBtn.disabled = true; | ||
| } | ||
| }); | ||
|
|
||
| // Event handler for when "to language" selection is updated | ||
| selectToLang.passedElement.element.addEventListener('change', (event) => { | ||
| if (isTranslationActive(selectFromLang.getValue().value, event.detail.value)) { | ||
| showClearBtn(true); | ||
| } | ||
| }) | ||
| </script> |
1 change: 1 addition & 0 deletions
1
hypha/apply/funds/templates/submissions/partials/submission-title.html
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| <h1 class="mt-2 mb-0 font-medium">{{ object.title }}<span class="text-gray-400"> #{{ object.public_id|default:object.id }}</span></h1> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,18 @@ | ||
| from django import template | ||
| from django.conf import settings | ||
|
|
||
| register = template.Library() | ||
|
|
||
|
|
||
| @register.filter | ||
| def can_translate_submission(user) -> bool: | ||
| """Verify that system settings & user role allows for submission translations. | ||
| Args: | ||
| user: the user to check the role of. | ||
| Returns: | ||
| bool: true if submission can be translated, false if not. | ||
| """ | ||
| return bool(settings.APPLICATION_TRANSLATIONS_ENABLED and user.is_org_faculty) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.