Make submission language selection and metadata forms independent from website language settings

[jyhein]

Related issues:

• Fix submission language/languages metadata #5000
• Restrict what languages are available for new submissions #7431

Issue description

The language settings should be separated into two independent entities. This allows us to serve more use cases and makes a clear distinction between UI and website language selection and the language settings for submissions.
The Website Languages entity works like before. It controls which languages are available for the user interface. The forms selection is used to control the language versions for context settings fields. The available languages are based on the available application translations.
The Submission Language is controlled separately. Instead of being tied to the limitation of available application locales, it will use a standard ISO 639-1 list for available languages. It will also allow the managers to activate just the metadata fields for a language, but activating Submissions should also activate the Forms automatically.

The current Submission Language settings only reflect the current language policy of the journal. This means that if a journal imports German language back content but does not publish in German anymore, the metadata forms and the submission primary language should always work independently. If therefore the primary language of a submission is for example German, the submission metadata should be editable without activating the German language metadata forms in the settings. Technically this means that the UI for the publication metadata fields will always take into account the submission metadata when loading available forms. The editor can be notified with a visual cue that the submission language is no longer used in the context.

Changes

• In Website > Setup > Language settings, add a new html table for submission languages and remove the Submissions column from the Languages table. The table includes an ‘Add Language' button to install submission languages. Clicking the button opens a modal window to select and install available languages from the list.
• Use the Submission Language settings when showing available submission metadata forms
• Always load the submission metadata forms for the selected submission language. The Submission Language settings do not affect it in any way.

Questions

Q: Could we not use the journal primary locale for supportedDefaultSubmissionLocale? Also, should maybe journal primary locale always be part of supportedAddedSubmissionLocales, supportedSubmissionLocales, and supportedSubmissionMetadataLocales?
A: The idea is to separate the two: UI language and submission language. The journal might publish using a language not available as UI language. We should not force them to accept submissions also with the UI language they end up using.

The attached PR's includes the following code changes:

• new getters for submission metadata
• submission locale added to supported locales. E.g. in workflow > publication, submission metadata forms include submission locale
• new setting (form and handler) to add/remove languages to/from submission locales and metadata in Website > Setup > Language settings

PRs:
PKP: #9309
OJS: pkp/ojs#4040
OMP: pkp/omp#1461
OPS: pkp/ops#569

[bozana]

Hi @jyhein, I went over pkp-lib and ojs changes. Lots of work! Thanks a lot, it looks great!
I would like to hear your opinion on this: Could we user the journal primary locale for supportedDefaultSubmissionLocale?

[bozana]

What about the case when a journal removes a (metadata) locale that was already used and the data exist in (and is not submission primary locale)?
EDIT: I suppose then those data cannot be edited. Finding out all locales a submission's metadata exist for seems to be a little bit complex...

[jyhein]

Thanks for the review, @bozana !

I try to answer your questions:

Q: Could we user the journal primary locale for supportedDefaultSubmissionLocale?
A: The idea is to separate the two: UI language and submission language. The journal might publish using a language not available as UI language. We should not force them to accept submissions also with the UI language they end up using.

Q: What about the case when a journal removes a (metadata) locale that was already used and the data exist in (and is not submission primary locale)?
A: Yes, the forms should include not just submission locale but also other metadata locales. I add the possibility. I cannot say about the complexity yet...

[bozana]

Hi @jyhein, I went through the code once again. The changes in PKP-LIB and OJS looks very good! I left a few questions in OMP and OPS PRs -- I think you still need to finish them, right?
However, sometimes I am not sure if submission->getPublicationLanguages() should be used or publication->getLanguages. If you have it clear in your head maybe to write a few comments in the code, why this or that? -- It will surely be very helpful for other developers!
One more thing is left: now that we use Weblate locales, I think we would need to consider the old UI locales in the upgrade that do not match with Weblate locales and are used in submission and metadata. I think you already created a list and how those UI locales needs to be changed. This is not changing the whole UI locales, just those used in submissions and metadata, so that the data are correct after the upgrade.
And: The fully new files created by you should now have 2024 instead of 2023 in copyright in header :-)
I have not tested it all again, but I am going to also test it once again/now...
Thanks a lot for the great work! I believe we can merge it very soon :-)

[jyhein]

"-- sometimes I am not sure if submission->getPublicationLanguages() should be used or publication->getLanguages. "
submission->getPublicationLanguages() is used because of different publication versions (between versions there could be different set of metadata locales). Seemed to me that it would be better to filter locales in the front-end than back-end (I didn't add any code to filter locales because I'm not sure if it is really needed).

[bozana]

Yes... I was just a little bit confused because it was differently in OMP and OPS -- but I suppose OMP and OPS need to be adapted still?
Also in the lib/pkp/classes/publication/Repository.php, in the edit() function the publication->getLanguages() is used -- differently to add() function -- but I suppose it is OK/correct, because the publication already exists and we can only consider the languages of that publication?

[jyhein]

Yes, I haven't updated OMP/OPS yet.

lib/pkp/classes/publication/Repository.php: add() vs edit(): I'm not sure about this

[bozana]

Hi @jyhein,
I took a look at all changes and all looks good now 👏 🙌 😃
I would like to test it once again, also an upgrade... Then we can merge it... 🎉 😃

[bozana]

Hi @jyhein, I roughly tested it and all is good, except that one comment for the upgrade script in the pkp-lib: It seems it should be TABLE_CATALOG instead of TABLE_SCHEMA for postgresql.
After this change you can then rebase all applications and when the tests pass we can merge :-)