Add translations of language names to locale install list

[amandastevens]

New problem description (3.4)
The language lists (both in the software installer and e.g. on the language toggle plugin) list language names in the current language:

In cases where the same language is available in 2 character sets (e.g. Serbian), it's not clear which is which.

For users who find themselves using the software in a language they don't speak, they may not know their language's name in the current UI language.

Suggestion (Alec): Include the language name in both the current UI and listed language's own locale.

Old problem description (3.3.x)

When I install a new locale on an OJS 3.3 site as an Administrator in the Site Settings, the list of locales presented shows the locale names in their native script. If I can't read that language, it's difficult to identify which locale I need to install. I was only able to identify which locale was Simplified Chinese by consulting the registry/locales.xml file.

I suggest that the locale list show the language name in its native script and a translation of it into English, German, Spanish, Indonesian, etc., depending on the interface language being used.

This would be especially helpful for multi-OJS site administrators.

This is the current locale list:

[NateWr]

I'm not sure if we have every language translated into every other language. @asmecher do you know if this is part of the ISO library we're using?

(Also, I didn't know we have a Scottish Gaelic translation of OJS!! 😂)

[asmecher]

The php-isocodes library we use for countries and currencies does include 639-3 language list. Note that it's harder to customize entries once they're delegated to that library, though; see https://github.com/asmecher/customCountryList for a plugin that allows adjustments to the country list.

[amandastevens]

Thanks, I didn't know about that plugin.

[asmecher]

In the main branch (pre-3.4), the language lists have now been converted to the current language (probably since #6328). However, this isn't a great solution either. I've updated the issue to show both formulations of the problem and scheduled this issue for 3.4; for 3.3.x, you can customize registry/locales.xml as a work-around.

Some examples we can look to on major websites:

debian.org:

un.org:

wordpress.org (probably relying on #7512):

Note that many major sites do not appear to have language selectors on them -- probably due to their reliance on Accept-Language headers (7512), making it appear transparent to the user. However, while that would be nice for an automatic pre-selection, IMO there's value in making our multilingual support explicit.

[NateWr]

So is the idea to have the list show in the current locale, then it's own locale?

Change Language
✔ English
Arabic (عربى)
Ukrainian (українська)

[asmecher]

Yes, IMO that would be the best result. The encoding would be implicit (e.g. Latin vs. Cyrillic) but I don't see any benefit in exposing that more explicitly.

[touhidurabir]

So is the idea to have the list show in the current locale, then it's own locale?

Change Language
✔ English
Arabic (عربى)
Ukrainian (українська)

@asmecher In that case do we plan to remove the country name ? Right now we shows as

Arabic (Iraq)
Spanish (Spain)
Spanish (Mexico)
French (Canada)
French (France)

we have same language with different country . if we plan to remove the country, then it make no sense to have multiple language like French for Canada and France . in that case, we will have something like this

Arabic (عربي)
Spanish (Español)
French (Français)

but I think that will be a breaking change . Should we do something like

Arabic (Iraq)(العراق - عربي)
Spanish (Spain)(España - Español)
French (France)(France - Français)

so basically the format will be like
LANGUAGE_NAME_IN_ENGLISH(COUNTRY_NAME_IN_ENGLISH)(COUNTRY_NAME_IN_LANGUAGE_LOCALE - LANGUAGE_NAME_IN_LANGUAGE_LOCALE)

what do you think ?

[NateWr]

We should remove the country name. This causes a fair amount of confusion and sometimes controversy.

[touhidurabir]

We should remove the country name. This causes a fair amount of confusion and sometimes controversy.

@NateWr in that case how are we going to show the locale lists as French (Canada) as locale value fr_CA and French (France) has locale value fr_FR ? So if a journal/context has already installed fr_CA but in our list we can only show

French (Français)

without country . so what value it should be ? we will have no way to show option for both value fr_CA and fr_FR .

[NateWr]

I did an informal survey on Mattermost with some large hosts and I found that this case is not really used. The only cases they found where two variants of the same language with the same character set were installed were a few pt_BR and pt_PT. The publishers thought this was likely an attempt at inclusion because before our language always included the country.

In other words, they just wanted Portuguse but because our UI said "Portugese Brazil" and "Portugese Portugal" they felt they needed to enable both so that authors weren't led to believe they only supported Brazilian or Portugese authors.

That's what I mean about confusion. Countries are a bad proxy for language and rarely map onto real-world usage.

[asmecher]

@touhidurabir, the country is already excluded in the main branch; see #7871 for details about that change.

[asmecher]

Touhidur and I talked about this today and especially the case where we do have to present several languages with different country codes alongside each other (e.g. pt_PT and pt_BR). This won't be common for end users but there is the case above that Nate mentioned (#7352 (comment)) and it will be a problem e.g. on the configuration forms and installation form.

We decided that a good approach that covers all cases might be to examine the list before presentation to the user, determine whether there are multiple candidates for a language (e.g. with 2 countries), and if there are, then include the country name in the resulting list.

For most users, this will mean only showing the language name -- but for those with multiple candidates, it will also include the country.

[NateWr]

The case that I mentioned in my comment was actually a case against using country names. Journals were enabling the same language in two countries because the country names made the language seem exclusive to one country or another.

Do we know of a real-world use case where a journal wants to have two country options for the same language with the same character set?

[asmecher]

On the installation form (and the settings forms) we need something that'll distinguish between fr_CA and fr_FR, and can't just list "French" twice. So those lists need to be specific.

For end-user-facing content, they should be as general as possible -- listing only the language Portuguese / Português unless the journal has chosen to enable both Portuguese / Português (Brazil) and Portuguese / Português (Portugal). In the latter case the country should be presented too, so it's not just Portuguese / Português twice. I agree that this is a boundary case but I think it would be nicer to go with an option that wouldn't leave existing users up a creek.

There probably aren't many cases where two countries should be listed for the same language, but I'm willing to bet one exists that'll be important to someone. Thinking of cases where...

• the language is formally standardized in two places e.g. Swiss Standard German (de_CH): https://en.wikipedia.org/wiki/Swiss_Standard_German, and/or
• places where it's politically important to distinguish between locales (I'm sure you can think of a few of these).

@NateWr, for a more general solution to the issue that led a journal to enabling two Portugueses, I'd like to stop forcing language codes to specify countries in the first place, in line with standard locale code usage. See #8210.

[NateWr]

Ok, makes sense. It won't "fix" things for existing users who have already enabled two languages to compensate. But it will improve things going forward. Maybe we can mention it in our release notebook, too.

[asmecher]

All merged -- thanks, @touhidurabir!

[asmecher]

@touhidurabir, I still see the install form showing with ambiguous names, e.g. Uzbek:

[asmecher]

Also, I think the original goal of this (described at the top) has been lost -- the language selection drop-down in the back-end is no longer showing translated language names.

[touhidurabir]

I still see the install form showing with ambiguous names, e.g. Uzbek:

@asmecher this is how it's looks for me

here Uzbek is shown as Uzbek/Uzbek as the translated name of Uzbek returns as Uzbek by our used package sokil/php-isocodes . even setting the current lang to uz does not impact , run the following code

putenv('LANGUAGE=uz_UZ.UTF-8');
putenv('LC_ALL=uz_UZ.UTF-8');
setlocale(LC_ALL, 'uz_UZ.UTF-8');

use \Sokil\IsoCodes\IsoCodesFactory;

$isoCodes = new IsoCodesFactory();

$languages = $isoCodes->getLanguages();

$language = $languages->getByAlpha2('uz');

dump($language->getLocalName()); // print Uzbek

// $isoCodes->getCountries()->getByAlpha2('UZ')->getLocalName();

Also as we have removed the country postfix e.g. __2_LETTER_COUNTRY_CODE, we have no way to determine the country code from lang code only so no country name also shown .

I think the original goal of this (described at the top) has been lost -- the language selection drop-down in the back-end is no longer showing translated language names.

Yes, thats got reverted back in our last patch PR . But as we have some temp cache via static variable for the Site class, we have to fallback to direct use of Locale::getFormattedDisplayNames in one place .

[touhidurabir]

@asmecher PRs to review

pkp-lib --> #8742
ojs --> pkp/ojs#3812

I will forward port to OMP/OPS once we finalize .

[asmecher]

@touhidurabir, are you looking at e.g. the main OJS installation form? The top of the page looks like this:

I'm still not seeing the locale codes there, which should be added to help disambiguate between similarly-named locales.

(I'm referring to the locale list with checkboxes, not the dropdown in the screenshot above.)

[touhidurabir]

referring to the locale list with checkboxes, not the dropdown in the screenshot above.

added . But I think we decided not to make any modification in the installation page

[asmecher]

@touhidurabir, thanks, that looks good; I've merged to main, and the OMP and OPS parts were trivial enough so I patched those apps too.