Warnings appears during training related to phonemizer on arabic language

[JRMeyer]

>>> khalilrhouma
[December 4, 2020, 9:48am]

Working branch : dev

I am trying to fine-tune Tacotron-DDC-130k on LJSpeech on the Arabic
dataset using the phonemizer. Arabic is supported under ar tag. slash
During the training, these warnings didn't stop for the entire epoch:

[WARNING] extra phones may appear in the 'ar' phoneset
[WARNING] language switch flags have been removed (applying 'remove-flags' policy)
[WARNING] 1 utterances containing language switches on lines 1
[WARNING] extra phones may appear in the 'ar' phoneset
[WARNING] language switch flags have been removed (applying 'remove-flags' policy)
[WARNING] 1 utterances containing language switches on lines 1
[WARNING] extra phones may appear in the 'ar' phoneset
[WARNING] language switch flags have been removed (applying 'remove-flags' policy)
[WARNING] 1 utterances containing language switches on lines 1
[WARNING] extra phones may appear in the 'ar' phoneset
[WARNING] language switch flags have been removed (applying 'remove-flags' policy)
[WARNING] 1 utterances containing language switches on lines 1
............

Where the problem came from? I mean is there anything to change before
training except these parameters:

text_cleaner: phoneme_cleaners
use_phonemes: true
phoneme_language: ar

I think it may be related to text cleaning Any help? slash
Thank you

[This is an archived TTS discussion thread from discourse.mozilla.org/t/warnings-appears-during-training-related-to-phonemizer-on-arabic-language]

[JRMeyer]

>>> khalilrhouma
[December 4, 2020, 11:10am]

slash
yes I posted here because I didn't get a replay for the issue , I close
it now as you told me. slash
for the dataset it is about 3h, this is a sample of diacritic Arabic
which is supported by the phonemizer:

> إِضَافَةً إِلَى عَرْضِ أَنْوَاعٍ عَدِيدَةٍ مِنَ الْأَثَاثِ
> الْمُزَخْرَفِ وَالصَّنَادِيقِ وَالْأَوَانِي وَالْأَدَوَاتِ -
> وَالْأَغْطِيَةِ وَاللَّوْحَاتِ الْفَنِّيَّةِ الْعَاكِسَةِ
> لِتِقْنِيَّاتٍ مُدْهِشَةٍ - تُجَسِّدُ مَهَارَاتٍ حِرَفِيَّةً
> عَرَفَتْهَا الْأَنْدُلُسْ

we tested the phonemizer on the Arabic sentence directly and the
phonemizer didn't output any warning. This is the phonemizer output:

> ʔidˤa.ːfatan ʔilaː ʕardˤi. ʔanwaːʕin ʕadiːdatin mina alʔaθaːθi
> almuzaχɹafi was̪ːnaːdiːqi walʔauaːniː walʔadauaːti walʔaɣt̪i.ːati
> wallwħaːti alfanniːti alʕaːkisati litiqniːaːtin mudhiʃatin tudʒassdu
> mahaːɹaːtin ħiɹafiːtan ʕaɹafathaː alʔandulus

which is correct without any error or warnings, that's why I am thinking
it is related to text cleaning form mozilla tts

[Archived Post]

[JRMeyer]

>>> othiele
[December 4, 2020, 12:51pm]

Put in some logging where TTS uses the phonemizer and find out what
utterances cause the messages. For us we then knew what we had to
change.

[Archived Post]

[JRMeyer]

>>> erogol
[December 4, 2020, 2:21pm]

I'd suggest to run phonemizer over all of your dataset, list unique
phones and compare it with the phoneme set under symbols.py. If there
are extras, you can add them there and send a PR for us to update it.

[Archived Post]

[JRMeyer]

>>> khalilrhouma
[December 4, 2020, 4:59pm]

slash
We tested the whole dataset with phonemizer only, it gives it's output
as expected without any warnings. slash
Actually, the problem was related to phonemizer_cleaner , and exactly
when the ASCII conversion happens: text = convert_to_ascii(text) slash
Phonemizer accepts the Arabic letters as it is , and when you make that
conversion it does change the letters completely to an other format
which is not Arabic. slash
So, for now we are using basic_cleaners in place of phoneme_cleaners
and the warnings gone (until this moment) slash
Is there any method to disable cleaners ?

[Archived Post]

[JRMeyer]

>>> erogol
[December 4, 2020, 5:08pm]

you can pass a empty function which only returns the value as it is.

[Archived Post]

[JRMeyer]

>>> mrthorstenm
[December 4, 2020, 6:42pm]

.

Didn't have time to answer more early. slash
Here are some info what we did in german phoneme cleaning.

We appended following lines in the end of 'TTS/TTS/tts/utils/text'

def german_phoneme_cleaners(text):
return GermanTransliterate(replace={';': ',', ':': ' '}, sep_abbreviation=' -- ').transliterate(text)

Maybe that's the place where you could make your own function returning
the original value. slash
Reference your method name in your config.json as we did:

'text_cleaner': 'german_phoneme_cleaners',

We had some discussion on phoneme topic in another thread, maybe this is
helpful for you. slash

{.avatar
tts](https://discourse.mozilla.org/t/contributing-my-german-voice-for-tts/48150/141)
drop-close='true'
.box}

> Hello. It's time for another short update. We're currently preparing a
> new training run and figured out some 'issues' with phoneme handling
> with mixed english/german wording. This warning occurs quite often:
> slash [WARNING slash ] fount 1 utterances containing language switches on lines 1
> slash [WARNING slash ] extra phones may appear in the slash 'de slash ' phoneset slash [WARNING slash ]
> language switch flags have been kept (applying slash 'keep-flags slash ' policy)
> We analyzed where these warnings come from and found out that our
> dataset (metadata.c...

[Archived Post]

[JRMeyer]

>>> khalilrhouma
[December 5, 2020, 7:15am]

Thank you all for your support and your time. slash
said, we just need a
function that returns the input as it is. The basic_cleaners function
seems to do that properly, and we check that it gives the input as it is
without issues. slash
So thanks again, I really appreciate your work in Mozilla TTS

[Archived Post]

[JRMeyer]

>>> Pak
[February 10, 2021, 12:34pm]

> basic_cleaners

Hello, slash
Can you share your pipeline for training arabic? have own DB, dont need
data. slash
Thanks you.

[Archived Post]

[JRMeyer]

>>> erogol
[February 10, 2021, 1:24pm]

so did it work? How you got your Arabic model?

[Archived Post]

[JRMeyer]

>>> Pak
[February 11, 2021, 6:03am]

I install espeak-ng to get 'ar' support, but get
'xcb_connection_has_error() returned true' so fix by make 'unset
DISPLAY' now train. Will report result.

[Archived Post]

[JRMeyer]

>>> erogol
[February 11, 2021, 1:12pm]

Google is your friend. Not related to TTS.

[Archived Post]

[cod3r0k]

Hi , do you have any update about preliminaries steps for Arabic language?
I mean, do I need to set guut as phonemizer with ar language? and then use phoneme_cleaners ?
Also do we need to extends punctuations as below?

tokenizer, config = TTSTokenizer.init_from_config(config)
tokenizer.characters.punctuations = "!'(),-.:;? ؟،؛ ‌ \"“”«»"
```  @JRMeyer @all