Vietnamese samples

[phineas-pta]

demo: https://huggingface.co/spaces/doof-ferb/MatchaTTS_ngngngan

checkpoints: https://huggingface.co/doof-ferb/matcha_ngngngan

the training is much faster than vits, audio sample is also awesome

@shivammehta25 if possible can u check my training tensorboard: https://huggingface.co/doof-ferb/matcha_ngngngan/tensorboard ? 🙏 im newbie in this domain 🙏 tysm

the loss curve & mel spectrogram is weird, is it a data problem ?

[shivammehta25]

Hi! Thank you for your interest in 🍵 Matcha-TTS :D It's good to hear that it worked for you and you got good results.
Unfortunately, I don't speak Vietnamese to evaluate the prosody well enough. Looking at the mel spectrograms of the referenced audio, the dataset seems quite noisy. The loss seems to be behaving fine for the flow matching loss and enc loss. In terms of duration loss overall for epoch, it is also decreasing but, for step, it is very noisy I suspect some text and audio mismatch in some batches, keep an eye on that and train the model for a bit more, I think it will improve further :)

One hack that you can do is after some training, train the model further with a batch size 1 for 1 epoch and see which samples are giving absurd loss and investigate the reason behind them.

[phineas-pta]

tysm for the suggestions 🤗

evaluate the prosody

the intonation is pretty good, but some words sound a bit cracked, i suspect training data didnt cover all possible phonemes (tbh i have no idea how many phonemes there are 😅)

the most problematic is rhythm: some words are usually grouped together but the pause happen intra-group instead of inter-group

some text and audio mismatch in some batches

yes, it's the big problem with data, i use whisper to transcribe then spend time hand-correcting spelling, but seem like there's still a lot of misspelling, i'll look for an automated solution

train the model for a bit more

im confused when i see the loss on validation set increased then decreased, idk what the expected behaviour of loss curve so i can stop

One hack that you can do is after some training, train the model further with a batch size 1 for 1 epoch and see which samples are giving absurd loss and investigate the reason behind them.

very interesting, i have to edit file matcha/data/text_mel_datamodule.py to print audio file name ?

---

another question is how i test accuracy on test set ? i see in #25 i need to define a def test_step(self): …

so i can just copy the content of validation_step ? do i need to define test_filelist_path to the yaml configs/data/███.yaml ?