[TTS] initial commit containing tts tutorials

[vpraveen-nv]

• Tutorial to finetune TTS models using TAO
• Tutorial to deploy finetuned models using RIVA

[LynseyFabel]

My review is complete. I've left quite a few comments. After 22.04 is released, this will need another review.

[LynseyFabel]

the previous tutorials that I reviewed a few days ago did not include this section

[vpraveen-nv]

Would you like to remove this? or retain this section?

[NikhilSrihari]

@vpraveen-nv

Training NB:

1. Notebook file name to "tts-python-advanced-pretrain-tts-tao-training.ipynb"
2. Notebook title to "How to train Riva TTS models (FastPitch and HiFiGAN) with TAO Toolkit"
3. "This tutorial walks you through the steps to train a FastPitch and HiFiGAN model from scratch with LJSpeech dataset using TAO Toolkit." -> "This tutorial walks you through the steps to train Riva TTS models (FastPitch and HiFiGAN) from scratch with LJSpeech dataset using TAO Toolkit."
4. Pls add an "Nvidia Riva Overview" section. Refer to https://github.com/nvidia-riva/tutorials/blob/dev/22.04/asr-python-advanced-finetune-am-citrinet-tao-finetuning.ipynb . In this section, we would refer to the Riva TTS documentation as well.
5. "NVIDIA TAO Toolkit Overview" - We need to improve this section. The Transfer line is just hanging there without any context. The TAO toolkit diagram, a pretty complicated diagram, is just present there without any explanation
6. "Text to Speech" section, "In TAO Toolkit, TTS is made up with two models: FastPitch for spectrogram generation and HiFiGAN as vocoder." -> Drop the "In TAO Toolkit" part. Just say "TTS Pipeline is made up with...". Can we also add a line or 2 above these models and how the pipeline connects?
7. I think we should make it clearer what exactly we are going to do here. Like include the info that "We are trying to train TTS on a voice from scratch" or something of that sort
8. "TTS model export": Pls put "Export to Riva" section on top. And call out that export to onnx is needed for riva - You can do this in the title "Export to ONNX (Note: Export to ONNX is not needed for Riva)". Refer to https://github.com/nvidia-riva/tutorials/blob/dev/22.04/asr-python-advanced-finetune-am-citrinet-tao-finetuning.ipynb
9. "TTS Inference" -> "TTS Inference with TAO Toolkit"
10. Add section header right under "TTS Inference with TAO Toolkit", called "TTS Inference using TLT checkpoint". Also make "TTS Inference using ONNX" a section under "TTS Inference with TAO Toolkit"
11. Under "TTS Inference using TLT checkpoint", say something like "In this section, we are going to run inference on the tlt checkpoint with TAO Toolkit. For real-time inference and best latency, we need to deploy this model on Riva - Refer to How to deploy custom TTS models (FastPitch and HiFiGAN) trained with TAO Toolkit on Riva tutorial.". Example https://github.com/nvidia-riva/tutorials/blob/dev/22.04/asr-python-advanced-finetune-am-citrinet-tao-finetuning.ipynb
12. "What's Next?" - You can use TAO to build custom models for your own applications, or you could deploy the custom model to NVIDIA Riva. Example https://github.com/nvidia-riva/tutorials/blob/dev/22.04/asr-python-advanced-finetune-am-citrinet-tao-finetuning.ipynb

[NikhilSrihari]

@vpraveen-nv

Deployment NB:

1. Notebook file name to "tts-python-advanced-pretrain-tts-tao-deployment.ipynb"
2. Notebook title to "How to deploy custom TTS models (FastPitch and HiFiGAN) trained with TAO Toolkit on Riva"
3. Add 1 line under the title to mention what we are doing. Example https://github.com/nvidia-riva/tutorials/blob/dev/22.04/asr-python-advanced-finetune-am-citrinet-tao-deployment.ipynb
4. Can you add an "NVIDIA Riva Overview" section pls. Example https://github.com/nvidia-riva/tutorials/blob/dev/22.04/asr-python-advanced-finetune-am-citrinet-tao-deployment.ipynb
5. Add a "Train, Adapt, and Optimize TAO Toolkit Overview" section. Example https://github.com/nvidia-riva/tutorials/blob/dev/22.04/asr-python-advanced-finetune-am-citrinet-tao-deployment.ipynb .
6. "Learning Objectives" - Training nb didnt have this. So either have this in both nbs or neither of them
7. "Pre-requisites": Call out docker as a requirement.
8. "Start Riva Server" - "Once the model repository is generated, we are ready to start the Riva server. From this step onwards you need to download the Riva QuickStart Resource from NGC.": Details on ow to download the QSG guide. Links as well. Refer/Point to the repo's README for example
9. "Run Inference" - "To send GRPC requests, you can install Riva Python API bindings for client. This is available as a pip .whl with the QuickStart." - Be more specific here. Refer/Point to the repo's README for example
10. "Connect to Riva server and run inference" - wave library is not needed. Refer to the https://github.com/nvidia-riva/tutorials/blob/dev/22.04/asr-python-basics.ipynb nb for example
11. "Pre-requisites" - "a .riva model file that you want to deploy. You can obtain this from tao export (with export_format=RIVA). For more information on training and exporting a .riva model, refer to the [Speech Synthesis using Train Adapt Optimize (TAO) Toolkit]." - Link pls

[NikhilSrihari]

@vpraveen-nv

All looks great. Just 2 minor things.

Training NB:
12. "You can use TAO to build custom models for your own applications, or you could deploy the custom model to NVIDIA Riva":
The hyperlink is broken

Deployment NB:
9. "Run Inference": We have improved the requirements section here, yes. But we still dont talk about installing numpy. The "Requirements" section is already covered in Readme.md. Refer to the "Requirements and Setup" section here - https://github.com/nvidia-riva/tutorials/blob/stable/tts-python-basics.ipynb . We basically do the same thing here.