[JARS-2115] Add TAO asr-citrinet model to Riva tutorial

[chan8972nv]

Update and add TAO NBs to Riva tutorials repo - ASR (https://jirasw.nvidia.com/browse/JARS-2115)

[NikhilSrihari]

@chankyu-nvda

Training NB:

1. Change nb name to something like "asr-python-advanced-finetune-am-citrinet-tao-finetuning" and "asr-python-advanced-finetune-am-citrinet-tao-deployment"
2. The TAO toolkit overview can be improved. The line about Transfer learning just seems out of place. The TAO framework image seems pretty complicated and we dont provide any explanation to it.
3. In the Automatic Speech Recognition section - We talk about Quartznet and Jasper models. We shouldnt be doing that. Lets talk about Citrinet and refer them to the Riva docs (or TAO docs) for more details. Refer to https://github.com/nvidia-riva/tutorials/blob/dev/22.04/asr-python-advanced-finetune-am-citrinet-for-noisy-audio-withtao.ipynb 's "Fine-tuning Riva Acoustic Model (Citrinet) with NVIDIA TAO" section for an example
4. "Installing and setting up TAO" - Do we wanna call out how to create the python env? And also call out how to run this nb in a ve? The Readme file of the Tutorials already does (Readme file will be present in the Riva Documentation page, under Tutorials section as Getting Started)
5. Again under the training section we talk about QuartzNet and Jasper models, but not Citrinet - This is a problem. We can have a reference to them I guess, but focus should be on the Citrinet model
6. "ASR model export" - Put the "Export to Riva" section on top. Call out that "Export to ONNX" is not needed for Riva deployment at all. Its something else entirely.
7. "ASR model export" - I dont see speech_to_text_citrinet used anywhere
8. "ASR inference" - Call out that this is inference with TAO only. And for best performance in latency for infeence, we should deploy to Riva. Refer to the riva deployment nb here
9. "What's Next?" - Provide link to deployment nb here

[NikhilSrihari]

@chankyu-nvda

Deployment NB:

1. I would change the title to "How to deploy custom Acoustic Model (Citrinet) trained with TAO Toolkit on Riva"?
2. "This tutorial walks you through the deployment of Riva ASR Acoustic Model (Citrinet) with TAO Toolkit." and "In this tutorial, we will deploy Riva ASR Acoustic Model (Citrinet) with TAO Toolkit." -> Please update this according to [notebooks] Add Riva speech API and QA notebooks #1 from above.
3. I like the TAO intro section better in this nb than the training nb. But may be we can add a couple more lines here. And also add link to TAO documentation. Example https://github.com/nvidia-riva/tutorials/blob/dev/22.04/asr-python-advanced-finetune-am-citrinet-for-noisy-audio-withtao.ipynb
4. "Train Adapt Optimize (TAO) Toolkit" section - "This tutorial explores taking a .riva model, the result of tao speech_to_text_citrinet command, and leveraging the Riva ServiceMaker framework to aggregate all the necessary artifacts for Riva deployment to a target environment. Once the model is deployed in Riva, you can issue inference requests to the server. We will demonstrate how quick and straightforward this whole process is." -> Can we call out and link to the training nb here?
5. As stated in [apps] Add Riva Contact App #4 above, when we can, I think we can have a tighter integration and crosss-reference between the training and inference nbs. Say something like "if you are coming from training nb, here we continue from where we stopped in the training nb"
6. "Learning Objectives" - Training nb didnt have this. So either have this in both nbs or neither of them
7. "Pre-requisites": Call out docker as a requirement.
8. "Start Riva Server" - "Once the model repository is generated, we are ready to start the Riva server. From this step onwards you need to download the Riva QuickStart Resource from NGC.": Details on ow to download the QSG guide. Links as well. Refer to the repo's README for example
9. "Run Inference" - "To send GRPC requests, you can install Riva Python API bindings for client. This is available as a pip .whl with the QuickStart." - Be more specific here. Refer to the repo's README for example
10. "Connect to Riva server and run inference" - wave library is not needed. Refer to the https://github.com/nvidia-riva/tutorials/blob/dev/22.04/asr-python-basics.ipynb nb for example
11. "Pre-requisites" - "Have a .riva model file that you wish to deploy. You can obtain this from tao export (with export_format=RIVA). Please refer the tutorial on Automatic Speech Recognition using Train Adapt Optimize (TAO) Toolkit for more details on training and exporting a .riva model." - Link pls

[chan8972nv]

Thanks for the comment @NikhilSrihari. Here is the point-by-point response to the finetuning nb comments.

Training NB:

1. Change nb name to something like "asr-python-advanced-finetune-am-citrinet-tao-finetuning" and "asr-python-advanced-finetune-am-citrinet-tao-deployment"
   Response) Done

2. The TAO toolkit overview can be improved. The line about Transfer learning just seems out of place. The TAO framework image seems pretty complicated and we dont provide any explanation to it.
   Response) This is general to TAO notebooks. I will get back to this later after discussing with TAO folks.

3. In the Automatic Speech Recognition section - We talk about Quartznet and Jasper models. We shouldnt be doing that. Lets talk about Citrinet and refer them to the Riva docs (or TAO docs) for more details. Refer to https://github.com/nvidia-riva/tutorials/blob/dev/22.04/asr-python-advanced-finetune-am-citrinet-for-noisy-audio-withtao.ipynb 's "Fine-tuning Riva Acoustic Model (Citrinet) with NVIDIA TAO" section for an example
   Response) Sorry for confusion. Modified for Citrinet Version.

4. "Installing and setting up TAO" - Do we wanna call out how to create the python env? And also call out how to run this nb in a ve? The Readme file of the Tutorials already does (Readme file will be present in the Riva Documentation page, under Tutorials section as Getting Started)
   Response) While the “readme file” has the hardware/software requirement for TAO toolkit, this Installing “nvidia-pyindex/tao” process is general to all tao notebook. If these packages are already installed, it passes to the next step.

5. Again under the training section we talk about QuartzNet and Jasper models, but not Citrinet - This is a problem. We can have a reference to them I guess, but focus should be on the Citrinet model
   Response) Modified for Citrinet Version.

6. "ASR model export" - Put the "Export to Riva" section on top. Call out that "Export to ONNX" is not needed for Riva deployment at all. Its something else entirely.
   Reponse) Put the “Export to Riva” page on the top and “Export to ONNX” page on the bottom.

7. "ASR model export" - I dont see speech_to_text_citrinet used anywhere
   Response) Modified for Citrinet Version.

8. "ASR inference" - Call out that this is inference with TAO only. And for best performance in latency for infeence, we should deploy to Riva. Refer to the riva deployment nb here
   Response) Added the “citrinet deployment page” here.

9. "What's Next?" - Provide link to deployment nb here
   Response) Added the “citrinet deployment page” here.

[chan8972nv]

Thanks for the comment @NikhilSrihari. Here is the point-by-point response to the deploy nb comments.

@chankyu-nvda

Deployment NB:

1. I would change the title to "How to deploy custom Acoustic Model (Citrinet) trained with TAO Toolkit on Riva"?
   Response) Done

2. "This tutorial walks you through the deployment of Riva ASR Acoustic Model (Citrinet) with TAO Toolkit." and "In this tutorial, we will deploy Riva ASR Acoustic Model (Citrinet) with TAO Toolkit." -> Please update this according to [notebooks] Add Riva speech API and QA notebooks #1 from above.
   Response) Done

3. I like the TAO intro section better in this nb than the training nb. But may be we can add a couple more lines here. And also add link to TAO documentation. Example https://github.com/nvidia-riva/tutorials/blob/dev/22.04/asr-python-advanced-finetune-am-citrinet-for-noisy-audio-withtao.ipynb
   Response) Done

4. "Train Adapt Optimize (TAO) Toolkit" section - "This tutorial explores taking a .riva model, the result of tao speech_to_text_citrinet command, and leveraging the Riva ServiceMaker framework to aggregate all the necessary artifacts for Riva deployment to a target environment. Once the model is deployed in Riva, you can issue inference requests to the server. We will demonstrate how quick and straightforward this whole process is." -> Can we call out and link to the training nb here?
   Response) Done

5. As stated in [apps] Add Riva Contact App #4 above, when we can, I think we can have a tighter integration and crosss-reference between the training and inference nbs. Say something like "if you are coming from training nb, here we continue from where we stopped in the training nb"
   Response) Added the links in TAO Toolkit session.

6. "Learning Objectives" - Training nb didnt have this. So either have this in both nbs or neither of them
   Response) Removed this and combined with “TAO toolkit”.

Response to below) I will address them later since it's all Riva related.

7. "Pre-requisites": Call out docker as a requirement.
8. "Start Riva Server" - "Once the model repository is generated, we are ready to start the Riva server. From this step onwards you need to download the Riva QuickStart Resource from NGC.": Details on ow to download the QSG guide. Links as well. Refer to the repo's README for example
9. "Run Inference" - "To send GRPC requests, you can install Riva Python API bindings for client. This is available as a pip .whl with the QuickStart." - Be more specific here. Refer to the repo's README for example
10. "Connect to Riva server and run inference" - wave library is not needed. Refer to the https://github.com/nvidia-riva/tutorials/blob/dev/22.04/asr-python-basics.ipynb nb for example
11. "Pre-requisites" - "Have a .riva model file that you wish to deploy. You can obtain this from tao export (with export_format=RIVA). Please refer the tutorial on Automatic Speech Recognition using Train Adapt Optimize (TAO) Toolkit for more details on training and exporting a .riva model." - Link pls

[NikhilSrihari]

@chankyu-nvda

Training NB:

2. Got it
3. Got it
4. Can we also call out in thee "Export to ONNX" section - "Note: Export to ONNX is not needed for Riva"
5. RE "You might have to work with the infer.yaml file to select the files you want for inference. For more information about inference with Riva, refer the How to deploy custom Acoustic Model (Citrinet) trained with TAO Toolkit on Riva." -> "In this section, we are going to run inference on the tlt checkpoint with TAO Toolkit. \n For real-time inference and best latency, we need to deploy this model on Riva - Refer to How to deploy custom Acoustic Model (Citrinet) trained with TAO Toolkit on Riva tutorial. \n You might have to work with the infer.yaml file to select the files you want for inference."
6. New comment "ASR Inference" -> "ASR Inference with TAO Toolkit".
7. New comment New section header right under "ASR Inference with TAO Toolkit" called "ASR Inference using TLT checkpoint"
8. New comment "ASR Inference using ONNX" should be a section under "ASR Inference with TAO Toolkit"

[NikhilSrihari]

@chankyu-nvda

Deployment NB: Sounds good :-)

[LynseyFabel]

My review is complete. I've left quite a few comments. After 22.04 is released, this will need another review.

[LynseyFabel]

Suggested change

	"# How to deploy Riva ASR Acoustic Model (Citrinet) with TAO Toolkit?\n",
	"# How to Deploy a Riva ASR Acoustic Model (Citrinet) with TAO Toolkit\n",

[LynseyFabel]

Suggested change

	"This tutorial walks you through the deployment of Riva ASR Acoustic Model (Citrinet) with TAO Toolkit."
	"This tutorial walks you through the deployment of the Riva ASR acoustic model (Citrinet) with TAO Toolkit."

[LynseyFabel]

Suggested change

	"In this tutorial, we will deploy Riva ASR Acoustic Model (Citrinet) with TAO Toolkit. <br> \n",
	"In this tutorial, we will deploy Riva ASR acoustic model (Citrinet) with TAO Toolkit. <br> \n",

[LynseyFabel]

this doesn't say anything different than what we say on line 10. let's delete this.

[NikhilSrihari]

@chankyu-nvda
Merging this PR now. Will make the Riva-related changes in a new PR