Skip to content

Fix restoring from checkpoint for case when model.common_dataset_parameters.label_vocab_dir is provided#3562

Merged
ekmb merged 20 commits intomainfrom
fix/punctuation/working_with_label_vocab_dir_to_main
May 9, 2022
Merged

Fix restoring from checkpoint for case when model.common_dataset_parameters.label_vocab_dir is provided#3562
ekmb merged 20 commits intomainfrom
fix/punctuation/working_with_label_vocab_dir_to_main

Conversation

@PeganovAnton
Copy link
Contributor

@PeganovAnton PeganovAnton commented Jan 30, 2022

What does this PR do?

Please have one of the following:

Before your PR is "Ready for review"

  • Make sure you read and followed Contributor guidelines
  • Did you write any new necessary tests?
  • Did you add or updated any necessary documentation?

If you haven't finished some of the above items you can still open "Draft" PR.

Who can review?

Anyone in the community is free to review the PR once the checks have passed.
Contributor guidelines contains specific people who can review PRs to various areas.

@PeganovAnton PeganovAnton added the bug Something isn't working label Jan 30, 2022
@PeganovAnton PeganovAnton requested a review from ekmb January 30, 2022 05:46
@PeganovAnton PeganovAnton self-assigned this Jan 30, 2022
@PeganovAnton PeganovAnton marked this pull request as draft January 30, 2022 05:46
Signed-off-by: PeganovAnton <peganoff2@mail.ru>
Signed-off-by: PeganovAnton <peganoff2@mail.ru>
Signed-off-by: PeganovAnton <peganoff2@mail.ru>
Signed-off-by: PeganovAnton <peganoff2@mail.ru>
@PeganovAnton PeganovAnton marked this pull request as ready for review February 14, 2022 17:44
ekmb
ekmb previously approved these changes Feb 14, 2022
@okuchaiev
Copy link
Collaborator

@PeganovAnton it is not passing CI anymore, could you please have a look?

Signed-off-by: PeganovAnton <peganoff2@mail.ru>
Signed-off-by: PeganovAnton <peganoff2@mail.ru>
@PeganovAnton PeganovAnton requested a review from ekmb May 8, 2022 16:27
@ekmb ekmb merged commit 5ed583c into main May 9, 2022
@ekmb ekmb deleted the fix/punctuation/working_with_label_vocab_dir_to_main branch May 9, 2022 14:23
stevehuang52 pushed a commit to stevehuang52/NeMo that referenced this pull request May 9, 2022
…ameters.label_vocab_dir` is provided (NVIDIA-NeMo#3562)

* Fix RANK env variable check in global rank check

Signed-off-by: PeganovAnton <peganoff2@mail.ru>

* Fix restoring from checkpoint with label vocab dir

Signed-off-by: PeganovAnton <peganoff2@mail.ru>

* Add tests for various ways to pass label ids to model

Signed-off-by: PeganovAnton <peganoff2@mail.ru>

* Fix typo

Signed-off-by: PeganovAnton <peganoff2@mail.ru>

* Fix typo

Signed-off-by: PeganovAnton <peganoff2@mail.ru>

* Do not create tmp directory

Signed-off-by: PeganovAnton <peganoff2@mail.ru>

* Fix parameter name

Signed-off-by: PeganovAnton <peganoff2@mail.ru>

* Try CUDA launch blocking

Signed-off-by: PeganovAnton <peganoff2@mail.ru>

* Fix labels errors

Signed-off-by: PeganovAnton <peganoff2@mail.ru>

Co-authored-by: Oleksii Kuchaiev <okuchaiev@users.noreply.github.com>
Signed-off-by: stevehuang52 <heh@nvidia.com>
titu1994 pushed a commit that referenced this pull request May 9, 2022
* Tn tutorial (#4090)

* refactor tn data folder, and update of measure

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* udpate jenkins

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* added whitelist with spaces for asr

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* save

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* save

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* electronic updated

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* added file support

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* add ip prompt

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* add missing file

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* update grammars

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* fix header

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* fixed electronic review added ssn

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* udpated money

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* added more formats for year range

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* fix pytest

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* style fix

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* fix

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* fix

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* remove run predict

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* add images

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* add images

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* remove redundant notebook

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* remove readme

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* change requirement name for tn

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* updated wfst tut

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* added final notes

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* added final notes

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* redirect readme

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* remove files

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* deleted redundant docs

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* finished deployment docs

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* fix style

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* update tutorials

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* add more test cases

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* update

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* update

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* add

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* remove image

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* add link

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* lgtm

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* sort imports

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* changed to language support matrix

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* changed to language support matrix

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* fix branch

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>
Signed-off-by: stevehuang52 <heh@nvidia.com>

* fix error (#4120)

* fix error

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* fix image

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* update image

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>
Signed-off-by: stevehuang52 <heh@nvidia.com>

* fix typo

Signed-off-by: stevehuang52 <heh@nvidia.com>

* Fix restoring from checkpoint for case when `model.common_dataset_parameters.label_vocab_dir` is provided (#3562)

* Fix RANK env variable check in global rank check

Signed-off-by: PeganovAnton <peganoff2@mail.ru>

* Fix restoring from checkpoint with label vocab dir

Signed-off-by: PeganovAnton <peganoff2@mail.ru>

* Add tests for various ways to pass label ids to model

Signed-off-by: PeganovAnton <peganoff2@mail.ru>

* Fix typo

Signed-off-by: PeganovAnton <peganoff2@mail.ru>

* Fix typo

Signed-off-by: PeganovAnton <peganoff2@mail.ru>

* Do not create tmp directory

Signed-off-by: PeganovAnton <peganoff2@mail.ru>

* Fix parameter name

Signed-off-by: PeganovAnton <peganoff2@mail.ru>

* Try CUDA launch blocking

Signed-off-by: PeganovAnton <peganoff2@mail.ru>

* Fix labels errors

Signed-off-by: PeganovAnton <peganoff2@mail.ru>

Co-authored-by: Oleksii Kuchaiev <okuchaiev@users.noreply.github.com>
Signed-off-by: stevehuang52 <heh@nvidia.com>

Co-authored-by: Yang Zhang <yzhang123@users.noreply.github.com>
Co-authored-by: PeganovAnton <peganoff2@mail.ru>
Co-authored-by: Oleksii Kuchaiev <okuchaiev@users.noreply.github.com>
yidong72 pushed a commit that referenced this pull request May 12, 2022
…ameters.label_vocab_dir` is provided (#3562)

* Fix RANK env variable check in global rank check

Signed-off-by: PeganovAnton <peganoff2@mail.ru>

* Fix restoring from checkpoint with label vocab dir

Signed-off-by: PeganovAnton <peganoff2@mail.ru>

* Add tests for various ways to pass label ids to model

Signed-off-by: PeganovAnton <peganoff2@mail.ru>

* Fix typo

Signed-off-by: PeganovAnton <peganoff2@mail.ru>

* Fix typo

Signed-off-by: PeganovAnton <peganoff2@mail.ru>

* Do not create tmp directory

Signed-off-by: PeganovAnton <peganoff2@mail.ru>

* Fix parameter name

Signed-off-by: PeganovAnton <peganoff2@mail.ru>

* Try CUDA launch blocking

Signed-off-by: PeganovAnton <peganoff2@mail.ru>

* Fix labels errors

Signed-off-by: PeganovAnton <peganoff2@mail.ru>

Co-authored-by: Oleksii Kuchaiev <okuchaiev@users.noreply.github.com>
Signed-off-by: Yi Dong <yidong@nvidia.com>
yidong72 pushed a commit that referenced this pull request May 12, 2022
* Tn tutorial (#4090)

* refactor tn data folder, and update of measure

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* udpate jenkins

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* added whitelist with spaces for asr

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* save

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* save

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* electronic updated

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* added file support

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* add ip prompt

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* add missing file

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* update grammars

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* fix header

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* fixed electronic review added ssn

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* udpated money

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* added more formats for year range

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* fix pytest

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* style fix

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* fix

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* fix

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* remove run predict

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* add images

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* add images

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* remove redundant notebook

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* remove readme

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* change requirement name for tn

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* updated wfst tut

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* added final notes

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* added final notes

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* redirect readme

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* remove files

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* deleted redundant docs

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* finished deployment docs

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* fix style

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* update tutorials

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* add more test cases

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* update

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* update

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* add

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* remove image

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* add link

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* lgtm

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* sort imports

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* changed to language support matrix

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* changed to language support matrix

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* fix branch

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>
Signed-off-by: stevehuang52 <heh@nvidia.com>

* fix error (#4120)

* fix error

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* fix image

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* update image

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>
Signed-off-by: stevehuang52 <heh@nvidia.com>

* fix typo

Signed-off-by: stevehuang52 <heh@nvidia.com>

* Fix restoring from checkpoint for case when `model.common_dataset_parameters.label_vocab_dir` is provided (#3562)

* Fix RANK env variable check in global rank check

Signed-off-by: PeganovAnton <peganoff2@mail.ru>

* Fix restoring from checkpoint with label vocab dir

Signed-off-by: PeganovAnton <peganoff2@mail.ru>

* Add tests for various ways to pass label ids to model

Signed-off-by: PeganovAnton <peganoff2@mail.ru>

* Fix typo

Signed-off-by: PeganovAnton <peganoff2@mail.ru>

* Fix typo

Signed-off-by: PeganovAnton <peganoff2@mail.ru>

* Do not create tmp directory

Signed-off-by: PeganovAnton <peganoff2@mail.ru>

* Fix parameter name

Signed-off-by: PeganovAnton <peganoff2@mail.ru>

* Try CUDA launch blocking

Signed-off-by: PeganovAnton <peganoff2@mail.ru>

* Fix labels errors

Signed-off-by: PeganovAnton <peganoff2@mail.ru>

Co-authored-by: Oleksii Kuchaiev <okuchaiev@users.noreply.github.com>
Signed-off-by: stevehuang52 <heh@nvidia.com>

Co-authored-by: Yang Zhang <yzhang123@users.noreply.github.com>
Co-authored-by: PeganovAnton <peganoff2@mail.ru>
Co-authored-by: Oleksii Kuchaiev <okuchaiev@users.noreply.github.com>
Signed-off-by: Yi Dong <yidong@nvidia.com>
titu1994 pushed a commit to titu1994/NeMo that referenced this pull request May 13, 2022
…ameters.label_vocab_dir` is provided (NVIDIA-NeMo#3562)

* Fix RANK env variable check in global rank check

Signed-off-by: PeganovAnton <peganoff2@mail.ru>

* Fix restoring from checkpoint with label vocab dir

Signed-off-by: PeganovAnton <peganoff2@mail.ru>

* Add tests for various ways to pass label ids to model

Signed-off-by: PeganovAnton <peganoff2@mail.ru>

* Fix typo

Signed-off-by: PeganovAnton <peganoff2@mail.ru>

* Fix typo

Signed-off-by: PeganovAnton <peganoff2@mail.ru>

* Do not create tmp directory

Signed-off-by: PeganovAnton <peganoff2@mail.ru>

* Fix parameter name

Signed-off-by: PeganovAnton <peganoff2@mail.ru>

* Try CUDA launch blocking

Signed-off-by: PeganovAnton <peganoff2@mail.ru>

* Fix labels errors

Signed-off-by: PeganovAnton <peganoff2@mail.ru>

Co-authored-by: Oleksii Kuchaiev <okuchaiev@users.noreply.github.com>
titu1994 pushed a commit to titu1994/NeMo that referenced this pull request May 13, 2022
* Tn tutorial (NVIDIA-NeMo#4090)

* refactor tn data folder, and update of measure

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* udpate jenkins

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* added whitelist with spaces for asr

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* save

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* save

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* electronic updated

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* added file support

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* add ip prompt

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* add missing file

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* update grammars

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* fix header

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* fixed electronic review added ssn

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* udpated money

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* added more formats for year range

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* fix pytest

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* style fix

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* fix

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* fix

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* remove run predict

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* add images

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* add images

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* remove redundant notebook

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* remove readme

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* change requirement name for tn

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* updated wfst tut

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* added final notes

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* added final notes

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* redirect readme

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* remove files

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* deleted redundant docs

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* finished deployment docs

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* fix style

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* update tutorials

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* add more test cases

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* update

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* update

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* add

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* remove image

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* add link

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* lgtm

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* sort imports

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* changed to language support matrix

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* changed to language support matrix

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* fix branch

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>
Signed-off-by: stevehuang52 <heh@nvidia.com>

* fix error (NVIDIA-NeMo#4120)

* fix error

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* fix image

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>

* update image

Signed-off-by: Yang Zhang <yangzhang@nvidia.com>
Signed-off-by: stevehuang52 <heh@nvidia.com>

* fix typo

Signed-off-by: stevehuang52 <heh@nvidia.com>

* Fix restoring from checkpoint for case when `model.common_dataset_parameters.label_vocab_dir` is provided (NVIDIA-NeMo#3562)

* Fix RANK env variable check in global rank check

Signed-off-by: PeganovAnton <peganoff2@mail.ru>

* Fix restoring from checkpoint with label vocab dir

Signed-off-by: PeganovAnton <peganoff2@mail.ru>

* Add tests for various ways to pass label ids to model

Signed-off-by: PeganovAnton <peganoff2@mail.ru>

* Fix typo

Signed-off-by: PeganovAnton <peganoff2@mail.ru>

* Fix typo

Signed-off-by: PeganovAnton <peganoff2@mail.ru>

* Do not create tmp directory

Signed-off-by: PeganovAnton <peganoff2@mail.ru>

* Fix parameter name

Signed-off-by: PeganovAnton <peganoff2@mail.ru>

* Try CUDA launch blocking

Signed-off-by: PeganovAnton <peganoff2@mail.ru>

* Fix labels errors

Signed-off-by: PeganovAnton <peganoff2@mail.ru>

Co-authored-by: Oleksii Kuchaiev <okuchaiev@users.noreply.github.com>
Signed-off-by: stevehuang52 <heh@nvidia.com>

Co-authored-by: Yang Zhang <yzhang123@users.noreply.github.com>
Co-authored-by: PeganovAnton <peganoff2@mail.ru>
Co-authored-by: Oleksii Kuchaiev <okuchaiev@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

PunctuationCapitalizationModel restoring from checkpoint is not working if a config field model.common_dataset_parameters.label_vocab_dir is set

3 participants

Comments