es and es_en changes for unified models#143
Conversation
Signed-off-by: Mariana Graterol Fuenmayor <mgrafu@gmail.com>
Signed-off-by: Mariana Graterol Fuenmayor <mgrafu@gmail.com>
Signed-off-by: Mariana <47233618+mgrafu@users.noreply.github.com>
for more information, see https://pre-commit.ci
Signed-off-by: Mariana Graterol Fuenmayor <mgrafu@gmail.com>
Signed-off-by: Mariana Graterol Fuenmayor <mgrafu@gmail.com>
for more information, see https://pre-commit.ci
|
This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days. |
Signed-off-by: Mariana Graterol Fuenmayor <mgrafu@gmail.com>
|
This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days. |
|
@mgrafu is this PR WIP? |
|
@ekmb yes! the majority of the changes are done, but I'm reviewing bugs before final review |
|
This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days. |
|
This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days. |
|
This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days. |
Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com>
for more information, see https://pre-commit.ci
Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com>
Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com>
Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com>
Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com>
Signed-off-by: Mariana <47233618+mgrafu@users.noreply.github.com>
tbartley94
left a comment
There was a problem hiding this comment.
Left comments and requested changes. There's a few mispellings, questions about edge cases, and request for variable renaming.
I trust the test cases are boilerplate, didn't go into much detail there.
nemo_text_processing/inverse_text_normalization/es/data/dates/year_suffix_cased.tsv
Show resolved
Hide resolved
nemo_text_processing/inverse_text_normalization/es/data/roman/hundreds.tsv
Show resolved
Hide resolved
nemo_text_processing/inverse_text_normalization/es/verbalizers/time.py
Outdated
Show resolved
Hide resolved
nemo_text_processing/inverse_text_normalization/es_en/data/es_whitelist.tsv
Show resolved
Hide resolved
nemo_text_processing/inverse_text_normalization/es_en/taggers/tokenize_and_classify.py
Show resolved
Hide resolved
Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com>
Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com>
Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com>
Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com>
Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com>
Signed-off-by: Mariana <47233618+mgrafu@users.noreply.github.com>
for more information, see https://pre-commit.ci
Signed-off-by: Mariana <47233618+mgrafu@users.noreply.github.com>
|
This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days. |
tbartley94
left a comment
There was a problem hiding this comment.
Waiting on merge conflicts to be resolved then will approve
nemo_text_processing/inverse_text_normalization/es/taggers/ordinal.py
Outdated
Show resolved
Hide resolved
nemo_text_processing/inverse_text_normalization/es/taggers/telephone.py
Outdated
Show resolved
Hide resolved
Signed-off-by: Mariana <47233618+mgrafu@users.noreply.github.com>
This reverts commit 5e238fa.
* enable capitalized itn for es Signed-off-by: Mariana Graterol Fuenmayor <mgrafu@gmail.com> * enable capitalized itn for es_en Signed-off-by: Mariana Graterol Fuenmayor <mgrafu@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * style changes Signed-off-by: Mariana Graterol Fuenmayor <mgrafu@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix imports Signed-off-by: Mariana Graterol Fuenmayor <mgrafu@gmail.com> * mod eval scripts Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update whitelist Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> * update currencies for es itn Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> * bugfix for es time grammar Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> * update cache for es Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> * ordinal tagger fix Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> * telephone tagger fix Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> * time tagger fix Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> * time verbalizer fix Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> * update cache Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Mariana Graterol Fuenmayor <mgrafu@gmail.com> Signed-off-by: Mariana <47233618+mgrafu@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Alex Cui <alexcui1994@gmail.com>
* enable capitalized itn for es Signed-off-by: Mariana Graterol Fuenmayor <mgrafu@gmail.com> * enable capitalized itn for es_en Signed-off-by: Mariana Graterol Fuenmayor <mgrafu@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * style changes Signed-off-by: Mariana Graterol Fuenmayor <mgrafu@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix imports Signed-off-by: Mariana Graterol Fuenmayor <mgrafu@gmail.com> * mod eval scripts Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update whitelist Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> * update currencies for es itn Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> * bugfix for es time grammar Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> * update cache for es Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> * ordinal tagger fix Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> * telephone tagger fix Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> * time tagger fix Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> * time verbalizer fix Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> * update cache Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Mariana Graterol Fuenmayor <mgrafu@gmail.com> Signed-off-by: Mariana <47233618+mgrafu@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Alex Cui <alexcui1994@gmail.com>
* enable capitalized itn for es Signed-off-by: Mariana Graterol Fuenmayor <mgrafu@gmail.com> * enable capitalized itn for es_en Signed-off-by: Mariana Graterol Fuenmayor <mgrafu@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * style changes Signed-off-by: Mariana Graterol Fuenmayor <mgrafu@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix imports Signed-off-by: Mariana Graterol Fuenmayor <mgrafu@gmail.com> * mod eval scripts Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update whitelist Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> * update currencies for es itn Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> * bugfix for es time grammar Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> * update cache for es Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> * ordinal tagger fix Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> * telephone tagger fix Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> * time tagger fix Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> * time verbalizer fix Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> * update cache Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Mariana Graterol Fuenmayor <mgrafu@gmail.com> Signed-off-by: Mariana <47233618+mgrafu@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Alex Cui <alexcui1994@gmail.com>
* enable capitalized itn for es Signed-off-by: Mariana Graterol Fuenmayor <mgrafu@gmail.com> * enable capitalized itn for es_en Signed-off-by: Mariana Graterol Fuenmayor <mgrafu@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * style changes Signed-off-by: Mariana Graterol Fuenmayor <mgrafu@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix imports Signed-off-by: Mariana Graterol Fuenmayor <mgrafu@gmail.com> * mod eval scripts Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update whitelist Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> * update currencies for es itn Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> * bugfix for es time grammar Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> * update cache for es Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> * ordinal tagger fix Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> * telephone tagger fix Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> * time tagger fix Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> * time verbalizer fix Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> * update cache Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Mariana Graterol Fuenmayor <mgrafu@gmail.com> Signed-off-by: Mariana <47233618+mgrafu@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Alex Cui <alexcui1994@gmail.com>
* enable capitalized itn for es Signed-off-by: Mariana Graterol Fuenmayor <mgrafu@gmail.com> * enable capitalized itn for es_en Signed-off-by: Mariana Graterol Fuenmayor <mgrafu@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * style changes Signed-off-by: Mariana Graterol Fuenmayor <mgrafu@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix imports Signed-off-by: Mariana Graterol Fuenmayor <mgrafu@gmail.com> * mod eval scripts Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update whitelist Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> * update currencies for es itn Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> * bugfix for es time grammar Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> * update cache for es Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> * ordinal tagger fix Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> * telephone tagger fix Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> * time tagger fix Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> * time verbalizer fix Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> * update cache Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Mariana Graterol Fuenmayor <mgrafu@gmail.com> Signed-off-by: Mariana <47233618+mgrafu@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Alex Cui <alexcui1994@gmail.com>
* enable capitalized itn for es Signed-off-by: Mariana Graterol Fuenmayor <mgrafu@gmail.com> * enable capitalized itn for es_en Signed-off-by: Mariana Graterol Fuenmayor <mgrafu@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * style changes Signed-off-by: Mariana Graterol Fuenmayor <mgrafu@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix imports Signed-off-by: Mariana Graterol Fuenmayor <mgrafu@gmail.com> * mod eval scripts Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update whitelist Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> * update currencies for es itn Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> * bugfix for es time grammar Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> * update cache for es Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> * ordinal tagger fix Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> * telephone tagger fix Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> * time tagger fix Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> * time verbalizer fix Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> * update cache Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Mariana Graterol Fuenmayor <mgrafu@gmail.com> Signed-off-by: Mariana <47233618+mgrafu@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Namrata Gachchi <ngachchi@nvidia.com>
* enable capitalized itn for es Signed-off-by: Mariana Graterol Fuenmayor <mgrafu@gmail.com> * enable capitalized itn for es_en Signed-off-by: Mariana Graterol Fuenmayor <mgrafu@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * style changes Signed-off-by: Mariana Graterol Fuenmayor <mgrafu@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix imports Signed-off-by: Mariana Graterol Fuenmayor <mgrafu@gmail.com> * mod eval scripts Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update whitelist Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> * update currencies for es itn Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> * bugfix for es time grammar Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> * update cache for es Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> * ordinal tagger fix Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> * telephone tagger fix Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> * time tagger fix Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> * time verbalizer fix Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> * update cache Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Mariana Graterol Fuenmayor <mgrafu@gmail.com> Signed-off-by: Mariana <47233618+mgrafu@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
What does this PR do ?
Add a one line overview of what this PR aims to accomplish.
Before your PR is "Ready for review"
Pre checks:
git commit -sto sign.pytestor (if your machine does not have GPU)pytest --cpufrom the root folder (given you marked your test cases accordingly@pytest.mark.run_only_on('CPU')).bash tools/text_processing_deployment/export_grammars.sh --MODE=test ...pytestand Sparrowhawk here.__init__.pyfor every folder and subfolder, includingdatafolder which has .TSV files?Copyright (c) 2023, NVIDIA CORPORATION & AFFILIATES. All rights reserved.to all newly added Python files?Copyright 2015 and onwards Google, Inc.. See an example here.try import: ... except: ...) if not already done.PR Type:
If you haven't finished some of the above items you can still open "Draft" PR.