Skip to content

Update ASR Scores and Results #5254

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
titu1994 merged 1 commit into from
Oct 27, 2022
Merged

Update ASR Scores and Results #5254

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions docs/source/asr/data/scores/be/conformer_be.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
Model Name,Language,MCV Test-Set v10 (be)
stt_be_conformer_ctc_large,be,4.7 %
stt_be_conformer_transducer_large,be,3.8 %
6 changes: 3 additions & 3 deletions docs/source/asr/data/scores/ca/conformer_ca.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
Model Name,Language,MCV Test-Set v9.0 (ca)
stt_ca_conformer_ctc_large,ca,4.27
stt_ca_conformer_transducer_large,ca,3.85
Model Name,Language,MCV Dev-Set (v??) (ca),MCV Dev-Set v9.0 (ca),MCV Test-Set v9.0 (ca)
stt_ca_conformer_ctc_large,ca,,4.70,4.27
stt_ca_conformer_transducer_large,ca,,4.43,3.85
4 changes: 2 additions & 2 deletions docs/source/asr/data/scores/ca/quartznet15x5_ca.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
Model Name,Language,MCV Dev-Set (v??) (ca)
stt_ca_quartznet15x5,ca,6.0
Model Name,Language,MCV Dev-Set (v??) (ca),MCV Dev-Set v9.0 (ca),MCV Test-Set v9.0 (ca)
stt_ca_quartznet15x5,ca,6.0,,
4 changes: 2 additions & 2 deletions docs/source/asr/data/scores/en/conformer_en.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
Model Name,Language,Librispeech Dev-Clean,Librispeech Dev-Other,Librispeech Test-Clean,Librispeech Test-Other,MCV Test-Set v8.0 (en),MLS Dev (en),MLS Test (en),NSC Part1,NSC Part6,Peoples Speech Test v1,SLR 83 Test,WSJ Dev 93,WSJ Eval 92
stt_en_conformer_ctc_small,en,3.6,8.1,3.7,8.1,,,,,,,,,
stt_en_conformer_ctc_medium,en,2.5,5.8,2.6,5.9,,,,,,,,,
stt_en_conformer_ctc_large,en,2.0,4.4,2.1,4.3,,,,,,,,,
stt_en_conformer_ctc_large,en,1.9,4.4,2.1,4.5,,,,,,,,,
stt_en_conformer_ctc_xlarge,en,1.77 %,3.79 %,2.00 %,3.74 %,7.88 %,,5.99 %,,6.44 %,22.90 %,5.50 %,2.36 %,
stt_en_conformer_ctc_small_ls,en,3.3,8.8,3.4,8.8,,,,,,,,,
stt_en_conformer_ctc_medium_ls,en,2.7,7.4,3.0,7.3,,,,,,,,,
stt_en_conformer_ctc_large_ls,en,2.4,6.2,2.7,6.0,,,,,,,,,
stt_en_conformer_transducer_small,en,2.8,6.6,2.5,6.6,,,,,,,,,
stt_en_conformer_transducer_medium,en,2.0,4.6,2.1,4.7,,,,,,,,,
stt_en_conformer_transducer_large,en,1.5,3.5,1.7,3.6,,,,,,,,,
stt_en_conformer_transducer_large,en,1.6,3.5,1.7,3.7,,,,,,,,,
stt_en_conformer_transducer_large_ls,en,2.1,5.0,2.3,5.1,,,,,,,,,
stt_en_conformer_transducer_xlarge,en,1.48 %,2.95 %,1.62 %,3.01 %,6.46 %,4.59 %,5.32 %,5.70 %,6.47 %,21.32 %,,2.05 %,1.17 %
stt_en_conformer_transducer_xxlarge,en,1.52 %,3.09 %,1.72 %,3.14 %,,5.29 %,5.85 %,6.64 %,,,,2.42 %,1.49 %
7 changes: 7 additions & 0 deletions docs/source/asr/data/scores/en/squeezeformer_en.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
Model Name,Language,Librispeech Dev-Clean,Librispeech Dev-Other,Librispeech Test-Clean,Librispeech Test-Other,MCV Test-Set v8.0 (en),MLS Dev (en),MLS Test (en),NSC Part1,NSC Part6,Peoples Speech Test v1,SLR 83 Test,WSJ Dev 93,WSJ Eval 92
stt_en_squeezeformer_ctc_xsmall_ls,en,3.6 %,9.7 %,3.8 %,9.4 %,,,,,,,,,
stt_en_squeezeformer_ctc_small_ls,en,2.9 %,7.4 %,3.1 %,7.4 %,,,,,,,,,
stt_en_squeezeformer_ctc_small_medium_ls,en,2.7 %,7.0 %,2.8 %,7.1 %,,,,,,,,,
stt_en_squeezeformer_ctc_medium_ls,en,2.4 %,6.2 %,2.6 %,6.3 %,,,,,,,,,
stt_en_squeezeformer_ctc_medium_large_ls,en,2.3 %,6.0 %,2.5 %,5.9 %,,,,,,,,,
stt_en_squeezeformer_ctc_large_ls,en,2.3 %,5.7 %,2.4 %,5.7 %,,,,,,,,,
8 changes: 5 additions & 3 deletions docs/source/asr/data/scores/enes/conformer_enes.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
Model Name,Language,Fisher-Dev-Es,Librispeech Dev-Clean,Librispeech Dev-Other,Librispeech Test-Clean,Librispeech Test-Other,MCV Dev-Set v7.0 (en),MLS Dev (es),VoxPopuli Dev (es)
stt_enes_conformer_ctc_large,enes,16.7 %,2.2 %,5.5 %,2.6 %,5.5 %,5.8 %,3.5 %,5.7 %
stt_enes_conformer_transducer_large,enes,16.2 %,2.0 %,4.6 %,2.2 %,4.6 %,5.0 %,3.3 %,5.3 %
Model Name,Language,Fisher-Dev-En,Fisher-Dev-Es,Fisher-Test-En,Fisher-Test-Es,Librispeech Dev-Clean,Librispeech Dev-Other,Librispeech Test-Clean,Librispeech Test-Other,MCV Dev-Set v7.0 (en),MCV Dev-Set v7.0 (es),MCV Test-Set v7.0 (en),MCV Test-Set v7.0 (es),MLS Dev (en),MLS Dev (es),MLS Test (en),MLS Test (es),VoxPopuli Dev (en),VoxPopuli Dev (es),VoxPopuli Test (en),VoxPopuli Test (es)
stt_enes_conformer_ctc_large,enes,,16.7 %,,,2.2 %,5.5 %,2.6 %,5.5 %,5.8 %,,,,,3.5 %,,,,5.7 %,,
stt_enes_conformer_ctc_large_codesw,enes,,16.51 %,,16.31 %,2.22 %,5.36 %,2.55 %,5.38 %,,5.00 %,,5.51 %,,3.46 %,,3.73 %,,5.58 %,,6.63 %
stt_enes_conformer_transducer_large,enes,,16.2 %,,,2.0 %,4.6 %,2.2 %,4.6 %,5.0 %,,,,,3.3 %,,,,5.3 %,,
stt_enes_conformer_transducer_large_codesw,enes,15.70 %,,15.66 %,,1.97 %,4.54 %,2.17 %,4.53 %,4.51 %,,5.06 %,,3.27 %,,3.67 %,,5.28 %,,6.54 %,
4 changes: 2 additions & 2 deletions docs/source/asr/data/scores/enes/contextnet_enes.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
Model Name,Language,Fisher-Dev-Es,Librispeech Dev-Clean,Librispeech Dev-Other,Librispeech Test-Clean,Librispeech Test-Other,MCV Dev-Set v7.0 (en),MLS Dev (es),VoxPopuli Dev (es)
stt_enes_contextnet_large,enes,14.8 %,2.2 %,5.6 %,2.3 %,5.5 %,4.7 %,3.0 %,5.0 %
Model Name,Language,Fisher-Dev-En,Fisher-Dev-Es,Fisher-Test-En,Fisher-Test-Es,Librispeech Dev-Clean,Librispeech Dev-Other,Librispeech Test-Clean,Librispeech Test-Other,MCV Dev-Set v7.0 (en),MCV Dev-Set v7.0 (es),MCV Test-Set v7.0 (en),MCV Test-Set v7.0 (es),MLS Dev (en),MLS Dev (es),MLS Test (en),MLS Test (es),VoxPopuli Dev (en),VoxPopuli Dev (es),VoxPopuli Test (en),VoxPopuli Test (es)
stt_enes_contextnet_large,enes,,14.8 %,,,2.2 %,5.6 %,2.3 %,5.5 %,4.7 %,,,,,3.0 %,,,,5.0 %,,
2 changes: 1 addition & 1 deletion docs/source/asr/data/scores/hr/conformer_hr.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
Model Name,Language,ParlaSpeech-HR v1.0 (dev),ParlaSpeech-HR v1.0 (test)
Model Name,Language,ParlaSpeech Dev-Set v1.0 (hr),ParlaSpeech Test-Set v1.0 (hr)
stt_hr_conformer_ctc_large,hr,4.43,4.70
stt_hr_conformer_transducer_large,hr,4.56,4.69
4 changes: 2 additions & 2 deletions docs/source/asr/data/scores/rw/conformer_rw.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
Model Name,Language,MCV Test-Set v9.0 (rw)
stt_rw_conformer_ctc_large,rw,18.22
stt_rw_conformer_transducer_large,rw,16.19
stt_rw_conformer_ctc_large,rw,18.2 %
stt_rw_conformer_transducer_large,rw,16.2 %
44 changes: 42 additions & 2 deletions docs/source/asr/scores.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,23 +42,40 @@ EN

--------------------

CA
.. csv-table::
:header-rows: 1
:align: left
:file: data/scores/en/squeezeformer_en.csv

--------------------

BE
^^

.. csv-table::
:header-rows: 1
:align: left
:file: data/scores/ca/quartznet15x5_ca.csv
:file: data/scores/be/conformer_be.csv

--------------------

CA
^^

.. csv-table::
:header-rows: 1
:align: left
:file: data/scores/ca/conformer_ca.csv

--------------------

.. csv-table::
:header-rows: 1
:align: left
:file: data/scores/ca/quartznet15x5_ca.csv

--------------------

DE
^^

Expand Down Expand Up @@ -189,6 +206,16 @@ IT

--------------------

KAB
^^^

.. csv-table::
:header-rows: 1
:align: left
:file: data/scores/kab/conformer_kab.csv

--------------------

PL
^^

Expand All @@ -209,6 +236,16 @@ RU

--------------------

RW
^^

.. csv-table::
:header-rows: 1
:align: left
:file: data/scores/rw/conformer_rw.csv

--------------------

ZH
^^

Expand All @@ -223,3 +260,6 @@ ZH
:header-rows: 1
:align: left
:file: data/scores/zh/conformer_zh.csv

--------------------