module request: piper

[KiaraGrouwstra]

piper is 'a fast, local neural text to speech system' (samples here).
it would be nice to have speechd support this as well.

cross-post: rhasspy/piper#265

/cc @Elleo who has done some work integrating these thru pied.

[csukuangfj]

I suggest that you also have a look at
https://github.com/k2-fsa/sherpa-onnx

It is implemented in C++ and has various APIs for different languages, e.g., Python/C/Go/C#/Swift/Kotlin, etc.

You can find Android APKs for it at
https://k2-fsa.github.io/sherpa/onnx/tts/apk.html

You can also try it in our huggingface space without installing anything.
https://huggingface.co/spaces/k2-fsa/text-to-speech

---

By the way, it supports models from piper as well.

[csukuangfj]

Also cc @Elleo . You may find sherpa-onnx interesting.
It supports both speech-to-text and text-to-speech.

[Elleo]

Just for a little context on what I'm doing, Pied can currently configure speech dispatcher to work with Piper through the sd_generic module, but my long term plan is to create a piper speech dispatcher module that can be kept loaded to further reduce latency and add support for speed/pitch/etc. changes

@csukuangfj Thanks, that's interesting, I'll check it out!

[coderalpha]

I'm trying to integrate Piper through the sd_generic module. But I get the error:
speechd: Error: Module reported error in request from speechd (code 3xx): 300-Opening sound device failed. Reason: Cannot open plugin server. error: file not found.

I can't find any information on this error. Any help will be appreciated!

I added the module in speechd.conf:
AddModule "piper" "sd_generic" "piper.conf"'
DefaultVoiceType "FEMALE1"
DefaultModule "piper"
DefaultLanguage "en"
AudioOutputMethod "libao"

And created the piper.conf file in the /etc/speech-dispatcher/modules directory:
AddVoice "en" "FEMALE1" "en_US-amy-medium.onnx"
DefaultVoice "en_US-amy-medium.onnx"
GenericExecuteSynth "echo \'$DATA\' | /home/dev/Apps/piper/piper --model /home/dev/Apps/piper/models/en_US-amy-medium.onnx --output_raw | paplay"

[sthibaul]

As mentioned in the issue template, add Debug 1 to the speechd config file and the module config file, and get the corresponding log files, so we get to know what exactly went wrong.

[sthibaul]

(of course, the issue template knows better, that's why we write documentation, so we don't have to rely on our memory: it's LogLevel 5 in the speechd config, and indeed Debug 1 in the module config)

[coderalpha]

I set the LogLevel to 5 and Debug to 1 and attached the files
speechd.zip

[coderalpha]

I've obviously been trying everything to get this going and in the process I've changed a lot of the config and it might not be optimal. Since it seems to be an issue with the loading of the sound plugin, I changed the AudioOutputMethod back to "pulse". I'm running Ubuntu 22.04 and according to the output of inxi, Pulse is running:

System:
Host: dev Kernel: 6.2.0-36-generic x86_64 bits: 64 Desktop: N/A
Distro: Ubuntu 22.04.3 LTS (Jammy Jellyfish)
Machine:
Type: Desktop Mobo: ASUSTeK model: WS X299 SAGE/10G v: Rev 1.xx
serial: <superuser required> UEFI: American Megatrends v: 3601
date: 09/24/2021
Audio:
Device-1: Intel 200 Series PCH HD Audio driver: snd_hda_intel
Device-2: AMD Navi 21 HDMI Audio [Radeon RX 6800/6800 XT / 6900 XT]
driver: snd_hda_intel
Device-3: AMD Navi 21 HDMI Audio [Radeon RX 6800/6800 XT / 6900 XT]
driver: snd_hda_intel
Sound Server-1: ALSA v: k6.2.0-36-generic running: yes
Sound Server-2: PulseAudio v: 15.99.1 running: yes
Sound Server-3: PipeWire v: 0.3.48 running: yes

Then I get the following error:
speechd: Error: Module reported error in request from speechd (code 3xx): 300-Opening sound device failed. Reason: Couldn't open pulse plugin.

I noticed that there is an Ubuntu package speech-dispatcher-audio-plugins, and it is installed, and contains the following:
/usr/lib/x86_64-linux-gnu/speech-dispatcher
/usr/lib/x86_64-linux-gnu/speech-dispatcher/spd_alsa.so
/usr/lib/x86_64-linux-gnu/speech-dispatcher/spd_libao.so
/usr/lib/x86_64-linux-gnu/speech-dispatcher/spd_oss.so
/usr/lib/x86_64-linux-gnu/speech-dispatcher/spd_pulse.so

So, the Pulse plugin is installed.

[sthibaul]

Since it is using the generic module, piper.conf is just passing audio to paplay (though it should rather be $PLAY_COMMAND so it works automatically with pulse, ao, alsa, etc.), so there is no need for an audio plugin.

Reason: Cannot open plugin server. error: file not found : that happens with other generic modules actually: the server is just trying to make audio go through it, and notices the error and falls back to making the module open audio itself. The warning is indeed confusing, I have now fixed it.

Your speechd.log seems to be showing various attempts, I can't see how to know what corresponds to what configuration you used.

Actually, at the end of your speechd.log there doesn't seem to be any issue?

[coderalpha]

Yes, there aren't any issues logged in speech-dispatcher.log or piper.log, but it isn't working. There is no sound played. If I run the command directly, i.e.
echo "hello" | /home/dev/Apps/piper/piper --model /home/dev/Apps/piper/models/en_US-amy-medium.onnx --output_raw

it works.

[sthibaul]

Your command is missing the paplay part?

Also, in your speech-dispatcher.log I don't see any speech attempt, how do you actually test it?

[coderalpha]

Yes, the full command I run on the command-line is:
"echo 'hello' | ./piper --output-raw --model models/en_US-amy-medium.onnx | aplay -r 22050 -f S16_LE -t raw"

Through speech-dispatcher, I test it on the command-line with spd-say "hello"