ReSpeaker – First Impressions + Simple Offline Voice Recognition

I had the opportunity to get a free ReSpeaker core during their Kickstarter in exchange for an honest review—an offer I couldn’t pass up.

You can think of a ReSpeaker as something like an Amazon Echo, but it’s open-source and you can re-configure it to do whatever you want. You can hook it up to the online cloud voice service of your choice and have it handle complex questions like, “What’s the weather like in Grand Rapids?” or “What’s the average air velocity of a swallow?”

One of the features I was particularly interested in is that you can configure it to detect keywords and simple commands (“play music,” “launch the missiles”) offline with no internet connection.

Here is an example of how to do that:

Basic ReSpeaker Setup

First, we need to get the ReSpeaker connected to your wifi. You’ll also need a microSD card.

  • Insert your microSD card, plug in your ReSpeaker, and power it on. Give it a few seconds to boot.
  • From a phone or computer, join the “Linkit Smart _xxxxxx” wifi network (the “xxxxxx” will be some random text specific to your device).
  • Use a browser to connect to http://192.168.100.1.
  • Follow the prompt to set up a root password.
  • Select the “Network” tab, then select “Station mode.”
  • Choose the wifi network you want the ReSpeaker to use from the drop-down menu and enter its password. Then click “configure.”
  • Connect your phone or computer to the same wifi network where you connected the ReSpeaker.
  • If you have a Mac or Linux machine, you should now be able to SSH to your ReSpeaker:
    • ssh [email protected]
    • Give it the root password you configured earlier
    • If you have a Windows machine, you’ll probably have to find another way to figure out its IP address, and you’ll probably want to use PuTTY for SSH.

There is more “getting started” documentation here.

Built-In AirPlay/DLNA Support

At this point, if you want, you should be able to use AirPlay to stream music to your ReSpeaker. On your phone or computer, just select “Shairport Sync on mylinkit” in AirPlay. Then play some music via the audio jack on your ReSpeaker.

Offline Command Recognition

Here’s how I set this up:

  • pip install pyusb
  • cd /tmp/run/mountd/mmcblk0p1
  • git clone https://github.com/respeaker/respeaker_python_library
  • cd respeaker_python_library
  • python setup.py develop
  • python examples/offline_voice_assistant.py

It will take a while to start up and spit out some warning from ALSA lib. But when it starts to spit out some 0s, yell, “HEY, RESPEAKER,” and you should see output something like this:

root@mylinkit:/tmp/run/mountd/mmcblk0p1/respeaker_python_library# python examples/offline_voice_assistant.py
ALSA lib pcm.c:2239:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.front
ALSA lib pcm.c:2239:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.rear
ALSA lib pcm.c:2239:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.center_lfe
ALSA lib pcm.c:2239:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.side
ALSA lib pcm.c:2239:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.surround21
ALSA lib pcm.c:2239:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.surround21
ALSA lib pcm.c:2239:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.surround40
ALSA lib pcm.c:2239:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.surround41
ALSA lib pcm.c:2239:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.surround50
ALSA lib pcm.c:2239:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.surround51
ALSA lib pcm.c:2239:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.surround71
ALSA lib pcm.c:2239:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.iec958
ALSA lib pcm.c:2239:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.iec958
ALSA lib pcm.c:2239:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.iec958
ALSA lib pcm.c:2239:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.hdmi
ALSA lib pcm.c:2239:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.hdmi
ALSA lib pcm.c:2239:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.modem
ALSA lib pcm.c:2239:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.modem
ALSA lib pcm.c:2239:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.phoneline
ALSA lib pcm.c:2239:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.phoneline
00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001110000000000000000000000000000000000001111+111111000111100000000000000000000000000000000000000000000000-
recognized hey respeaker , analyzed 27 chunks
wakeup

Hooray, it works! :D Go ahead and press “ctrl+c” to kill the script.

At the moment, our offline recognizer only knows how to recognize “Hey, ReSpeaker” and “Alexa.”

I found the default mic capture volume to be set too low (hence the yelling). You can use alsamixer to set it higher.

Adjust the Volume

To make this adjustment:

  • Run alsamixer.
  • Press the tab key to switch to the capture mixer (as opposed to the default playback mixer).
  • Turn the capture volume and ADC PCM volume up quite a bit. I’m not sure if both are needed though.
  • Try python examples/offline_voice_assistant.py again. It should be much more sensitive now.

Add a Custom Command

Of course, we want our ReSpeaker to be able to recognize more than just “Hey, ReSpeaker” and “Alexa.” To help our script recognize a new command, we first need to add it to the keywords.txt file. To do this:

  • Edit the keywords file with a command like: vi respeaker/pocketsphinx-data/keywords.txt.
  • Add your command, for example “launch missiles” or “activate omega thirteen” or “play music,” with a threshold (you can copy one of the existing examples in the file). A larger threshold lowers the chance of false positives, but it may make the phrase harder to recognize.
  • Don’t put any blank lines in the keywords file or you might get segfaults.

Then you’ll need to edit the examples/offline_voice_assistant.py and add code to do something when the new command is recognized. The code has an example for “play music” here, which you can copy, replace “play music” with your command, and add something like: print “it worked!”. Then you can start up the script again python examples/offline_voice_assistant.py and see it recognize your voice command!

Why ReSpeaker?

There’s nothing on the ReSpeaker core board that you couldn’t conceivably also do with a Raspberry Pi or a C.H.I.P (you can use PocketSphinx pretty much anywhere), and the ReSpeaker’s CPU is quite a bit slower. So why use it? I think the add-on microphone array (which I didn’t have an opportunity to try out) is where ReSpeaker has the potential to really stand out. The microphone array should add the ability to easily recognize commands anywhere in a room. Making a similar setup on a Raspberry Pi would require quite a bit more setup and hardware.

Basically, if you want a cheap, general-purpose SBC with wifi, there might be better options. But if you’ve wanted a hackable voice control system, you should consider giving ReSpeaker a try.