Made a quick NPC announcer for this virtual pokemon style gathering tech (aka Online Town)
Online Town was used for a previous conference, ICLR, where people even “went to the beach” (there’s several environments, I chose the conference hall one).
ICLR Town: Pokemon-esque environment to wander around and bump into people, which syncs almost seamlessly with video-chatting capabilities. — maithra_raghu
However there is no chat history, and no way to set a description or announcements. In order to do so, Ondrej Biza suggested I actually just have a repeating audio announcement character: the Town Announcer.
After some googling, I quickly whipped together a set of bash scripts to do so. This is is Ubuntu 18.04.
1. Create recording of the announcement
I used `festival`, since I already had that installed for my terminal timer (blinks the terminal red, plays a loud sound, and speaks “n minutes are up”).
sudo apt install festival
text2wave onlinetown.txt | lame - text.mp3
Inside the text file I had
$ vi onlinetown.txt
Welcome! The next scheduled event is: Happy Hour at 5pm Eastern. Again that
is at 5pm Eastern. Move away
to stop hearing this voiceover. To add to this voiceover email me at a b c at m i t dot edu
(Side note: Somehow festival is really bad at pronouncing email, oh well).
2. Create “virtual microphone”
I can then select this as my mic input for the videoconference
As per sebpiq on stackoverflow
pactl load-module module-pipe-source source_name=virtmic file=/tmp/virtmic format=s16le rate=16000 channels=1
3. Play speed recording over virtual mic
As per sebpiq on stackoverflow
ffmpeg -re -i text.mp3 -f s16le -ar 16000 -ac 1 - > /tmp/virtmic
4. Create bash script to continuously do so
$ vi loop_mic.sh
#!/bin/bash
while true
do
echo "Press [CTRL+C] to stop.."
ffmpeg -re -i text.mp3 -f s16le -ar 16000 -ac 1 - > /tmp/virtmic
sleep 0.5
done
5. Run it
chmod +x loop_mic.sh
sh loop_mic.sh
Check that it’s in the settings
Then open online.town, and when it asks for which microphone input to select, select the Unix FIFO.
To check that it’s all working, I joined from another computer & could hear the announcer. Sometimes it does seem like I can’t hear the announcer when joining, but I can’t reliably reproduce it.
Another sanity check is to go back to Sound settings and check that the “input level” bars are going up and down.
Appendix
I did try speeding up the audio, but it did introduce new audio artifacts (squeaks). If you want to do so, add a setting when generating the mp3. Below it speeds up to 2x.
text2wave -eval "(Parameter.set 'Duration_Stretch 0.5)" onlinetown.txt -o text.mp3
( I also tried speeding it up in the `ffmpeg` command, it sounded horrible).
Better text to speech
This was a pain… ! Not sure why. Very confusing instructions.
Helped in part As per. https://levelup.gitconnected.com/installing-mozilla-tts-on-a-raspberry-pi-4-e6af16459ab9
These are final records of what worked.
2022 apt-get install -y espeak libsndfile1 python3-venv
0000 python3 -m venv env
0000 source env/bin/activate
2029 pip3 install -U pip setuptools wheel
2050 pip install https://github.com/reuben/TTS/releases/download/ljspeech-fwd-attn-pwgan/TTS-0.0.1+92aea2a-py3-none-any.whl
2001 git clone git@github.com:mozilla/TTS.git
0000 cd TTS
2053 pip3 install packaging
2052 python -m TTS.server.server
0000 firefox localhost:5002
# FAILS unless inside TTS git folder!
Traceback from failure:
# FAILURE CASE
$ python -m TTS.server.server
> Loading TTS model ...
| > model config: None
| > checkpoint file: None
Traceback (most recent call last):
File "/usr/lib/python3.6/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.6/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/home/chai/projects/TTS/server/server.py", line 62, in <module>
synthesizer = Synthesizer(args)
File "/home/chai/projects/TTS/server/synthesizer.py", line 36, in __init__
self.config.use_cuda)
File "/home/chai/projects/TTS/server/synthesizer.py", line 52, in load_tts
self.tts_config = load_config(tts_config)
File "/home/chai/projects/TTS/utils/io.py", line 16, in load_config
with open(config_path, "r") as f:
TypeError: expected str, bytes or os.PathLike object, not NoneType
Some other commands I tried (didn’t work)
https://github.com/mozilla/TTS/wiki/Released-Models#simple-packaging---self-contained-package-that-runs-an-http-api-for-a-pre-trained-tts-model
sudo apt-get install -y espeak libsndfile1 python3-venv
python3 -m venv env
source env/bin/activate && which python3
pip install -U https//example.com/url/to/python/package.whl
pip3 install -U pip setuptools wheel
$ python -m TTS.server.server --help
It will say synthesizing, After a minute or two it will finish and start playing the audio. Right click and “save audio”
Output is a wav file.
What successful TTS.server.server looks like:
(env) 15:53:25 chai@W530:~/projects/TTS (master %)$ python -m TTS.server.server
/home/chai/projects/env/lib/python3.6/site-packages/librosa/util/decorators.py:9: NumbaDeprecationWarning: An import was requested from a module that has moved location.
Import of 'jit' requested from: 'numba.decorators', please update to use 'numba.core.decorators' or pin to Numba version 0.48.0. This alias will not be present in Numba version 0.50.0.
from numba.decorators import jit as optional_jit
> Loading TTS model ...
| > model config: /home/chai/projects/env/lib/python3.6/site-packages/TTS/server/model/tts/config.json
| > checkpoint file: /home/chai/projects/env/lib/python3.6/site-packages/TTS/server/model/tts/checkpoint.pth.tar
> Setting up Audio Processor...
| > sample_rate:22050
| > num_mels:80
| > min_level_db:-100
| > frame_shift_ms:12.5
| > frame_length_ms:50
| > ref_level_db:20
| > num_freq:1025
| > power:1.5
| > preemphasis:0.98
| > griffin_lim_iters:60
| > signal_norm:True
| > symmetric_norm:True
| > mel_fmin:0
| > mel_fmax:8000.0
| > max_norm:4.0
| > clip_norm:True
| > do_trim_silence:True
| > trim_db:60
| > sound_norm:False
| > n_fft:2048
| > hop_length:275
| > win_length:1100
> Using model: Tacotron2
> Loading PWGAN model ...
| > model config: /home/chai/projects/env/lib/python3.6/site-packages/TTS/server/model/pwgan/config.yml
| > model file: /home/chai/projects/env/lib/python3.6/site-packages/TTS/server/model/pwgan/checkpoint.pkl
> Setting up Audio Processor...
| > sample_rate:22050
| > num_mels:80
| > min_level_db:-100
| > frame_shift_ms:12.5
| > frame_length_ms:50
| > ref_level_db:20
| > num_freq:1025
| > power:None
| > preemphasis:0.98
| > griffin_lim_iters:None
| > signal_norm:True
| > symmetric_norm:True
| > mel_fmin:0
| > mel_fmax:8000.0
| > max_norm:4.0
| > clip_norm:True
| > do_trim_silence:True
| > trim_db:60
| > sound_norm:False
| > n_fft:2048
| > hop_length:275
| > win_length:1100
* Serving Flask app "server" (lazy loading)
* Environment: production
WARNING: This is a development server. Do not use it in a production deployment.
Use a production WSGI server instead.
* Debug mode: off
[INFO] * Running on http://0.0.0.0:5002/ (Press CTRL+C to quit)
> Model input: Welcome! The next scheduled event is: Happy Hour at 5pm Eastern. Again that is at 5pm Eastern. Move away to stop hearing this voiceover.
['Welcome!', 'The next scheduled event is: Happy Hour at 5pm Eastern.', 'Again that is at 5pm Eastern.', 'Move away to stop hearing this voiceover.']
[INFO] 127.0.0.1 - - [01/Jun/2020 15:54:03] "GET /api/tts?text=Welcome!%20The%20next%20scheduled%20event%20is%3A%20Happy%20Hour%20at%205pm%20Eastern.%20Again%20that%20is%20at%205pm%20Eastern.%20Move%20away%20to%20stop%20hearing%20this%20voiceover. HTTP/1.1" 200 -
Hurray. ! The output sounds much nicer.
Note:
Undo with
pactl unload-module module-pipe-source
