Category Archives: Uncategorized

Town Announcer NPC for Online Town (ICRA2020 virtual happy hour)

Made a quick NPC announcer for this virtual pokemon style gathering tech (aka Online Town)

Online Town was used for a previous conference, ICLR, where people even “went to the beach” (there’s several environments, I chose the conference hall one).

ICLR Town: Pokemon-esque environment to wander around and bump into people, which syncs almost seamlessly with video-chatting capabilities. — maithra_raghu

However there is no chat history, and no way to set a description or announcements. In order to do so, Ondrej Biza suggested I actually just have a repeating audio announcement character: the Town Announcer.

After some googling, I quickly whipped together a set of bash scripts to do so. This is is Ubuntu 18.04.

1. Create recording of the announcement

I used `festival`, since I already had that installed for my terminal timer (blinks the terminal red, plays a loud sound, and speaks “n minutes are up”).

sudo apt install festival
text2wave onlinetown.txt | lame - text.mp3

Inside the text file I had

$ vi onlinetown.txt

Welcome! The next scheduled event is: Happy Hour at 5pm Eastern. Again that
is at 5pm Eastern. Move away
to stop hearing this voiceover. To add to this voiceover email me at a b c at m i t dot edu

(Side note: Somehow festival is really bad at pronouncing email, oh well).

2. Create “virtual microphone”

I can then select this as my mic input for the videoconference

As per sebpiq on stackoverflow

pactl load-module module-pipe-source source_name=virtmic file=/tmp/virtmic format=s16le rate=16000 channels=1

3. Play speed recording over virtual mic

As per sebpiq on stackoverflow

ffmpeg -re -i text.mp3 -f s16le -ar 16000 -ac 1 - > /tmp/virtmic

4. Create bash script to continuously do so

$ vi loop_mic.sh

#!/bin/bash
while true
do 
echo "Press [CTRL+C] to stop.."
ffmpeg -re -i text.mp3 -f s16le -ar 16000 -ac 1 - > /tmp/virtmic
sleep 0.5
done

5. Run it

chmod +x loop_mic.sh
sh loop_mic.sh

Check that it’s in the settings

Then open online.town, and when it asks for which microphone input to select, select the Unix FIFO.

To check that it’s all working, I joined from another computer & could hear the announcer. Sometimes it does seem like I can’t hear the announcer when joining, but I can’t reliably reproduce it.

Another sanity check is to go back to Sound settings and check that the “input level” bars are going up and down.

Appendix

I did try speeding up the audio, but it did introduce new audio artifacts (squeaks). If you want to do so, add a setting when generating the mp3. Below it speeds up to 2x.

 text2wave -eval "(Parameter.set 'Duration_Stretch 0.5)" onlinetown.txt -o text.mp3

( I also tried speeding it up in the `ffmpeg` command, it sounded horrible).

Better text to speech

This was a pain… ! Not sure why. Very confusing instructions.

Helped in part As per. https://levelup.gitconnected.com/installing-mozilla-tts-on-a-raspberry-pi-4-e6af16459ab9

These are final records of what worked.

 2022  apt-get install -y espeak libsndfile1 python3-venv
 0000 python3 -m venv env
 0000 source env/bin/activate
 2029  pip3 install -U pip setuptools wheel
 2050  pip install https://github.com/reuben/TTS/releases/download/ljspeech-fwd-attn-pwgan/TTS-0.0.1+92aea2a-py3-none-any.whl
 2001 git clone git@github.com:mozilla/TTS.git
 0000 cd TTS
 2053 pip3 install packaging
 2052 python -m TTS.server.server 
 0000 firefox localhost:5002
# FAILS unless inside TTS git folder!

Traceback from failure:

# FAILURE CASE
$ python -m TTS.server.server
> Loading TTS model ...
| > model config: None
| > checkpoint file: None
Traceback (most recent call last):
File "/usr/lib/python3.6/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.6/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/home/chai/projects/TTS/server/server.py", line 62, in <module>
synthesizer = Synthesizer(args)
File "/home/chai/projects/TTS/server/synthesizer.py", line 36, in __init__
self.config.use_cuda)
File "/home/chai/projects/TTS/server/synthesizer.py", line 52, in load_tts
self.tts_config = load_config(tts_config)
File "/home/chai/projects/TTS/utils/io.py", line 16, in load_config
with open(config_path, "r") as f:
TypeError: expected str, bytes or os.PathLike object, not NoneType

Some other commands I tried (didn’t work)

https://github.com/mozilla/TTS/wiki/Released-Models#simple-packaging---self-contained-package-that-runs-an-http-api-for-a-pre-trained-tts-model

sudo apt-get install -y espeak libsndfile1 python3-venv
python3 -m venv env
source env/bin/activate && which python3
pip install -U https//example.com/url/to/python/package.whl
pip3 install -U pip setuptools wheel
$ python -m TTS.server.server --help

It will say synthesizing, After a minute or two it will finish and start playing the audio. Right click and “save audio”

Output is a wav file.
What successful TTS.server.server looks like:

(env) 15:53:25 chai@W530:~/projects/TTS (master %)$ python -m TTS.server.server
/home/chai/projects/env/lib/python3.6/site-packages/librosa/util/decorators.py:9: NumbaDeprecationWarning: An import was requested from a module that has moved location.
Import of 'jit' requested from: 'numba.decorators', please update to use 'numba.core.decorators' or pin to Numba version 0.48.0. This alias will not be present in Numba version 0.50.0.
  from numba.decorators import jit as optional_jit
 > Loading TTS model ...
 | > model config:  /home/chai/projects/env/lib/python3.6/site-packages/TTS/server/model/tts/config.json
 | > checkpoint file:  /home/chai/projects/env/lib/python3.6/site-packages/TTS/server/model/tts/checkpoint.pth.tar
 > Setting up Audio Processor...
 | > sample_rate:22050
 | > num_mels:80
 | > min_level_db:-100
 | > frame_shift_ms:12.5
 | > frame_length_ms:50
 | > ref_level_db:20
 | > num_freq:1025
 | > power:1.5
 | > preemphasis:0.98
 | > griffin_lim_iters:60
 | > signal_norm:True
 | > symmetric_norm:True
 | > mel_fmin:0
 | > mel_fmax:8000.0
 | > max_norm:4.0
 | > clip_norm:True
 | > do_trim_silence:True
 | > trim_db:60
 | > sound_norm:False
 | > n_fft:2048
 | > hop_length:275
 | > win_length:1100
 > Using model: Tacotron2
 > Loading PWGAN model ...
 | > model config:  /home/chai/projects/env/lib/python3.6/site-packages/TTS/server/model/pwgan/config.yml
 | > model file:  /home/chai/projects/env/lib/python3.6/site-packages/TTS/server/model/pwgan/checkpoint.pkl
 > Setting up Audio Processor...
 | > sample_rate:22050
 | > num_mels:80
 | > min_level_db:-100
 | > frame_shift_ms:12.5
 | > frame_length_ms:50
 | > ref_level_db:20
 | > num_freq:1025
 | > power:None
 | > preemphasis:0.98
 | > griffin_lim_iters:None
 | > signal_norm:True
 | > symmetric_norm:True
 | > mel_fmin:0
 | > mel_fmax:8000.0
 | > max_norm:4.0
 | > clip_norm:True
 | > do_trim_silence:True
 | > trim_db:60
 | > sound_norm:False
 | > n_fft:2048
 | > hop_length:275
 | > win_length:1100
 * Serving Flask app "server" (lazy loading)
 * Environment: production
   WARNING: This is a development server. Do not use it in a production deployment.
   Use a production WSGI server instead.
 * Debug mode: off
[INFO]  * Running on http://0.0.0.0:5002/ (Press CTRL+C to quit)
 > Model input: Welcome! The next scheduled event is: Happy Hour at 5pm Eastern. Again that is at 5pm Eastern. Move away to stop hearing this voiceover.
['Welcome!', 'The next scheduled event is: Happy Hour at 5pm Eastern.', 'Again that is at 5pm Eastern.', 'Move away to stop hearing this voiceover.']
[INFO] 127.0.0.1 - - [01/Jun/2020 15:54:03] "GET /api/tts?text=Welcome!%20The%20next%20scheduled%20event%20is%3A%20Happy%20Hour%20at%205pm%20Eastern.%20Again%20that%20is%20at%205pm%20Eastern.%20Move%20away%20to%20stop%20hearing%20this%20voiceover. HTTP/1.1" 200 -

Hurray. ! The output sounds much nicer.

 

Note:

Undo with

pactl unload-module module-pipe-source

Pandemic Diary #10, 23 May 2020

Productivity levels (0-3), about a 1
Very sleepy this week, been disorganized the past two weeks. Not sure if due to pulling an all nigher this past week (I slept enough the days after though). Just crashed on the floor for an hour, trying to fight it felt like a stone in a lake trying to swim up against gravity; kinda given up on the sleep doctor but maybe I’ll try again.

Still have some skin rash too, now shifting to more parts of my body. Running out of the hydrocortisone. I wonder if I should try getting an air purifier?

Need to do: dental insurance paperwork, braces, vote (yay absentee ballots all arrived — filed the forms via email too)

I’ve been posting cat pictures to instagram now. Quite questionable, entering the social media of the youngsters.

Visited a bamboo grove last week — super tall bamboo, and shoots everywhere as tall as my hip.. Just randomly in the middle of a normal forest. Quite tasty. Bamboo looks interesting inside. Reminds me of fin gripper from TRI.

ICRA video – mild disaster. Happy with slides, but final microphone take was bad — recorded over zoom, so there’s artifacts, and lots of clothes rustling noise ironically (put mic on myself to get rid of keyboard noise). Can’t find my Zoom h1 camera for narwhaledu days. It’s $100 ! Better get some other solution.

Had fire alarm battery die and make chirpy noises. Cat was so scared she cried! She also seems to have a penchant for eating paper. Seems harmless enough. Her missing hair has recovered! I put a collar on her and she tolerated it okay.

She’s got high-five and sit kinda down, working on a jump. I know I’m not teaching her properly, she gets her paws confused. Maybe parents teaching her differently.

Read no COVID news nor activism lately. Feel quite depressed about mandatory masks op-ed being buried. Having no luck finding a subletter. Maybe that’s okay, sounds like MIT might start re-opening. Need to finish classwork. 🙁

Todo: find fellowship and scholarship and grant opportunities. Finish ISER paper. Update quals committee. (Keep focus on how far I’ve come vs compare to other people).

Got monoprice printer working again. Realized need to stop being so perfectionist about bed leveling, just roughly level and print, the print will tell you if need to adjust bed leveling further.

https://www.thingiverse.com/thing:2245341

https://satoshinm.github.io/blog/180122_3dprint1_3d_printing_with_the_monoprice_select_mini_v2_first_impressions_and_prints_a_saddle_knob_guard_guide_spinner_a_cube_and_clip_squeezer_holder_dish_case_stand_fingers_and_carousel.html

Thinking I need to create another blog. Maybe give up on fixing formatting on old posts. Also thinking ridiculous to have two blogs and a portfolio. Academic life, notes to self, portfolio, and random blog. Do papers and class projects define who I am as a person?

Feeling lousy over old projects I left in a huff. Current lab partner was really nice even though he did 90% of the work.

Learned to spray the fungicide for the black spot mildew.

Cat is:  138.6 – 125.6 = 13 lbs exactly. Slowly moving needle in right direction.

Pandemic Diary #9, 17 May 2020

Too much watching of animal hoarder confessions

Stressed about many things… ICRA videos, homework, apartment, not enough done on covid front

Looking for new “learning chinese drama” to watch with parents

Uprooted some bamboo shoots today- – they’re fuzzy!!

AH finally found the drama about a crossdressing chinese general – it’s actually a web series not a drama, apparently, but one with 60 episodes!!

https://en.wikipedia.org/wiki/Oh_My_General

https://www.viki.com/tv/35829c-oh-my-general?locale=en

https://mydramalist.com/20114-oh-my-general

 

https://reelrundown.com/tv/top-22-best-historical-chinese-dramas-you-must-watch 7. God of War, Zhao Yun is loosely based on the 14th-century Chinese novel Romance of the Three Kingdoms

https://mydramalist.com/discussions/recommendation-forum/33510-dramas-where-female-lead-is-highly-skilled-intelligent-resourceful

https://mydramalist.com/discussions/recommendation-forum/31889-chinese-japanese-dramas-with-strong-female-leads