Getting started with AprilTags on Ubuntu 16.04

I recently started working with April tags, since they seem cool & you see them all over the place (used as fiducials for robots trying to walk around a somewhat unstructured environment).


the internet fell short (feel free to skip this section)

It was surprisingly hard to find instructions to get started, my search-fu was failing me. My search results turned up the original “official” website from the April Robotics Laboratory at University of Michigan,

This had no “getting started” instructions on it. Same for the C++ wrapper by a researcher at CSAIL (now a professor who I met at CMU!)

And same for the ROS wrapper around the Apriltags, which also confusingly seems to have several version that may or may not now be the same. and

(oh wait neat, there are instructions at However, I’m still not terribly familiar with ROS, so I wasn’t too enthused about using this wrapper.

Fortunately Patrick over at Kuindersma’s lab above me was able to get me started.

getting started

  1. Download and install C++ wrapper as per instructions at
    See below:
  2. sudo apt-get install subversion cmake libopencv-dev libeigen3-dev libv4l-dev
    sudo port install pkgconfig opencv eigen3
    svn co
    cd apriltags
  3. Yay, now a window pops open (see “troubleshooting” if it doesn’t, as was the case for me) with a video stream. But we need tags for it to recognize!

Getting tags

I actually found this pretty annoying, the zipped files on those sites give you a thousand options and it’s not clear which ones will work. So for me, I actually had my friend give me four tags that definitely work.

ID = #0 #1 #6 and #7 tags.

  1. Print out tag
  2. Run
  3. Now stick the tag in front of your camera. In the videostream you should now see a circle. In the terminal you should now see data streaming out.
  4. The data display shows distance (from the camera to the tag), the xyz location of the center of the tag, as well as the roll, pitch, and yaw. These coordinates will depend on which side you put pointing up when you pasted the tag on, so beware.  In fact, none of the data should be taken as absolute until you calibrate your camera.
2 tags detected: 
Id: 1 (Hamming: 0) distance=0.079741m, x=0.000532, y=0.006102, z=-1.487915, yaw=-0.134615, pitch=0.071828, roll=-0.041146
Id: 7 (Hamming: 0) distance=0.079741m, x=0.000532, y=0.006102, z=-1.487915, yaw=-0.134615, pitch=0.071828, roll=-0.041146
14.9312 fps
Two apriltags
Example output

calibrate camera

I wandered for a while lost in the lands of

Fortunately, eventually I found my way to a python library that made the whole process super simple. I ignored the above link (to official openCV docs) entirely. Instead, I used the following python package. All I had to do was print out the checkerboard pattern included in the repository, wave it in front of the camera and record a short video, run the python file, and bam! I had the configuration.yaml file I needed.

To get it working, I did have to make sure I had pyyaml installed
 (venv) nrw@earlgrey:$ sudo -H pip install pyyaml --upgrade

This project is super awesome and included an example you can run right away and inspect. The following line, run in the root takes in the video included in the repo (chessboard.avi) and outputs the resulting configuration file to “calibration.yaml”.

(venv) nrw@earlgrey:~/projects/video2calibration$ ./ example_input/chessboard.avi calibration.yaml --debug-dir out

At this point I printed out the checkboard pattern included in the repository, put it on a stiff surface, and then opened cheese (you don’t have to use cheese, we just need to record a video).

Then I waved my board around in front of the camera and recorded a short video.



Anyhow, I record ten or fifteen seconds of video. Then I ran

nrw@earlgrey:~/projects/video2calibration$ ./ ~/Videos/Webcam/2018-03-26-112657.webm calibration.yaml --debug-dir out

Performing calibration...
 RMS: 0.442700776066
 camera matrix:
 [[ 666.78668352    0.          343.73827809]
 [   0.          665.79103853  227.19081685]
 [   0.            0.            1.        ]]
 distortion coefficients:  [  6.06301194e-02  -1.94620209e-02   1.45555284e-04   1.24410189e-03

Input calibration parameters into source code

Edit into the demo file

nrw@earlgrey:~/projects/apriltags/example$ vi apriltags_demo.cpp

Specifically, we want to change the following section. Note that we are using the definition of the rotation matrix to pull out (from the calibration.yaml output) the focal point and principal point parameters.


  // default constructor
  Demo() :
    // default settiwgs, most can be modified through command line options (see below)
 [...excerpted section...]
    m_tagSize(0.00944), // in meters
    m_fx(667), // in pixels
    m_fy(666), //
    m_px(344), // principal point

Ah! I forgot, we also needed to measure, using a ruler (or calipers), the size of the apriltag in real life. So just measure one of the sides of the tag (which should be square…) and put it inoto m_tagSize. (The width and height should be the size in pixels of the image from the video camera).

Compile and run (use “make clean” if the build fails, then run “make” again)

nrw@earlgrey:~/projects/apriltags/example$ cd ..
nrw@earlgrey:~/projects/apriltags/$ nrw@earlgrey:~/projects/apriltags/example$ make

Then run the program:

nrw@earlgrey:~/projects/apriltags/$ ./build/bin/apriltag_demo

One easy way to double-check whether the camera is roughly calibrated is to physically measure the distance between the camera and the tag, and then compare to the “distance” output in your terminal. Hopefully they match…


The roll, pitch, and yaw are reported in radians. To convert into degrees, multiply by 57.3 (approximately).


A framerate of 17fps or so is totally reasonable, since the apriltags demo is decently compute intensive. I had a problem with lag, where the video ran smoothly but with a significant lag — this may have been a result of me running the entire thing in a virtual machine. Let me know if you don’t have lag!


I had a somewhat frustrating beginning where I couldn’t get the example program to run.

-- Found OpenCV: /opt/ros/lunar (found version "3.3.1")
 framerate (17fps raesonable, but lag???)
 output is in radians
 9.23346 fps
 0 tags detected:
 0 tags detected:
 0 tags detected:
 0 tags detected:
 0 tags detected:
 0 tags detected:
 0 tags detected:
 0 tags detected:
 1 tags detected:
 OpenCV Error: Assertion failed (mtype == type0 || (((((mtype) & ((512 - 1) << 3)) >> 3) + 1) == 1 && ((1 << type0) & fixedDepthMask) != 0)) in create, file /tmp/binarydeb/ros-lunar-opencv3-3.3.1/modules/core/src/matrix.cpp, line 2542
 terminate called after throwing an instance of 'cv::Exception'
 what():  /tmp/binarydeb/ros-lunar-opencv3-3.3.1/modules/core/src/matrix.cpp:2542: error: (-215) mtype == type0 || (((((mtype) & ((512 - 1) << 3)) >> 3) + 1) == 1 && ((1 << type0) & fixedDepthMask) != 0) in function create
Id: 15 (Hamming: 1)Aborted (core dumped)

It turned out that because I had ROS installed, or perhaps also because I installed the “apriltags” ROS wrapper, I was having openCV version conflicts.



nrw@earlgrey:~$ pkg-config --modversion opencv

To solve, I simply had to edit one line in the CMakeLists.txt to force it to use the right version of openCV. I added an “exact required” tag, along with my openCV version (, to the appropriate line.

nrw@earlgrey:~/projects/apriltags$ vi CMakeLists.txt 
(line 14)
find_package(OpenCV EXACT REQUIRED)

Then I ran “make” and

nrw@earlgrey:~/projects/apriltags/$ ./build/bin/apriltag_demo

and the example program worked! Huzzah.

the end.




How to create a timelapse (both timed pictures and post-processed video) live viewable

edit: Note that physically, the set-up is that I have an extra desktop which is always on and connect to the internet which is running Ubuntu (16.04).  I have an external webcam. This also requires a dropbox account.

I’ve often wanted to monitor something remotely, but have had issues in the past trying to set up a video stream. For monitoring experiments remotely (or for instance a 3d printer), I really don’t need 30 frames per second.

I suppose nowadays, in theory I could set up a youtube live stream, especially since I could just disable audio. But for more control over the end result, my solution is a one-liner bash command, combined with a dropbox account.

I was having a terrible time with all my search results describing ways to create a timelapse animation, instead of detailing how to take the pictures for the timelapse in the first place.


use streamer

streamer -c /dev/video1 -t 10 -r 1 -o 0000.jpeg &

The settings for the above command are as follows:

  • Using the “second” camera (ince my desktop has a built-in webcam mapped to /dev/video0, whereas I wanted to take pictures with a USB webcam)
  • 10 frames total
  • 1 frame a second (rate)
  • Name starts at 0000.jpeg (it will automatically count up)
streamer -c /dev/video1 -t 300 -r 0.0033 -s 640x480 -o 0000.jpeg &

i used the first command to test whether the command is working.

  • For the actual timelapse, I wanted to take a picture every 5 minutes, or in other words every (1/360 = 0.0033) seconds. (I don’t think there’s an option to define the rate in anything other than seconds).
  • I wanted about 24 hours of timelapse. At every 5 minutes, I take 12 an hour or 12*24 a day, so around 300 frames
  • Streamer was defaulting to 320×240 pixel images, so I asked streamer to instead take 640×480 pictures, which are what my webcam supports.


Note: I did not really play around with this, but I think you can also create video with streamer, e.g.

streamer -c /dev/video1 -t 0:30 -o movie.avi -f jpeg

installing headless dropbox

Following the online instructions from the dropbox site

cd ~ && wget -O - "" | tar xzf -
Run the synchronizer

making the timelapse persistent

screen allows our session (within which we are running streamer) to persist, even if someone closes the terminal window.

Screen Basic commands:
screen -r (reattach)
screen -d (inside screen session, to detach)

ctrl-a w (list windows)
ctrl-a 1 (switch to window #1)
ctrl-a c (new window)
exit (close window) — after last window is closed, screen terminates.

Commands to handle background process

For instance, if you messed up your settings and streamer is now taking 300 frames at a rapid rate,
$ ps x | grep streamer
$ sudo killall streamer

Putting it all together

$ nohup ~/.dropbox-dist/dropboxd&
$ screen
> cd ~/Dropbox
> streamer -c /dev/video1 -t 300 -r 0.0033 -s 640x480 -o 0000.jpeg &

Now, I can access my setup online at I can also use the “share folder using link” option on dropbox to share with multiple people.

Make a QR code

This was purely for fun, but I turned the dropbox link into a QR code which I printed out and stuck next to the experiment



I find it really handy to include labeling information not just in the filenames but also physically in the image. Below you can see I labeled the image with the date of the timelapse.0233

Making a timelapse (animation / video of the resulting images) using ffmpeg

Use ffmpeg.

 cat *.jpg | ffmpeg -r 20 -f image2pipe -i - output.mp4

Takes all jpg files and turns it into an mp4 with 20 frames per second.

I was having a lot of issues following the instructions online, but somehow using the image2pipe solved everything. (I was getting errors like “would you like to overwrite this .jpeg file y/n”).

[~/Downloads/buse/rotate]$ ffmpeg -framerate 1/2 -i *.JPG -c:v libx2^C -r 30
File '0010_rotated.JPG' already exists. Overwrite ? [y/N] nNot overwriting - exiting

One thing that surprised me on this page, is that a common command such as “all images in this folder” is a little trickier than I’d expect. (I was working with another set of images that used timestamps as filenames, instead of an orderly 0001.jpeg like other examples onlin).
1) You must use “glob option”
2) You must surround your search pattern in quotes

 ffmpeg -pattern_type glob -i "image-*.png" video.webm

note to self: to rotate all images in a directory 90 degrees clockwise:

for file in ./*.jpg do
 convert "$file" -rotate 90 "${file%.JPG}"_rotated.JPG

There you have it!

Thanks to the one-liner “streamer” command, and by using dropbox as a syncing service. you can accomplish a real-time viewable timelapse with just 4 or 5 lines of code, and then use just one more line to create a video of the resulting images.

$ nohup ~/.dropbox-dist/dropboxd&
$ screen
> cd ~/Dropbox
> streamer -c /dev/video1 -t 300 -r 0.0033 -s 640x480 -o 0000.jpeg &
 cat *.jpg | ffmpeg -r 20 -f image2pipe -i - output.mp4

Drawing Automata and the History of Computer Science

Note: This is mostly for me to remind myself of the name of the drawing automata that I still love to look at videos of and wonder at how amazing it is.


Currently I’m taking this Computer Science 191: Classics of Computer Science, where we read a bunch of famous papers tracing the evolution of computer science.

Today we talked about the Analytical Engine. (Lovelace wrote the lengthy “Notes” part which takes up the bottom 2/3). Some people at my table thought it was crazy to go from an abacus to this complicated setup. Due to my interest in mechanical engineering and robot art, I knew of this mechanical writing machine where you could slot letters in and out. I thought this was Maillardet’s (so 40 years before Babbage), but I think the letter-selecting device is actually from Jaquet-Droz automata.

The video I remember is the BBC one.


Anyhow, these were made to sell more watches (at least according to wikipeida)! Just some bored watchmakers, making crazy intense automata.

Here is another documentary which explains in the first 5 minutes or so how Jaquet made little dioramas with mechanically spinning and quacking birds (really, reminds me of the Fabergé eggs!), traveled and got a bunch of money after waiting months to see a king, and then got to go make complicated things like this automata.


Maillardet’s is at the Franklin Institute in Philadelphia. Here is some more about it from FI. Here’s a CBS show about it. Apparently it is possible it has the most “read-only memory”, about 300 kb.

Also apparently the drawing automata in Hugo was created in real life! This pleases me immensely.

Oh yea they also passed around some Jacquard’s loom punchcards. I don’t know much about weaving and so the complexity of programmable looms goes over my head, but I think the drawing automata makes it really easy to comprehend how complicated these things can get. At least, it really impressed the folks at my table 🙂

jacquardpunchcard jacquardpunchcarddetail


What else… oh I learned what squaring the circle means, since Hilbert talks about it in his “Mathematical Problems” lecture.

I also love how the wikipedia article talks about “debunking illogical circle-squaring theories“. I guess we have perpetual motion theories still! Apparently “squaring the circle” is also an idiom for “trying to do the impossible.”

Oh, and there’s a lady mathematician who formed an integral part of the 10th problem, which is the one people go gaga about trying to figure out if Hilbert would have approved of (the solution to). That’s the one where we learned that we cannot know the solution

I also thought it was cool how the work to solve this problem built up over two decades. (and one of the four contributors was Julia Robinson, who had a J.R. hypothesis that wasn’t proven until 20 years later).

Oh, and then we read Turing’s paper, the one with That Word. Entscheidungsproblem. Anyhow, I started browsing briefly and decided I really still have no idea what it means for something to be computable. The wikipedia article states “Computability is the ability to solve a problem in an effective manner”. Really, wikipedia? It’s probably similar to the question of what counts as a robot, I suppose. Anyhow, that article briefly mentions both automata (heh, sounds like automaton) and also hypercomputability. Such a fun word.

Okay, now back to fixing the problem of why my alarm clock app, Timely, no longer goes off reliably :((((((((((( I thought Timely might have updated, but it says the last update was in September of 2017. So maybe it’s some Tmobile or Android or Samsung update that borked my alarm clock 🙁 I’ll try the only other version of Timely, a 2015 update, and report back.


connecting to eduroam & harvard wireless without having to install sketchy things

UPDATE 27 SEPT 2019 (checked current as of 27 Dec 2019 as well) – Ubuntu 19.04 / Android version 9

It appears eduroam settings have changed (CA certificate *must* be installed)

On my laptop (Ubuntu 19.04)

Where the CA file comes from


(thanks old blog post)

Note: Eduroam appears to be the same settings as Harvard Secure


I also had to install the certs on my phone

  1. Put above file onto phone (I transferred using bluetooth> send files, but usb would work too I’m sure)
  2. Install on phone
  3. And then type some sensible name
  4. Then, connect to Harvard Secure wifi with similar settings as in the desktop case. Select the CA certificate we just installed.

And hopefully it will work.

  PROTIP: You have to be fully disconnect from the network (not attempting to connect to it) in order to edit the settings.

I’m still working out how to reconnect to eduroam… sigh

Legitimately, when I have friends co-work, it’s so annoying for them to connect to the internet that I end up just giving them my password. Which is wrong and bad for all kinds of reasons, but that’s life as an end-user…

Troubleshooting / via CLI for the nerds


rui@chaiX1YG2:/ $ nmcli con 
eduroam 9196d2be-df9e-4f92-8781-c5c47c60e90d wifi wlp4s0
rui@chaiX1YG2:/etc/NetworkManager/system-connections$ $ sudo cat eduroam
[sudo] password for rui: 






OUTDATED – Old post below:

Harvard is very insistent about me downloading random software in order to connect to their wireless network, even though I have Harvard credentials (and eduroam credentials).

I have no idea why they do this. Why do I need to download software to connect to a wifi network? Anyhow, the solution is to just ignore it and figure out the settings on my own.

I just use eduroam. Here is what the internet told me:

  • Security : WPA & WPA2 Enterprise
  • Authentication : Tunnelled TLS
  • Anonymous Identity : (I left this blank,but I think they have configured the network that way so it won’t ask for this kind of Identity)
  • CA certificate : (I left this blank as well,but again this has to do with the configuration of the wireless network,so it might be different in other educational institutes)
  • Inner Authentication : PAP
  • Username : (the one given from your school)
  • Password : (the one given from your school)



Identity & password = my harvard credentials.


Right now I am using Windows 10, and I have discovered that the wireless situation is not really better and maybe somewhat worse than on Ubuntu. *sigh*

Anyhow, I had expired MIT credentials. I finally figured out how to change my password: Go to settings -> Wifi -> Manage known networks (this option is kind of hard to see) -> eduroam -> Forget.


(Jeebus knows why “properties” doesn’t tell me / let me do anything useful)

Then, the next time I click on “eduroam” and “Connect”, it will prompt me for a username and password. Enter in my harvard credentials and voila.


pom poms galore

poms poms? those things cheerleaders wave around?

Kind of, but made of yarn!

one of the first pom poms i ever made!

i was looking at some knitting blog, when i was interested in making open source versions of textile manufacturing machines, and the author briefly mentioned trying to find a book for sale by trikotri. as i’m familiar with chinese / hanzi characters, i was able to find it easily on ebay.

(if you mispell their name, by the way, you end up with pictures of tractors! I remember their name as “trick or try” now).

some inspiration

since then, making pom poms has grown into an obsession… in particular, I wanted to try to figure out how this tiger was made

tiger by trikotri
tiger by trikotri

Wow, right?? That’s one intensely detailed pompom.





Okay, so how do you make these cute fluffy things?
They’re actually pretty straightforward, essentially you make a dense loops of yarn and then tie a tight knot around the center, then cut through the tops and bottoms of the loops, and voila, you have a  yarn pom pom.


Watch this video. She uses cardboard to make the “pom pom tool”. Note in particular that with the custom tool, she can make elliptical instead of spherical pom poms (though you can always shave spherical ones into elliptical ones, of course).

nekoatsume_pom nekoatsume_pom2 nekoatsume_pom3


That should give you an idea of how to “customize” a pom pom.

If you just want to make gifts for friends, that’s it! Cut out some eyes, ears, and a mouth from felt, and in ten minutes you’ve created a really adorable gift that doesn’t take up much space. There’s ZERO need to make more detailed designs, the cute ones are simple and will be adored.

a step up — flowers

but then… how to make the more detailed designs?

After some searching, I finally came across a tutorial online (since then I’ve found several books that also cover the topic!) by

Specifically, he shows in his Flower Pom Poms article how to go about designing a pom pom (again using a home-made cardboard tool).

Essentially, the way we make pom poms is symmetrical (I’m still trying to figure out how asymmetrical designs work, though maybe they all are just by using tweezers to fiddle the yarn around…). The things to consider are where you are wrapping, and how “deep” (how many loops) you are wrapping, and what “layer” you are wrapping (on top or under what other layers). Succinctly, here is the main ideas from the blog.

Take the flower design you want, and find the axis of symmetry.



Next, we see that we start out by wrapping the innermost layer, #1 (yellow). Next, we wrap the red layer, #2, around it. Then we wrap the green layer, #3, around that — but note that we don’t want the green part to not completely overlap the red part, and wrap accordingly. Finally, we finish off with a white layer. This is just half of the pom pom — the back half is made by doing a full white layer of similar thickness.


credit to
credit to


here we can see the flower design after it is cut


Next step — alphabet

Okay, so now maybe we want to go even more detailed with the pom poms. Flowers are great, but what about the alphabet?

Mr. Printables shows us the way again.



more magic! credit to my printables

I spent many hours on my “ABC”s of pom pom making!

The “A” is quite instructive on how to think about this.

a bunch of my failures…

I struggled a lot!! with keeping the yarn threads from slipping off of each other, and have the A come out with the bottom legs stuck together (more like an eye-shape than an “A”), and a lot of other problems.

mr printables version


I think part of it is the yarn I used, so I have to look into that, as I don’t know anything about the different types of yarn. On the leftmost pom pom you can clearly see that the “denim yarn” I got was not suitable. And I’d try to put too many winds on, so I couldn’t close the pom pom maker (I got some clover pom pom makers). Or the letters weren’t consistently fat. Or the letters were falling off the edges of the pom pom. I’m really not sure…  I should try it with bigger pom poms, maybe that’d make life easier.

I think in the end, the key is to not be too perfectionist… and to be okay with using tweezers and scissors to “post-process” your pom poms!


It turns out that the dirty little trick is to make the “asymmetrical” letters like “F” and “P” and “S” still symmetric by having a reversed (mirror) version of the letter on the other side. The symmetry is created by carving the letter into two parts and mirroring those parts individually.


Full listing of Mr. Printables pom pom posts:




mrprintables-fruit-pom-poms-tutorial (1) mrprintables-kiwi-pompom-step-by-step





Oh nice, here’s a tutorial showing how they made it! Notably it looks like their pom poms are a lot bigger than what I was going for! So maybe that accounts for how easy they had it. Also, they do say 20-30 minutes per pom pom… I feel like I can rest easier now.

the secret to trikotri ‘s pom poms!

Some months (a year?) later, trikotri put up a youtube video of how she makes her pom poms.

My puzzlement was over not just the intricacy of her tiger pom poms, but also the asymmetry in some of her pom poms


It turns out to be a combination of pom poms and felting!

Perhaps the tiger’s asymmetrical features are made by poking wool in to make the stripes? I will have to try to make the tiger!

pom pom parties

okay, that was an info dump about how *other* people make pom poms. What about the beginner stuff — what you can make in 2 or 3 hours?

You can definitely make a lot and get really far! I organized a pom pom party where I got a bunch of friends together and we all made pom poms, everyone except me for the first time.

Doing so is easy, just bring colors of yarn, sharp scissors (!! This is important, probably at least 1 pair of scissors per two people, and they need to be sharp). If you want to be fancy, bring hot glue, googly eyes, and felt (for ears, mouths, etc.) (note — I got a variety pack of felt on amazon, that didn’t include white and black! so be careful what you get).


orangenarwhals_pompom_party orangenarwhals_bee_pompom_party

My friend Renee Bell did too!  Here are the creations of some of her friends.







orangenarwhals_tentcruel_final orangenarwhals_bear orangenarwhals_i<3u orangenarwhals_macaroon orangenarwhals_heart

About the teddy bear — Nacho Bell pom pom . From the website: “In Fall 2015, I tried to teach 18.01 (single-variable calculus). However, since I have no fingers, I could not pick up the chalk, and consequently got very poor reviews.”

neat note

you can make pom poms on the go using nothing but yarn, your fingers, and scissors




Well then.

where to go from here?

a few other interesting things I found online…

Making a blanket, with a design!, out of pom poms



Making a panda pom pom



How to make multiple pom poms at once (the same idea as the duck blanket, but shows you the idea with just one line)







Appendix: more photos

Screenshots of youtube videos