websocketd – a UNIX Utility for Turning a Script/Utility into a Chart/Animation

websocketd may be my new favorite UNIX utility. It’s extremely useful for taking the ugly output from a script or utility and turning it into a chart or animation, or even adding a nice interactive UI. And it does it all without changing any code or adding any dependencies to your script or utility.

I was recently working on a fun side project with Shawn Anderson and wanted to visualize/compare a couple grid search algorithms. I really wanted to draw/plot the grid cells that were touched by the algorithm, but I didn’t want to add a dependency to SDL or something just to do that. So we just used print. We were either printing the sequence of grid coordinates (which is tricky to visualize in your head) or printing out “grids” as rows of text, which was better, but quite ugly.

Wouldn’t it be nice if there was something as dirt-simple as print, that could still run anywhere, but would allow for more intuitive visualizations when called for? It turns out you can do this with websocketd!

Algorithm Visualization

Here is the output from the two scripts. They are both super simple algorithms designed to find the closest (in very loose terms) point to another point by searching outwards.

The first few lines of the raw output looks like this:

“Box” search

0, 0
-1, -1
-1, 0
-1, 1
1, -1
1, 0
1, 1
0, -1
0, 1
-2, -2

“Spiral” search

0, 0
0, 1
1, 1
1, 0
1, -1
0, -1
-1, -1
-1, 0
-1, 1
-1, 2

Looking at the raw output, you can tell that they do something different, but it’s hard to tell what’s different.

Then to visualize the output, I can run the script with websocketd + a tiny .html file, with a command like this:


websocketd --port 8080 --staticdir=./ ruby search_box.rb

Then I point my browser at http://localhost:8080/ and websocketd serves up my tiny .html file which creates a websocket, when the websocket connects, websocketd launches my script, and pipes the output into the websocket. The the code in the .html converts then converts the output into something that looks like this:

“Box” search

“Spiral” search

spiral search

Now it’s much easier to see the differences between the two algorithms.

Websocketd for Charts

In one of my earlier posts I wrote about using a Raspberry Pi as a simple current meter. When I made that tool, I had a little script that would print out current measurements to the console as fast as it could. The output looked something like this:


  14.00uA:
  14.00uA:
  14.00uA:
 314.00uA: ---------------
  16.00uA:
 114.00uA: -----
 314.00uA: ---------------
  14.00uA:
  14.00uA:
 314.00uA: ---------------

Which worked really well, but was a little ugly. This would have a been an excellent use for websocketd so I made an example that mimicks the same behavior (it fakes out the ADC reading so you don’t need any special hardware to run it) and then use websocketd with another tiny html + smoothie charts to make something like this:

One of my faviorite things about websocketd, is that the scripts that I use with it don’t lose any of their usefulness as console utilities, but I can get a much nicer looking output with only a tiny bit of extra effort.

Working around stdout Buffering

The biggest gotcha with websocketd is actually one shared by all command line utilities: output buffering. Basically print actually behaves differently depending on whether it’s printing to a terminal (aka a tty) or a unix pipe. When it’s printing to a terminal, print is generally unbuffered, but when printing to pipe the output is usually buffered by default. This results in “strange” behavior where your script will seem to work fine when run in a terminal manually, but appears to hang when the output is piped to websocketd (or other utility), with the output only appearing in large bursts. This behavior is actually controlled by the libc of whatever platform you are using.

There are a few ways to fix this problem.
The easiest way: Manually flush. This is what I do in my examples and it’s pretty simple, just call flush on stdout after you’ve finished writing to it. But what if you don’t want, or can’t, change the code?

There are a few other utilities that solve this problem by creating a virtual tty that your script can print to. The ones I’ve found are:

For example, on my OSX machine, I can do:
websocketd --port 8080 --staticdir=./ script -a /dev/null python fake_read_adc.py
and it will work nicely, even without the sys.stdout.flush().

The best one to use will depend on your platform. For information on the other options, see:

Examples

I packages these examples, along with instructions oh how to run them, into a repo here.