Callaloo Radio System: Part 3 – Publishing Status over MQTT and HTTP

Publishing over MQTT and HTTP

Part 2 finished with the radio receiver circuit’s data made available via a UART-to-USB bridge and a libftdi-based client on the host computer. The data may be available on the network now, but how do people access it?

The USB client program is also running an MQTT client (via libmosquitto). This publishes status updates as the radio messages arrive. A radio payload of 0x11 A0 01 means “status message (0x11), upstairs door (0xA0), currently open (0x01)”, for example, and a corresponding open message is published on the callaloo/upstairs topic. This message has a Quality-of-Service level of 0 (“fire and forget”), because there isn’t much point in ensuring it is reliably delivered – it will just be updated in a few seconds. It does have the retain flag set, however, so the most recent status will be kept and immediately delivered to new subscribers. (My previous post on MQTT explains these.)

With a bit of rework, this same messaging infrastructure could be used for other sensors and transmitters throughout the office. Instead of having the USB client know that 0xA0 and 0xD1 refer to the upstairs and downstairs door switches, it could just publish the message as-is to callaloo/endpoint/A0 and let a small client with a name lookup table publish a translated message to callaloo/upstairs. The overhead would be small, but the real benefit would be in separating message delivery from the physical transport layer, in the same way that DNS allows more flexible networks than working with IPs directly.

People throughout the office can subscribe to these endpoints for push notifications. This could be as simple as:

$ mosquitto_sub -h radiator -t "callaloo/#" -v

To display a feed of incoming radio status messages:


Programs such as mqttwarn can also deliver these MQTT messages to twitter, growl, and many other notification services.

For people who are more interested in the current state than push notifications, I also wrote a tiny web server in C. It subscribes to the callaloo MQTT topics, listens for incoming requests, and responds with the current door info in text/plain or text/json. People can read them with curl radiator:8080. (“radiator” is the host computer for the USB client.)

I wasn’t originally going to write a whole web server, of course — I had planned on making a quick server with bottle — but WSGI’s synchronous design meant it would be a bit trickier to juggle HTTP/1.1 and MQTT traffic than originally anticipated. Since this was a learning project anyway, I decided to dig deeper and write a web server from scratch in C. (I’d written an asynchronous, select-based server in Lua before, but not C.) Obviously, this isn’t production-ready; it’s not 100% RFC 2616-compliant yet, and currently lacks infrastructure like logging. Still, it’s going to be run in a controlled environment, and only accepts specific input. I may generalize the code to something more complete and do thorough security auditing, but it’s only about 400 lines of C, and does everything my project needs.

Mike English also made a webserver with websocket-based updates in Go, and Ryan Abel made an OSX taskbar notifier. We decided it’d be fun to build a bunch of other servers and explore the design space of lightweight web services; I might do one with Erlang/OTP or Ocsigen (OCaml) next.

It may have started with something a bit silly (“is the bathroom downstairs available?”), but exploring infrastructure to answer that question has taught us quite a bit along the way!