Getting Started with MQTT

As more and more things around us become networked, the communication protocols tying them together need careful rethinking. This network of devices, sometimes called the “Internet of Things” or “Machine-to-Machine” network (though it could also just be called “the Internet”), includes many embedded devices with very limited resources.

Protocols designed for typical ethernet networks, such as HTTP, are based around assumptions that no longer fit: they expect more bandwidth, processing power, and network reliability than may be available, and that networked devices will be on most of the time.

There is a more appropriate alternative, however: MQTT. It’s much lower in overhead, with only a 2-byte header for many messages. Its design suits devices that are suspended most of the time, with only occasional network activity. It also has support for reliable delivery built into the protocol, so simple sensors can just flag an outgoing message as requiring confirmed delivery and let the message broker take care of delivery reattempts. Using a standard messaging protocol for all communication also greatly reduces the surface area for possible security vulnerabilities.

MQTT’s Pub/Sub Model

MQTT is based on a pub/sub (publisher/subscriber) model, which helps decouple who is communicating from what messages are available. Rather than being resource based, like HTTP, messages are organized around a tree of topics, and clients can subscribe to messages delivered to particular topics or groups of topics.

Specific clients can come and go, as long as the topics used to communicate are loosely agreed upon. Messages posted to new topics will automatically create branches in the tree as necessary, so the set of topics used can evolve over time.

MQTT Extensions

MQTT adds a few extensions to the pub/sub model, which make it significantly more expressive.

First, messages are published with a Quality of Service (QoS) level, which specifies delivery requirements. An embedded system sets a couple bits in the message to indicate delivery style, and the broker takes care of the rest.

A QoS-0 message is fire-and-forget. For example, a notification from a doorbell may only matter when immediately delivered. With QoS-1, the broker stores messages on disk and retries until clients have acknowledged their delivery. (Possibly with duplicates.) It’s usually worth ensuring error messages are delivered, even with a delay. QoS-2 messages have a second acknowledgement round-trip, to ensure that non-idempotent messages can be delivered exactly once.

Second, messages have a ‘retain’ flag, indicating that they should be stored for delivery to new subscribers. For topics that are infrequently updated, it’s often useful to give new subscribers the last-known message right away (with an indication that it isn’t fresh). Newer retained messages replace previously ones.

Third, when clients connect, they can specify an optional “will” message, to be delivered if they are unexpectedly disconnected from the network. (In the absence of other activity, a 2-byte ping message is sent to clients at a configurable interval.) This “last will and testament” can be used to notify other parts of the system that a node has gone down, somewhat like EXIT messages in Erlang.

Finally, topic subscriptions can have wildcards: ‘+’ matches anything at a given tree level so the topic “sensor/+/temp” would match “sensor/dev1/temp”, “sensor/dev2/temp”, etc. ‘#’ matches a whole sub-tree, so “sensor/#” would match all topics under “sensor/”. These enable nodes to subscribe to groups of topics that don’t exist yet, allowing greater flexibility in the network’s messaging structure.

Implementations

MQTT is an open standard with a short and readable protocol specification. (There is also the closely related MQTT-SN for wireless sensor networks using datagrams such as Zigbee, rather than TCP/IP.) This has led to multiple, interoperable implementations.

My examples will use the C-based mosquitto implementation, which includes a message broker, a C client library, and command-line tools. The C library has wrappers for languages such as Python and Lua. There are also other brokers and client libraries for Java, Erlang, and many others.

Starting the Broker

$ mosquitto

This will start an MQTT server on localhost:1883 with defaults. An optional configuration file can set options such as persistence and client authentication. Brokers can also bridge to other brokers, relaying (and optionally re-mapping) topics.

Subscribing to a Topic, CLI Client

$ mosquitto_sub -t "topic/path"

The client can subscribe to multiple topics, downgrade topic subscriptions to a lower QoS, provide an ID for persistent sessions, define a will message, and many other options. (mosquitto_sub -h will list them.)

Publishing to a Topic, CLI Client

$ mosquitto_pub -t "topic/path" -m "message payload" -q 1 -r

This will publish a message to a given topic at QoS 1, retained. This, too, has many options listed by mosquitto_pub -h.

Example C Client Using libmosquitto

I wrote a small example client using libmosquitto. The linked code creates a client that connects to a broker at localhost:1883, subscribes to the topics tick, control/#{PID}, and control/all, and publishes its process ID and uptime (in seconds) on tock/#{PID} every time it gets a tick message. If the message halt is sent to either control endpoint, it will disconnect, free its resources, and exit.

The shell script:

#!/bin/sh
while true; do
    echo tick
    sleep 1
done | mosquitto_pub -l -t "tick"

publishes a “tick” message once a second to the topic tick, driving the client program and the command:

mosquitto_sub -t "tock/#"

This will print the output from all running tick-tock clients.

While only a small example, this already covers most of the core functionality needed to connect systems using MQTT.