Getting Started with MQTT

mqttorgAs more and more things around us become networked, the communication protocols tying them together need careful rethinking. This network of devices, sometimes called the “Internet of Things” or “Machine-to-Machine” network (though it could also just be called “the Internet”), includes many embedded devices with very limited resources.

Protocols designed for typical ethernet networks, such as HTTP, are based around assumptions that no longer fit: they expect more bandwidth, processing power, and network reliability than may be available, and that networked devices will be on most of the time.

There is a more appropriate alternative, however: MQTT. It’s much lower in overhead, with only a 2-byte header for many messages. Its design suits devices that are suspended most of the time, with only occasional network activity. It also has support for reliable delivery built into the protocol, so simple sensors can just flag an outgoing message as requiring confirmed delivery and let the message broker take care of delivery reattempts. Using a standard messaging protocol for all communication also greatly reduces the surface area for possible security vulnerabilities.

MQTT’s Pub/Sub Model

MQTT is based on a pub/sub (publisher/subscriber) model, which helps decouple who is communicating from what messages are available. Rather than being resource based, like HTTP, messages are organized around a tree of topics, and clients can subscribe to messages delivered to particular topics or groups of topics.

Specific clients can come and go, as long as the topics used to communicate are loosely agreed upon. Messages posted to new topics will automatically create branches in the tree as necessary, so the set of topics used can evolve over time.

MQTT Extensions

MQTT adds a few extensions to the pub/sub model, which make it significantly more expressive.

First, messages are published with a Quality of Service (QoS) level, which specifies delivery requirements. An embedded system sets a couple bits in the message to indicate delivery style, and the broker takes care of the rest.

A QoS-0 message is fire-and-forget. For example, a notification from a doorbell may only matter when immediately delivered. With QoS-1, the broker stores messages on disk and retries until clients have acknowledged their delivery. (Possibly with duplicates.) It’s usually worth ensuring error messages are delivered, even with a delay. QoS-2 messages have a second acknowledgement round-trip, to ensure that non-idempotent messages can be delivered exactly once.

Second, messages have a ‘retain’ flag, indicating that they should be stored for delivery to new subscribers. For topics that are infrequently updated, it’s often useful to give new subscribers the last-known message right away (with an indication that it isn’t fresh). Newer retained messages replace previously ones.

Third, when clients connect, they can specify an optional “will” message, to be delivered if they are unexpectedly disconnected from the network. (In the absence of other activity, a 2-byte ping message is sent to clients at a configurable interval.) This “last will and testament” can be used to notify other parts of the system that a node has gone down, somewhat like EXIT messages in Erlang.

Finally, topic subscriptions can have wildcards: ‘+’ matches anything at a given tree level so the topic “sensor/+/temp” would match “sensor/dev1/temp”, “sensor/dev2/temp”, etc. ‘#’ matches a whole sub-tree, so “sensor/#” would match all topics under “sensor/”. These enable nodes to subscribe to groups of topics that don’t exist yet, allowing greater flexibility in the network’s messaging structure.

Implementations

MQTT is an open standard with a short and readable protocol specification. (There is also the closely related MQTT-SN for wireless sensor networks using datagrams such as Zigbee, rather than TCP/IP.) This has led to multiple, interoperable implementations.

My examples will use the C-based mosquitto implementation, which includes a message broker, a C client library, and command-line tools. The C library has wrappers for languages such as Python and Lua. There are also other brokers and client libraries for Java, Erlang, and many others.

Starting the Broker

$ mosquitto

This will start an MQTT server on localhost:1883 with defaults. An optional configuration file can set options such as persistence and client authentication. Brokers can also bridge to other brokers, relaying (and optionally re-mapping) topics.

Subscribing to a Topic, CLI Client

$ mosquitto_sub -t "topic/path"

The client can subscribe to multiple topics, downgrade topic subscriptions to a lower QoS, provide an ID for persistent sessions, define a will message, and many other options. (mosquitto_sub -h will list them.)

Publishing to a Topic, CLI Client

$ mosquitto_pub -t "topic/path" -m "message payload" -q 1 -r

This will publish a message to a given topic at QoS 1, retained. This, too, has many options listed by mosquitto_pub -h.

Example C Client Using libmosquitto

I wrote a small example client using libmosquitto. The linked code creates a client that connects to a broker at localhost:1883, subscribes to the topics tick, control/#{PID}, and control/all, and publishes its process ID and uptime (in seconds) on tock/#{PID} every time it gets a tick message. If the message halt is sent to either control endpoint, it will disconnect, free its resources, and exit.

The shell script:

#!/bin/sh
while true; do
    echo tick
    sleep 1
done | mosquitto_pub -l -t "tick"

publishes a “tick” message once a second to the topic tick, driving the client program and the command:

mosquitto_sub -t "tock/#"

This will print the output from all running tick-tock clients.

While only a small example, this already covers most of the core functionality needed to connect systems using MQTT.
 

Conversation
  • Dominik Obermaier says:

    Wow, great blog post to get started with MQTT!

    Nowadays most modern web browsers support Websockets, which means we can also publish MQTT messages directly to web applications. That has some nice featues like we can send a LWT messages when a client closes the browser. A simple example how to do this is available here http://www.hivemq.com/build-javascript-mqtt-web-application/

    The only prerequisite would be a MQTT broker which supports MQTT and websockets out of the box like HiveMQ. Also mosquitto can be used with websockets but there is some extra software needed.

    I’m looking forward to the next MQTT posts. Keep the MQTT posts coming! :)

  • James says:

    Many thanks for these examples. One small observation – the following line:
    mosquitto_sub -t “tock/#”
    should be
    mosquitto_sub -t “tick/#”

    • Scott Vokes Scott Vokes says:

      I just double checked it — the current setup is correct. The shell script is publishing to “tick”, the client demo is subscribing to “tick” and publishing on “tock/${PID}”, and then mosquitto_sub on the command line is subscribing to all “tock/${PID}” output using “tock/#” (the ‘#’ is a wildcard).

      • florian says:

        you can’t subscribe to tock and send to tick.. i tested and it doesn’t work. This would work mosquitto_pub -t “tock/tick” -m “tick”

        • Scott Vokes Scott Vokes says:

          I think you’re misunderstanding the setup a bit:

          – The shell script publishes “tick” on the topic “tick” once a second.
          – The C client subscribes to “tick” and publishes its uptime on “tock/{process ID}” when it sees a message published on the topic “tick” by the shell script.
          – The command-line `mosquitto_sub` command being run prints every message posted on “tock/{anything}”, which should print out a stream of process IDs and uptimes from the running client(s).

          Does that help?

  • Matt says:

    Hi Scott,

    Getting compilation error due to missing “mosquitto.h”.

    I have just installed mosquitto using
    sudo apt-get update
    sudo apt-get install mosquitto python-mosquitto
    sudo apt-get install mosquitto-clients

    and tested mosquitto_sub & mosquitto_pub which are working fine

    Please help to resolve this issue.

    Thanks in advance!

    • Scott Vokes Scott Vokes says:

      Did you install the package that has the development headers? Since you mentioned apt-get, you’re probably using Debian or Ubuntu. According to their conventions, there’s probably a related package called something like mosquitto-devel or libmosquitto-devel with the files you need. If that doesn’t work, try searching for info about mosquitto + your distro.

    • Dimitris says:

      I face the same problem. Where you able to find a solution?

  • Rafael says:

    Great job and thanks to share it.

  • Comments are closed.