Headless Server Worth Having

I use Unix‚ĄĘ-like systems everywhere (I’m not going to get into a religious argument about this …I’ve been using various sorts of Unix for dang-near 40 years now and I just never quite got the hang of the GUI paradigm, so call me an old fart if you like; the moniker fits quite well ūüôā ). ¬†Recently I’ve replaced a few of the bigger, more power-hungry boxen around the house with ARM boards, which are inexpensive to buy and very cheap to run. ¬†Some of the Linux offerings (Armbian, Debian, Ubuntu-core and the various siblings, family and friends) are doing an amazing job of keeping up with the ever-changing, disparate array of hardware coming out of the Middle Kingdom (and FreeBSD isn’t far behind), so using these tiny boards as headless servers is quite viable nowadays.

Recently I bought a Nano-Pi M1 (plus 3D printed case) from the FriendlyARM on-line shop, with the intention of pressing it into service as a 24×7 web-cam support machine and general anything-else-I-can-get-to-run-on-it server. ¬†As an ESP8266 user, MQTT was one of the leading candidates in the “anything-else” category. ¬†Well, the good news is that it’s really not that difficult to get a distribution to run on one of these $11 boards. ¬†FriendlyARM do a lot better than some of the other manufacturers in making relatively up-to-date releases available from their web site and again, unlike some of the others, the choice of working distributions is actually greater than just one.

While I still haven’t got the web-cam up and running (anyone want to recommend a decent, cheap camera with reasonable resolution for landscape shots?), I have had a lot of fun playing about with this small board and trying out the various flavours of Linux available for it.

Having mentioned the Nano-Pi M1, I have to say that it is not the¬†subject of this post, though. ¬†While browsing my normal favourite collection of sites over breakfast this morning I came upon this little beauty (courtesy of CNX-Software …one of the better sites for cutting-edge info on embedded systems).Nano-Pi Neo board imageThis is the new, NanoPi NEO from FriendlyARM; the latest stable-mate to the NanoPi M1 (and for that reason, I have a quite reasonable expectation that this board will have good support from day-1 …and as my mouse hovers over the “Add to Cart” button, I hope that FriendlyARM won’t prove me wrong).

This tiny board (40mm x 40mm) has 100Mb ethernet, a USB-2 port (with two more ports available on header pins), a micro-USB socket for power and data, a Micro-SD card socket, a 36-pin header for GPIO (which includes I2C, UART, SPI and general I/O pins) all driven by a quad-core Cortex A7 running at 1.2GHz and 256MB of DDR3 RAM (512MB of RAM for an extra $2).

So how much is it?  $7.99 for the 256MB version.

Before I get screamed at, I need to qualify that price and tell you that the last time I bought anything ¬†(the M1) from FriendlyARM, the postage was $10 (via DHL to most of the world, ¬†so reasonably speedy), so you need to add that into your budget. ¬†As far as I can see, there’s no case available yet (Update:-¬†Just heard back from FriendlyARM that the case is in the works and should be available towards the end of this month …July 2016).

Update:-¬† It looks as though FriendlyARM have changed their shipping options. ¬†When I went to “checkout” this time, the cheapest option was plain old “China Post”, with Fedex just a couple of dollars extra and the DHL/TNT option a few dollars more.

Update:-¬†I mentioned in the original version of this post that it might be worth adding FriendlyARM’s $1 heat-sink to your order. ¬†Better hold off on that for the time being. ¬†The processor on the NEO is on the bottom of the board and we still have no ¬†idea of what the case is going to look like. ¬†Oops!

It’s going to be hard to beat this as an off-the-shelf, ethernet-connected, low power (as in watts used, not computing power) MQTT/DHCP/DNS server for your ESP8266 network.

Last word… for GUI-centric people, please do note that there is no video output connector on this board; it’s serial-console only.

Update (Aug 8th 2016):- ¬†As of today, the printed case for the NEO is now ¬†available for sale on the FriendlyArm site. ¬†There’s also a fairly chunky heat-sink.¬† Both are priced at $2.99. ¬†The pricing is a little bit of a downer, as together they bump up the total cost of the unit by a fairly hefty margin. ¬†The other downer is the colour …do you really want a neon-pink case sitting anywhere around your house?

 

NanoPi NEO layout and specs

 

 

Using “TASMOTA” [Part – 2, MQTT]

Update @ 20th February 2018¬† –¬† There’s now a newer post available on using TASMOTA, which I’d recommend as a follow-up to this one.

[Apologies for those of you have been checking back in regularly for this second article on using Theo Arends’ Sonoff-MQTT-OTA-Arduino package. ¬†Not only has Real Life‚ĄĘ been getting in the way, but I’ve purposely been delaying hitting the big, red “publish” button until I could sort out a nagging little problem I’ve been having (more on that later). ¬†Anyway, here we go with part two…]

In the previous article, we looked at how to access some of the features of TASMOTA (Theo Arends’ Sonoff-MQTT-OTA-Arduino package) using the button/LED and serial-console interfaces. ¬†In this article we’ll take a look at controlling the unit using MQTT.

[…and if you don’t really know what MQTT is, or what it does, Elliot Williams has recently published a clear, easy to read and easy to understand article on Hackaday which is worth looking at…]

The main selling point (other than the extraordinarily low price) of the Sonoff is the fact that it is controllable across the ‘net. ¬†Unfortunately, ITEAD Studios have tied control of the device to their own cloud service which, while it may be great for most people, doesn’t work for those of us who are still using “dumb” phones and who don’t have the option of using a service such as Oogleg’s “voice” to get around that limitation. ¬†Theo’s firmware implements MQTT on the Sonoff which, while it doesn’t help my “dumb” phone, does mean that I can use existing devices on my own network to control it. ¬†I already have a couple of MQTT servers on the local network providing time services to various battery-powered, ESP8266 enabled sensors and collecting data (temperature, humidity, barometric pressure, battery voltage) from them. ¬†It would be a trivial matter to have the internal MQTT servers push that data to one of the free data aggregation services out on the ‘net and only a little more work to make the MQTT interface available through the firewall for external access when we’re away from home (Okay, a lot more work to do it properly ūüôā ). ¬†In my case, the need for MQTT is purely pragmatic; it’s a neat device, but I need MQTT to get it to work without a smart-phone. ¬†For others, there may be a reluctance to entrust data access and control to a cloud service which might disappear at some time in the future. ¬†Whichever it happens to be, Theo’s package “hits the spot” and broadens the possible usage window for the Sonoff (which, long term, is good for ITEAD’s sales, too).

In the previous article we already saw how to set up the “topic” keyword to give your Sonoff devices individual addresses, as well as using “grouptopic” to set a group address to which all devices will listen. ¬†Before you can use MQTT though, you need to configure the MQTT broker (server) details for your network, too. ¬†The recommended way of doing this (unless you have a gazillion MQTT servers on multiple networks) is to simply compile-in the hostname/IP and port number when you initially build the package. ¬†If you’re using Theo’s original package, edit the user configurable details at the top of the sonoff.ino file and change these lines:-

// MQTT
#define MQTT_HOST "sidnas2"
#define MQTT_PORT 1883

…to point to your own MQTT server address and port (if you’re using the version which I put up on GitHub with the function defines added, these settings broken out into user_config.h instead).

You can still change this setting (along with most others) from the serial console, or from MQTT, later on.  Using the serial console (as per the previous examples), you can see that the Sonoff unit automatically reboots after the change:-

host
HOST = blind.nthld.org

host hazeltonrig.nthld.org
HOST = hazeltonrig.nthld.org

ets Jan 8 2013,rst cause:1, boot mode:(3,7)
chksum 0x42

Project sonoff (Topic 2F-toilet-gas-sensor, Fallback DVES_0CB0CB, GroupTopic calling-all-units) Version 1.0.6 (Boot 4, SDK 1.5.1(e67da894))

First we used the “host” command without arguments to interrogate the unit for the current setting. Secondly, we used the “host hazeltonrig…” command to change the compiled-in default. TASMOTA then automatically saves the new value to non-volatile memory and restarts the ESP8266 using the newly updated parameters.

Mosquitto notes

Before we start using MQTT let’s just take a brief look at Mosquitto and the utilities which come bundled with it, mosquitto_sub and mosquitto_pub. ¬†If you’re not using Mosquitto as your MQTT server, feel free to skip this section.

As you’d expect from the names, mosquitto_sub is a command-line utility which allows you to subscribe to a topic, while mosquitto_pub allows you to publish. ¬†They¬†are invaluable tools when setting-up and debugging MQTT installations, allowing you to send arbitrary data and commands and to monitor the output.

You can also install the Mosquitto package on a desktop or laptop machine to make these applications available when you just want to connect to a remote MQTT server on some other machine (both utilities accept a “-h” option with an argument specifying the hostname to connect to and a “-p” option with a numeric argument to specify the port number). ¬†However, typing in the full command names, plus options and arguments can get tedious very quickly, so I’d suggest adding aliases for both commands. ¬†These can be added to your .bashrc (assuming you’re using the common, Bourne Again SHell) to make your life a little easier. Here’s an example (you need to customize it for your specific installation):-


##
## Aliases for Mosquitto utilities.
##
## Simple alias for server on this machine.
alias mp=/usr/bin/mosquitto_pub;
alias ms=/usr/bin/mosquitto_sub;
##
## Aliases for server on remote machine.
## (You can comment-out this whole section if you
## only use the server running on this machine).
MQTT_SRV="hazeltonrig.nthld.org";
MQTT_PRT=1883;
alias rmp="mp -h ${MQTT_SRV} -p ${MQTT_PRT}";
alias rms="ms -h ${MQTT_SRV} -p ${MQTT_PRT}";
unset MQTT_SRV MQTT_PRT; ## Prevent pollution.
##
## End of Mosquitto aliases.
##

In the .bashrc excerpt above, we are first setting “mp” to be an alias for the full pathname of the mosquitto_pub command and then doing the same for “ms” and mosquitto_sub command.

You only need the second part (from the line “## Aliases for server on remote machine.”) if you are running the MQTT server on a different machine to the one where you’ll be running the mosquitto_pub and ¬†mosquitto_sub commands. ¬†If you do want to use it, you’ll need to change “hazeltonrig.nthld.org” (and possibly the 1883 port number) to point to the MQTT server on your network. ¬†The aliases here build on the “mp” and “ms” aliases which we’ve already defined to provide “rmp” (“r”emote “mp”) and “rms” to access our remote server with minimal typing.

The “unset” command simply deletes the MQTT_SRV and MQTT_PRT variables to prevent any unwanted clashes if those names are used by you later.

At the end of that, you can now simplify and shorten your commands.

From:- /usr/bin/mosquitto_sub -h hazeltonrig.nthld.org -p 1883 -t stat/#

    To:-    rms -t stat/#

 

Using the Mosquitto utilities

Now that we have our server defined and some aliases set up, let’s go ahead and dive into using MQTT with the Sonoff and TASMOTA. ¬†I’m going to assume at this point that, if you already have an MQTT broker (server) set up, you basically know how the publish/subscribe model works. ¬†Let’s also assume that our MQTT server (the machine running Mosquitto) is called “hazeltonrig” (as we saw above) and that we’re actually using a laptop to type in these examples and display the output.

Because our laptop is not the MQTT server, we need to use the remote aliases which we set up above to channel all of our publish and subscribe requests through “hazeltonrig”, because for the most part, we are actually interested in interacting with a third machine, the Sonoff module. ¬†So, just the same as with the serial console, let’s start off by getting the status from the device. ¬†The easiest way to do this is to use an MQTT wildcard and listen in on all status broadcasts from all devices on the network:-

rms -t "stat/#"

Going back to our aliases, we can see that this actually expands to the command:-

/usr/bin/mosquitto_sub -h hazeltonrig.northld.org -p 1883 -t "stat/#"

The word “stat” is the top-level topic used by TASMOTA clients to publish their status data. ¬†The “#” is a wildcard, meaning match-anything-from-here onwards.

What happens when you type in this command? ¬†Well, unless your Sonoff happens to be transmitting some status information at the time you hit <CR>, absolutely nothing. ¬†You won’t get a prompt and you won’t see any text on the screen after the command itself. ¬†Not very exciting, but you have just started a subscriber process which will sit there quite happily and spit out any data that comes along with the “stat/” topic header.

If you power-cycle your Sonoff at this point, you  will see a couple of power-up status messages displayed on your screen; something like:-

Sonoff switch
1.0.6
DVES_0CB0CB

You’ll also notice that, if you still have the serial monitor connected via the USB<->TTL converter, you will start seeing status messages appearing on your screen when you type in virtually any command. ¬†So, if you type “power 1” into the serial monitor, you’ll get output on the serial monitor itself and you’ll also see the word “on” appear in the window where your subscribe command is running (actually, even if you don’t type in commands, the subscribe window will still show messages at power-up, restart or power-down events; for instance, pulling the plug on the Sonoff will produce an “offline” message approximately 10 seconds after the unit is switched off).

You could type a <CTRL>z to put the command into background and carry on typing MQTT commands into the same window where you’ve just started your subscriber process, but that would begin to get a little confusing once that process started to return data, so at this point I’d recommend opening a new window to be able to enter further commands.

Now we need to command the Sonoff to send out some status, so that we can read it with our patiently listening subscriber process. ¬†For this we use our remote publish alias, “rmp”:-

rmp  -t  "cmnd/2f-toilet-gas-sensor/status"  -m  ""

This expands to:-

/usr/bin/mosquitto_pub  -h  hazeltonrig.nthld.org  -p  1883  -t  "cmnd/2f-toilet-gas-sensor/status"  -m  ""

…which means, publish the command “status” (request status) to the device currently known as “2f-toilet-gas-sensor”. ¬†The -m “” part of the command is simply a null argument to keep mosquitto_pub happy, otherwise it will print a usage summary to try and persuade you that you need to use a message (-m) as an argument to any topic (-t).

The output you’ll see in your subscriber window from the status command with a null (-m “”) option will be something like:-

1.0.6, 2f-toilet-gas-sensor, POWER, 0, 9

This is telling you that the Sonoff is running firmware version 1.0.6, has a topic (addressable name) of “2f-toilet-gas-sensor”, that the last sub-command was “POWER” and the status of the relay is off (“0”). The last “9” indicates the timezone which is currently configured (in this case GMT+9 for Tokyo).

Up Next…

This article has been languishing in the deeper recesses of the WordPress draft cloud for far too long, and has a lot of information for the reader to digest, so I’m going to save some of the other MQTT commands for yet another article and push the big, red “publish” button to get this one out into the wild.

In the next part, we’ll use some of the more useful commands which Theo has provided and look at the difference between normal and group commands.

 

No-Phone Sonoff stuff…

Sonoff with 3v3 USB programming adapter attachedI’m still quite enamoured of the Sonoff, from ITEAD Studios, despite the fact that I can’t get their eWelink to work with a tablet (the app refuses to register you unless you have a phone number — which my tablet doesn’t have, of course — and the excellent suggestion from Jon Chandler in the support forum, “use Oogleg Voice” may well work, but only for ‘merkins living in merika). ¬†Sniffle. ¬†I really shouldn’t be surprised, though; previous experience has shown that anything connected to female sheep usually leaves me with a cold, wet arm or a body to bury (occasionally both).

Anyway, playing around with the unmodified device showed it as coming up on an IP address of 10.10.7.1 and happily assigning my laptop 10.10.7.2, but that’s as far as I got. All attempts to connect to it from the laptop failed miserably and I was also unable to get the Android “Smartconfig” app to do anything with it (if you’ve had success with any other method of getting the Sonoff to work with an Android tablet, please do leave a note in the comments section below).

So, I disconnected the wee beastie, ripped it apart and set about adding some pins to the existing programming header. A couple of warnings are in order here before I go any further, though:-

  • Your warranty is null and void if you start screwing around with your hardware (but you knew that already, right!?!)
  • You need to switch the unit off, unplug it from the mains and remove any cables connected to it before you start work (hopefully you knew that, too …if you didn’t, give yourself a sharp rap across the knuckles with wooden ruler and stop reading this right now!).
  • You should let the unit stand for half an hour or so to let the capacitors on the mains side of the PSU discharge before taking the top off.
  • DO NOT under any circumstances, touch any part of the PCB or component with a finger, soldering iron or USB-to-TTL adapter until all of the above instructions have been complied with.

ITEAD Studios have designed the board of the Sonoff with a socket for a programming header.

ITEAD Studios' Sonoff, with no clothes onAs it ships on the current version that header is unpopulated (this is ITEAD Studios telling you and me not to mess with their board …which is very sound advice).

The available schematic and the board itself don’t quite match up. ¬†The schematic shows this as a five-pin header, but the PCB very obviously only has four holes. ¬†What are we going to be connecting? ¬†Well, we need pins for ground, receive data, transmit data and +3v3. ¬†We need the latter simply because we do not want to attempt connecting another device (PC/Laptop/UFO-anti-gravity-generator, or whatever) to something which is already connected to the mains (do NOT do that! ¬† I mean it!). ¬†At any rate, four pins is all we need and four pins is what we’ve got. ¬†We also need something that I’d been missing for a while …a 3v3 compatible USB<->TTL adapter (I finally gave in and ordered a couple a while back).

Adding the pins is fairly straightforward, but you should note that the ground pin is the one closest to the camera in the shot above, not the one with the squared-off hole. ¬†The¬†squared-off hole is the 3v3 connection. ¬†Close-up shot of the unpopulated programming headerIf in doubt, get your meter out and test. ¬†Before you put the pin header in place it’s also fairly easy to see the “star” connections from the ground connector to the ground plane around it. ¬†The pin on the opposite end of the connector (next to the “J1” label and the big switch) is the 3v3 connector.

The long row of holes across the bottom of the picture is the connector for the 433MHz remote control board which is fitted to the Sonoff “RF” models, but missing on the base model.

With the pin headers in place, it’s not very easy to see either the silk-screen around pin 1 or the “J1” label, so it might be a good idea to label the ground pin using something like an indelible marker Programming socket populated with pinspen so that you’ll know which way round to connect the USB<->TTL converter leads the next time you take the top off.

Now that we have the header pins in place, we need some firmware to burn to the ESP8266 on the board. ¬†I’d like to point you towards¬†Theo Arends’¬†Sonoff-MQTT-OTA-Arduino package on GitHub. ¬†It’s a drop in replacement which has a very versatile interface, allowing control of the relay, as well as status reporting and some configuration, from both MQTT and from the Arduino-IDE serial console window. ¬†In addition, the button on the Sonoff will also provide manual control of the relay, start SmartConfig mode, start OTA update mode and force a reset to defaults, depending upon how many presses or how long you hold it down. ¬†Changes to the relay state triggered by the button also automatically generate an MQTT state-change message, so your application should always be able to track state of external switch events. ¬†Neat! ¬†Theo also maintains a non-Arduino version if any of you masochists out there prefer fiddling with the SDK.

To update your Sonoff, make sure that no mains cables are attached, plug your 3v3 USB<->TTL converter cables onto the newly installed header pins (+3v3 to the pin next to the switch, ground to the pin next to the row of holes for the 433MHz module) and then hold down the switch button while plugging the USB connector into your computer. ¬†Burn Theo’s firmware to the board using whichever method you are familiar with.

Note that the USB converters don’t usually supply enough current to run the ESP8266 when it fires up the radio (and it definitely won’t supply enough current to power the relay coil), so don’t be surprised if you see a couple of start-up messages followed by a crash or watchdog reset; just disconnect your USB cable and put the unit back together again (including all of the covers), before applying mains power. ¬†Once it’s all back in one piece and powered on, you can check basic functionality by pressing the button. ¬†A single press should produce an audible click from the Sonoff as the relay toggles state, followed by two brief flashes of the LED. ¬†If you have “mosquitto_sub” available (or one of the Android MQTT monitor apps), you can fire it up and monitor the topic “stat/#” to see output from your newly updated Sonoff (assuming that you set up your own MQTT server details in the firmware defaults). ¬†The mosquitto_sub command would be:-

mosquitto_sub -h Your.MQTT.Server -t sub/#

Use mosquitto_pub to publish an MQTT message to the cmnd/sonoff topic to have your Sonoff toggle the state of the relay:-

mosquitto_pub -h Your.MQTT.Server -t cmnd/sonoff/power -m "toggle"

Hopefully you’re now in business and your Sonoff is happily clicking away as you play with MQTT.

Update –¬† If you’re having trouble compiling Theo’s package (CFG_Default not defined, etc), I’ve put together a very slightly modified package with the function prototypes broken out into separate header files which should compile cleanly for most people. ¬†It also has a minor pre-processor tweak which will reset the MQTT_MAX_PACKET_SIZE and MQTT_KEEPALIVE values to Theo’s recommended settings of 1024 and 120, without¬†the need to modify the original PubSub library file.

Next …the next article in this series is a tutorial on using Theo’s new firmware.

Reworking the AI Thinker T5 – Part IV.V

The board should now be capable of running a somewhat useful program. ¬†I’ve used the Arduino-ESP8266 core to bang this together, not because I’m an Arduino enthusiast (I’m not …I’ve never owned one, borrowed one or even stolen one), but because I found it the easiest environment to program in, given my cack-handed use of C and preference for the “vi” editor. ¬†It just happens to handle libraries and compile code in a way which is easy to install, use and understand and¬†it fits things into the ESP8266’s overly-complex memory footprint without requiring the user to edit SDK files. ¬†It’s just a pity that the IDE sucks so much. ūüôā

Any-hoo (see, I can speak Canadian, too!), this mini project uses the ESP8266 to interrogate the DHT11 sensor for temperature and humidity and then spits those readings out, together with a VCC voltage reading and a unique module ID, to MQTT.  After the MQTT publish is confirmed, the ESP8266 will drop into deep-sleep mode, waking itself up (assuming you remembered to connect GPIO16 to the RESET pin) after 10 minutes to start the whole process over again.

The code contains a ton of Serial.print’s, surrounded by #ifdef DEBUG #endif’s. ¬†So as long as you have “DEBUG” defined to something in user_config.h, you’ll also see all of the data on the serial output, too.

Also in the user_config.h file are a few other settings which you must configure for your specific environment.  They include the SSID and password for your WiFi access point and static IP addresses for the ESP8266, gateway and DNS servers, as well as your MQTT server hostname, port number and topic IDs.  Everything else in that file can be tweaked too, but the items mentioned above are pretty much essential changes to get the ESP into operational mode on your specific network.

The code is available from GitHub:- PuceBaboon/ESP8266-AIT-T5

Do you need a T5 board to use this code? ¬†Heck, no! ¬†You can use just about any of the ESP modules out there, from the ESP-03, ESP-07, ESP-12 through to the WROOM-02. ¬†The ESP-03 (and some of the other, older modules) may give you a bit of trouble when it comes to connecting GPIO16 to RESET, but it is doable. ¬†The original ESP-01 is just about the only module I can’t recommend for this project. ¬†You’ll need a DHT11 (or DHT22) and an LED to replicate the original T5 project, but all in all it’s probably easier going with a new ESP-12 than trying to rework the actual T5 board.

Libraries – The code makes use of the Adafruit DHT library and the “esp-mqtt” library from Ingo Randolf, which is a version of TuanPM’s library modded for the Arduino-ESP environment. ¬†You’ll need to add those to your local, Arduino “libraries” directory to be able to compile the code.

A big “Thank you!” to all of those folks above for making their work available to the ESP community.

In the next instalment, we’ll do a simple update to convert one of the switches to a “program” switch (hold it down while powering on to put the ESP into programming mode) instead of having to fiddle with that jumper every time.

 

Mosquitto “invalid protocol MQTT in CONNECT” errors (and ESP8266 continuously trying to reconnect to broker service, without success)

[Edit: See “Stop Press” note at bottom of article before embarking on this fix]

This is a problem which is common to Ubuntu based Linux systems running the “mosquitto” broker.

Unfortunately, the default MQTT package available from the Ubuntu Software Centre is an ancient revision, v0.15, which equally unfortunately, shows up as being “3.1/3.1.1 compatible” …when it isn’t.

For us ESP8266 esp_mqtt users, that translates to a constant stream of “reconnecting…” messages on the ESP8266 console and “Invalid protocol MQTT” messages in the broker’s log (probably /var/log/syslog on your system, but check your mosquitto.conf file to verify). ¬†The end result is that the ESP8266 fails to establish an MQTT session with the broker and is therefore unable to publish or subscribe.

What changed and why did it suddenly stop working?  Well, when Tuan (who has done an absolutely fantastic job on producing this package and on responding to issues, including this one) updated esp_mqtt to run with the nodemca LUA package, he updated (as I understand it) esp_mqtt to be fully 3.1.1 compliant.  Unfortunately, because the stock Ubuntu package is so long in the tooth, this bit most Ubuntu users (and users of Ubuntu derived distributions, such as ElementaryOS) in the arse (so blame the Ubuntu package maintainers, not Tuan).

Because there hasn’t been an update to the official Ubuntu package for so long, Roger Light, the author and maintainer of the mosquitto package, has gone to the trouble of providing a local repository for the package which the Ubuntu Update Manager can be configured to poll and download from (it will show up in the Update Manager just the same as any other third-party package, such as the Google “Chrome” browser). ¬†You can configure the Update Manager to use Roger’s repository (and to automatically update your stock, outdated version of mosquitto) in just a few simple steps. ¬†Please note that these steps, described below, add a non-Canonical repository to your Update Manager, which means you are explicitly giving Roger Light and the mosquitto development team permission to change package contents on your system; if you are uncomfortable with this then you should not follow these instructions and should probably remove the existing (now useless) mosquitto package, too (your call).

Okay, now we’ve got the small-print out of the way, lets get going. ¬†Before starting the upgrade, you should stop your existing mosquitto daemon:-

  service mosquitto stop

and back-up your existing mosquitto.conf file:-

  cp /etc/mosquitto/mosquitto.conf /etc/mosquitto/mosquitto.conf_ORIG

and then start the Update Manager (icon, menu, cli, whatever your particular Ubuntu variant uses). ¬†Leave the Update Manager to sort itself out (it takes a few seconds to start up) and go back to your web-browser and go to¬†Roger’s excellent update page, at:-

https://launchpad.net/~mosquitto-dev/+archive/ubuntu/mosquitto-ppa

(it will open in a new tab). ¬†Click on the green “Technical details about this PPA” close to the top of the page. ¬†This will expose a pull-down menu, ¬†“Choose your Ubuntu version”. ¬†Once you’ve selected your version, say 12.04, the text box below will change to display the package manager entry for that specific version of Ubuntu. ¬†Two lines will be displayed. ¬†One starting with just “deb”, which is the main package repository and the other starting with “deb-src” which is (duh!) the package source-code repository. ¬†We need to use these addresses to configure the Update Manager to use Roger’s repository, so select the first, “deb” line and then swap back from the browser to the Update Manager window.

In the Update Manager window, click on the “settings” button to bring up the configuration window and select the “Other Software” tab. ¬†Near the bottom of the settings window you should now see an “add” button and clicking on that will bring up another window with the natty heading “Enter the complete APT line of the repository that you want to add as source“. ¬†Paste the “deb” line into the text box, making sure there are no trailing characters after the last word, “main” and then hit the “Add Source” button to add it. ¬†Optionally, you can now go back and repeat the same procedure with the “deb-src” line, if you want to download the source-code, too (again, your call).

Going back once again to Roger’s page, take a note of the number displayed in the “Signing key” line. ¬†Flip back to the Update Manager settings window and select the “authentication” tab. ¬†Scroll down the list of Trusted Software Suppliers and somewhere close to the bottom you should find an entry for “Launchpad mosquitto“. ¬†Check that it is the same key number as the one you just noted. ¬†The Update Manager will automatically download the public key specified by this number from Roger’s site to enable secure updates.

Okay, we’re done with the heavy lifting. ¬†You can now close the “Software Sources” (settings) window. ¬†If your mosquitto update has been successful, the close will trigger an automatic update of the available software cache (so click the “reload” button, when prompted). ¬†The Update Manager should (again, automatically) go off and check for new software and find the newly available mosquitto package. ¬†Select the package in the normal way and click “Install Updates” to have the Update Manager remove the old version of mosquitto and install the latest available version from Roger’s site.

One thing you need to be aware of is that the current version (as of Feb 2015) of the mosquitto package has a very minimal configuration file, so before starting the new daemon you should check your original mosquitto.conf file for settings specific to your site and import them into that new file. ¬†A simple method for pulling those settings out of your old file (you did back it up, didn’t you!?!) is to use this command pipe:-

grep -v "^#" /etc/mosquitto/mosquitto.conf_ORIG | cat -s

The `-v “^#”‘ sequence tells grep not to match any comment lines and the “cat -s” suppresses multiple blank output lines.

Make sure you put the settings from the old configuration file into the new, minimal file (especially any “user” or authentication settings) before you restart the new daemon. ¬†You can use¬†man mosquitto.conf¬†to get help with the available settings.

Stop Press: Literally a few minutes before I pressed the “publish” button on this missive, Tuan rolled out an update (told you he was good :-)) to esp_mqtt, adding a protocol define option to the include/user_config.h file:-

#define PROTOCOL_NAMEv31      /* MQTT version 3.1 compatible with Mosquitto v0.15 */
//#define PROTOCOL_NAMEv311   /* MQTT version 3.11 compatible with Paho & Mosquitto > v0.15 */

…so now you don’t have to upgrade if you don’t want to. ¬†Yay!