No-Phone Sonoff stuff…

Sonoff with 3v3 USB programming adapter attachedI’m still quite enamoured of the Sonoff, from ITEAD Studios, despite the fact that I can’t get their eWelink to work with a tablet (the app refuses to register you unless you have a phone number — which my tablet doesn’t have, of course — and the excellent suggestion from Jon Chandler in the support forum, “use Oogleg Voice” may well work, but only for ‘merkins living in merika).  Sniffle.  I really shouldn’t be surprised, though; previous experience has shown that anything connected to female sheep usually leaves me with a cold, wet arm or a body to bury (occasionally both).

Anyway, playing around with the unmodified device showed it as coming up on an IP address of 10.10.7.1 and happily assigning my laptop 10.10.7.2, but that’s as far as I got. All attempts to connect to it from the laptop failed miserably and I was also unable to get the Android “Smartconfig” app to do anything with it (if you’ve had success with any other method of getting the Sonoff to work with an Android tablet, please do leave a note in the comments section below).

So, I disconnected the wee beastie, ripped it apart and set about adding some pins to the existing programming header. A couple of warnings are in order here before I go any further, though:-

  • Your warranty is null and void if you start screwing around with your hardware (but you knew that already, right!?!)
  • You need to switch the unit off, unplug it from the mains and remove any cables connected to it before you start work (hopefully you knew that, too …if you didn’t, give yourself a sharp rap across the knuckles with wooden ruler and stop reading this right now!).
  • You should let the unit stand for half an hour or so to let the capacitors on the mains side of the PSU discharge before taking the top off.
  • DO NOT under any circumstances, touch any part of the PCB or component with a finger, soldering iron or USB-to-TTL adapter until all of the above instructions have been complied with.

ITEAD Studios have designed the board of the Sonoff with a socket for a programming header.

ITEAD Studios' Sonoff, with no clothes onAs it ships on the current version that header is unpopulated (this is ITEAD Studios telling you and me not to mess with their board …which is very sound advice).

The available schematic and the board itself don’t quite match up.  The schematic shows this as a five-pin header, but the PCB very obviously only has four holes.  What are we going to be connecting?  Well, we need pins for ground, receive data, transmit data and +3v3.  We need the latter simply because we do not want to attempt connecting another device (PC/Laptop/UFO-anti-gravity-generator, or whatever) to something which is already connected to the mains (do NOT do that!   I mean it!).  At any rate, four pins is all we need and four pins is what we’ve got.  We also need something that I’d been missing for a while …a 3v3 compatible USB<->TTL adapter (I finally gave in and ordered a couple a while back).

Adding the pins is fairly straightforward, but you should note that the ground pin is the one closest to the camera in the shot above, not the one with the squared-off hole.  The squared-off hole is the 3v3 connection.  Close-up shot of the unpopulated programming headerIf in doubt, get your meter out and test.  Before you put the pin header in place it’s also fairly easy to see the “star” connections from the ground connector to the ground plane around it.  The pin on the opposite end of the connector (next to the “J1” label and the big switch) is the 3v3 connector.

The long row of holes across the bottom of the picture is the connector for the 433MHz remote control board which is fitted to the Sonoff “RF” models, but missing on the base model.

With the pin headers in place, it’s not very easy to see either the silk-screen around pin 1 or the “J1” label, so it might be a good idea to label the ground pin using something like an indelible marker Programming socket populated with pinspen so that you’ll know which way round to connect the USB<->TTL converter leads the next time you take the top off.

Now that we have the header pins in place, we need some firmware to burn to the ESP8266 on the board.  I’d like to point you towards Theo Arends’ Sonoff-MQTT-OTA-Arduino package on GitHub.  It’s a drop in replacement which has a very versatile interface, allowing control of the relay, as well as status reporting and some configuration, from both MQTT and from the Arduino-IDE serial console window.  In addition, the button on the Sonoff will also provide manual control of the relay, start SmartConfig mode, start OTA update mode and force a reset to defaults, depending upon how many presses or how long you hold it down.  Changes to the relay state triggered by the button also automatically generate an MQTT state-change message, so your application should always be able to track state of external switch events.  Neat!  Theo also maintains a non-Arduino version if any of you masochists out there prefer fiddling with the SDK.

To update your Sonoff, make sure that no mains cables are attached, plug your 3v3 USB<->TTL converter cables onto the newly installed header pins (+3v3 to the pin next to the switch, ground to the pin next to the row of holes for the 433MHz module) and then hold down the switch button while plugging the USB connector into your computer.  Burn Theo’s firmware to the board using whichever method you are familiar with.

Note that the USB converters don’t usually supply enough current to run the ESP8266 when it fires up the radio (and it definitely won’t supply enough current to power the relay coil), so don’t be surprised if you see a couple of start-up messages followed by a crash or watchdog reset; just disconnect your USB cable and put the unit back together again (including all of the covers), before applying mains power.  Once it’s all back in one piece and powered on, you can check basic functionality by pressing the button.  A single press should produce an audible click from the Sonoff as the relay toggles state, followed by two brief flashes of the LED.  If you have “mosquitto_sub” available (or one of the Android MQTT monitor apps), you can fire it up and monitor the topic “stat/#” to see output from your newly updated Sonoff (assuming that you set up your own MQTT server details in the firmware defaults).  The mosquitto_sub command would be:-

mosquitto_sub -h Your.MQTT.Server -t sub/#

Use mosquitto_pub to publish an MQTT message to the cmnd/sonoff topic to have your Sonoff toggle the state of the relay:-

mosquitto_pub -h Your.MQTT.Server -t cmnd/sonoff/power -m "toggle"

Hopefully you’re now in business and your Sonoff is happily clicking away as you play with MQTT.

Update –  If you’re having trouble compiling Theo’s package (CFG_Default not defined, etc), I’ve put together a very slightly modified package with the function prototypes broken out into separate header files which should compile cleanly for most people.  It also has a minor pre-processor tweak which will reset the MQTT_MAX_PACKET_SIZE and MQTT_KEEPALIVE values to Theo’s recommended settings of 1024 and 120, without the need to modify the original PubSub library file.

Next …the next article in this series is a tutorial on using Theo’s new firmware.

I before E, except after C

Working away with the hardware and software for the modified AI-Thinker T5 board, I decided that it was time to shoehorn the thing into a project case and throw it outside, to measure our night-time lows.  Following this line of thought along, I decided I’d have a go at implementing OTA (Over-The-Air) updates, as it would start to get difficult to do updates if I had to climb a ladder with a screwdriver in one hand and the laptop in the other (following the thought even further along, I realised that it was going to be pretty tough to do an OTA update on a device that sits in deep-sleep mode for 99% of the time …but we’ll cross that particular bridge if we ever get to it).  Now anyone who’s worked with any of the ESP8266 toolchains over the past year or so can probably guess what I’m going to say next.  When you change anything with the ESP, you always run into, not just another problem, but a whole extended family (including in-laws) of problems.  I haven’t found any exceptions to this rule to date and I didn’t this time, either.

I was already using v1.6.6 of the Arduino-IDE, but the ArduinoOTA program for the ESP requires v1.6.7 so, not without a little trepidation, I dutifully upgraded.  There were no massive problems.  The IDE fired up again straight away and recompiled my existing application code with n’er a murmur.  I started to get suspicious.  Where were the big red flashing warnings?  Why hadn’t my cat spontaneously combusted?  It’s too quiet!

Moving cautiously on with the plan, I opened the demo application, BasicOTA.ino, in the IDE, compiled it (successfully, again) and then, still using the USB serial adapter, squirted it to the ESP8266 on the T5.  A few seconds afterwards I saw the “Ready” message and an IP address for the board appear in the IDE serial console window.  At this point I had to heave a fairly major sigh of relief …I’d finally hit a problem …all was as it should be in the ESP world.  It was a very, very minor problem, though.  The newly minted IP address for the ESP should have shown up in the Arduino-IDE “Tools” menu as an available programmer, but it didn’t.  However, the OTA documentation warns that this might happen and suggests simply restarting the IDE, which works just fine.  After the restart, I was able to load the original DHT_Test.ino firmware to the ESP8266, hands free.  No holding down the program switch while restarting; just choose the newly available network device as the “programmer”, hit the button and away it goes.  It’s fast, too!  The OTA update only took roughly half the time of the USB method (and I’m not even counting “fiddling with switches” time).

Okay, next step.  We need (according to the documentation) to put the code for the OTA functionality into the target application itself, because the OTA code doesn’t sit in some separate, compartmentalized box somewhere in the ESP memory space, it just gets mixed right in there with everything else (so when I programmed my original DHT_Test code in that previous step as a test of the OTA functionality, it overwrote the OTA which was in memory and I lost the ability to do OTA updates again.  Duh!).

The documentation suggests that we should take the BasicOTA.ino code and just pop it into the target application, giving a fairly easy-to-follow layout, too.  So, we need to add a couple of extra includes.  Nothing too exciting there.  And then we need to add the OTA callbacks into the setup() section of the code.  This looks a bit weird to me.  I’m worried that the code in the example may have been corrupted by the markdown editor, because there are far too many square-brackets and parentheses in there.  However, the code looks the same as the BasicOTA.ino code and I know that worked, so I just quickly check with Oogleg and, having found the identical code in another tutorial, do a quick cut-and-paste.  Having dropped it into what looks approximately like the right-ish area, I save the file and, out of habit, run indent on it.  Oops! very bad move.  Indent says, “There’s far too many square-brackets and parentheses in there.  It’s given me a right headache and I’m going to puke all over your shoes.”, which it promptly did (figuratively, anyway).  After cleaning up the mess (not that difficult when you have a dog an automatic back-up file), I went back in and inserted those magic, indent incantations, /* *INDENT-OFF* */ and /* *INDENT-ON* */ above and below the weird OTA code and ran indent again, without a problem this time.

Now then, I need to put an ArduinoOTA-handle() call into the loop() section of the code.  Oh dear!  The only thing in my loop is a needlessly sarcastic comment and a call to ESP.restart(), just in case the program ever somehow, mistakenly drops through into it. Bugrit! Never mind, I’ll just try a quick compile to see whether what I already have blows anything up. Tra-la-la, IDE, reload, compile, BLAM! Futtocks! Now what?  Maybe indent was right after all!

I’m staring at a simple error message which tells me that I have an error in my lambda function (Wow! Kinky!  I don’t think I’ve ever knowingly used a lambda before) and that OTA_RECEIVE isn’t defined in this scope.  Eh?  I check that I have ArduinoOTA.h included.  I quickly check the include file itself and note that there’s an error enum define which looks okay to me.  I check the order of the includes and lose some more time re-compiling with different combinations of includes, all to no avail.  I’m just about to do what I should have done in the first place (copy the whole BasicOTA.ino file into my code and use the sections verbatim) when I got the urge to fire up “screen” and use the side-by-side mode (CTL-a followed by the pipe “|” character, if yuz didn’t already knaa) to check those small sections of code directly.  Whaddyaknow!  My cut-and-paste code, “RECEIVE”.  BasicOTA.ino, “RECIEVE”.  Bugrit!  I go back and take a closer look at the include file.  Same spelling error.  Rats!  I fire up GitHub, go to the ESP8266/Arduino/libraries/ArduinoOTA …and find that Stavros Korokithakis has already fixed it a couple of months back (thanks for that, Stavros.  The tiny ESP8266 breakout board looks great, by the way).

Onwards and upwards!  So, I fire up the library manager in my squeaky new 1.6.7 Arduino-IDE and have it update da-woyks!  I restart the IDE, fire up another compile and find that the OTA_RECEIVE error is magically gone …to be replaced by the much more verbose:-

/tmp/build319.tmp/libraries/esp-mqtt/mqtt/mqtt.c.o:(.bss.default_private_key_len+0x0): multiple definition of `default_private_key_len'
/tmp/build319.tmp/libraries/ESP8266WiFi/WiFiClientSecure.cpp.o:(.bss.default_private_key_len+0x0): first defined here
/tmp/build319.tmp/libraries/esp-mqtt/mqtt/mqtt.c.o:(.bss.default_certificate_len+0x0): multiple definition of `default_certificate_len'
/tmp/build319.tmp/libraries/ESP8266WiFi/WiFiClientSecure.cpp.o:(.bss.default_certificate_len+0x0): first defined here
collect2: error: ld returned 1 exit status

Hmmm!  I suppose that the OTA_RECEIVE was just a second cousin (think extended families).  This one is definitely a wicked uncle-Ernie, fiddling about (insert air guitar here …only old codgers need apply).

I don’t have much of a clue on where to start with this one, other than the obvious MQTT library.  Unfortunately, that particular library hasn’t been updated for a few months and definitely wasn’t changed by my library update.  I’m starting to run out of patience with the extended family though, so after some more “fiddling about” of my own, I take the path of least resistance and comment out the defines in mqtt.c.  Why that file and not the ESP8266WiFi stuff?  Well, ESP8266WiFi is core and if the lengths are defined there, they’ll get carried through to any other lib in a project which depends upon them (that might not be true in Arduino land, where you need a separate bit of hardware for networking, but I’m fairly sure I’ll be using wireless in everything I do with the ESP …and definitely if I’m using MQTT).

And finally, a clean compile.

It still doesn’t work, of course (no call to the ArduinoOTA-handle() routine), but I’ve had enough for one day and decide to end the evening on a relatively high point.

 

Reworking the AI Thinker T5 – Part IV.V

The board should now be capable of running a somewhat useful program.  I’ve used the Arduino-ESP8266 core to bang this together, not because I’m an Arduino enthusiast (I’m not …I’ve never owned one, borrowed one or even stolen one), but because I found it the easiest environment to program in, given my cack-handed use of C and preference for the “vi” editor.  It just happens to handle libraries and compile code in a way which is easy to install, use and understand and it fits things into the ESP8266’s overly-complex memory footprint without requiring the user to edit SDK files.  It’s just a pity that the IDE sucks so much. 🙂

Any-hoo (see, I can speak Canadian, too!), this mini project uses the ESP8266 to interrogate the DHT11 sensor for temperature and humidity and then spits those readings out, together with a VCC voltage reading and a unique module ID, to MQTT.  After the MQTT publish is confirmed, the ESP8266 will drop into deep-sleep mode, waking itself up (assuming you remembered to connect GPIO16 to the RESET pin) after 10 minutes to start the whole process over again.

The code contains a ton of Serial.print’s, surrounded by #ifdef DEBUG #endif’s.  So as long as you have “DEBUG” defined to something in user_config.h, you’ll also see all of the data on the serial output, too.

Also in the user_config.h file are a few other settings which you must configure for your specific environment.  They include the SSID and password for your WiFi access point and static IP addresses for the ESP8266, gateway and DNS servers, as well as your MQTT server hostname, port number and topic IDs.  Everything else in that file can be tweaked too, but the items mentioned above are pretty much essential changes to get the ESP into operational mode on your specific network.

The code is available from GitHub:- PuceBaboon/ESP8266-AIT-T5

Do you need a T5 board to use this code?  Heck, no!  You can use just about any of the ESP modules out there, from the ESP-03, ESP-07, ESP-12 through to the WROOM-02.  The ESP-03 (and some of the other, older modules) may give you a bit of trouble when it comes to connecting GPIO16 to RESET, but it is doable.  The original ESP-01 is just about the only module I can’t recommend for this project.  You’ll need a DHT11 (or DHT22) and an LED to replicate the original T5 project, but all in all it’s probably easier going with a new ESP-12 than trying to rework the actual T5 board.

Libraries – The code makes use of the Adafruit DHT library and the “esp-mqtt” library from Ingo Randolf, which is a version of TuanPM’s library modded for the Arduino-ESP environment.  You’ll need to add those to your local, Arduino “libraries” directory to be able to compile the code.

A big “Thank you!” to all of those folks above for making their work available to the ESP community.

In the next instalment, we’ll do a simple update to convert one of the switches to a “program” switch (hold it down while powering on to put the ESP into programming mode) instead of having to fiddle with that jumper every time.

 

Arduino-IDE revision 1.6.6

There’s a new version of the Arduino-IDE out which has a major new feature, arduino-builder.  What’s so good about it?  Well, for anyone else who, like me, hankers after good ole’ command-line builds, this is what you’ve been waiting for.  This version of the IDE actually folds in full support for arduino-builder, so once you’ve created your Makefile, you’re all set to ditch the clunky IDE window, fire up vi or emacs (or whatever) and compile …except…  Bugrit!  There’s always an “except” nowadays, isn’t there?!

Well, the show stopper in this case is that the newly minted 1.6.6 takes your old code and immediately fires back the error message:-

Board generic (platform esp8266, package esp8266) is unknown

WTDF?!?*

Checking the preferences (either from the aforementioned clunky IDE window, or in your ~/.arduino15/preferences.txt file) shows that the board manager “additional URLs” are still there (these are the links back to the arduino.esp8266.com repository and should contain at least:- http://arduino.esp8266.com/stable/package_esp8266com_index.json).

Checking the selected board type under the “tools” pull down in the window (or in the “fqbn” line in the build.options.json file in the /tmp/buildxxxx directory, following a failed compile) also shows that the generic ES8266 module type is selected, so it should work.   But it doesn’t.

This seems to be one of the top reasons for the suggestion that you “stay with 1.6.5 for now” in the esp8266/Arduino GitHub installation instructions, but actually it turns out that this is simply an incompatibility between the old 1.6.5 and new 1.6.6 releases.

The fix** is quite simple; move your existing ~/.arduino15 directory to one side and create a new one, keeping only the original preferences.txt file.  First, close down the IDE window, if you happen to be running it (and it doesn’t matter whether you’re running the 1.6.5 version or the 1.6.6 version — close it!).

Move the original .arduino15 directory to one side, create a new ./arduino15 directory and then copy the preferences.txt file from the old directory into your newly created one.  Here’s a cut-and-paste’able version of that sequence (you might want to paste this into an xterm window and make sure that there are no line-breaks or odd HTML characters in there before pressing the <CR> key). Don’t try this on Windows!  :-

cd ${HOME} && mv ./.arduino15 ./.arduino15_ORIG-1.6.5 && mkdir ./.arduino15 && cp ./.arduino15_ORIG-1.6.5/preferences.txt ./.arduino15
[Just a quick aside on what that line means if you’re unfamiliar with Unix… The ${HOME} refers to your home directory and the “&&” characters simply tell the shell not to execute the next command (after the “&&”) unless the previous command was successful, so if any one of the commands in that long line fails, you’ll get an error message and execution will stop at that point… and you’ll have to sort it out yourself. 🙂 ]

Assuming that there weren’t any problems with the move, you should be able to fire up the 1.6.6 version of the Arduino-IDE and select the Board Manager from the Tools->Board submenu.  Select the “ESP8266 by ESP8266 Community” (which will be way down at the bottom of the scrollable list of options in the Board Manager window) and hit install.  That will trigger a download of the ESP8266 packages (including the latest updates to “stable”) and, when you start off a compile, it will all now work as expected.

Next, how do we get the arduino-builder to work.  More on that later.

Update #1 – 1.6.6 is a new version, so things are different.  One of the first things I noticed was that 1.6.6 tried to compile in all of the files in the project directory which I’d stored using filenames starting with a “.” character  — mostly chunks of older code or example code which I wanted to keep around for reference and which the previous versions up to 1.6.5 had considered to be “hidden”.  1.6.6 considers them fair game.

* = Double

** Thanks to Marvin Roger for pointing us all in the right direction with his posting to issue #973 on GitHub.