ESPurna updated to 1.9.4.

Tinkerman (Xose Pérez) has just updated his ESPurna package (an alternative to Theo’s TASMOTA) to version 1.9.4.  The change log shows additions for the Huacanxing H802 LED controller and the V9261F and ECH1560 energy metering ICs as well as fixes to ensure that all ESP8285 based devices are forced to use esp01_1m  (limited memory) and updates to MQTT handling.

If you’re using any version less than 1.9.0, it’s probably worth upgrading anyway, as that was the last major update, where Xose added support for a whole bunch of newer Sonoff products (including the RF Bridge, T1 light switch and 4CH Pro).

If you haven’t visited Xose’s site yet, it’s definitely worth checking out his recent RF Bridge article, as well as his tutorial on how to secure your IOT communications with an nginx reverse proxy and Let’s Encrypt certificates, both of which are very good reads.

Advertisements

Sonoff-TASMOTA Updated to 5.8.0

Theo has just released version 5.8.0 of the Sonoff-TASMOTA package into the wild, with a fairly varied collection of additions, updates and fixes.

There are several changes related to WS2812 LED control, a fix for a watchdog timeout problem, as well as other fixes for MQTT, language support and Domoticz and addition of support for the Yunshan Wifi Relay and Witty Cloud.  The complete changelog is available here:- https://github.com/arendst/Sonoff-Tasmota/blob/master/sonoff/_releasenotes.ino.

Owen Duffy has a nice rundown on the pros and cons of the Yunshan WiFi Relay board on his blog (and it’s worth taking a look at some of his other ESP stuff, too).

Yay! ESP32 SPIFFS Arrives.

This is one that seems to have been blocking a few projects up until now (going by the number of “any updates?” emails to the forum thread).

The official SPIFFS implementation is now available from the Arduino-ESP32 repository.

That’s the good news.  The bad news is that it’s flagged as “initial” and doesn’t seem to have things like “info” support yet.  Never mind, we have SPIFFS;  thanks Hristo and Ivan!

 

T-bao R8 15.6″ $185 Laptop – Installing Linux

-Preface-   I’ve hit a couple of roadblocks (specifically regarding partial hangs of the system when mounting large filesystems via NFS) which have delayed this post for a couple of weeks now.  Rather than delay any further, I’m posting this cut-down article on the basic installation details and will create a new article on running Linux once I’ve found a fix for my issues.


The T-bao R8 comes with Windows 10 preloaded on the internal, 64GB EMMC.  I’m not going to say anything more about Windows 10 as I’m completely unable to drive it (I just don’t “get” the Windows paradigm) and am therefore unqualified to give an informed opinion, one way or the other.  I intended from the outset that this would be a replacement for my existing, ailing laptop, which runs Elementary OS, a very usable version of Linux.

Anyway, the version of Elementary I’m running on my current laptop is fairly long in the tooth now (it’s based on Ubuntu 12.04 LTS), so it was time to upgrade the OS, too.  So, what we’re going to do here is:-

  • Break into the BIOS on the laptop and do some simple set-up.
  • Download a known, good ISO from the `net.
  • Boot the ISO image on our laptop.
  • Optionally back-up the pre-installed Windows-10 partition.
  • Install Linux.

What I discovered previously, with the Z8350-based Z83-II mini-PC (and the T-bao and Chuwi laptops are the same), was that the BIOS was strongly biased towards a Windows peculiarity where a native 64-bit system uses a 32-bit bootloader at switch-on.  This quirk, together with the UEFI requirement, means that many “UEFI enabled” distributions won’t actually boot on this device, as they only provide a 64-bit bootloader.  The answer to this boot problem for Atom equipped machines has been provided by Ian Morrison over at Linuxium in Australia.  Ian has developed a script, isorespin.sh, which will take the normal distribution ISO image file from several of the most popular Linux variants, unpack the ISO into a temporary directory structure, fix the bootloader, upgrade the kernel and add in required drivers for this type of machine (WiFi adapter and mouse-pad) and then roll the whole thing back up into a new ISO file which you can then dd to a USB thumb-drive to use as a combined bootable live demo and installation drive.  Ian has put a ton of work into this handy script and into re-spinning some ready-to-use example ISOs, so if you use his work, please be kind and buy him a beer.

Going back to the laptop, there are a couple of things you need to do and a couple of things you need to know to prepare for your install.  The BIOS on this laptop advertises itself as being a version from “American Megatrends”, but it has a very limited number of changeable settings.  You can get into the BIOS menu by hitting the escape key within a couple of seconds of powering on.  Do that first, go into general settings and set the “Num Lock” to “OFF” (if you don’t do this your keyboard will appear to be broken when you go into the UEFI shell, for instance).  You should also change the “Boot Delay” setting from “2” (seconds) to “5”, to give yourself a chance of reading any text displayed during the start-up process.  You may also find it helpful to change the boot messages option from “Quiet” to “Verbose” (which will give you the American Megatrends BIOS help messages at switch on).  Save those new settings and reset.


-IMPORTANT-

The boot device menu will not display any device that doesn’t have a 32-bit UEFI bootloader file.  This means that if you plug in a USB thumb-drive with (say) a standard 64-bit Ubuntu distribution on it, not only will the distribution be ignored, but the thumb-drive itself will not show up in the menu.  The same holds true for an external DVD-drive or any other bootable device.


Here’s where I wasted a lot of time during my initial attempts at building my chosen distribution.  I downloaded Ian’s script and ran it on my old Inspiron in an effort to build a bootable ISO for the new laptop.  I gained a lot of experience of running isorespin.sh from the command line, but never managed to produce an image which was bootable on the new laptop.  Strangely, all of Ian’s pre-built ISOs which I tried booted perfectly, so there was something about my ancient version of Linux (or just my ancient laptop) which didn’t cut the mustard in this case.

Anyway, to cut a tedious experience short, what I ended up doing was using one of Ian’s pre-built Ubuntu images to do a quick install on the T-bao R8 and then using the laptop itself to run isorespin.sh on the latest Elementary OS ISO to produce a bootable, working USB thumb-drive to do the “real” install.  This method worked perfectly first time. You don’t need to do this unless you want to install Elementary or some other version which Ian doesn’t have a pre-built ISO for.  The instructions below assume you’re doing a simple, straight install from Ian’s pre-built Lubuntu image (find the Lubuntu section and link to the image about 1/3 of the way down this page).

Load the image onto a USB thumb-drive (using dd) and connect it to your T-bao (L/H side connector is the USB-3.0 socket).  Power on and hit <F7> to get the boot selection menu and boot from USB.  Select the USB thumb-drive (if it’s not on the menu then, as per the info above, there’s something wrong with your download or copy procedure) and hit <CR>, followed by “Try Lubuntu without installing”.  Your machine should boot into the GUI within a couple of minutes.  At this point you should be able to play around with Lubuntu and get a feeling for how the keyboard and mouse perform before starting the actual install.

Once you’re comfortable with the keyboard and mouse, you can select “Install Lubuntu” either from the “System Tools” menu or by clicking on the install icon on the desktop.

A small warning here before you start the install …if you’ve chosen one of Ian’s other image files which has a “persistent” save area enabled on the thumb-drive and then have finger trouble when you enter (say) disk partitioning information, that same (bad) information will show up again the when you reboot from the thumb-drive.  It’s generally easier to reload the original, clean ISO image to the thumb-drive again, rather than try to work around it.

Note:-  The “disk” in the R8 is actually a 64GB eMMC memory module.  This is not the same as a SSD and is just one more of the compromises made when producing a budget laptop.  I found the eMMC on the Z83-II mini-PC to be quite slow and, although the T-bao unit isn’t a walking-through-waist-deep-treacle-nightmare, it still doesn’t break any speed records.

The Lubuntu install ISO which I linked to above will do a very good job of installing Linux onto the eMMC with the existing Windows-10 intact, without needing any special input from you.  In that case though, you’ll only end up with about 20GB of disk space dedicated to Linux.  You can choose to delete the Windows data and dedicate the whole disk to Lubuntu to get the whole 57-odd-GB of actual available disk space.

One other option you have while booted from the live ISO on the thumb-drive and before starting a Linux install, is to do a full copy of the eMMC drive using “dd” to another machine across the network (or to a local, USB disk drive), thus allowing you the option of restoring the machine to its default (Windows-10) state at any time in the future if, for instance,  you should decide to give it to your nephew for his school work.  This will take a bit of time (probably a couple of hours, depending upon your network and the target machine) and will use roughly 6GB of disk space on the remote machine if you use “xz” as the compression method (more for gzip, bzip2 or others).  If you have a very powerful remore server (muli-core CPU) you can compress remotely, otherwise I’d advise compressing on the laptop to get the added advantage of passing less data over your network.  A sample command (local compression on the laptop) might look something like this:-

dd   if=/dev/mmcblk0   bs=4M   |    xz   -T2   -c   |   ssh    UserName@SmallServer   "(   cd   /BigDisk/LotsaSpace   &   cat   ->   T-bao-Laptop_Windows-10_bkup_mmcblk0_dd.xz   )"

If you have some major iron on the server side and want to do the compression there instead, it might look like this instead:-

dd   if=/dev/mmcblk0   bs=4M   |   ssh    UserName@BigServer   "(   xz   -T4   -c   |   cd   /BigDisk/LotsaSpace   &   cat   ->   T-bao-Laptop_Windows-10_bkup_mmcblk0_dd.xz   )"

In the first example, the “-T2” option to the xz command tells it to use two threads for compression and the xz command itself comes before the ssh section and so is running on the laptop (a four core machine).  In the second version the xz command has moved into the ssh section and is being executed on the remote server.  Note that in this case the option has changed to “T4” to use four threads on that (imaginary) eight core machine.  In both cases the command is run as root on the laptop and the remote user “UserName” needs to have write permission in the /BigDisk/LotsaSpace directory on the remote server.

Installing from the live image on the thumb-drive after this doesn’t really require any explanation (as the prompts are informative and I’m also pretty sure you’ve already installed Linux once or twice yourself already).  The only place you need to pause and consider your answer is at the disk partitioning prompt; I don’t have any qualms about wiping Windows completely, but you might.

The actual install progresses very quickly and, as far as I can tell, selecting “download updates during install” doesn’t have any untoward effects on the (already updated) kernel and driver files from Ian’s respin (but please do let us all know in the comments if your experience differs from mine).

I had planned to limit the eMMC use on this machine purely to the initial boot of the kernel and have the laptop mount the live filesystems via NFS from one of my servers.  However, to date I’ve been unable to get the system to run reliably with even just /home mounted via NFS (the window manager appears to freeze, but shells on alternative console devices with, for instance, CTRL-ALT-F2, seem to carry on working).  This is obviously directly related to NFS traffic, as triggering multiple read/write operations on the /home filesystem with a non-trivial compile will reliably cause the hang.  “nfsstat” on the server shows an initial burst of activity which immediately gives way to just a single read operation (by the client) every couple of seconds.  Right now I’m stalled on this issue.


For anyone interested, the output from “dmesg” on the T-bao R8 running ElementaryOS 4.1 is available here.

 

Keeping your local ESP8266 source trees up to date

I have a burgeoning directory on my laptop which holds all of the source code for any ESP8266 project I’ve ever come across and thought “This looks interesting…”.  Through the magic of Git, it usually only takes a few seconds to clone a complete project to local disk and have a permanent local copy which you can play with.  The only problem being that the code usually gets stale fairly quickly and you never know when the original author has pushed out an update.  To this end, I knocked together a very simple script which simply lists all of the subdirectories in my ESP8266 master directory and then goes off and does a “git pull” for everything it finds …assuming that the subdirectory is actually a git structured repository and is not flagged with “Do Not Update”.

Git being git, it doesn’t actually care whether the remote, original repository is on GitHub, PasteBin or Bob’s_basement_box, it’ll happily update from the stored URL, whatever and where-ever it is.

The “Do Not Update” feature is there in case you’re lazy (like me) and tend to make working directories with local modifications (which would fail to update and cause errors), or just because you want to maintain a pristine, original snapshot of a repository at some particular point in time.  You can tell the script not to update a repository in two ways.  The first is to add the name of the subdirectory to the file Do_Not_Update.txt in the top-level ESP8266 directory.  The second is to create a file named “Do_Not_Copy” in the actual repository subdirectory itself (the file can be empty, or you can use it to store a note on why you wanted to prevent updates).

The script will keep a (fairly verbose) log of the update process (by default, it is written to /var/tmp/ESP8266_auto_update.log).

The only thing you must change for your particular installation is the path to the top-level ESP8266 directory on your machine (which can be called anything you wish, BTW), “WK_DIR”, which is right at the top of the script.  There’s a sample cron entry in the README.md file which will run the script once per week.

I’ve been running this early every morning for my ESP8266, ESP32 and a couple of other directories for well over a year now and my morning routine now includes a quick “ls -alrt” of those directories to see what’s been updated in the last 24 hours  …a good way of staying abreast of Theo’s amazing update rate on TASMOTA.

An interesting new Sonoff (ITEAD) product on the way

Browsing through the wiki pages on the ITEAD site is always a good way to pass a few idle minutes and usually rewards the curious reader with interesting stuff (like schematics, for instance) which ITEAD are kind enough to publish for our edification.  Sonoff 433MHz to WiFi bridge, block diagramToday’s snippet was some information on what looks like an as-yet unannounced product, a WiFi to 433MHz gateway module.  The schematic shows this as an ESP8266-based unit, but there’s no separate flash memory chip that I can see and the block diagram refers to an ESP8285 (shame!).  There are both transmitter and receiver sections on the 433MHz side and it appears to use an EFM8BB1 “Busy Bee” 8-bit microcontroller to interface between the 433MHz RX/TX section and the ESP UART, with what looks like a slide switch (S2 on the diagram) to disconnect the Busy Bee to allow for programming of the ESP.  The device itself receives external power via a micro-USB socket.

Depending upon the price (and ITEAD prices are usually pretty reasonable) and the range of the 433MHz components, this could be a neat little device to have around. Front and back views (photo courtesy of ITEAD) It’s not just all of those older 433MHz switch modules that have been available for years, but also the slew of devices which just transmit (doorbells, weather stations, window interlocks, etc).  There does seem to be a four device limit on the number of remote 433MHz modules supported by the stock firmware though, according to the User’s Guide.

Update – ITEAD have just sent out a “Mid-Year Carnival Sale” promotion which features this unit (with the photo above) but, bizarrely for a sale, without a price.

Update 8th Aug 2017 – The main sales page is available on ITEAD’s site now and, for a time anyway, the unit is available at an introductory price of $9.90 (down from $12.90).  There are some clarifications of the details too, with the supported device limits being shown as “up to sixteen 433MHz RF devices” or “up to four 1-4 button 433MHz RF Remotes” (so basically 16 addressable channels).

As expected, Theo isn’t far behind and Sonoff-TASMOTA has had support for the 433MHz RF Bridge incorporated since the 5.5.0 version (released on July 30th), with further updates to the code added in v5.5.1.

Save power by (reliably) switching the ESP WiFi on and off

I have an ongoing project to add an ESP8266 to a solar trickle charger for a lead-acid battery.  Really, the only set-in-concrete requirement is that the controller must switch off the feed from the solar panel before the battery voltage gets to dangerous (“gassing”) levels, but it would be nice to have remote data logging of voltages (battery and solar) and temperature, as well as some control over the load (a small impeller pump in an external sump well, used for orchard irrigation during the summer and autumn).

Because the target battery is a standard (wet plate) car battery, I have quite a bit of capacity to play with, but even so, my back-of-the-envelope calculations show that the ESP8266 is quite capable of completely discharging the battery in under a week (assuming no sunshine and an ESP8266 module with WiFi enabled and transmitting on a more or less continuous basis), so I’d been looking for a reliable way to save power, but still have the ESP8266 do real time monitoring of the battery and solar voltages and interrupt the charge process when the battery was already fully charged (I should probably note that we get a lot of sunshine here during the summer and autumn months and our usage of the pump is only very intermittent, so despite having a mere 300mA output from the solar panel, it is possible to overcharge the battery over time).

I didn’t want to use deep-sleep mode (although it’s the least power hungry), because I want the ESP to be the actual charge controller and it can’t do that if it’s off in la-la land for the best part of its day.  I still want to use the WiFi capability (otherwise there’d be little point in using the ESP over a PIC), but I don’t need to have 24/7 connectivity.  Turning the WiFi on and off has always been a bit of a hit and miss business for me, though (and with quite a few other people too, judging by the number of hits a web search on the subject generates).  So I was pleasantly surprised to find some answers in a GitHub “issue” raised against the ESP8266/Arduino project.  The thread contained some excellent answers and testing experiences from some of the well known members of the ESP8266 community (thanks guys!), so I felt fairly sure I was onto a winner, but there were one or two twists along the way.  So here is a brief rundown on how I’m doing it with those kinks ironed out and here is a program which will demonstrate the practicality and reliability of using the “Issue 644” method.

First, let me assure the nervous reader that, despite what follows, we are not putting the ESP to sleep at all, we do not need to use an interrupt or a reset to restart the WiFi and we also do not need to make any hardware changes to the ESP module for this method to work.

The crux of this method is to use WiFi.forceSleepBegin() to put the ESP into “modem sleep” mode.  It’s important to realize that this does not put the core into any form of sleep; it simply turns the modem (radio) off.  There are some other incantations in there to ensure that WiFi is shut down in  an orderly fashion:-


bool WiFi_Off() {
WiFi.disconnect();
WiFi.mode(WIFI_OFF);
WiFi.forceSleepBegin();

One of the issues at this point though, is that the WiFi can take a finite, but variable time to shut down after the final forceSleepBegin() call, so at this point I’ve just added a timeout loop, checking the status of the connection until it goes down (returning without an error) or times out (returning with an error):-


while ((WiFi.status() == WL_CONNECTED)
&& (conn_tries++ < WIFI_RETRIES)) {
delay(100);
#ifdef DEBUG
Serial.print(".");
#endif
}
if (WiFi.status() != WL_CONNECTED)
return (true);
else
return (false);

The default timeout of 3 seconds is more than enough for the WiFi_Off() function.

The WiFi_On() function is pretty much a reversal of the “off” code:-


bool WiFi_On() {
conn_tries = 0;

WiFi.forceSleepWake();
WiFi.mode(WIFI_STA);
wifi_station_connect();
WiFi.begin(STA_SSID, STA_PASS, WIFI_CHANNEL, NULL);

while ((WiFi.status() != WL_CONNECTED)
&& (conn_tries++ < WIFI_RETRIES)) {
delay(100);
#ifdef DEBUG
Serial.print(“.”);
#endif
}
if (WiFi.status() == WL_CONNECTED)
return (true);
else
return (false);
}

The call to “wifi_station_connect()” seems to be the equivalent of a sacrificial chicken for most people, though. Without it, the restart of the WiFi doesn’t always work; with it, everything is hunky-dory (whatever that means).

Unfortunately, actually getting the call to wifi_station_connect() to compile turned out to be a bit problematic for me (from within the PlatformIO build environment). Just where the heck is it? Well, some searching found a define for it in the user_interface.h file in Espressif’s SDK. Adding a simple:-

#include "user_interface.h"

…in the header of my ESP_Power_Save.ino file got me past the initial “not found” messages, but then, at the last stage, the linker failed with “undefined reference” errors. Grrrr! The magic sauce for that though, is simply to let the C++ compiler know that it’s looking for a plain C function (buried deep in the nether regions of the Espressif SDK, in this case) by modifying that include to be:-

extern "C" {
#include "user_interface.h"
}

Finally, it compiled …and worked, too.

Before you flash ESP_Power_Save.ino to your ESP8266, you’ll need to configure the IP addresses, access-point SSID and password in the user_config.h file.

After flashing, connect to the ESP8266 console as normal and you’ll find yourself in a mini test environment where you can use a few simple, single character commands:-

  • “w” — to toggle the state of the WiFi between on and off.
  • “s” — to display the current state of the WiFi connection.
  • “c” — to display a simple count on the console, just to prove that the ESP core is still operational.

I’d recommend opening a second window and using it to ping the ESP’s IP address, to verify whether it is up or not as you toggle the WiFi.

Finally, a word of caution …If you try to connect to a non-existent access-point, or one which is out of range, or you use an incorrect password, then the WiFi will not turn off. That seems pretty counter-intuitive to me and I haven’t yet worked out why it happens. At any rate, it’s obviously worth doing some testing (I use a hand-held ESP8266 rig running on three AA batteries) to make sure that you have connectivity before deploying this power-save code to your remote units.


Update — The latest version of the code now includes a cut-down version of the “FSWebServer” example (from the ESP8266WebServer library) which will server up a 750kB JPEG file from SPIFFS on the ESP.  It takes a non-trivial amount of time for the little ESP to retrieve such a substantial file from SPIFFS, chunk it up and push it out across the network, which hopefully will give you adequate time to measure the current consumption of your module while busy transmitting, for comparison with when the WiFi is toggled to the “off” state.


Update #2 –  Real world testing has brought some interesting “features” to light.  Most problematical is that the WiFi doesn’t seem to always switch off completely, which is peculiar.

First, you need to know that the ESP8266 returns its current status when interrogated with the WiFi.printDiag() request.  When running normally, the results look something like this:-


Mode: STA
PHY mode: N
Channel: 11
AP id: 0
Status: 5
Auto connect: 1
SSID (9): Mainwaring
Passphrase (10): D0ntPan1c!
BSSID set: 0

The two most important indicators are the Mode and Status lines (although the SSID and Passphrase entries will also change when the WiFi is disabled).

The current draw on  my test module is generally in the area of 75ma when WiFi is enabled, with very brief peaks when actually transmitting (so brief that they’re impossible to measure with an ordinary digital multimeter).  That value drops off to 20ma when the WiFi is disabled and at the same time, WiFi.printDiag() changes to show “Mode: NULL” and “Status: 255” (and the module stops servicing web requests or even  a simple “ping”).

Frequently, toggling the WiFi (with the “w” command) will give the same result in terms of connectivity (ie:- the unit stops responding to web requests or pings) and the result of a WiFi.printDiag() is still “Mode: NULL” and “Status: 255”, but the current draw remains in the 75ma region.  I suspected that the auto-connect or auto-reconnect settings may have been responsible, but after further testing that doesn’t seem to be the case.

So, after using the word “reliably” in the title of this post (which, admittedly, referred specifically to being able to turn WiFi back on after a modem sleep), I now find that the current reduction isn’t reliable at all.  Does anyone have any ideas?