Support for ESP32 and Ethernet in TASMOTA

Development on TASMOTA continues apace (with the ongoing C-19 movement restrictions, possibly even more rapidly than usual) and lots of new and interesting stuff has been popping up in the code recently (for instance, if you want to connect an anemometer, to add wind-speed to your weather-station, Matteo Albinola has you covered). However, one of the most exciting recent additions has been the arrival of baked-in support for the ESP32, based on Jörg Schüler-Maroldt’s work. This first appeared in version 8.2.0.6, back at the beginning of May, after Jörg created the libesp32 compatibility library and an initial pull request back in April. Since that time there have been a lot of “#ifdef ESP32” lines added to the code.

Now (version 8.3.1.5, as of June 17th 2020) we not only have the ESP32 compatibility and compile additions but also the addition of ESP32 hardware Ethernet support with the recently added xdrv_82_ethernet.ino driver file. The new, tasty goodness doesn’t end there, though. Olimex ESP32-POE  If you take a quick look at the headers of that driver file, you’ll find the pinout defines and TASMOTA template for the Olimex ESP32-POE, so not only do we get the ESP32 and ethernet, we also get PoE thrown in (the Olimex board currently sells from their site for €17.95, but is also available from the likes of Mouser and even Amazon in some areas).

If you bought one of the exceedingly cheap ESP32-CAM boards (AIThinker/Geekcreit), you now have the option of TASMOTA enabling  it, too.  Look for a second new driver file, xdrv_81_webcam.ino, to see the pinout details and TASMOTA template for that device.

Okay, so new, sexy additions to the code, but how do we compile TASMOTA for the ESP32. Well, if you’re using PlatformIO (and you should be!), it couldn’t be much simpler. Copy the Sonoff-Tasmota code into a clean directory (or git clone the repository) and then:-

  • In that directory, copy the platformio_override_sample.ini file to platform_override.ini.
  • Edit your new platform_override.ini file and uncomment line number 29 so that “; tasmota32” becomes just “ tasmota32“.
  • Type “pio run“.

Sit back and watch for a couple of minutes while the compile runs. It will compile two different versions. The vanilla “tasmota” (ESP8266 version) will be created in .pioenvs/tasmota/firmware.bin. The tasmota32 (ESP32 version) will be created in .pioenvs/tasmota32/firmware.bin.

[ For anyone who hasn’t compiled TASMOTA before, the customizations for your local network settings and for the target device itself are made in the ./tasmota directory. The simplest method for a first time compile is to use the user_config_override.h file to update only the bare essentials. Once you’ve made sure that you can compile successfully, you can modify the (very much more complex) ./tasmota/my_user_config.h file for full customization. ]

(Blowfish) Virus Protection

Hopefully this will lift your spirits a little bit (if you’re a techie, anyway).  Here’s a quick snap of the tabletop, taken this morning while my better half was taking a break.

A section of the OpenBSD "Blowfish" T-shirt being cut up for use as a facemask

Just in case you don’t know what you’re looking at, it’s her workspace, which is currently dedicated to producing DIY face-masks and the donor material is an iconic, OpenBSD “Blowfish” T-shirt.  This shirt had a picture of Puffy on the front, along with the text:- “So long and thanks for all the passwords”.  The back was what contributed to it becoming mildly infamous at the time, as it was printed with the Blowfish algorithm and was (allegedly) illegal to export from the U.S. (although I don’t remember ever hearing of anyone being arrested for walking through customs wearing one).

So, what used to protect your ssh tunnel‡ is now helping to protect (in a small way, I know) against the SARS-CoV-2 virus and, just in case it’s still on the banned list (and we ever get to travel again), the offending algorithm will be folded into the inside of the mask, keeping our secret safe.

 


 

‡  —  If you’re still using Blowfish, you should know that Bruce Schneier, the designer, now recommends upgrading to “Twofish”.

Amazon Smart Bulb

Csndice RGBWW Smart Bulb (box, front view)

We have a little log house (which we built ourselves and used to live in) which we let out as a holiday cottage.  It’s something that is (generally) fun to do and also pays a few of the bills, but one of the big mistakes we made when we put it together our large, wooden Lego kit was to put too many light switches in a single location (if you live in a house long enough, you’ll eventually remember what most of them do, but for short-term visitors it’s just plain confusing, even with little stick-on labels).  So one of the things we find happens quite often is that one (or two …or five) of the outside lights are left on all night (and might be left on all day too, if our guests head out in a hurry in the morning).  Long ago we changed out all of the bulbs for the little CFL corkscrew types to reduce the overall power usage and we thought we might get around to replacing them all with LEDs, if the prices ever come down.

To compound the mistake with the switches, we also bought almost all of the external lights with E17 size fittings (about half the size of a normal household bulb), not because we really wanted them, but because that was the only size the maker supplied the fittings in.  This latter issue came back to bite us when I decided I could fix the lights-left-on problem by replacing the CFLs not just with LED bulbs, but with “Smart” LED bulbs (Wow-weee!).

Nope!  No smart bulbs in E17 form-factor (at the time).  So I did what any other self-respecting ESP hacker would do and, when one of the CFLs died a noisy, sparky death one day; I chopped off its head, ripped out the charred remains of the inverter and replaced it with and ESP01S, a TRIAC and one of the brightest (and cheapest) E17 LED bulbs I could find.  I stuck my little FrankenESP monster (I admit, it wasn’t particularly pretty) into the fitting right outside our kitchen door, so that we could play with it and monitor it without inconveniencing any of our guests during the initial testing and burn-in period.

(FrankenESP) Photo of DIY E17 ESP01S light assembly

The FrankenESP in all of its glory …The ESP01S and other DC components are at the extreme R/H end of the board, with the opto-isolator to the left of the programming header.  The TRIAC is closest to the camera (slightly left of centre) with the PSU sitting over at the back.  You can just see the PROG/RUN mode jumper partially hidden by the big, black electrolytic.

I set up a TASMOTA rule to switch it on at dusk and off again a few hours later and coached my better half in how to communicate her wishes to the nice lady stuck inside that tiny, pinky-orangey UFO thingy that landed on the kitchen counter a couple of years ago (…and always say “Thank you”, because we want our grandchildren to grow up to be polite).  So, (her) “Hey Gewjull!  Turn on the kitchen door”.  (GH, pertly) “I’m sorry, I don’t know how to turn on the kitchen door”.  “Hey Gewjull, turn on the effin’ kitchen door!!”.  (GH, sullen) “Okay, turning on the effin’ kitchen door”.  “Thank you!”.  (GH, stilted and mechanically) “That makes me so ‘effin happy”.  Ah, another successful foray into the world of home subjugation.

For a while, everything went along swimmingly.  In fact we got so used to coming home at night and being able to see the keyhole that the transfer of FrankenESP to the cottage was delayed, several times.  Then one evening we came back to a dim, flickering welcome …uh-oh!  A power-cycle cured it, but only for about five minutes and then it was back to flickering again.  A PSU problem?  More likely a TRIAC issue.  Gottverdeckel!  I pulled Franky out and put the LED bulb back in the fitting (yup, works okay).  Unfortunately, as usual with these intermittent problems, Franky worked fine on the test bench, with not a flicker to be seen.  I went back outside and gave the light fitting a couple of dunts with the flat of my hand to test for loose connection, but no flickering or dimming was evident.  And so everything remained as it was for (quite) a while  …until I decided to replace the CFL in the worst offending cottage light fitting with an LED bulb, anyway (Franky or no Franky).  The light itself was identical to the test fitting next to our kitchen door, but was some four years older.  I opened it up and, lo and behold, it was an E26 holder assembly.  “Oh crikey!” said I (or maybe something similar), all of this mucking around and I could have just gone out and bought an E26 “smart” bulb and flashed it with TASMOTA.  At about the same time, the E17 bulb in the kitchen door fitting couldn’t contain its hilarity any longer and started flickering and dimming intermittently again (no Franky involved). “Oh double crikey!” (or words to that effect).

Anyway, suitable chastened with my unsuccessful techie adventure and unreliable memory, I went off to Amazon (Japan) and searched for an E26 bulb with a high output (lumens) and cool-white hue, suitable for an outside fitting.  Box (side-panel, specifications)What I finally decided on was a “Csndice” branded,  RGBWW bulb that had a 900lm stated output and an adjustable colour temperature between 2700 and 6500K.  In addition, it was tagged as being compatible with Google Home and Alexa, with “no hub required” (which is usually a pretty good indication that there’s an ESP variant in there and that it can be flashed with TASMOTA).  It wasn’t particularly cheap, but I can confirm that it is easily converted to TASMOTA (with the excellent Tuya-Convert) and seems to work reliably (my better-half has started to complain about the “Geriatric disco mode” of the colour sequencer test).

This particular bulb works with the Qualitel ALS08 setting from the TASMOTA templates repository.

Photo of CSNDICE smart bulbThe light (in WW mode) is very bright and easily equals that of the CFL.  It has the added bonus of being easily adjustable (from sliders on the TASMOTA main-menu page) if you prefer a particular hue of white output (from “cool” blue through to “warm” yellow).  If the fancy takes you, you can also fiddle with the RGB settings to have a particular colour and shade, instead of that boring old white.

To help you (well, okay, to help me) test out the functionality and reliability, I’ve put up a simple, command-line exerciser on GitHub.  You’ll need to change the variables (at the top of the file) to use your TASMOTA MQTT topic name (variable: BULB_ID) and your MQTT broker (server) IP-address or name (variable: MQTT_SERV).  After that, you can just run the program from the command line with no options or arguments (to turn the bulb on in white, neutral hue mode), or use one of the following options:-

  • -c — “C”ool white. Switches on the WW LEDs in the bulb with a blue hue.
  • -w — “W”arm white. Switches on the WW LEDs with a yellow hue.
  • -n — “N”eutral white. Switches on the WW LEDs with the hue set mid-way between cool and warm.
  • -o or -0 — Switches all LEDs (RGB and WW) off (that’s a zero, by the way).
  • -s — “S”equence. Turns on various colour mixes of RGB for 2 seconds before fading to the next colour (Control-C to quit).
  • -d — “D”ebug. Fairly quiet debug output.
  • -D — “D”EBUG. Very verbose debug output.
  • -h — “H”elp. Basically this command listing.

Now you too can have fun at the geriatric disco.

 

 

 

 

Using a GPS with TASMOTA (Part II)

This is part-2 of a two part series.  If you missed it, the first part is here.


CONFIGURATION

Once you have your build of TASMOTA installed and the physical connections made, you need to configure which ESP pins TASMOTA will use to communicate with the GPS module.  There are a couple of points to make here …first, the TASMOTA pin configuration refers to the data flow with respect to the ESP8266 itself, while the labelling on the GPS module pins refers to data flow with respect to the GPS.  This means that the pin you define as GPS_TX on the ESP actually connects to “RX” on the GPS module and GPS_RX connects to “TX”.  The second point is “Don’t Panic!”, as I mentioned earlier, you can’t harm either the GPS or the ESP by connecting or configuring them the wrong way round.  The last point is that if your GPS module doesn’t support the UBX protocol, it ain’t going to work, no matter which way round you connect things.

Okay, boot your ESP8266 and connect to the TASMOTA web interface.  Go to “Configure” and then “Configure Module” and select the “Module type” to be “Generic (18)” (you’ll need to scroll right down to the bottom of the list of modules to find it).The TASMOTA pin configuration menu, showing GPS_RX highlighted  Save this change.  When the save is complete you can go back to the “Configure Module” tab and will now find all of the available pins on your module displayed.  Select GPIO12 and again, scroll right to the bottom of the pull-down listing.  You should see the options “GPS_RX (190)” and “GPS_TX (191) close to the bottom of the list (if you don’t see them, then the “#define GPS” option in the tasmota/my_user_config.h file has not been un-commented, or you’ve flashed the wrong image to your ESP).  Remembering that “GPS_RX” refers to data received by the ESP, select that option for pin GPIO12 (which is connected to the “TX” pin of the GPS module) and then “GPS_TX” for pin GPIO_13 (connected to “RX” on the GPS module).  Save those changes.

The ESP8266 will restart itself as part of the “save” process and when it comes back up again you should see several GPS status lines above the normal buttons on the main menu.

TASMOTA main menu showing GPS status lines, but all readings zero.
No Data From GPS

If your GPS module hasn’t acquired satellite data (GPS LED not flashing). or if it doesn’t support the UBX protocol, the GPS status lines will be present, but the data will be zeroed out (as shown above).  It’s also possible that your TX and RX configuration is reversed, so it’s worth trying either changing the pin configuration in TASMOTA, or just swapping the wires over (whichever is easiest).

TASMOTA main menu with live GPS data shown
GPS With Live Data

If your GPS is correctly configured, has already acquired data and supports the UBX protocol, you should see something like the screenshot above, with the data updating once per second.

There are several commands available via the TASMOTA console to manipulate the GPS data handling functions.  Commands are entered at the console and have a format of sensor60  n[n] where “n” is a number and “sensor60” specifies that this command is  for sensor number 60 (which is reserved for the GPS module).  So an example might be:-

sensor60  9   —   Start the NTP server process (using time data from the GPS module).

Here’s the complete list of commands, as of early Feb, 2020.

  • COMMAND     DESCRIPTION
  • sensor60  0  —  Write log data to all available storage, then stop
  • sensor60  1  —  Write log data to all available storage, then restart from the beginning (overwrite mode)
  • sensor60  2  —  Filter horizontal noise from GPS signal
  • sensor60  3  —  Stop horizontal noise filter
  • sensor60  4  —  Start recording log data to storage in append data mode
  • sensor60  5  —  Start recording log data to storage in overwrite data mode
  • sensor60  6  —  Stop recording log data to storage
  • sensor60  7  —  Send an MQTT update on each positional change (very noisy)
  • sensor60  8  —  Stop sending MQTT updates on positional changes
  • sensor60  9  —  Start the NTP server on port 123 (GPS web updates stop)
  • sensor60  10 — Stop the NTP server (GPS web updates start again)
  • sensor60  11 — Force a TASMOTA time update on each GPS packet
  • sensor60  12 — Stop time updates on GPS packets
  • sensor60  13 — Set TASMOTA lat/long values from GPS positional data
  • sensor60  14 — Start virtual serial TCP server (UBX data) on port 1234
  • sensor60  15 — Pause virtual serial TCP server

USES
LATITUDE/LONGITUDE

Now that we have our commands down pat, we can put the GPS data to use.  The first (and possibly the the most immediately useful) is command 13, setting the latitude and longitude values from the GPS reported values.  This will allow the TASMOTA sunrise/sunset timer functions to work without any manual intervention (that is, even if your timezone isn’t correctly set, having longitude and latitude available from the GPS allows the sunrise/sunset algorithm to calculate those values).  Note that there was an issue with this function in the initial release, so you should use TASMOTA version 8.1.0.9 or above (the link in part-I of this article is to a good, working version).

NTP SERVER

Don’t be confused by this.  TASMOTA already has NTP client capability baked-in (it will listen to other NTP servers to set its own time), but this option, command 9, starts an NTP server process which allows other systems to interrogate your ESP8266 on port 123 for time data (your ESP becomes a network time provider, instead of just being a consumer).

As you might expect with such constrained memory limits, this implementation has some limits.  The stratum level  (the measure of how many levels, or hops, we are removed from the physical source of the time signal) is hard-coded to “2”.  In a normal configuration, the time source itself counts as stratum 0 and the machine connected to it would be stratum 1, which would be treated as a preferred source by downstream clients.  However, our ESP8266 isn’t using  pulse-per-second synchronization of the time data (it’s simply publishing it on arrival), so marking it as a stratum-1 primary source could lead to problems (especially if you happen to be a day-trader).  The good news is that it is still a very accurate source of time data in certain situations.  For instance, if you have an isolated network of IoT devices, accuracy to the nearest second is probably acceptable and having an ESP8266+GPS combination providing NTP services is a low cost, low power option for a network of wireless data-loggers.

There’s another point you need to be aware of if you’re setting this up on a network with other NTP servers present; the ESP needs to be configured as a “server” in the ntp.conf file on the other machines, not as a “peer”.  This is because being a peer requires two-way communications between the machines and this implementation only has the capability of replying to a simple request for a time packet.  If you configure the ESP as a peer, the other machines in the network will ignore it and show it as being stuck in the “INIT” state.  If you configure it as a server however, the other machines will happily request time data from it …although, in my case anyway, the data from the ESP is so far off the time reported by the other systems in the NTP group (that is, local NTP servers on my network as well as “upstream” NTP sources on the internet) that the ESP is almost immediately marked as an “outlier” (with a “-” character in the first column of the “ntpq -p” output) and subsequently ignored.  It could still be quite useful, though.  If you are relying on upstream NTP servers on the internet (which is generally a good thing), then you would lose synchronization if your internet connection was down for any significant amount of time.  Your ESP though, would still be providing a local time source with one second accuracy, which is still pretty good (day-traders excepted).

The load of updating the GPS readout (as seen in the photo above) and providing time critical replies to NTP requests is also a little bit too much for the ESP8266, so you’ll see a message, “NTP Active” appear in that window and the GPS updates will stop while the NTP server process is running.  Once the “Sensor60 10” command is given to stop NTP, the GPS updates will restart and the “NTP Active” message will disappear.

One final wrinkle to the NTP implementation is worth noting (although it’s more of an “interesting characteristic” than a bug).  The “refid” field in the output of  “ntpq -p” command (on another NTP client on your network, not on the TASMOTA device) has two different modes of output.  Normally it will display the IP address of the machine to which the indicated peer is currently listening, but if the peer is a stratum-1 clock source, it will print one of a limited number of strings to let you know to what type of a clock device it is connected (ie:- radio-clock, atomic-clock, etc).  Now because our ESP is directly connected to a clock-like device (the GPS) but, as mentioned earlier, is pegged as a stratum-2 device, the refid output field displays the highly unlikely IP address of “71.80.83.0” (hint:- if you haven’t already guessed why it should do this, try “man ascii” for the answer).

LOGGING

If you happen to take your ESP8266/GPS combo with you when you’re hiking, paragliding or orienteering, then this section might be of interest.  Christian has added the option of logging GPS data to the ESP8266 flash, so you can save your experience for perpetuity (well, you can probably save some of it anyway …there’s not that much flash on an ESP8266).  This is the reason for the very oddly named “#FLOG” setting in the TASMOTA config file (nothing at all, as far as I know, to do with fifty shades of black and blue).  If that setting is un-commented, then flash-logging of GPS data is enabled and commands “sensor60 0” and “sensor60 1” set the log handling mode to “overwrite” or “stop writing when flash is full”, respectively.  Commands 4 and 5 both start the log recording, but 4 starts in append mode, while 5 starts a completely new log, overwriting whatever data was there.  When you start a recording the “Flash-Log” status line in the TASMOTA main menu window (see photo, above) changes from “ready” to “recording”.  Command number 6 stops the recording.

Once you stop a recording, a new, blue button will appear above the GPS information in the main menu.  There is no text displayed on the button (at least there isn’t in either Firefox or Chrome) until you move your mouse pointer over it, at which point the message “Download GPX-File” will appear, flashing on and off in time with the updates to the GPS data (perhaps this works as a normal button in some other browser?).  In any case, clicking on the button will download the flash-log data to your system.

VIRTUAL SERIAL PORT

You can also connect to a virtual serial port on the ESP8266 to get a live view of the (binary!) data streaming from the GPS unit.  The “sensor60 14” command will start this virtual serial connection on port 1234 and command 15 will stop it.  If you’d like to check the data, you can connect to your ESP using:-

nc  IP-ADDRESS  1234  |  od -x

Christian mentions that the intended use for this (other than trying to read hex data very quickly) is to feed into the u-blox “u-center” application, which is a debug and management application, freely available from the u-blox site (recent revisions are for  Windows only, though).

The Bottom Line

Christian has produced a very versatile and interesting application, adding an exciting new sensor option to the TASMOTA family.  He’s not finished yet, either.  The most obvious challenge is trying to get PPS working (which should significantly increase the accuracy of the NTP server function) without making the ESP8266 completely unusable for any of the other options running on it.  Even without that addition though, this is still a useful addition to TASMOTA and is certainly a lot more fun to play with than most other sensors.

If the idea of an ESP interfaced to a GPS module tickles your fancy (as it did mine), there’s also Chris Liebman’s stand-alone ESPNTPServer project, which I’ve mentioned before in these pages and which now has its own PCB and 3d-printed case.

 

Using a GPS with TASMOTA (Part I)

Towards the end of last year, Christian Baars integrated GPS-as-a-sensor support into TASMOTA with the addition of the xsns_60_GPS.ino file and ancillary code.  You can now easily attach a cheap GPS unit to your ESP8266, although just exactly how to connect it and how to communicate with it may not be immediately obvious.

In this two-part series,we’ll look at some of the issues influencing Christian’s design choices before getting down to the nitty-gritty of physical connections, TASMOTA configuration and (finally) uses.


TIP  —  Before we start …if you have a new GPS module‡ which you’ve never used before, plug in the antenna and connect it to a 3v3 supply while you’re reading the rest of this article (it’ll save you a little bit of time and perhaps some frustration, too).  You’ll notice that the LED is off when it’s first powered-on, but after a while (and it could take up to thirty minutes) it will start flashing at 1Hz.  This is letting you know that it has acquired signal from multiple satellites and is now ready to send data.


BACKGROUND

First of all, just a tiny bit of background on Christian’s implementation.  He is using the u-blox binary format (“UBX”) to transfer serial data from the GPS module.  GPS modules from most suppliers, including u-blox, normally come with ASCII format (“NMEA”) data transfer enabled by default, but this format produces long, fairly verbose messages which are not particularly well suited to the ESP8266’s limited memory and single core processor (remembering that the CPU must attend to WiFi housekeeping tasks on a regular basis).  To give TASMOTA the space and time to run its own tasks, Christian initializes the GPS board to use the short-form UBX binary data transfer instead.  Unfortunately, there’s a catch  —  not all GPS modules will support the “UBX” native protocol which this implementation requires.  As far as I know (from reading the u-blox web site),  it seems as though the majority of modules for sale on Alibaba, EBay or Aliexpress are probably fakes in one way or another (older modules being labelled as newer types, or just plain knockoffs with no u-blox chips in them), so it’s no surprise that a lot of them don’t speak the UBX protocol.

My own experience pretty much mirrors this.  Across three different modules, I have one (blue PCB. four-pin connector) which works with NMEA (the most common protocol), but not with UBX, Two GPS modules; one black, one blue.a second (red, narrow PCB, five-pin connector, with PPS¹) which just didn’t work at all and, lastly, a (black PCB, five-pin connector, with PPS) board marked as “WAVGAT NEO-6M” which does actually work using both NMEA and UBX protocols.  None of these modules pass the u-blox label test .  The WAVGAT module came from the WAVGAT Aliexpress shop though, so if nothing else, they may be more consistent than oddball modules from oddball suppliers (please treat this as a comment, not a recommendation).  Christian mentions in the header notes of the xsns_60_GPS.ino file that in addition to a (genuine) u-blox NEO-6M, the Beitian 220 module has also been proven to work.


CONNECTIONS

Okay, so now we know that why not all GPS modules are compatible with this TASMOTA addition and why that is, but how do we connect up our module to see whether it works or not?  The first thing you need to know about the connection is that serial communication with the GPS module from the ESP8266 is via the TasmotaSerial library (a TASMOTA-specific software-serial implementation) and not the ESP hardware serial port.  This means that, excluding the usual suspects of GPIO-15, and GPIO-02, we can connect to just about any available pin.  I’d recommend GPIO-12 (D6 on most ESP8266 boards) to the TX pin on your GPS board and GPIO-13 (D7) to RX on the GPS board.  Black WAVGAT PCB, showing pin connectorNote that it is very, very easy to reconfigure your ESP8266 (through the TASMOTA web interface) to change these pin settings, so don’t worry too much about mixing up TX and RX …it won’t hurt either the ESP or the GPS.  Finally, you need to supply GND and 3v3 connections to the GPS board (do be careful with these… there doesn’t seem to be too much consistency between the different GPS makers when it comes to positioning the power connections).   Those four connections are all we need (the current implementation doesn’t use the PPS signal).

Next comes TASMOTA.  I’m going to assume here that you’re familiar enough with it to be able to flash a binary to your ESP8266 (and if you don’t, there’s a ton of documentation on the web site).  If you’re comfortable building TASMOTA from source, then you should build with your normal “my_user_configs.h” file, but uncomment:-

  • . #USE_GPS
  • . #USE_FLOG

If you’re more comfortable just flashing a pre-built binary, you can download an 8.1.0.9 version built with just the GPS/FLOG defines (otherwise bog-standard “tasmota.bin”) from this link (it’s the “tasmota_GPS.bin” file).  Note that it is an unconfigured binary (just like you’d pick up from the normal TASMOTA release site) and will need to be configured from scratch on your network.  Please also note that the source server is using HTTPS protocol, so it is not suitable for an OTA update pull  …you’ll need to download the binary file and then use the “manual upload” method (assuming your ESP is already running a fairly recent version of TASMOTA).  There’s also a “minimal.bin” firmware image on the same server, but you can just as easily use the minimal image available on either the main release page or from Andre Thomas’ excellent thehackbox.org site (which is an OTA source).


This is part-1 of a two part series.  Click here for part-2.

 


FOOTNOTES

¹ – PPS = Pulse-Per-Second.  Blue GPS PCB showing 220-ohm resistor addition for PPS feedThis signal is used to synchronize the time data output – the hardware equivalent of “At the tone, it will be 3 AM, exactly“.  All of the u-blox modules which I have (including a USB dongle) have PPS outputs from pin-3 of the RF module, but many modules connect it to an LED as an activity/lock indicator.  However, you can easily add a PPS feed to your micro by simply adding a 220Ω resistor on the RF module side of the 1k LED current limiting resistor.  Click on the photo to the left and zoom in on the bottom, L/H corner to see where the resistor is connected.  Note that this module only has a four pin connector (at the top, centre of the board).

† – GPS modules can generally be configured to emit only those “sentences” of NMEA data which you specifically want, however it is still long-form, human readable data, compared to the UBX binary format.

‡ – When you first connect a new GPS module to power, nothing appears to happen.  This is because the module needs to receive a certain amount of data from satellites before it can determine the current time and it’s own location.  This can take up to thirty minutes in some locations.  Once the module has downloaded enough data to initialize, the “PPS” LED will start flashing (even if there are no serial data connections).  The data received on this initial “cold start” is automatically saved by the module itself to battery-backed memory, so on subsequent power cycles the start-up sequence will be much shorter.


 

Tuya compatible smart-bulbs TASMOTA-ized

[Via Hackaday]  —  Don Howdeshell picked up a couple of “Merkury” branded smart bulbs in a Black Friday sale and, discoverering that they were Tuya compatible, has very kindly blogged not just his update experience and a tear-down, but also the TASMOTA template for the bulbs he found (just in case your’s are different, you can find more templates for Merkury smart bulbs in the template repository).

Although you might find availability and shipping a bit patchy at the moment because of the ongoing virus outbreak,  there are lots of smart bulbs on AliExpress and Banggood which are now showing “Tuya Compatible” flags, which should be a good indicator of ESP-based hardware.  We’re also starting to see some E14 base (candle/candelabra sized) smart bulbs appearing, too.

Don used Tuya-Convert to re-flash his bulbs and this is just a reminder that Tuya-Convert also ships complete with a recent version (8.1.0.2 at the time of writing) of TASMOTA.

 

Recovering data from TASMOTA config back-up files

Okay, so you have a TASMOTA-enabled ESP8266 board in a place which is difficult to access and the latest upgrade doesn’t quite go as planned.  The upgrade itself appears to work, but the configuration back-up from the previous revision doesn’t take (your ESP is missing some peripheral devices).  Unfortunately, it’s also a custom build, not a shop bought device with a pre-rolled config already in the (vast) array of choices already shipped with TASMOTA.  What to do (other than retrieve the device and open it up)?  [“Document your projects better, you pillock!”]

Well, as I found out earlier, there is help at hand in the form of the command-line Python program by Norbert Richter:-  decode-config.py.

The good news is that it does (even more than) what you’d expect and can be used for recovering configs directly from a device, or converting between input and output formats.  The bad news is that it isn’t immediately obvious how to use it and it’s also very fussy about what version of Python you have (more especially right now, with the impending doom of v2.7 rushing towards us at the speed of a new year’s party-popper).

DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade….

…etc, etc. So, before you do anything else, grab the latest version of decode-config.py from it’s own repository, as that one is already Python3-ified. You’ll probably need to install dependencies, too (check the header comments).

Anyway, here’s a really, really short intro to using decode-config.py to recover data from older config files, so that you can plug that data straight back into your latest release.

decode-config.py --full-help

Will, as you expect, give a fairly verbose listing of the options.

decode-config.py -f /path/to/saved/configs/Config_Sonoff_6.3.0.dmp

Will give you an extremely verbose (and virtually indecipherable) JSON listing of the whole configuration (better than nothing, but not by much).

decode-config.py --indent-json 4 -f /path/to/saved/configs/Config_Sonoff_6.3.0.dmp

Will give you a reasonably formatted version of that same, JSON output (we’re getting there!).

decode-config.py --indent-json 4 -f /path/to/saved/configs/Config_Sonoff_6.3.0.dmp -g rules

Will give you a much shorter, much more useful, formatted JSON output of your saved rules (assuming you have any) preceded by a bunch of housekeeping information about the decode-config.py program itself and the system you’re running it on.

The sections which you can use with the “-g” flag are listed as:- “Control, Devices, Display, Domoticz, Internal, Knx, Light, Management, Mqtt, Power, Rf, Rules, Sensor, Serial, Setoption, Shutter, System, Timer, Wifi”, most of which are self-explanatory, but not all of which seem to work as you’d expect when trying to recover data from much older revisions of TASMOTA (as an example of this, the GPIO information which I was searching for was tagged onto the end of the “Mqtt” data). So, eventually I happened on this final invocation, which seemed to produce sane (and much more readable) output from the same back-up file.

decode-config.py -T command -f /path/to/saved/configs/Config_Sonoff_6.3.0.dmp

This (the winner!) provides output which you can scroll through directly, or as I did, save to a file for future reference. The data is grouped under the same headings as listed above in plain ASCII text. Everything appears to be grouped together logically (unlike the JSON output) and the only problem I found was an apparent off-by-one error in the numbering of the GPIOs (starting from GPIO1, rather than GPIO0) …which was fixed before I even finished this article (world record support times!).

So, that’s my brief introduction to decode-config.py, a bacon-saver if ever there was one.


Late News  —  Norbert (the author of decode-config) has just let me know that binary versions of the utility are also available for multiple operating systems, so there’s no longer any need to update Python.

PicoLibc adds hooks for ESP8266/ESP32 builds

Keith Packard has been putting in a lot of work over the past few months on PicoLibc, his lightweight libc replacement, which is specifically aimed at memory-limited embedded systems.  Keith has been steadily increasing the functionality of PicoLibc and has attracted some additional contributors along the way.  One of those contributors, Jonathan McDowell in Belfast, has just added the hooks for compiling xtensa-lx106-elf binaries.  Jonathan notes that it has been tested on an ESP8266 with the 2.2.1 + 3.0.1 Espressif NONOS SDKs + GCC 9.2.1 from Debian.

Over at CNX Software, Siji Sunny recently put together a (non-ESP specific) description of how to configure, compile and build code using PicoLibc under Linux, so it should now be possible (with Jonathan’s additions) to cross-compile for the ESP8266 using that handy guide.  In the meantime, Keith is continuing with additional work on integration of the ESP8266 and ESP32 targets.

Netatalk/AFPD/TimeMachine server set-up on FreeBSD-12.1

This entry is more of a memo to myself to remind me of  what the syntax is for the /usr/local/etc/afp.conf file for FreeBSD-12.1.

I’m consolidating a couple of my servers onto one physical machine (the Beelink AP35 which I mentioned a few months back)†, using FreeBSD as the OS.  Not surprisingly, the syntax for the afpd daemon‡ set-up is very different from the ARM-Linux box which is currently running that particular service (which provides a local TimeMachine server for all of the Mac users in the family).  This is the magic incantation which worked for me:-

[Global]
login message = “AFP Service on \”herons\””
vol preset = TimeMachine
log level = default:debug
log file = /var/log/afp.log
uam list = uams_guest.so,uams_dhx.so,uams_dhx2.so
mac charset = MAC_JAPANESE
unix charset = UTF8
mimic model = Xserve
zeroconf = yes
zeroconf name = “Herons-Store”

[Herons TimeMachine]
path = /store/TimeMachine
cnid scheme = dbd
ea = ad
time machine = yes

The “log level” setting of “default:debug” is quite verbose, but really helps with debugging problems during set-up.  Once your configuration is up and running you can remove that line completely to go back to the default logging level of “default:note”.

The “Mimic model” setting tells the server which Apple device icon to display on the client machine.  Using “Xserve” shows a rack-mount server, but you can also try “TimeCapsule8,119” or “TimeCapsule6,116” to see two different versions of a TimeCapsule icon.

The “zeroconf” lines were essential (in my environment, anyway) to having the clients recognize the new server.

The “charset” lines are probably unnecessary (or at least will need local modification) for users outside of Japan.


†  The AP35 is classed as a “mini-pc” (basically a device aimed at the set-top box market, but with an Intel CPU, rather than ARM).  This model comes with a dual-core J3355 CPU and 4 x USB3 ports, making it quite a versatile little machine.

‡  Note that the current, “approved” method for implementing a non-Mac hosted TimeMachine is the Samba (SMB) suite.  However, the older netatalk method described above is still simpler and a lot lighter (in terms of the number of support packages pulled in when using “pkg install”) than Samba if you only want to implement a TimeMachine instance (as opposed to a general, network file server).

Flashing the Sonoff TX/T0 US Version

The latest version of the Sonoff “smart” light-switch, the TX, is significantly different from the older ones. For a start, Itead have taken notice and created the front-panel for the US version in portrait, rather than landscape orientation. However, as one of the other changes was to remove all of the graphics (barring a small “Sonoff” label along the bottom), the orientation hardly matters now, anyway.  There’s also a black version available (and the Itead  site photo of the new, graphics-free front panel with blue, LED backlighting shows this version).Black version showing switch back-light

The new, plain white (or black)  front panel is fairly thick, with a single slot on the bottom edge for levering it open from the main body of the switch  —  which is required before you can fit it in the wall. The thickness of the front panel means that removing it from the body before it is fitted into the wall is a little difficult; while the front panel doesn’t flex, the thinner plastic of the body twists alarmingly. The best way to overcome this seems to be to lay the whole device on a flat surface, face down, then firmly press down with one thumb on the screw mounting hole (on the back-plate, just above the slot) before using a large, flat-bladed screwdriver to lever the back-plate away from the front panel. It does take quite a bit of force to open it.Press firmly & lever open

Removing the front panel allows you to screw the main body into the wall cavity, but also gives us access to the PCB so that we can flash TASMOTA onto the ESP8285 controller before fixing it into its permanent home.

The top PCB (with the switches and the ESP8285) needs to be disconnected from the main body of the device before programming and pulls off fairly easily (the design is asymmetric, so the PCB can only be inserted back into the body in one way).  The PCB layout also seems to have changed quite a bit since the previous version and there’s no “TP2/GPIO0” test point on the reverse side of this board.  Luckily, the other common connection point, the pad on R19, is still available, but in a different position on the PCB.Position of GPIO0 on R19

GPIO0 is on the end of R19 which is closest to the ESP8285 chip (the larger of the two chips, on the R/H side of the photo).

The contacts for a (3.3 volt) USB adapter are visible on the bottom, R/H side of the PCB photo (click to enlarge) and, as usual, GPIO0 needs to be briefly connected to ground when powering up the device to put it into programming mode.  I just used a flying lead clipped to ground at one end, touching the other end against the resistor pad while connecting the 3v3 supply.  Programming instructions for the “T1” version of the switch still work for this version.

Version 7 of TASMOTA is now available and the template for the T1/TX is also available from GitHub,  simply choose the correct template for your specific device and copy the string from the web page, then select Configure->Configure Other from the main menu on your newly installed device, paste the text into the Template input section, tick the Activate box and hit save.  Once the ESP has rebooted you will see the toggle boxes for each of the individual switches at the top of the main menu.