So, what used to protect your ssh tunnel‡ is now helping to protect (in a small way, I know) against the SARS-CoV-2 virus and, just in case it’s still on the banned list (and we ever get to travel again), the offending algorithm will be folded into the inside of the mask, keeping our secret safe.
‡ — If you’re still using Blowfish, you should know that Bruce Schneier, the designer, now recommends upgrading to “Twofish”.
We have a little log house (which we built ourselves and used to live in) which we let out as a holiday cottage. It’s something that is (generally) fun to do and also pays a few of the bills, but one of the big mistakes we made when we put it together our large, wooden Lego kit was to put too many light switches in a single location (if you live in a house long enough, you’ll eventually remember what most of them do, but for short-term visitors it’s just plain confusing, even with little stick-on labels). So one of the things we find happens quite often is that one (or two …or five) of the outside lights are left on all night (and might be left on all day too, if our guests head out in a hurry in the morning). Long ago we changed out all of the bulbs for the little CFL corkscrew types to reduce the overall power usage and we thought we might get around to replacing them all with LEDs, if the prices ever come down.
To compound the mistake with the switches, we also bought almost all of the external lights with E17 size fittings (about half the size of a normal household bulb), not because we really wanted them, but because that was the only size the maker supplied the fittings in. This latter issue came back to bite us when I decided I could fix the lights-left-on problem by replacing the CFLs not just with LED bulbs, but with “Smart” LED bulbs (Wow-weee!).
Nope! No smart bulbs in E17 form-factor (at the time). So I did what any other self-respecting ESP hacker would do and, when one of the CFLs died a noisy, sparky death one day; I chopped off its head, ripped out the charred remains of the inverter and replaced it with and ESP01S, a TRIAC and one of the brightest (and cheapest) E17 LED bulbs I could find. I stuck my little FrankenESP monster (I admit, it wasn’t particularly pretty) into the fitting right outside our kitchen door, so that we could play with it and monitor it without inconveniencing any of our guests during the initial testing and burn-in period.
The FrankenESP in all of its glory …The ESP01S and other DC components are at the extreme R/H end of the board, with the opto-isolator to the left of the programming header. The TRIAC is closest to the camera (slightly left of centre) with the PSU sitting over at the back. You can just see the PROG/RUN mode jumper partially hidden by the big, black electrolytic.
I set up a TASMOTA rule to switch it on at dusk and off again a few hours later and coached my better half in how to communicate her wishes to the nice lady stuck inside that tiny, pinky-orangey UFO thingy that landed on the kitchen counter a couple of years ago (…and always say “Thank you”, because we want our grandchildren to grow up to be polite). So, (her) “Hey Gewjull! Turn on the kitchen door”. (GH, pertly) “I’m sorry, I don’t know how to turn on the kitchen door”. “Hey Gewjull, turn on the effin’ kitchen door!!”. (GH, sullen) “Okay, turning on the effin’ kitchen door”. “Thank you!”. (GH, stilted and mechanically) “That makes me so ‘effin happy”. Ah, another successful foray into the world of home subjugation.
For a while, everything went along swimmingly. In fact we got so used to coming home at night and being able to see the keyhole that the transfer of FrankenESP to the cottage was delayed, several times. Then one evening we came back to a dim, flickering welcome …uh-oh! A power-cycle cured it, but only for about five minutes and then it was back to flickering again. A PSU problem? More likely a TRIAC issue. Gottverdeckel! I pulled Franky out and put the LED bulb back in the fitting (yup, works okay). Unfortunately, as usual with these intermittent problems, Franky worked fine on the test bench, with not a flicker to be seen. I went back outside and gave the light fitting a couple of dunts with the flat of my hand to test for loose connection, but no flickering or dimming was evident. And so everything remained as it was for (quite) a while …until I decided to replace the CFL in the worst offending cottage light fitting with an LED bulb, anyway (Franky or no Franky). The light itself was identical to the test fitting next to our kitchen door, but was some four years older. I opened it up and, lo and behold, it was an E26 holder assembly. “Oh crikey!” said I (or maybe something similar), all of this mucking around and I could have just gone out and bought an E26 “smart” bulb and flashed it with TASMOTA. At about the same time, the E17 bulb in the kitchen door fitting couldn’t contain its hilarity any longer and started flickering and dimming intermittently again (no Franky involved). “Oh double crikey!” (or words to that effect).
The light (in WW mode) is very bright and easily equals that of the CFL. It has the added bonus of being easily adjustable (from sliders on the TASMOTA main-menu page) if you prefer a particular hue of white output (from “cool” blue through to “warm” yellow). If the fancy takes you, you can also fiddle with the RGB settings to have a particular colour and shade, instead of that boring old white.
To help you (well, okay, to help me) test out the functionality and reliability, I’ve put up a simple, command-line exerciser on GitHub. You’ll need to change the variables (at the top of the file) to use your TASMOTA MQTT topic name (variable: BULB_ID) and your MQTT broker (server) IP-address or name (variable: MQTT_SERV). After that, you can just run the program from the command line with no options or arguments (to turn the bulb on in white, neutral hue mode), or use one of the following options:-
-c — “C”ool white. Switches on the WW LEDs in the bulb with a blue hue.
-w — “W”arm white. Switches on the WW LEDs with a yellow hue.
-n — “N”eutral white. Switches on the WW LEDs with the hue set mid-way between cool and warm.
-oor-0 — Switches all LEDs (RGB and WW) off (that’s a zero, by the way).
-s — “S”equence. Turns on various colour mixes of RGB for 2 seconds before fading to the next colour (Control-C to quit).
An Intel-based mini-pc (not even an Atom, but a fairly decent quad-core Celeron) with a case and power supply for less than the price of a Raspberry Pi. Can it be real?
[Update 11th March] — Looks as though these (assuming there ever was more than one) have sold out. 😦 Scroll down for some other, not quite as cheap, but still available mini-pc bargains. — …and the answer is, probably not! Check the delivery cost very carefully before clicking the order button. As far as I can tell, this seller is hoping to charge you €999.00 for express delivery of the first item and €80.00 for each additional item. It’s not clear to me whether that is the default, or whether you can have it shipped “normal delivery” for €4.99; perhaps someone more familiar with on-line shopping in French can comment on that?— Reader “raspi” has confirmed that delivery in France is €4.99 (see the comments section, below), so it does look as though this is a bargain after all (but you should still check delivery costs and final price before confirming your order).
Unfortunately, as I’ve mentioned before, Cdiscount only sell/deliver to a small number of western European countries, so the rest of us are out of luck, but if you’re in France, Spain, Germany or Belgium …get them while you can (and let us know in the comments if you manage to snag one of these at the advertised price). [IMPORTANT – If you didn’t read the update at the top of the page, go back and read it now — verify the shipping cost before you buy!!]
Silence is golden — Although the adverts are usually vaguely worded, you can safely assume that most of these thin, Celeron-based systems have a laptop-style fan hidden away somewhere inside (generally the earlier generation, Atom-based mini-pcs didn’t). The Coofun advertisment above very straightforwardly displays a photo of the heatsink and fan unit, so there’s no guesswork involved.
AMAZON N3150/J3160 BAREBONES
Currently (11th March 2020), most Amazon regions (including the U.S.) are advertising an N3150 or J3160 (virtually identical) based fanless model with 4 x USB3, 2 x USB2, 2 x HDMI and 2 x GbE. Note that this is the barebones price (no memory, no SSD/HDD). The price seems to be pretty much the same for Amazon US, Japan and UK at roughly $135 with free shipping:-
It seems that you’ll be shipped whichever CPU is at hand when your order is received (in other words, you can’t choose). It’s still very attractive, though; this is a Celeron machine at the current Atom-Z8350 price. From personal experience with similar machines, you can expect this beastie to run fairly hot; not hot enough to burn you, but hot enough to be uncomfortable to the touch.
As usual, these are not “monetized” links and I have no relationship at all with CDiscount.com (because they won’t sell or deliver to me). I am an occasional customer of Amazon.com, but I haven’t ever bought a “Coofun” branded computer (from them, or anyone else), so please do treat this as a simple pointer and not a recommendation. [ END OF THE SMALL PRINT ]
Once you have your build of TASMOTA installed and the physical connections made, you need to configure which ESP pins TASMOTA will use to communicate with the GPS module. There are a couple of points to make here …first, the TASMOTA pin configuration refers to the data flow with respect to the ESP8266 itself, while the labelling on the GPS module pins refers to data flow with respect to the GPS. This means that the pin you define as GPS_TX on the ESP actually connects to “RX” on the GPS module and GPS_RX connects to “TX”. The second point is “Don’t Panic!”, as I mentioned earlier, you can’t harm either the GPS or the ESP by connecting or configuring them the wrong way round. The last point is that if your GPS module doesn’t support the UBX protocol, it ain’t going to work, no matter which way round you connect things.
Okay, boot your ESP8266 and connect to the TASMOTA web interface. Go to “Configure” and then “Configure Module” and select the “Module type” to be “Generic (18)” (you’ll need to scroll right down to the bottom of the list of modules to find it). Save this change. When the save is complete you can go back to the “Configure Module” tab and will now find all of the available pins on your module displayed. Select GPIO12 and again, scroll right to the bottom of the pull-down listing. You should see the options “GPS_RX (190)” and “GPS_TX (191) close to the bottom of the list (if you don’t see them, then the “#define GPS” option in the tasmota/my_user_config.h file has not been un-commented, or you’ve flashed the wrong image to your ESP). Remembering that “GPS_RX” refers to data received by the ESP, select that option for pin GPIO12 (which is connected to the “TX” pin of the GPS module) and then “GPS_TX” for pin GPIO_13 (connected to “RX” on the GPS module). Save those changes.
The ESP8266 will restart itself as part of the “save” process and when it comes back up again you should see several GPS status lines above the normal buttons on the main menu.
If your GPS module hasn’t acquired satellite data (GPS LED not flashing). or if it doesn’t support the UBX protocol, the GPS status lines will be present, but the data will be zeroed out (as shown above). It’s also possible that your TX and RX configuration is reversed, so it’s worth trying either changing the pin configuration in TASMOTA, or just swapping the wires over (whichever is easiest).
If your GPS is correctly configured, has already acquired data and supports the UBX protocol, you should see something like the screenshot above, with the data updating once per second.
There are several commands available via the TASMOTA console to manipulate the GPS data handling functions. Commands are entered at the console and have a format of sensor60 n[n] where “n” is a number and “sensor60” specifies that this command is for sensor number 60 (which is reserved for the GPS module). So an example might be:-
sensor60 9 — Start the NTP server process (using time data from the GPS module).
Here’s the complete list of commands, as of early Feb, 2020.
sensor60 0 — Write log data to all available storage, then stop
sensor60 1 — Write log data to all available storage, then restart from the beginning (overwrite mode)
sensor60 2 — Filter horizontal noise from GPS signal
sensor60 3 — Stop horizontal noise filter
sensor60 4 — Start recording log data to storage in append data mode
sensor60 5 — Start recording log data to storage in overwrite data mode
sensor60 6 — Stop recording log data to storage
sensor60 7 — Send an MQTT update on each positional change (very noisy)
sensor60 8 — Stop sending MQTT updates on positional changes
sensor60 9 — Start the NTP server on port 123
sensor60 10 — Stop the NTP server
sensor60 11 — Force a TASMOTA time update on each GPS packet
sensor60 12 — Stop time updates on GPS packets
sensor60 13 — Set TASMOTA lat/long values from GPS positional data
sensor60 14 — Start virtual serial TCP server (UBX data) on port 1234
sensor60 15 — Pause virtual serial TCP server
Now that we have our commands down pat, we can put the GPS data to use. The first (and possibly the the most immediately useful) is command 13, setting the latitude and longitude values from the GPS reported values. This will allow the TASMOTA sunrise/sunset timer functions to work without any manual intervention (that is, even if your timezone isn’t correctly set, having longitude and latitude available from the GPS allows the sunrise/sunset algorithm to calculate those values). Note that there was an issue with this function in the initial release, so you should use TASMOTA version 22.214.171.124 or above (the link in part-I of this article is to a good, working version).
Don’t be confused by this. TASMOTA already has NTP client capability baked-in (it will listen to other NTP servers to set its own time), but this option, command 9, starts an NTP server process which allows other systems to interrogate your ESP8266 on port 123 for time data (your ESP becomes a network time provider, instead of just being a consumer).
As you might expect with such constrained memory limits, this implementation has some limits. The stratum level (the measure of how many levels, or hops, we are removed from the physical source of the time signal) is hard-coded to “2”. In a normal configuration, the time source itself counts as stratum 0 and the machine connected to it would be stratum 1, which would be treated as a preferred source by downstream clients. However, our ESP8266 isn’t using pulse-per-second synchronization of the time data (it’s simply publishing it on arrival), so marking it as a stratum-1 primary source could lead to problems (especially if you happen to be a day-trader). The good news is that it is still a very accurate source of time data in certain situations. For instance, if you have an isolated network of IoT devices, accuracy to the nearest second is probably acceptable and having an ESP8266+GPS combination providing NTP services is a low cost, low power option for a network of wireless data-loggers.
There’s another point you need to be aware of if you’re setting this up on a network with other NTP servers present; the ESP needs to be configured as a “server” in the ntp.conf file on the other machines, not as a “peer”. This is because being a peer requires two-way communications between the machines and this implementation only has the capability of replying to a simple request for a time packet. If you configure the ESP as a peer, the other machines in the network will ignore it and show it as being stuck in the “INIT” state. If you configure it as a server however, the other machines will happily request time data from it …although, in my case anyway, the data from the ESP is so far off the time reported by the other systems in the NTP group (that is, local NTP servers on my network as well as “upstream” NTP sources on the internet) that the ESP is almost immediately marked as an “outlier” (with a “-” character in the first column of the “ntpq -p” output) and subsequently ignored. It could still be quite useful, though. If you are relying on upstream NTP servers on the internet (which is generally a good thing), then you would lose synchronization if your internet connection was down for any significant amount of time. Your ESP though, would still be providing a local time source with one second accuracy, which is still pretty good (day-traders excepted).
One final wrinkle to the NTP implementation is worth noting (although it’s more of an “interesting characteristic” than a bug). The “refid” field in the output of “ntpq -p” command (on another NTP client on your network, not on the TASMOTA device) has two different modes of output. Normally it will display the IP address of the machine to which the indicated peer is currently listening, but if the peer is a stratum-1 clock source, it will print one of a limited number of strings to let you know to what type of a clock device it is connected (ie:- radio-clock, atomic-clock, etc). Now because our ESP is directly connected to a clock-like device (the GPS) but, as mentioned earlier, is pegged as a stratum-2 device, the refid output field displays the highly unlikely IP address of “126.96.36.199” (hint:- if you haven’t already guessed why it should do this, try “man ascii” for the answer).
If you happen to take your ESP8266/GPS combo with you when you’re hiking, paragliding or orienteering, then this section might be of interest. Christian has added the option of logging GPS data to the ESP8266 flash, so you can save your experience for perpetuity (well, you can probably save some of it anyway …there’s not that much flash on an ESP8266). This is the reason for the very oddly named “#FLOG” setting in the TASMOTA config file (nothing at all, as far as I know, to do with fifty shades of black and blue). If that setting is un-commented, then flash-logging of GPS data is enabled and commands “sensor60 0” and “sensor60 1” set the log handling mode to “overwrite” or “stop writing when flash is full”, respectively. Commands 4 and 5 both start the log recording, but 4 starts in append mode, while 5 starts a completely new log, overwriting whatever data was there. When you start a recording the “Flash-Log” status line in the TASMOTA main menu window (see photo, above) changes from “ready” to “recording”. Command number 6 stops the recording.
Once you stop a recording, a new, blue button will appear above the GPS information in the main menu. There is no text displayed on the button (at least there isn’t in either Firefox or Chrome) until you move your mouse pointer over it, at which point the message “Download GPX-File” will appear, flashing on and off in time with the updates to the GPS data (perhaps this works as a normal button in some other browser?). In any case, clicking on the button will download the flash-log data to your system.
VIRTUAL SERIAL PORT
You can also connect to a virtual serial port on the ESP8266 to get a live view of the (binary!) data streaming from the GPS unit. The “sensor60 14” command will start this virtual serial connection on port 1234 and command 15 will stop it. If you’d like to check the data, you can connect to your ESP using:-
nc IP-ADDRESS 1234 | od -x
Christian mentions that the intended use for this (other than trying to read hex data very quickly) is to feed into the u-blox “u-center” application, which is a debug and management application, freely available from the u-blox site (recent revisions are for Windows only, though).
The Bottom Line
Christian has produced a very versatile and interesting application, adding an exciting new sensor option to the TASMOTA family. He’s not finished yet, either. The most obvious challenge is trying to get PPS working (which should significantly increase the accuracy of the NTP server function) without making the ESP8266 completely unusable for any of the other options running on it. Even without that addition though, this is still a useful addition to TASMOTA and is certainly a lot more fun to play with than most other sensors.
If the idea of an ESP interfaced to a GPS module tickles your fancy (as it did mine), there’s also Chris Liebman’s stand-alone ESPNTPServer project, which I’ve mentioned before in these pages and which now has its own PCB and 3d-printed case.
Towards the end of last year, Christian Baars integrated GPS-as-a-sensor support into TASMOTA with the addition of the xsns_60_GPS.ino file and ancillary code. You can now easily attach a cheap GPS unit to your ESP8266, although just exactly how to connect it and how to communicate with it may not be immediately obvious.
In this two-part series,we’ll look at some of the issues influencing Christian’s design choices before getting down to the nitty-gritty of physical connections, TASMOTA configuration and (finally) uses.
TIP — Before we start …if you have a new GPS module‡ which you’ve never used before, plug in the antenna and connect it to a 3v3 supply while you’re reading the rest of this article (it’ll save you a little bit of time and perhaps some frustration, too). You’ll notice that the LED is off when it’s first powered-on, but after a while (and it could take up to thirty minutes) it will start flashing at 1Hz. This is letting you know that it has acquired signal from multiple satellites and is now ready to send data.
First of all, just a tiny bit of background on Christian’s implementation. He is using the u-blox binary format (“UBX”) to transfer serial data from the GPS module. GPS modules from most suppliers, including u-blox, normally come with ASCII format (“NMEA”) data transfer enabled by default, but this format produces long, fairly verbose messages which are not particularly well suited to the ESP8266’s limited memory and single core processor (remembering that the CPU must attend to WiFi housekeeping tasks on a regular basis). To give TASMOTA the space and time to run its own tasks, Christian initializes the GPS board to use the short-form UBX binary data transfer instead. Unfortunately, there’s a catch — not all GPS modules will support the “UBX” native protocol which this implementation requires. As far as I know (from reading the u-blox web site), it seems as though the majority of modules for sale on Alibaba, EBay or Aliexpress are probably fakes in one way or another (older modules being labelled as newer types, or just plain knockoffs with no u-blox chips in them), so it’s no surprise that a lot of them don’t speak the UBX protocol.
My own experience pretty much mirrors this. Across three different modules, I have one (blue PCB. four-pin connector) which works with NMEA (the most common protocol), but not with UBX, a second (red, narrow PCB, five-pin connector, with PPS¹) which just didn’t work at all and, lastly, a (black PCB, five-pin connector, with PPS) board marked as “WAVGAT NEO-6M” which does actually work using both NMEA and UBX protocols. None of these modules pass the u-blox label test . The WAVGAT module came from the WAVGAT Aliexpress shop though, so if nothing else, they may be more consistent than oddball modules from oddball suppliers (please treat this as a comment, not a recommendation). Christian mentions in the header notes of the xsns_60_GPS.ino file that in addition to a (genuine) u-blox NEO-6M, the Beitian 220 module has also been proven to work.
Okay, so now we know that why not all GPS modules are compatible with this TASMOTA addition and why that is, but how do we connect up our module to see whether it works or not? The first thing you need to know about the connection is that serial communication with the GPS module from the ESP8266 is via the TasmotaSerial library (a TASMOTA-specific software-serial implementation) and not the ESP hardware serial port. This means that, excluding the usual suspects of GPIO-15, and GPIO-02, we can connect to just about any available pin. I’d recommend GPIO-12 (D6 on most ESP8266 boards) to the TX pin on your GPS board and GPIO-13 (D7) to RX on the GPS board. Note that it is very, very easy to reconfigure your ESP8266 (through the TASMOTA web interface) to change these pin settings, so don’t worry too much about mixing up TX and RX …it won’t hurt either the ESP or the GPS. Finally, you need to supply GND and 3v3 connections to the GPS board (do be careful with these… there doesn’t seem to be too much consistency between the different GPS makers when it comes to positioning the power connections). Those four connections are all we need (the current implementation doesn’t use the PPS signal).
Next comes TASMOTA. I’m going to assume here that you’re familiar enough with it to be able to flash a binary to your ESP8266 (and if you don’t, there’s a ton of documentation on the web site). If you’re comfortable building TASMOTA from source, then you should build with your normal “my_user_configs.h” file, but uncomment:-
¹ – PPS = Pulse-Per-Second. This signal is used to synchronize the time data output – the hardware equivalent of “At the tone, it will be 3 AM, exactly“. All of the u-blox modules which I have (including a USB dongle) have PPS outputs from pin-3 of the RF module, but many modules connect it to an LED as an activity/lock indicator. However, you can easily add a PPS feed to your micro by simply adding a 220Ω resistor on the RF module side of the 1k LED current limiting resistor. Click on the photo to the left and zoom in on the bottom, L/H corner to see where the resistor is connected. Note that this module only has a four pin connector (at the top, centre of the board).
† – GPS modules can generally be configured to emit only those “sentences” of NMEA data which you specifically want, however it is still long-form, human readable data, compared to the UBX binary format.
‡ – When you first connect a new GPS module to power, nothing appears to happen. This is because the module needs to receive a certain amount of data from satellites before it can determine the current time and it’s own location. This can take up to thirty minutes in some locations. Once the module has downloaded enough data to initialize, the “PPS” LED will start flashing (even if there are no serial data connections). The data received on this initial “cold start” is automatically saved by the module itself to battery-backed memory, so on subsequent power cycles the start-up sequence will be much shorter.
Although you might find availability and shipping a bit patchy at the moment because of the ongoing virus outbreak, there are lots of smart bulbs on AliExpress and Banggood which are now showing “Tuya Compatible” flags, which should be a good indicator of ESP-based hardware. We’re also starting to see some E14 base (candle/candelabra sized) smart bulbs appearing, too.
Okay, so you have a TASMOTA-enabled ESP8266 board in a place which is difficult to access and the latest upgrade doesn’t quite go as planned. The upgrade itself appears to work, but the configuration back-up from the previous revision doesn’t take (your ESP is missing some peripheral devices). Unfortunately, it’s also a custom build, not a shop bought device with a pre-rolled config already in the (vast) array of choices already shipped with TASMOTA. What to do (other than retrieve the device and open it up)? [“Document your projects better, you pillock!”]
Well, as I found out earlier, there is help at hand in the form of the command-line Python program by Norbert Richter:- decode-config.py.
The good news is that it does (even more than) what you’d expect and can be used for recovering configs directly from a device, or converting between input and output formats. The bad news is that it isn’t immediately obvious how to use it and it’s also very fussy about what version of Python you have (more especially right now, with the impending doom of v2.7 rushing towards us at the speed of a new year’s party-popper).
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade….
Will give you a much shorter, much more useful, formatted JSON output of your saved rules (assuming you have any) preceded by a bunch of housekeeping information about the decode-config.py program itself and the system you’re running it on.
The sections which you can use with the “-g” flag are listed as:- “Control, Devices, Display, Domoticz, Internal, Knx, Light, Management, Mqtt, Power, Rf, Rules, Sensor, Serial, Setoption, Shutter, System, Timer, Wifi”, most of which are self-explanatory, but not all of which seem to work as you’d expect when trying to recover data from much older revisions of TASMOTA (as an example of this, the GPIO information which I was searching for was tagged onto the end of the “Mqtt” data). So, eventually I happened on this final invocation, which seemed to produce sane (and much more readable) output from the same back-up file.
This (the winner!) provides output which you can scroll through directly, or as I did, save to a file for future reference. The data is grouped under the same headings as listed above in plain ASCII text. Everything appears to be grouped together logically (unlike the JSON output) and the only problem I found was an apparent off-by-one error in the numbering of the GPIOs (starting from GPIO1, rather than GPIO0) …which was fixed before I even finished this article (world record support times!).
So, that’s my brief introduction to decode-config.py, a bacon-saver if ever there was one.