An ESP32 project worth looking at…

…especially if you’re a farmer.

You might have noticed that it has been fairly quiet around here recently. That’s mainly because real life has been getting in the way quite a lot, but also because what little spare time I have has been taken up by a new project. I’m not ready to spill the beans on what that project is quite yet, but you can get a couple of fairly strong hints on what direction I’m going (!) by taking a look at the beautifully executed work of Matthias Hammer (and his daughter) on their autonomous equipment project. Note that Matthias isn’t limiting his work just to autonomous control of the tractor, but is also extending the project to automate the other agricultural equipment that he uses.

Matthias is using ESP32s in his project as replacements for the Arduinos used in the base AgOpenGPS project. He has a few videos up on YouTube giving a guided-tour through the equipment, as well as showing the tractor at work. If you’re at all interested in GPS/RTK and autonomous vehicles, his (short) videos, GitHub repository and the AgOpenGPS web site are all worth a visit (even if you’re not a farmer).


Using a GPS with TASMOTA (Part II)

This is part-2 of a two part series.  If you missed it, the first part is here.


CONFIGURATION

Once you have your build of TASMOTA installed and the physical connections made, you need to configure which ESP pins TASMOTA will use to communicate with the GPS module.  There are a couple of points to make here …first, the TASMOTA pin configuration refers to the data flow with respect to the ESP8266 itself, while the labelling on the GPS module pins refers to data flow with respect to the GPS.  This means that the pin you define as GPS_TX on the ESP actually connects to “RX” on the GPS module and GPS_RX connects to “TX”.  The second point is “Don’t Panic!”, as I mentioned earlier, you can’t harm either the GPS or the ESP by connecting or configuring them the wrong way round.  The last point is that if your GPS module doesn’t support the UBX protocol, it ain’t going to work, no matter which way round you connect things.

Okay, boot your ESP8266 and connect to the TASMOTA web interface.  Go to “Configure” and then “Configure Module” and select the “Module type” to be “Generic (18)” (you’ll need to scroll right down to the bottom of the list of modules to find it).The TASMOTA pin configuration menu, showing GPS_RX highlighted  Save this change.  When the save is complete you can go back to the “Configure Module” tab and will now find all of the available pins on your module displayed.  Select GPIO12 and again, scroll right to the bottom of the pull-down listing.  You should see the options “GPS_RX (190)” and “GPS_TX (191) close to the bottom of the list (if you don’t see them, then the “#define GPS” option in the tasmota/my_user_config.h file has not been un-commented, or you’ve flashed the wrong image to your ESP).  Remembering that “GPS_RX” refers to data received by the ESP, select that option for pin GPIO12 (which is connected to the “TX” pin of the GPS module) and then “GPS_TX” for pin GPIO_13 (connected to “RX” on the GPS module).  Save those changes.

The ESP8266 will restart itself as part of the “save” process and when it comes back up again you should see several GPS status lines above the normal buttons on the main menu.

TASMOTA main menu showing GPS status lines, but all readings zero.
No Data From GPS

If your GPS module hasn’t acquired satellite data (GPS LED not flashing). or if it doesn’t support the UBX protocol, the GPS status lines will be present, but the data will be zeroed out (as shown above).  It’s also possible that your TX and RX configuration is reversed, so it’s worth trying either changing the pin configuration in TASMOTA, or just swapping the wires over (whichever is easiest).

TASMOTA main menu with live GPS data shown
GPS With Live Data

If your GPS is correctly configured, has already acquired data and supports the UBX protocol, you should see something like the screenshot above, with the data updating once per second.

There are several commands available via the TASMOTA console to manipulate the GPS data handling functions.  Commands are entered at the console and have a format of sensor60  n[n] where “n” is a number and “sensor60” specifies that this command is  for sensor number 60 (which is reserved for the GPS module).  So an example might be:-

sensor60  9   —   Start the NTP server process (using time data from the GPS module).

Here’s the complete list of commands, as of early Feb, 2020.

  • COMMAND     DESCRIPTION
  • sensor60  0  —  Write log data to all available storage, then stop
  • sensor60  1  —  Write log data to all available storage, then restart from the beginning (overwrite mode)
  • sensor60  2  —  Filter horizontal noise from GPS signal
  • sensor60  3  —  Stop horizontal noise filter
  • sensor60  4  —  Start recording log data to storage in append data mode
  • sensor60  5  —  Start recording log data to storage in overwrite data mode
  • sensor60  6  —  Stop recording log data to storage
  • sensor60  7  —  Send an MQTT update on each positional change (very noisy)
  • sensor60  8  —  Stop sending MQTT updates on positional changes
  • sensor60  9  —  Start the NTP server on port 123 (GPS web updates stop)
  • sensor60  10 — Stop the NTP server (GPS web updates start again)
  • sensor60  11 — Force a TASMOTA time update on each GPS packet
  • sensor60  12 — Stop time updates on GPS packets
  • sensor60  13 — Set TASMOTA lat/long values from GPS positional data
  • sensor60  14 — Start virtual serial TCP server (UBX data) on port 1234
  • sensor60  15 — Pause virtual serial TCP server

USES
LATITUDE/LONGITUDE

Now that we have our commands down pat, we can put the GPS data to use.  The first (and possibly the the most immediately useful) is command 13, setting the latitude and longitude values from the GPS reported values.  This will allow the TASMOTA sunrise/sunset timer functions to work without any manual intervention (that is, even if your timezone isn’t correctly set, having longitude and latitude available from the GPS allows the sunrise/sunset algorithm to calculate those values).  Note that there was an issue with this function in the initial release, so you should use TASMOTA version 8.1.0.9 or above (the link in part-I of this article is to a good, working version).

NTP SERVER

Don’t be confused by this.  TASMOTA already has NTP client capability baked-in (it will listen to other NTP servers to set its own time), but this option, command 9, starts an NTP server process which allows other systems to interrogate your ESP8266 on port 123 for time data (your ESP becomes a network time provider, instead of just being a consumer).

As you might expect with such constrained memory limits, this implementation has some limits.  The stratum level  (the measure of how many levels, or hops, we are removed from the physical source of the time signal) is hard-coded to “2”.  In a normal configuration, the time source itself counts as stratum 0 and the machine connected to it would be stratum 1, which would be treated as a preferred source by downstream clients.  However, our ESP8266 isn’t using  pulse-per-second synchronization of the time data (it’s simply publishing it on arrival), so marking it as a stratum-1 primary source could lead to problems (especially if you happen to be a day-trader).  The good news is that it is still a very accurate source of time data in certain situations.  For instance, if you have an isolated network of IoT devices, accuracy to the nearest second is probably acceptable and having an ESP8266+GPS combination providing NTP services is a low cost, low power option for a network of wireless data-loggers.

There’s another point you need to be aware of if you’re setting this up on a network with other NTP servers present; the ESP needs to be configured as a “server” in the ntp.conf file on the other machines, not as a “peer”.  This is because being a peer requires two-way communications between the machines and this implementation only has the capability of replying to a simple request for a time packet.  If you configure the ESP as a peer, the other machines in the network will ignore it and show it as being stuck in the “INIT” state.  If you configure it as a server however, the other machines will happily request time data from it …although, in my case anyway, the data from the ESP is so far off the time reported by the other systems in the NTP group (that is, local NTP servers on my network as well as “upstream” NTP sources on the internet) that the ESP is almost immediately marked as an “outlier” (with a “-” character in the first column of the “ntpq -p” output) and subsequently ignored.  It could still be quite useful, though.  If you are relying on upstream NTP servers on the internet (which is generally a good thing), then you would lose synchronization if your internet connection was down for any significant amount of time.  Your ESP though, would still be providing a local time source with one second accuracy, which is still pretty good (day-traders excepted).

The load of updating the GPS readout (as seen in the photo above) and providing time critical replies to NTP requests is also a little bit too much for the ESP8266, so you’ll see a message, “NTP Active” appear in that window and the GPS updates will stop while the NTP server process is running.  Once the “Sensor60 10” command is given to stop NTP, the GPS updates will restart and the “NTP Active” message will disappear.

One final wrinkle to the NTP implementation is worth noting (although it’s more of an “interesting characteristic” than a bug).  The “refid” field in the output of  “ntpq -p” command (on another NTP client on your network, not on the TASMOTA device) has two different modes of output.  Normally it will display the IP address of the machine to which the indicated peer is currently listening, but if the peer is a stratum-1 clock source, it will print one of a limited number of strings to let you know to what type of a clock device it is connected (ie:- radio-clock, atomic-clock, etc).  Now because our ESP is directly connected to a clock-like device (the GPS) but, as mentioned earlier, is pegged as a stratum-2 device, the refid output field displays the highly unlikely IP address of “71.80.83.0” (hint:- if you haven’t already guessed why it should do this, try “man ascii” for the answer).

LOGGING

If you happen to take your ESP8266/GPS combo with you when you’re hiking, paragliding or orienteering, then this section might be of interest.  Christian has added the option of logging GPS data to the ESP8266 flash, so you can save your experience for perpetuity (well, you can probably save some of it anyway …there’s not that much flash on an ESP8266).  This is the reason for the very oddly named “#FLOG” setting in the TASMOTA config file (nothing at all, as far as I know, to do with fifty shades of black and blue).  If that setting is un-commented, then flash-logging of GPS data is enabled and commands “sensor60 0” and “sensor60 1” set the log handling mode to “overwrite” or “stop writing when flash is full”, respectively.  Commands 4 and 5 both start the log recording, but 4 starts in append mode, while 5 starts a completely new log, overwriting whatever data was there.  When you start a recording the “Flash-Log” status line in the TASMOTA main menu window (see photo, above) changes from “ready” to “recording”.  Command number 6 stops the recording.

Once you stop a recording, a new, blue button will appear above the GPS information in the main menu.  There is no text displayed on the button (at least there isn’t in either Firefox or Chrome) until you move your mouse pointer over it, at which point the message “Download GPX-File” will appear, flashing on and off in time with the updates to the GPS data (perhaps this works as a normal button in some other browser?).  In any case, clicking on the button will download the flash-log data to your system.

VIRTUAL SERIAL PORT

You can also connect to a virtual serial port on the ESP8266 to get a live view of the (binary!) data streaming from the GPS unit.  The “sensor60 14” command will start this virtual serial connection on port 1234 and command 15 will stop it.  If you’d like to check the data, you can connect to your ESP using:-

nc  IP-ADDRESS  1234  |  od -x

Christian mentions that the intended use for this (other than trying to read hex data very quickly) is to feed into the u-blox “u-center” application, which is a debug and management application, freely available from the u-blox site (recent revisions are for  Windows only, though).

The Bottom Line

Christian has produced a very versatile and interesting application, adding an exciting new sensor option to the TASMOTA family.  He’s not finished yet, either.  The most obvious challenge is trying to get PPS working (which should significantly increase the accuracy of the NTP server function) without making the ESP8266 completely unusable for any of the other options running on it.  Even without that addition though, this is still a useful addition to TASMOTA and is certainly a lot more fun to play with than most other sensors.

If the idea of an ESP interfaced to a GPS module tickles your fancy (as it did mine), there’s also Chris Liebman’s stand-alone ESPNTPServer project, which I’ve mentioned before in these pages and which now has its own PCB and 3d-printed case.

 

Using a GPS with TASMOTA (Part I)

Towards the end of last year, Christian Baars integrated GPS-as-a-sensor support into TASMOTA with the addition of the xsns_60_GPS.ino file and ancillary code.  You can now easily attach a cheap GPS unit to your ESP8266, although just exactly how to connect it and how to communicate with it may not be immediately obvious.

In this two-part series,we’ll look at some of the issues influencing Christian’s design choices before getting down to the nitty-gritty of physical connections, TASMOTA configuration and (finally) uses.


TIP  —  Before we start …if you have a new GPS module‡ which you’ve never used before, plug in the antenna and connect it to a 3v3 supply while you’re reading the rest of this article (it’ll save you a little bit of time and perhaps some frustration, too).  You’ll notice that the LED is off when it’s first powered-on, but after a while (and it could take up to thirty minutes) it will start flashing at 1Hz.  This is letting you know that it has acquired signal from multiple satellites and is now ready to send data.


BACKGROUND

First of all, just a tiny bit of background on Christian’s implementation.  He is using the u-blox binary format (“UBX”) to transfer serial data from the GPS module.  GPS modules from most suppliers, including u-blox, normally come with ASCII format (“NMEA”) data transfer enabled by default, but this format produces long, fairly verbose messages which are not particularly well suited to the ESP8266’s limited memory and single core processor (remembering that the CPU must attend to WiFi housekeeping tasks on a regular basis).  To give TASMOTA the space and time to run its own tasks, Christian initializes the GPS board to use the short-form UBX binary data transfer instead.  Unfortunately, there’s a catch  —  not all GPS modules will support the “UBX” native protocol which this implementation requires.  As far as I know (from reading the u-blox web site),  it seems as though the majority of modules for sale on Alibaba, EBay or Aliexpress are probably fakes in one way or another (older modules being labelled as newer types, or just plain knockoffs with no u-blox chips in them), so it’s no surprise that a lot of them don’t speak the UBX protocol.

My own experience pretty much mirrors this.  Across three different modules, I have one (blue PCB. four-pin connector) which works with NMEA (the most common protocol), but not with UBX, Two GPS modules; one black, one blue.a second (red, narrow PCB, five-pin connector, with PPS¹) which just didn’t work at all and, lastly, a (black PCB, five-pin connector, with PPS) board marked as “WAVGAT NEO-6M” which does actually work using both NMEA and UBX protocols.  None of these modules pass the u-blox label test .  The WAVGAT module came from the WAVGAT Aliexpress shop though, so if nothing else, they may be more consistent than oddball modules from oddball suppliers (please treat this as a comment, not a recommendation).  Christian mentions in the header notes of the xsns_60_GPS.ino file that in addition to a (genuine) u-blox NEO-6M, the Beitian 220 module has also been proven to work.


CONNECTIONS

Okay, so now we know that why not all GPS modules are compatible with this TASMOTA addition and why that is, but how do we connect up our module to see whether it works or not?  The first thing you need to know about the connection is that serial communication with the GPS module from the ESP8266 is via the TasmotaSerial library (a TASMOTA-specific software-serial implementation) and not the ESP hardware serial port.  This means that, excluding the usual suspects of GPIO-15, and GPIO-02, we can connect to just about any available pin.  I’d recommend GPIO-12 (D6 on most ESP8266 boards) to the TX pin on your GPS board and GPIO-13 (D7) to RX on the GPS board.  Black WAVGAT PCB, showing pin connectorNote that it is very, very easy to reconfigure your ESP8266 (through the TASMOTA web interface) to change these pin settings, so don’t worry too much about mixing up TX and RX …it won’t hurt either the ESP or the GPS.  Finally, you need to supply GND and 3v3 connections to the GPS board (do be careful with these… there doesn’t seem to be too much consistency between the different GPS makers when it comes to positioning the power connections).   Those four connections are all we need (the current implementation doesn’t use the PPS signal).

Next comes TASMOTA.  I’m going to assume here that you’re familiar enough with it to be able to flash a binary to your ESP8266 (and if you don’t, there’s a ton of documentation on the web site).  If you’re comfortable building TASMOTA from source, then you should build with your normal “my_user_configs.h” file, but uncomment:-

  • . #USE_GPS
  • . #USE_FLOG

If you’re more comfortable just flashing a pre-built binary, you can download an 8.1.0.9 version built with just the GPS/FLOG defines (otherwise bog-standard “tasmota.bin”) from this link (it’s the “tasmota_GPS.bin” file).  Note that it is an unconfigured binary (just like you’d pick up from the normal TASMOTA release site) and will need to be configured from scratch on your network.  Please also note that the source server is using HTTPS protocol, so it is not suitable for an OTA update pull  …you’ll need to download the binary file and then use the “manual upload” method (assuming your ESP is already running a fairly recent version of TASMOTA).  There’s also a “minimal.bin” firmware image on the same server, but you can just as easily use the minimal image available on either the main release page or from Andre Thomas’ excellent thehackbox.org site (which is an OTA source).


This is part-1 of a two part series.  Click here for part-2.

 


FOOTNOTES

¹ – PPS = Pulse-Per-Second.  Blue GPS PCB showing 220-ohm resistor addition for PPS feedThis signal is used to synchronize the time data output – the hardware equivalent of “At the tone, it will be 3 AM, exactly“.  All of the u-blox modules which I have (including a USB dongle) have PPS outputs from pin-3 of the RF module, but many modules connect it to an LED as an activity/lock indicator.  However, you can easily add a PPS feed to your micro by simply adding a 220Ω resistor on the RF module side of the 1k LED current limiting resistor.  Click on the photo to the left and zoom in on the bottom, L/H corner to see where the resistor is connected.  Note that this module only has a four pin connector (at the top, centre of the board).

† – GPS modules can generally be configured to emit only those “sentences” of NMEA data which you specifically want, however it is still long-form, human readable data, compared to the UBX binary format.

‡ – When you first connect a new GPS module to power, nothing appears to happen.  This is because the module needs to receive a certain amount of data from satellites before it can determine the current time and it’s own location.  This can take up to thirty minutes in some locations.  Once the module has downloaded enough data to initialize, the “PPS” LED will start flashing (even if there are no serial data connections).  The data received on this initial “cold start” is automatically saved by the module itself to battery-backed memory, so on subsequent power cycles the start-up sequence will be much shorter.


 

ESP8266 GPS NTP Server in the making

Chris Liebman is my kind of maker.  His projects are interesting, practical and almost always have the words “work in progress” somewhere in the (usually brief) description.  I was led to the NTP-server project when following up on his (ESP8266-based) AnalogClock creation (a method of keeping el-cheapo analog clocks in perfect time, even across power outages and daylight-savings time changes, using an ESP8266 and NTP).

The NTP-server is another “work in progress” project, with very little information, Breadboard prototype NTP serverother than the code, available.  It does hit a couple of sweet spots for me, though:-  NTP from a GPS PPS (pulse-per-second) signal and being ESP based (although the breadboard picture shows an ESP32, rather than ESP8266).

Chris is using an Adafruit “Ultimate GPS Breakout” board to provide the PPS signal, as well as serial time and date information to the ESP (the GPS communicates with the ESP via the standard serial pins, while debug-terminal output goes to Serial1).

I have to admit that I haven’t managed to get Chris’ code to compile successfully, yet (you’ll need to download some of Chris’ modified versions of several required libraries to get even close), but this certainly looks like one to keep an eye on (and I’m wondering if a lead soldered to the ‘status’ LED on one of those cheap, USB, GPS dongles would work reliably as a PPS signal source in this case).

 

 

A neat toy (but not an ESP)

Here I go with yet another not-quite-ESP-related diversion, but one which the audience of this blog this should find interesting, I hope.

A couple of weeks back I bought a cheap USB GPS dongle from Amazon.

U-Blox USB GPS Dongle

It was amazingly cheap (the Yen equivalent of $8 U.S.) and I wasn’t expecting too much, but I have a yearning to get a local reference clock for NTP set up on my local network and I thought that, if nothing else, this would be a good starting point for fiddling around with GPS before paying for (and possibly breaking) something more expensive.USB GPS dongle, antenna side  I have to say that I’m really, really impressed with this little thing.  I’m sitting in a valley with major mountain ranges to both the east and west of me (~3,000m on each side), so I have a fairly narrow view of the sky, but this little unit acquires and holds nine satellites from inside the house and about 1.5m from the window.

The second thing I did (after verifying that the unit did actually work) was to rip the covers off to verify the make and model of the receiver chip and, as you can see from the photo above, the built-in antenna covers most of the board space on the top side of the PCB (the single, other component, top L/H side, is the green, status LED).

USB GPS dongle, component side

The component side has the U-Blox G7020-KT chip in the bottom,R/H corner.  Note that there’s no FDTI (or equivalent) chip, as the G7020 has a USB interface included.

There’s a back-up battery on the board to maintain memory contents between power cycles (which, it turns out, is critical for reasonable start-up times for GPS receivers).

Unlike another (non-GPS) dongle which I bought recently‡, this unit barely gets warm to the touch and there are no ventilation slots or holes in the plastic cover.

Anyway, for anyone else interested in playing with these beasties, here are some hints to get you started:-

  • The very first time you power on this unit, it will take up to 25-minutes to locate the satellites and lock on. Apart from a very brief flash of the green LED when you first connect it, it will appear to be dead. It’s not! Just leave it connected and it will download the data it needs (the “almanac”) from the satellites it can see and eventually the green LED will start flashing once per second to show that it is now operational.
  • After that initial connection, the unit will come to ready (green LED flashing) in only a few seconds the next time you power on. This is because it saves that initial data to internal memory (with battery back-up).
  • If, on that first power-up, you don’t see a green flashing LED after 30-minutes, it’s because the unit can’t get a decent signal. Move it as close as you can to a window, or if you have a laptop, take it outside. If that doesn’t work, grab a 1m or 2m USB extension cable from your local 100-Yen shop and move your GPS dongle as far away from the computer (and as close to a window) as you can.
  • If you’re using Windows, you can get the genuine Windows driver directly from the chip manufacturer’s web site:-        https://www.u-blox.com/ja/support      -or- https://www.u-blox.com/en/support
  • The chip inside this unit is the U-Blox G7020-KT.
  • If you’re using open-source, you will probably find that your OS correctly identifies the USB device as soon as you plug it in. The application you need (as already noted) is the GPS daemon, “gpsd”. When you install “gpsd” you’ll get some additional helper applications bundled in, such as the invaluable “gpsmon” and “gpspipe”, which allow you to view the GPS data in several formats.
  • The device which you use to connect to the USB dongle will be different between various OSes. OpenBSD and FreeBSD will see it as /dev/cuaU0. Linux will see it as /dev/ttyACM0 or /dev/USB0.
  • To run gpsd you should use the “-n” option (“no-wait” – don’t wait for a client to connect before handling GPS data) and the device. So something like this:- /usr/local/sbin/gpsd -n /dev/cuaU0
  • After starting gpsd you should be able to run gpsmon (just type “gpsmon”) and see data coming from the device.
  • If you want to run your GPS dongle as a local source for the NTP daemon, you still need to start “gpsd” (and it’s important to use that “-n” option in this case). The gpsd process will automatically populate shared memory segments, which the ntpd daemon can then read to get time information (you may have to check your OS for sysctl settings for PPS to get this working). There are notes in both the ntpd and gpsd manual pages on how to do this. So far I have only managed to get ntpd to use the GPS embedded time information, not PPS.

As you can see from the photos above, the board has most of the components (including the battery and the G7020-KT chip) on one side and the antenna on the other. The green LED is on the same side as the antenna, so when you position your USB dongle, you should always have the green LED facing –upwards-. If your USB forces it to be downwards (unusual) or sideways (fairly common), you might want to buy an adapter or extender cable to ensure that you can position it so that the antenna (and LED) are pointing upwards.

 

‡ – A generic, black SDR dongle, bought for the advertised “R820T” tuner chip and specifically for receiving 1090Mhz ADS-B traffic from aircraft flying overhead (see the previous post on “PlaneSpotter”).  Not only does this thing get hot, it -doesn’t- have an R820T tuner chip at all (they use a cheaper and completely useless NC00012 tuner, which has a frequency range which cuts off at 985Mhz, so it doesn’t even cover the ADS-B band).  If you want to play with SDR (and especially ADS-B), save your money and buy a NooElec “Mini 2+” bundle (directly from nooelec.com) instead.  It has an R820T2 tuner chip, plus a temperature controlled, 0.5PPM accuracy oscillator and comes with a “starter” antenna and a remote control unit (??? don’t ask me …I have no idea why!) for $21 U.S. + $5 postage to anywhere in the world.