esp-link revisited

Regular readers may remember my article on hardware implementation (and my associated balls-up) for a serial console using an ESP8266 and JeeLabs’ excellent esp-link firmware.  Well this weekend I had cause to revisit it.  Another fairly ancient, headless server (a Soekris box this time) was in dire need of an update, so I reached for my trusty console adapter.

Except (there’s always an “except”, or a “but” nowadays …have you noticed?) that the Soekris has a console which, at boot-up, defaults to a (very useful and quite speedy, for the time) baud rate of 19,200  …and my v2.0-beta version of esp-link didn’t handle that as one of the selectable baud rates.  No problem!  One of the nice features of JeeLabs’ package (which I mentioned in the original article) is that you don’t even have to compile the code; they very conveniently provide the pre-compiled binaries on their site and, because I already have esp-link running on my ESP, I don’t even need to reach for the USB cable and FDTI adapter,  I can just flash over the air using the built-in utility.  Except…

Instead of doing a “make flash”, I decided to download the new firmware package, which consists of the binary files you need, as well as a little shell-script called “wiflash”.  Wiflash is a little helper script which basically wraps “curl” to do the heavy lifting.  You just give your ESP’s hostname (or IP address) and it’ll go off and do the actual programming for you.  Except that in my case it came back with an error message (from curl):-

HTTP server doesn't seem to support byte ranges. Cannot resume.

…which seemed a little strange, as at that particular point, all it was doing was fetching a 9-byte long file, /flash/next, which contained the plain-text name of the area to program this time round, “user2.bin”, and I wasn’t requesting, or expecting it to have to resume anything.

The curl manual page doesn’t seem to elaborate on whether the -C (–continue-at) option is a default or not, but it does say that prefixing option names with “no-” will disable them, so it seemed worth trying as a quick fix (remembering that today’s objective is to upgrade a system, not debug curl or the esp-link web server), so I simply modified line #82 in wiflash from:-

next=`curl -m 10 $v -s "http://$hostname/flash/next"`

to:-

next=`curl --no-continue-at -m 10 $v -s "http://$hostname/flash/next"`

…and blindly hoped that all of the other calls to curl in the script would just work.

As it turned out, work they did and that one simple change was all that was needed (a quick trip to the “issues” tab on the esp-link git page confirmed that I’m the only one, so far, to have seen this issue, so I’m willing to  believe it’s just a quirk of my fairly ancient OS combined with the old, original v2.0-beta version of esp-link already on the device).

Anyway, a minor problem solved and a nice, shiny, new version of esp-link installed on the ESP, with lots of new features and (almost! ‡) all of the baud rates you could ask for.

Screen shot of esp-link v3 console page

Note that the baud-rate and parity/stop configurations are now on nice, drop-down menus and there’s a separate console input box, as well as selectable history for all of your previous commands.

In addition, on the side panel we have new pages for “WiFi Station”, “WiFi Soft AP”, “Services”, “Upgrade Firmware” and “Web Server”.  While some of that information was available with older builds, the new pages give an uncluttered, neater feel to the application and, for most of the pages, there are new additions to the existing configuration options, giving you, for example, the option to set up a server and timezone for SNTP services.

All in all, a worthwhile update for a very useful utility and, while we’re here, I notice that JeeLabs and Thorsten von Eicken have created a tiny (and much neater) hardware solution than mine, specifically for programming one of their new products.  Neat!

 

‡ 74880 anyone?  (Yes, it’s that annoyingly non-standard squirt of debug info at ESP power-on)

Implementing a wireless console using JeeLabs’ esp-link

JeeLabs have a deservedly popular ESP8266 software package, esp-link which, as with all of the best ideas, is based on a relatively simple concept, a transparent, bi-directional pass-through between serial on one side and wireless on the other.  The main thrust of their development seems to be aimed towards OTA programming of Arduinos, ARMs and even ESP8266’es, but it’s just as easy to implement a “wireless console” for a remote device and, because JeeLabs have done such a good job of creating and packaging their software, you don’t even need to compile anything; just download their package, use esptool to load it to your ESP8266 and then connect to the device over the air to configure it.  Fantastic (especially for those of us who are more comfortable with hardware than mucking around with toolchains and eagle.app.v6 files)!

Note:-  If you’re going to configure your ESP8266 device as a transparent network-to-console link, you’ll need a version with enough pins.  The ESP-12 variants are good bets for this.  You can forget trying to use the ESP-01.

What started this quest was the need to update an older headless box (a mini server without a built-in frame-buffer) and the realization that I no longer had a working machine with a serial port to connect to its console.  Like everyone else in the ESP community, I’d seen JeeLabs’ project and was impressed by their detailed write-up, so it suggested itself as both a fun project and a solution to a pressing problem.

First of all, I knew that one thing that differed from the “standard” connect-to-an-Arduino scenario was that I would be dealing with RS232 voltages, not TTL, so whatever else, I was going to need a level shifter.  Time to troll through the spares box and look for anything with “Maxim” stamped on the top.  I eventually came up with a single, unused MAX232A.  This is the version which only needs 0.1u capacitors, but has a 5v TTL interface on the inner-facing side.  Never mind, I’ll just have to throw a level shifter in there between the 3v3 ESP and the Max (this, as it turned out, would be problematic).  Other than that, I had everything else immediately to hand:-  a WROOM-02 on a project board with extra space available, the D-type connector (and even a shell) and a length of ethernet cable (saved for just such jobs after the end connectors had both lost their lock-tabs).  We were good to gStripboard with ESP8266, Max232A and level-shifterso!

I already had a couple of 2N7000 MOSFETs working as level shifters on the WROOM-02 side of the board (my USB-to-serial adapter is only 5v capable) and I added another couple next to the Max232 chip.  The odd “transistor” close to the centre edge of the board is actually a DS18B20, which was already connected up to the ESP, doing duty as a battery operated remote sensor.

One of the many neat aspects of the JeeLabs package is that the serial pins used to communicate with the target can be reassigned.  This is useful if you want to use the standard port for programming and debug, but critically, it also means that you don’t have to worry about the power-up garbage, which the ESP spews out on the standard serial port, corrupting your data stream.  As mentioned earlier, you also don’t need to re-compile to make this change; you just blow the firmware files they ship with their package to your ESP and then reconfigure it from a web browser once it comes up.

Here though I hit a problem (and I need to emphasize here that it was -my- problem and nothing at all to do with the JeeLabs package).  Whatever I did, when I first plugged my newly modified hardware into the USB adapter and fired up esptool, I just couldn’t get it to program.  I went back to using the Arduino IDE and a totally different software package …just the same.  Although the serial monitor showed what appeared to be the normal power-up crap at-a-baud-rate-no-one-else-in-the-world-can-use, it just would not program, “Connect failed”.  Well, duh!

After double checking my connections and verifying the same with the meter, I went back to the “Square Doughnuts” fault-finding procedure  (U.K. techies of a certain age will probably know what I’m talking about) and asked myself “exactly what has changed?”.  Well, obviously there’s a Max sharing the bed with the ESP now, and it connects to the ESP through a couple of MOSFET level shifters and, because I don’t want the power-up crap (at-a-baud, etc), I’m connecting it to GPIOs 13 and 15.  Ding!  Ah!  Everyone knows that GPIO-15 is super special.  Thou shalt tie it to ground at switch-on, or verily there wilt be tears!  If you’re familiar with the normal MOSFET level shifter, you’ll recall thatBasic MOSFET level-shifter the gate of the MOSFET is tied to the 3v3 rail and there’s a pull-up resistor between the gate and the source, with the source being connected to the ESP pin (GPIO-15 in our case).  Oops! Another double-Duh! moment.  By habit, I normally tie-up and tie-down using moderately low value resistors.  In this case I’d thrown in a 4k7 between GPIO-15 and ground, but there was now a 12k (“R2” in the level-shifter schematic to the left) sitting between it and 3v3.  According to our mutual friend Georg Ω, that would put the voltage on GPIO-15 very marginally into the grey area between acceptable limits for logic high and low.  It shouldn’t be enough to cause a problem, but it is borderline and obviously was causing a problem in this configuration.

One option might be to add another resistor between the gate and the 3v3 railModified level-shifter circuit and decouple it with a large-ish value capacitor (and a reverse-biased diode across the new resistor for fast discharge of the cap at switch off), which ensures that the MOSFET is clamped in the off-state at switch-on and slows down the voltage rise across the (existing) voltage divider network to ensure that GPIO-15 is pulled to ground. Once C1 has charged up through R3, the level-shifter will function as it did before.  This worked fine for me, however the real answer is to not be a cheapskate and use the proper 3v3 version of the Maxim RS232 converter chip in the first place, so that you don’t need level shifters at all.  The Max3232E is a pretty good bet (and seems to be readily available), but there are now lots of other chips withCircuit for MAX3232E - Schematic capture courtesy of CadSoft EAGLE 3v3-to-5v functionality, so you  might want to do a little sleuthing (and price comparisons) before choosing.

Firmware – As mentioned previously, the esp-link package comes with pre-built binaries (in the “firmware” directory), so you don’t need to go through the whole build process yourself.  One thing you probably do want to do though, is to make a very small modification to the supplied Makefile to prevent the “make flash” command from triggering a full rebuild automatically.  To do this simply remove the “all” dependency from the “flash:” target at around line 384 in the Makefile.  Look for this block of code:-

flash: all
$(Q) $(ESPTOOL) --port $(ESPPORT) --baud $(ESPBAUD) write_flash -fs $(ET_FS) -ff $(ET_FF) \
0x00000 "$(SDK_BASE)/bin/boot_v1.5.bin" 0x01000 $(FW_BASE)/user1.bin \
$(ET_BLANK) $(SDK_BASE)/bin/blank.bin

That first line, “flash: all“, changes to just “flash:“.  This tells make to use the existing files in the firmware directory and not to try and update them by recompiling in your environment.  Note that you may have to update the path to “ESPTOOL” (at around line 64, at the top of the Makefile), so that make can find your particular installations’ esptool.py script if it happens to be in a different location from the default.  After that you’re good to go.  Connect up your ESP8266 as normal for programming and just do “make flash“.

Once programming is complete you can restart your ESP in normal (non-programming) mode and it will automatically bring up the ESP in STA+AP mode (normal client plus access point), with the access point named something like “ESP_B3769F”.  You need to connect your WiFi to that new access-point to configure your ESP.  Once connected to the access point you can point a web browser at “192.168.4.1” to access the configuration menus.  The initial screen looks like this:-

esp-link in STA+AP mode. Top page.

 

The “Pin Assignments” section on this initial page is where we’ll change the UART assignments for the console connection to our PC  (it’s important to understand that this does not change the debug output from the ESP8266, that data will still be squirted out on the normal UART-0 pins;  what it does is to redirect the “transparent” link between your network connection to the ESP and the RS232 hardware to use UART-1 on GPIO-15 and GPIO-13).  Note that there are a few different options in the pin assignment list, but only one of them (highlighted at the bottom) implements this “swap_uart” connection on alternate pins.

Option to select alternate UART pins

The other menus and set-up for configuring the device are all explained in detail in the GitHub read-me file for the project.

The baud-rate setting for your RS232 console must match the PC that you’re connecting to and that setting is at the very top of the page under the “uC Console” tab.

The “Debug log” tab shows the output from the ESP8266 console (not the RS232 device) and can be handy for initial troubleshooting.

One extra, handy function is the ability to have the ESP8266 log information to any MQTT server you have configured under the “REST/MQTT” tab.

esp-link board connected to an Alix2D3

So, after all of that, does it actually work? Heck yes!  And rather well, actually.  To the left is a photo of the board connected to an Alix-2D3 (from PC-Engines) while I re-installed the software on the system (the Alix-2D3 is a headless router/gateway).   I connected the esp-link board to the console port and then, from a laptop, telnet-ed into the ESP’s address and had immediate console access (after struggling to remember the baud-rate on the Alix for a few minutes I even had console access that I could read!).  The initial installation and configuration were all done via esp-link (and it looks rather as though I’m going to have to keep this little board in my toolkit for future, headless connections).   [Aside – If you look very closely at the photograph above, you’ll notice the additional components for the programming modification around the rightmost 2N7000, next to the MAX232]

After completing the installation I went off and completely forgot about the esp-link board being still plugged in.  I came back to it 4 days later and found that the batteries were still okay and it was still quite happy to communicate via telnet, which quite surprised me (I assumed that the batteries would have been flat after a day, or maybe two at most).  Anyway, if you have a room full of servers and want a permanent console connection to each one, then you’re going to have to provide a reliable supply to the ESP board (and the ITEAD Studio PSU which I mentioned a couple of weeks back looks like a good candidate at only $2.50 a pop).

Bottom line…  A big “Thank You!” to JeeLabs for their esp-link software which helped me out in a very practical way as well as providing an entertaining and worthwhile hardware project.

 

Ω – “That’s Dr Ohm to thee, lad!”.