ESP8266 Overclocking & Speedometer

A few days ago I posted the code for a(n almost unusable) version of Zork (or actually, the Zmachine) for the ESP8266.  It was (and is) really, really slow.  The main problem appears to be that it constantly re-writes the Zmachine stack back to SPIFFS which, in addition to causing the running-through-chest-deep-molasses effect, will also wear-out your flash in double-quick time.  The fix for this (obviously) is to shoehorn the whole thing into main memory (which is a work-in-progress), but while trying to squeeze enough performance out of the ESP8266 to make this early cut at least somewhat playable, I thought I’d take the easy option first and run the ESP at 160MHz instead of the default 80MHz.  I should warn you right now that as far as making Zork playable went, it was a total failure, but I was pleasantly surprised at just how easy the speed selection is when using PlatformIO.  I didn’t even have to sacrifice a chicken.

Here’s the simple incantation, which you just need to add to the bottom of your platformio.ini file in the project directory (the comment line is optional):-

; Set the ESP8266 clock frequency to 160MHz
board_f_cpu = 160000000L

…and then just recompile (and, because you’ve just made a change to the platformio.ini file, PlatformIO itself is smart enough to know that it needs to do a complete rebuild, not a partial).

That’s it …you’re done!


 

NOTE – You can also use the system call “system_update_cpu_freq()” to dynamically update the ESP8266 clock frequency from within the program itself, instead of using the platformio.ini “board_f_cpu” setting.  Using the system call will override the compile-time setting.

CODE – Ray Burnette’s adaptation of the Dhrystone test program is available from my repository in both static and dynamic versions.  The static version simply loops endlessly, reporting the performance of the current module (so effectively the compile-time clock setting from the platformio.ini file).  The dynamic version toggles the clock speed using the system_update_cpu_freq() system call on each iteration and displays the performance for whichever the current setting is.

 

 

Getting started with PlatformIO and the ESP32

Here’s the shortest “Getting Started” you’ve ever seen (disclaimer†  …I’m making the huge assumption that you already use PlatformIO as your development environment for your ESP8266 projects.  If you don’t, you should!).

Add support for the ESP32 with:-

platformio platform install espressif32

Create your new development (project) directory and, in that new directory, initialize the environment for the type of board you have‡  with:-

platformio init --board=esp32dev

Start writing code, as normal, in the newly created src directory and then compile with:-

platformio run

At this point, PlatformIO will go off and automatically download the framework support for your environment (this first time, only) and then compile your code.

You just can’t get any easier than that!


† I don’t actually have an ESP32 board yet.

‡ List the available target boards with “platformio boards

Ψ If you’re tired of typing “platformio” in full each time, you can shorten it to “pio” (“platformio” is used for clarity here).

ω For more information on getting started with PlatformIO, see the full documentation at:- http://docs.platformio.org/

PlatformIO and JSON woes…

Yesterday was, for the most part, quite a good day for me.  The weather was bad, so it gave me an excuse to stay inside and play, at least for a couple of hours in the morning.  I’m still slowly working away at implementing a simple, forced-air heat distribution system in the house (our only permanent source of heating is a wood-stove in the lounge/kitchen area, so I’ve added some ducting to take hot air from behind the stove and gently and quietly push it out through a vent in the bedroom).  The latest idea is that I’ll have an ESP8266 with a small screen and a DS18B20 (or similar) in the bedroom which will display the temperature and allow manual on/off override of the fan control via a couple of buttons.

Anyway, I’ve had one of those 2.2″ TFT displays sitting around for ages waiting for me to get going, so early yesterday morning I wire-wrapped it to one of the spare “Yellow Development” boards that I have hanging around and then looked around for something to burn into the ESP as a quick test.  I remembered Squix’s weather station and vaguely recalled seeing a TFT version too, so off I went and grabbed it from his GitHub repository.  After a couple of false starts (what is it with Windows and non-case-sensitivity?!?!) it fired up beautifully and (almost) everything sprung into life (be warned, this application uses a lot of flash memory space to store image files, so you might not be able to fit all of the moon phase .bmp files onto your particular ESP).  The display was impressive enough to divert me into going to the Weather Underground site and signing up for a developer’s API ID, just so that I could play with the application a bit more.

So after duly playing around for a while, I noticed from the comments section on Squix’ page that a couple of people had already added some new features, all of which looked interesting, so off I went to GitHub again and cloned Keith Fowler’s latest and greatest version.  This is where the fun started and my free time went down the plughole (and none of the palaver related below is in any way Keith’s fault, I hasten to add).

I’m using PlatformIO as my build environment (it generally works flawlessly and automatically does neat stuff, like shoehorning everything into memory, or finding your FDTI adapter, without having to be told).  When you need libraries you can just do a word search on, say “DS18B20” and then choose the most suitable looking candidate from the list returned.  Keith’s updated version of the application I was trying to install was looking for “simpleDSTadjust.h”, so I duly typed “simpleDSTadjust” into the library search, got one hit and installed that library.  The next run through produced exactly the same error message as previously, “simpleDSTadjust.h not found”.  Having been bitten once earlier in the morning with a case-mismatch between the #include line and the actual file name (just for refererence, “ArialRoundedMTBold_14.h” as opposed to (ArialRoundedMtBold_14.h”), I went back and scanned the #include and file name with a magnifying glass.  Nope, they’re both the same.  Hmmm….

I quickly looked at the library properties files, library.json and library.properties and discovered a minor typo with the repository URL in the .json file.  A quick check with the spec’ showed that this was a required field, so I corrected it and tried again.  Nope, same error.

Ran the platformio “run” (compile) command again, this time with the -v option.  I got a lot more information on the libraries which it had found, but nothing at all on the missing “simpleDSTadjust.h”.   Next, I went to GitHub and found the repository for simpleDSTadjust.  The most recent changes were to the library.json and library.properties files, but the GitHub diff showed only a single character change in both cases; the version number had changed from 1.0.0 to 1.1.0, unlikely to cause such a problem, I thought.

At this point I was beginning to suspect that maybe there was some sort of bug with PlatformIO that perhaps the length and composition of this particular library’s name was triggering.  On a lets-just-try-it-anyway hunch, I updated my platformio.ini file with a line to change the behaviour of the library dependency finder — lib_ldf_mode = deep+.  That didn’t help at all (so much for hunches!).

Next I removed the simpleDSTadjust library from the PlatformIO local cache (.piolibdeps) using the “lib uninstall” command and then downloaded the GitHub version to the local “lib” directory (normally used for your own, locally created libraries which aren’t available in the global, PlatformIO library system), giving it a different, shorter name.  I  updated both the library .cpp and .h files, as well as the #include lines in the TFT application source, to match the new name.  And …exactly the same.  PlatformIO ignored the newly created library and repeated the boring old “not found” error.  This was starting to get a little tiresome.

Working on the assumption now that the problem lay with the library itself and not, after all, with PlatformIO, I went into the library directory and moved all of the files which seemed to be non-essential into a newly created sub-directory.  The next compile actually worked.  It did fall over on some other errors, but we’d got past simpleDSTadjust.h for the time being.  Okay, what was the actual culprit?  Moving the non-essentials back, one by one, then re-running the compile proved the the library.json file was definitely at fault, but staring at it didn’t yield any obvious clues, so back to the on-line manual pages and the examples to see what a good file should look like (I’ve never knowingly created a .json file, so I definitely classify as a bona-fide novice).  It all looked fairly simple …quoted text, colon delimiters between defined name and data and comma delimiters at the end of data lines, except for the last one.  Okay, squint, squint, squint.  Nope nothing obvious.  Back to the examples — curly braces for multi-field data, comma after the closing curly bracket.  More squinting …and finally an answer; that comma delimiter after the closing curly is missing.  I quickly edited the file to include the errant comma and …Yay!  A working compile.  Finally!

Phew!  A couple of hours of precious free time wasted (but only because I was stupid enough to pursue the issue at hand instead of getting on with the actual project) mainly because there was no error message displayed (or perhaps generated) when parsing of the library.json file failed because of a syntax error.  Anyway, the author of the simpleDSTadjust library (Neptune2)updated the library.json file within a few hours of the issue being raised, so a very big thank you to him/her for that.

Now I suppose that I should buckle down and get back to work on that simple temperature display, now that I’ve proved that the hardware works.  🙂

 

PlatformIO, the ESP8266 and an ancient Linux version

A couple of the sites which I list in the links section (see the menu to the right) have been singing the praises of PlatformIO in connection with ESP8266 development recently.   Squix actually goes as far as propose using PlatformIO as part of a “Continuous Delivery” workflow for the ESP8266 (no, I didn’t know what it was either, but apparently it entails adding automatic testing to verify that a given release meets certain criteria before being released to the target device), while Jan Penninkhof was so impressed with PlatformIO that he put together a very nice “How To” video on getting started with it on the Mac.

But hold on, just what is PlatformIO and why would I want it?  PlatformIO is a build system; it’s like “make” on steroids, as implemented by Superman, with help from Jethro Tull†.  If you want to build a project for the ESP8266, PlatformIO will help you to do it by (for instance) downloading the toolchain automatically for you.  If you decide you’d like to build for two disparate architectures (lets say ESP8266 and TeensyLC) you can just let PlatformIO know that at initialization time and it will magically download and install both toolchains and support when you first attempt to build the project.  If you include libraries (via the normal “#include <whatever.h>” method) in your code, PlatformIO will help you find the latest version and automatically download any dependencies which your chosen library may have.  Basically, if you’re a newcomer to the ESP8266 world, this is the easiest way to get started compiling and flashing.  Don’t bother trying to follow the (mostly conflicting) instructions on the 50 odd (out-of-date) web sites on how to install the toolchain, or how to configure esptool for flashing, or how to set-up the Arduino-ESP environment… just install PlatformIO and let it do it all for you.

All in all, it looked like something worth trying to me, especially after Squix and Jan had explained some of the features in plain language (which is wonderful, because  buzzword-laden sentences containing “framework”, “integration”, “extensible” and friends put my brain into deep-freeze mode).  My main issue with anything “new” like this though, is usually my ageing primary work machine.  I’m running a fairly ancient version of Elementary OS, which is based on Ubunt 12.04.  Attempting to install anything written after the Magna Carta on this steady old workhorse usually results in a raspberry from the speaker as well as the normal, copious error messages about the kernel, libraries, disk spindle bearing and user all being past their “sell by” date.  I’ve given up even trying to install anything which mentions “Java” anywhere in the description.  PlatformIO though, is written in Python and although I’m not a Python programmer, I do know that the version installed on my system is quite a recent one, as the ESP8266 loader program required it.  A quick check with “python –version” showed that it was actually 2.7.3.  Almost all versions of Linux have some type of updater attached to the package-manager,  so you should be able to update your system (no matter how ancient) to a recent revision of Python without too much effort.

Moving on to getting PlatformIO working, the next step is even easier.  We now need to use the Python package installer, “pip”,  to do the grunt work of adding PlatformIO itself.  You can check to see whether “pip” is already available on your machine using a command such as “which pip” or “whereis pip“.  If “pip” exists, they should return the fully-qualified location; something like “/usr/bin/pip“.  If it doesn’t exist, you’ll need to use the system package-manager to add it (which might be as simple as “sudo apt-get install pip“) to install it.  Assuming that “pip” is now available to you, simply do:-

pip install -U platformio

Note that the “-U” option comes after the “install” command.  It forces pip to update the install target to the latest available version.   And that, basically, is that!  You now have platformIO installed on your ancient Linux system and it’s going to do all of the rest of the heavy lifting for you.

Now that you have it installed, you can create a work directory for your projects.  I just made a PlatformIO directory directly under my home directory as an easy way to get started, but basically you can call it whatever you like (“work”, “projects”, “Eric-woz-ere” will all work equally well).  What we’re going to do now is to create sub-directories in the work directory for each different project that you build.  This is good practice anyway, but PlatformIO makes it almost essential, as it supports a metric ton of embedded processors (as well as their different toolchains), so one project might be built for the ESP8266, while the next might be for the STM32-Discovery and the one after that for the TI MPS430.  PlatformIO will quite happily build each of your projects for a different processor (or even for multiple, different processors) if you want it to.

Okay, so to start off, we’re going to build a simple door-bell project for the ESP8266.  Basically, this is going to be a battery powered project and, because it’s housed inside an enclosure attached to a wall, we want to enable OTA programming to allow us to update it without ripping the enclosure apart each time.  We have some basic code already available which has been developed (to date) in the Arduino-ESP environment.  Lets start…

mkdir DoorBell  && cd DoorBell

mkdir ./src

cp ~/Arduino/DoorBell/DoorBell.ino ./src

cp ~/Arduino/DoorBell/user_config.h ./src

So, we’ve created the “DoorBell” sub-directory, changed into it and created a “src” directory, then copied over our code (and an include file) from the existing Arduino build tree into the new “src” directory.  Now we need to tell PlatformIO which target processor we want to build this project for.  You can get a (very long) list of the available targets using the “platformio boards” command.  You’ll need to scroll up through the output to find the  “Espressif” section.  Most of the common ESP boards are in the target listing, including things like the Huzzah and NodeMCUv2; there are also some that I’d never heard of, such as the Phoenix and WifInfo (which seems to be this).  I’m actually using the “esp12e” setting for my project (even though it’s just a generic ESP12 processor), with no ill effects.  To implement your board choice and have platformio write it’s own, minimal configuration file for this new project, use the “platformio  init  --board=phoenix_v2” command (which, in this example, is setting the board type to “Phoenix V2” …replace this with your own choice).

The init command (above) will create a bunch of files and directories under your newly created project directory.  The main ones are:-

  • platformio.ini      – The configuration  file for this project instance.
  • src      – The directory where PlatformIO looks for the source code for this project.
  • lib      – The directory where PlatformIO will look for custom, local libraries which you might add.

We already created the “src” directory in our initial steps, above; that’s okay though, the “init” command will not overwrite an existing directory.  The other contents of this newly initialized project directory can change, depending upon (for instance) which IDE you choose to use.  As an example, if we tell PlatformIO that we want to use the “Eclipse” IDE, using “platformio  init  --board=phoenix_v2 --ide=eclipse“, we will end up with an extra “.settings” directory and “.project” file, both of which contain template information specific to the Eclipse IDE.

The platformio.ini file is fairly sparse at initialization time, but it is important.  For instance, the default framework environment for the Espressif processor is “arduino”, but by editing your platformio.ini file you could change this to “simba” (that’s actually the only other valid framework available for Espressif targets).  If you now start a compile with “platformio run”, PlatformIO will immediately go off and automatically download and install the Simba framework package for you, before kicking off the build of your project with a totally different framework (if you try this, please be aware that not all of the Espressif boards are supported by Simba, whereas most of them seem to be fine with the default Arduino setting).

This semi-automatic mode of operation is an intrinsic part of PlatformIO and does make life easier for the novice user.  Another good example is when it comes to uploading your newly built code to the target board.  When we start off, we don’t generally have any other option than uploading via serial.  To kick-off the upload process, plug your ESP board into your computer, as normal and then use the command:-

platformio run -t upload

PlatformIO will go off and automatically detect your serial device (this works very well if you’ve already been using your system for programming an ESP board …if you’re starting from scratch in a clean environment, you might have to check the 99-platformio-udev.rules file settings – this is Linux-specific BTW), and  upload to the ESP.

That’s it.  You’ve just built and uploaded your project using PlatformIO.

Once your ESP board is up and running and connecting to the network, you can very easily change to using the Arduino-style OTA upload method by going back and editing the platformio.ini configuration file.  Just add the line:-

upload_port=192.168.1.212

(where “192.168.1.212” is replaced by the actual IP address of your ESP board) to have PlatformIO alter its behaviour to the “upload” command to use OTA instead of serial.  Of course, you need to add the Arduino OTA support code to your project file too, before this will work (see the example file BasicOTA.ino for the lines of code you need to add …the basic OTA functionality is already incorporated into the PlatformIO/Arduino libraries).

A word here on libraries.  Normally we use libraries created by others to add functionality to our projects.  A good example of this would be the PubSubClient library for MQTT.  As you might expect, PlatformIO provides a method for handling third-party libraries like this and, naturally enough, one of the first things a new user needs to do is find the libraries needed for a given project.  We can query PlatformIO for the MQTT libraries it knows about using:-

platformio lib search MQTT

This will return a numerically indexed list of libraries matching the keyword “MQTT”.  Currently this returns 13 hits on my system.  As it turns out, Nick O’Leary’s PubSubClient library is the first one in the list, with the reference number “89”.  Okay, lets get PlatformIO to download and install it:-

platformio lib install 89

Note that we use the reference number, not the name.  PlatformIO downloads and installs the library and adds it to it’s own database.  What’s nice about this is that it will check every week or so to see whether the library has been updated recently and prompt you to refresh your installation if it finds that you’re not running the latest revision.

As you might expect, PlatformIO also runs the “latest and greatest” revision check on itself and will also prompt you to upgrade if you’re not at the latest rev.

I’m impressed!  Since trying PlatformIO for the first time a couple of weeks back, I’ve been moving everything over to it.  No more messing around with the horridible Arduino IDE.  Yay!

 

 


† – The man, not the group.  Implying that it does some basic, but essential stuff for you, not that you’re as thick as a brick.