ESP32 + W5500 — Simple Working Example


NOTE [Nov 2020]  —  It seems as though the “June 2019” fix (immediately below this paragraph) is no longer required.  I downloaded the project directly from the GitHub repository and compiled with the latest version of PlatformIO (5.0.2) a few days ago and it compiled, uploaded and worked with no changes at all, so I’d suggest you try it and only use the following fix if you see the “invalid abstract” errors.

Update June 2019  —  A couple of people have mentioned (both here and on GitHub) that they are having difficulty compiling the code for this project, with “invalid abstract return type” errors.  This turns out to be an upstream problem caused by a non-backward compatible modification of the Espressif  arduino/ESP32 library.   You can force the build to use an older (compatible) version of the library by changing a single line in your platformio.ini file:-

—  platform = espressif32
++  platform = espressif32@1.6.0

Thanks to @maniekQ for documenting this workaround.


Over the past couple of days, in the course of answering questions on the popular article on adding an ethernet port to the ESP8266, I found myself putting up a link to some new code for a work-in-progress project which simply replaces the ESP8266 with an ESP32 (which seems to make a lot of sense, given the falling cost of the ESP32 modules, nowadays).  ESP32 + W5500 moduleWhile that project is for an ESP-Now gateway, it seemed like there was a need for a nice, simple test and verify project where people can do a minimum of work with the hardware (an ESP32, a W5500 module, some jumper leads and a breadboard) and get a working result in a reasonably short time.  I’d said in various places that it shouldn’t take too much work to modify the code for the ESP-Now project to handle any of the examples shipped by default with the Arduino Ethernet library, so that’s what I’ve done.

Here’s the code (along with the pinouts in the README) for the simple “UdpNtpClient” example, munged very slightly to work with the ESP32Output exampleAll it does is connect to an NTP server, retrieve the current timestamp and display the UTC time.  This is basically a 30-minute project to produce a working demostration of an ESP32 using hard-wired Ethernet.

The configuration uses a static IP and network setup (router/gateway, netmask and DNS), as the original ESP8266 project seemed to have problems with DHCP (and quite honestly, I just haven’t gotten around to trying it with the ESP32 version, yet …let me know how it goes if you do).  All of the configuration options are in the “local_config.h” file.

The original library example code uses a single NTP server, “time.nist.gov”, which seemed a little anti-social to me, so I’ve added several of the more popular geographical pools into the config file and updated the default target to be the main “pool.ntp.org”.   You should choose the one closest to you (unless you’d like to see how unreliable bare UDP really is, in which case you might like to try “antarctica.pool.ntp.org”  —  apologies if you’re reading this from McMurdo Station 🙂 ).

38 thoughts on “ESP32 + W5500 — Simple Working Example

  1. This is really great. Thank you for this article and code on GitHub. This helped me to get Ethernet running on ESP32 DevKitV1.
    I am using this setup to send and receive messages through MQTT, with MQTT PubSub library (https://github.com/knolleary/pubsubclient).
    However, for some reason, it gets disconnected every few hours. ESP32 stops receiving and sending new messages from MQTT. Ethernet client “connected” still returns “1”, but no MQTT messages are correctly sent. After a while it reconnects and keep working fine for few hours again.
    Have you seen any problems with connection stability like this?

    Liked by 1 person

    • @ManiekQ,

      Many thanks for your kind words (and apologies for my late reply …real life gets in the way again).

      I had the prototype sitting in the middle of my work desk for about a week and a half (before I had to move it to one side for the next, unrelated, great thing) and didn’t notice this issue. I do have two questions, though… Are you using a broker on your local LAN, or a remote machine (see McMurdo comment in the last paragraph of the article) and do you by any chance have an older OpenWRT (or similar) router on your network (I had an older device on my network which would occasionally lose ESP ARP entries for some unknown reason).

      Anyway, if I get some time free time (which probably won’t be until the second or third next rainy day (don’t ask!)), I’ll get it back out and fire it up to talk to the broker, instead of an NTP server.

      -John-

      Liked by 1 person

      • Thank you for taking your time to reply.

        I am using MQTT broker set up in the local network. My router is Asus with the latest firmware. I will look into the router logs to see if I can spot anything interesting.

        Inspired by your answer, I’ve created a very simple project which only subscribes to single MQTT topic and publish periodically to another topic.
        The problem also occurs within this simple project, so (unless I’ve done some stupid mistake in this simple code) I assume it is either something on my network or bug in one of the libraries.
        If you are interested, the project can be found here: https://github.com/maniekq/ESP32_W5500_MQTT .

        Mariusz

        Like

      • Mariusz,

        Thanks for that. I’ve cloned your project, so when I get some time I’ll load it up and start it running.

        Did your long messages make any difference to the repeatability of the problem, by the way?

        -John-

        Like

      • I’ve got a feeling that these long messages increase the frequency of the connection failures, but I am not sure yet. Occurrences of these failures looks very random to me. Once it was running for six hours without a single connection problem and the other time it has failed few times in an hour. I keep running those tests, so hopefully I will know more soon.

        I have also looked at my router logs, but I haven’t found anything interesting. I’ve set IP address per MAC static, just to be sure it is not getting in the way.

        Like

      • After running tests for days, it looks like ESP32 gets disconnected more often when sending long messages to MQTT (these messages are added here https://github.com/maniekq/ESP32_W5500_MQTT/blob/master/src/main.cpp#L112-L117)

        With mentioned lines commented out code worked once for around 24 hours without ESP being disconnected. Still problem occurred from time to time, but less frequently.
        When sending long messages, ESP got disconnected at least every few hours (often it was few times an hour).

        Mariusz

        Like

      • Have you had a chance to run the project on your setup?
        Sorry for bothering you, but I am sill fighting with this problem and I am loosing hope for solving this.

        Like

      • Mariusz,

        Sorry, I’m just well and truly snowed under with work at the moment (hence the lack of new content).

        Have you tried monitoring the connection with tcpdump or wireshark to verify what the failure is? If you didn’t already do it, you might also want to throw in a couple of calls to give you some idea of free memory, boot count and last reboot cause and then “tee” the console output to a file, so that you don’t miss anything.

        Are you using Mosquitto’s LW&T to monitor the ESP outages?

        -John-

        Like

      • …and thinking about it a little more, it may be worth recompiling your PubSubClient library with MQTT_MAX_PACKET SIZE set to something larger than the default 128 bytes (PubSubClient.h) and checking that your publish string doesn’t exceed it.

        Like

      • Thank you for all the good ideas and hints.
        I am monitoring free memory in ESP32 and look for reboots, but nothing wrong happens there. I don’t think I am hitting MQTT_MAX_PACKET SIZE – my messages are not that long, but I have increased it just to be sure.

        I’ve started monitoring connection with Wireshark with some interesting outcomes. I can see that at the time problem occurs, there is a weird packet send from ESP32 which is interpreted by Wireshark as “MQTT Disconnect Req” with a very high “Msg Len” (8186). My understanding is that all the correct packages/messages send by ESP after that packet, are considered as continuation of the “MQTT Disconnect Req”. It looks interesting to me, that this faulty “MQTT Disconnect Req” message looks almost the same every time the problem occurs, so it is not random gibberish (Msg Len is always 8168, content after MQTT message type is always the same looking at HEX).
        I’ve added file with captured packages to GitHub repo in case you’re interested: https://bit.ly/30aveb9 .

        MQTT broker (Mosquitto) doesn’t even see this “Disconnect Req” message, but ESP32 finally (after 30 seconds since the last correct communication) gets disconnected, because of “keep-alive” expiration.

        I think I will keep digging in MQTT and Ethernet libraries code, to understand why this “Disconnect Req” message is sent from ESP32.

        Mariusz

        Like

      • Mariusz,

        I’m guessing that packet 135 is where the MQTT disconnect is being flagged and it is kinda’ strange that we see the ESP using [PSH, ACK] from that point, despite the fact that it is still publishing exactly the same message sequence as before. I think you’re looking in the right direction with the libraries. It feels to me (no more than a hunch) that the ethernet driver may be reacting to a buffer overflow on the MQTT side.

        -John-

        Like

  2. Hi! I’m trying to do a project with a endpoint server to receive the data from sensors as a Json. But I want to use the WiFi to collect the data and send it with the ethernet, at the same time (maybe with a task schedule). Do you know if it’s possible, use the WiFi and the Ethernet at the same time? Now I’m using the esp8266, but I want to buy a esp32 to test because of yours 2 processors cores.

    Liked by 1 person

    • David,
      I think you need to investigate just how much latency you can live with in this process. Although neither the 8266 nor the 32 will run large numbers of concurrent processes in the way that something like the Raspberry-Pi can do, both of them are quite capable of handling the transfer of data between WiFi and ethernet fast enough to appear to be simultaneous (especially if the incoming data is very limited in size, as sensor data tends to be). The Espressif “ESP-Now” protocol dramatically reduces the amount of work both the sensor nodes and the ethernet gateway need to do when handing off the data from WiFi and then using something fairly lightweight like MQTT on the ethernet side helps keep those transmissions reasonably short, too. Your ESP should have absolutely no problem keeping up with dozens of sensors, if they’re only transmitting occasional data readings. You could increase that number by sending raw data packets (and passing the JSON conversion off to somewhere downstream of the MQTT server) and by having a simple tuning algorithm on the gateway to dynamically change the reporting time of each sensor module as your network gets more crowded (“Hey sensor, your next reporting slot has moved to xx:xx:xx”).

      -John-

      Like

  3. Hi John,

    I am currently getting this:
    Ethernet-Based MQTT Gateway v1.0

    This node AP mac: 02:60:0D:F0:0D:02
    This node STA mac: 18:FE:34:E1:34:93
    Starting Wiz W5500 Ethernet…
    Ethernet IP is: 192.168.0.11
    Ethernet local IP is: 0.44.44.44
    Attempting MQTT connection…
    Soft WDT reset

    >>>stack>>>

    ctx: cont
    sp: 3ffffb90 end: 3fffffc0 offset: 01b0
    3ffffd40: 00000401 00008000 3ffee604 40202bef

    I have my Ethernet connection plugged into a laptop (IP 192.168.0.1) with -but no MQTT server running.

    Any recommendations for the MQTT broker to run on the Laptop?

    Thanks,

    Paul

    Like

    • Paul,

      Mosquitto is really easy to install and set up. If you do a quick web search you’ll probably find an existing package for whatever OS you’re running.

      What does the “Ethernet local IP is: 0.44.44.44” line refer to?

      -John-

      Like

      • Hi Paul,

        I’m sending a reply to your original post, as the second comment you submitted was to a different project (this is “Simple Working Example”, your continuation comment was to the project “ESP-Now Gateway”).

        This project (“Simple Working Example”) is probably the one you actually want to use, so you should probably stick to the code from this repository. You definitely don’t want the ESP-Now code mixed in here. The ESP-Now project is based on the ESP8266, while the “Simple” project is based on the ESP32; the code is not interchangeable.

        On the “Ethernet local IP is:-” question …That line doesn’t exist in either of the above, so you may be using code from yet another source.

        The code you show is doing two completely different things:-

        Serial.println(eth_IP); // This only prints the variable “eth_IP”.

        Serial.println(Ethernet.localIP()); // This is a call to the ESP library requesting the chip to return its currently configured IP address.

        If the Ethernet.localIP() call is returning nonsense, then it’s a good bet that your ESP and the W5500 aren’t handshaking properly. The “Simple” code tries to give you some relevant hardware debugging info immediately after the Ethernet.localIP() call (in my original, unmodified code), which doesn’t seem to be working in your original submission (above). That also points to hardware problems between the ESP and the W5500.

        So, bottom line… Check that you’re using an ESP32 and use the GitHub repository code referred to in the “Simple” article. Double-check each individual connection between the modules (check the pin-to-pin connections are correct and check the continuity of each individual jumper wire).

        Good luck!

        -John-

        Like

  4. Thank you so much for your article. Very interesting!

    I’m doing my end-of-degree project and i want to send data through wifi and ethernet, for this reason, i use a wemos wifi and bluetooth based on uC ESP32 (see link: https://www.cytron.io/p-wifi-uno-based-esp32), and an arduino shield based on w5100 ( i check your example and ethernet libraries and it seems that it works for w5100) the problem comes when i compile… I see these messages:

    Hardware fault, or cable problem… cannot continue.
    Hardware Status: No hardware detected.
    Cable Status: Uknown status.

    I checked the hardwarestatus and when i use this board based in esp32, it shows a “0”, despite having the ethernet shield with 5100 connected. Instead, when i use arduino uno board harwarestatus method shows “1”, so with arduino uno it works correctly.

    I’ve seen that in local_config you use an special MAC for w500… Maybe that’s the problem, but if i use the MAC used in arduino ethernet examples “byte mac[] = { 0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED };” it doesn’t works either.

    I don’t know why my board doesn’t detect w5100 chip.

    I’m looking forward to hering from you.

    Like

    • Joan,

      It looks like your hardware is being detected by the SPI driver (typically, if there’s a wiring problem between the boards, you’d see a watchdog timeout or a strange IP address reported by the WizNet chip -before- the hardware check).

      I don’t have a W5100, but I do recall that there’s a second, small connector (a four or six-pin connector on one end of the board) which causes trouble when not connecting to an actual Arduino CPU board. Take a look at that as the first check.

      Second would be to verify that your local copy of the Ethernet library hasn’t been modified for the W5500 board (the easiest thing to do is simply delete the existing, -local- Ethernet library directory in your .pio/libdeps (or .piolibdeps) directory and let PlatformIO automatically re-install it).

      Let us know how it goes,

      -John-

      Like

    • Hi Hadi,

      Sorry to hear you’re having problems. It is most likely something very simple (the 0.0.0.0 is telling us that the ESP32 really can’t talk to the W5500 at all).

      You do need to check that the ground connection and power connection go to both boards (in addition to the actual data pins mentioned in the README). Are you seeing red LEDs lighting up on both boards?

      If it’s not the power supply, it must be one of the other connections. The best way to check is to take a known, good jumper cable and use it to replace each of your existing connections, one at a time. So for instance, turn the power off, disconnect the existing GPIO23 (ESP32) to MOSI (W5500) cable and replace it with your new one (take the time to double-check you have connected to the correct pins on each of the boards, too). Now power back on and check if anything has changed. If not, move on to the next connection (GPIO19 to MISO) and so on, until you’ve checked them all.

      Take your time and go slow.

      Good luck!

      -John-

      Like

  5. Hi John,

    it’s works.. Thank you for your help.
    Now the board run smooth and i’m able to ping the board. as you mention about the socket, in my case one of the cable have inconsistent connection.

    Regards,
    Hadi

    Like

  6. John,

    First, I should note that I am a controls engineer trying to rebuild myself as embedded systems, PCB design, microelectronics, so everything is pretty new still.

    I am trying to set up a webserver with esp32 that can either be connected with ethernet or wifi locally, using the esp32 as a soft AP. I intend to figure out the esp32 usage of its built-in MAC and only use a PHY built onto my board later, but I want to understand and demonstrate the W5500 first–as I might use both.

    One, do you have any insight into how to communicate on a local network that isn’t connected to the internet, without using a dedicated NWP server for the ethernet connection (imagine a Toughbook connected to the board in the middle of nowhere. Sometimes Wi-Fi is possible, sometimes ethernet is needed).

    Second, perhaps I’m missing it, but what are the passives used that differ from the ESP8266 example you did–which I did demonstrate. I only see one picture of the ESP32 board from the top, are there more in order to make sure I’m looking at the schematic right?

    Thank you,

    Ethan

    Like

  7. Hi Ethan,

    Let me answer the second (easy) question first. The stripboard is really no more than my way of keeping things permanently stuck together and permanently connected, without the “Oops! I moved it and now it doesn’t work anymore” issues of breadboard. I normally throw a 100u electrolytic and several 0.1u caps on, just for smoothing and noise suppression. Other than that, the wiring is straightforward point-to-point between the two boards, as outlined in the code (the ESP8266 version needed a buffer transistor, but that isn’t required for the ESP32).

    Your first question is a little tougher (especially as a Google tells me that NWP is “Numeric Weather Prediction”). I can really only repeat what I’ve said elsewhere, that the ESP doesn’t run any sort of OS, it’s only good for one or two distinct tasks. The limited hardware capabilities (especially memory) mean that it isn’t possible to have multitasking in the sense of (for instance) a Raspberry-pi running Linux. It should be possible to have an ESP receive (say) NTP packets from a not-too-distant WiFi network and be able to rebroadcast those packets on the ethernet connected LAN, but there’s no way to provide any equivalent of a normal commercial Access-Point, listening on dozens of sockets and handling an equivalent number of different protocols; it’s just not up to that sort of a task.

    Apologies if this isn’t what you were asking, but the bottom line is that yes, you can easily have a small web server (remembering that all of the content has to fit into that small memory space, along with the actual code), but it won’t be able to to do very much more than serve up limited, static content or handle simple websocket connections.

    -John-

    Like

    • John,

      Thank you for your reply. I mistyped in writing NWP instead of NTP, but I think I found my solution. Accurate time isn’t important for what I’m doing so I think a local crystal would work, or using the host devices internal clock.

      As to the other part, that clears up the component use, thanks, and I believe this would serve well for communicating to one or two devices, as from what you are saying I think I can handle a basic server for setting data elements. Long run time is more important than a strong OS capability for me, and I don’t need to read in a long history of items in memory, mainly control items read/write with the ability to use either Ethernet/Wi-Fi when to connect and do that.

      Ethan

      Like

      • Ethan,

        That sounds like a pretty reasonable solution to the network time issue. One of the cheap DS3231 boards attached to the ESP would give you pretty good accuracy, even over a year (search for Edward Mallon’s “Cave Pearl” project for lots and lots of details). You could also go with a GPS board, which would give you excellent accuracy over years, but at the cost of a little more power (the TASMOTA project supports GPS and has a simple NTP server capability built in).

        Both of these work over the I2C interface and are very simple to get running and both of them are lightweight enough to allow time for the ESP to perform other I/O tasks while still maintaining accurate time.

        Sounds like a really interesting project. Please do let us know how it goes.

        Best wishes,

        -John-

        Like

  8. I’ve spent about 5 hours trying to get the example to work–I assume my newness to platformIO is something to do with this.

    So, I have the local_config.h header under the EWS field include, and changed the ES32_NTP file to a .cpp as it kept erroring for C/C++ mismatch, and it’s under the EWS field src along with the sdkconfig.h file from “espressif IoT dev framework config”.

    Per issues with the ethernet library on github, in the platformio.ini file, I changed the platform to espressif32@1.7.0 and the framework to ESPIDF instead of Arduino. I included my router info in the header file as noted, but with these items done, when I try to run the platformio.ini file, I get a fail for #include errors and “cannot open” for all the headers arduino/spi/ethernet/ethernetudp along with the local_config header having the same error.

    Am I missing something obvious? The platformio.ini file should download the correct data, correct? Those headers should include under lib_deps = Ethernet?

    Thank you for any assistance, you’ve already been invaluable with the previous suggestion. This is a LOT of new for an EE working all the way down in HAL–hopefully it doesn’t freeze over 😉

    Like

    • Ethan,

      Sorry to hear you’re still having problems with this. Are you running on Windows? The phrase “…header under the EWS field include…” doesn’t mean anything to me at all.

      As a test, I just created a new directory and initialised it using “pio init –board=esp32doit-devkit-v1” and then did a git clone of the project (directly from github) and, using the latest 5.0.2 version of PlatformIO, it compiles without errors using just “pio run” (no changes to anything). I see that PlatformIO found and used this version of the Ethernet library:- 0.0.0-alpha+sha.9f41e8231b

      Just to make sure it wasn’t lying to me, I went and got the project board, plugged it in and did a “pio run -t upload” and then connected to the console to verify that it came up and announced its IP address as 192.168.1.100 (the default in the local_config.h file… which is not in my normal network address range). I then updated the local_config.h with my network details, recompiled and re-uploaded. When the ESP restarted it announced the corrected IP address and immediately started working.

      All of that kinda’ surprised me, as I hadn’t tried it for quite a while and thought that I’d at least have to change the “Platform=” setting in the platformio.ini file (as per the comment in the introduction at the very top of the article). Nope, it all just worked.

      If you can (and I say “if” because I genuinely don’t know if you can do this on Windows), just create a clean directory/folder and run through the manual steps outlined above. It should just work.

      Please let us know how it goes.

      -John-

      Like

      • John,

        Thanks again for the feedback, that was very helpful–more so than the literature. I know this is relatively simple considering, but given I had 2% idea what I was doing, and my coworkers are swamped, this type of explanation really shortened the curve. (plus I now learned a lot for moving forward, such as how the time server works, platformIO) etc. Anyway, I got it! Fully functional, ready to be manipulated so I can justify the big bucks.

        To others who are beginning and for reference, I had several issues to figure out.
        Note: I am on Windows 10

        1) Understanding PlatformIO fundamentals is important, probably start there. I had the toolchain setup wrong, and even had a couple items not positioned right in the folders for library access etc which throws spurious errors that make it seem like other stuff is wrong. So, checking that all the basic Windows 10 stuff is setup correctly.

        2) Ensure the right settings for your setup in platformio.ini, as I had the port setup incorrectly, as well as permission issues that had to do with the firmware relative to the driver. I also had to adjust the monitor settings, and adjust the baud rate.

        3) I was trying to port it in C++ without considering the way the arduino framework reads stuff. I had to declare the functions used appropriately and change the types. I basically manually changed stuff to .cpp After that, I had a bad USB cord…so connection issues as many people said can be a big part of stuff.

        4) I have no idea why, but the time server was trying to reference things wrong relative to the NTPpacket request. I used the North America server, and I’ll just say that reading stuff like this helped it make sense https://tttapa.github.io/ESP8266/Chap15%20-%20NTP.html

        Well, I’ll have to update again once I figure out how to get everything connected together.

        Ethan

        Like

  9. thank you for the article! i have one question. if i add an ethernet port (W5500 module) to my ESP8266 card, can i manage the wifi connection through the ethernet connection? i try to explain better my request. the ESP8266 has serial command to connect itself to a wifi network (AT+ CWSAP= ,). now i have connected the ES8266 to my Pc via serial port. adding a W5500 module to ESP8266, can i send the same command (AT+ CWSAP= ,) through ethernet connection (after connecting my pc to ESP8266 through ethernet cable)?

    thank you

    Like

    • Hi MicheleB,

      Theoretically, you could do this. You could build your own command menu to provide interactive access across the ethernet connection (I’ve done something similar in the past with PIC microprocessors and the forerunner to the W5500). However, I’m not aware of any library or existing code.

      I suspect, from your question, that what you’re really asking whether you can program the W5500 to send AT commands to the ESP8266 via serial. It would be rather difficult to do that (in the sample code here, the ESP32 is controlling the W5500, not the other way round).

      Best wishes,

      -John-

      Like

  10. Hi Michele,

    It’s difficult for me to give advice on this without knowing exactly what you’re planning and what your expertise level is, but yes, if you’re planning to talk to the ESP8266 serial port, then the 3v3-TTL version of this module should work (the default port settings are the same for both and a single 3v3 supply will work for both). You should read the user-command manual (https://docs.wiznet.io/Product/S2E-Module/WIZ750SR/command_manual-EN) to get some idea of what you’re dealing with before buying.

    BIG RED WARNING – The ESP8266 sends out some debug information on the serial port at a strange baud rate when it is first switched on. There is no way to stop this and it can cause problems when communicating to other devices hardwired to the serial port. You have two or three options — 1. Use Software serial on the ESP8266 and assign your software port to different output pins than the hardware serial port. 2. Use a pull-down resistor on the Wiznet nRESET pin and an ESP8266 GPIO pin to hold the Wiznet module in reset until the ESP8266 is ready to talk. 3. (Possibly the easiest) Use a large-ish value resistor and capacitor combination on the Wiznet nRESET pin to hold the module in reset for much longer than normal, to give the ESP8266 time to complete its own reset procedure first.

    Good luck!

    -John-

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s