Skip to content

Over-the-Air ESP8266 programming using PlatformIO

There were three items on my two-do list for quite some time already:

  1. Get an ESP toolstack running on my Mac
  2. Try out PlatformIO and see if it’s nice to work with
  3. Try out video blogging (or vlogging)

So, why not combined the three to-do items into one experiment? So yesterday I got my gear ready and started experimenting and recording and editing, almost all at the same time.

Here’s there result:

In this video I have installed an ESP toolstack using platform IO with the statements below:

Since PlatformIO version 2.7, PlatformIO doesn’t have a dependency on scons anymore, hence installation of scons is not necessary anymore.

And using platformIO I have then created a firmware skeleton using:

Of course the firmware needs some code to run, that’s why I copied the file mentioned below to the src directory that was created by the “platformio init” statement:

https://github.com/esp8266/Arduino/blob/master/libraries/ArduinoOTA/examples/BasicOTA/BasicOTA.ino

This firmware is a very basic Over-the-air (OTA) firmware listener. Once it is integrated in your firmware, it makes the ESP8266 listen for a special UDP command. Once that UDP has been received, it will download and install new firmware.

To compile the firmware and send it to the ESP8266, I used:

On first run, PlatformIO detects that the ESP8266 toolstack has not been installed yet, and proposes to do this on-the-fly. After that, it will build the firmware and upload it to the ESP8266 using a USB connection. It will do detection of the right USB port by itself, so there’s no real need to set the port if there is only one ESP8266 hooked up to the computer.

When the firmware has been deployed to the ESP8266, it is now possible to start adding features to the firmware and redeploy the firmware over-the-air by adding “upload_port = <esp-ip-address>” to platformio.ini. The platformio.ini file is located in the root-path of the firmware directory.

After adding the upload_port line, the command “platformio run –target upload” will now try to upload the firmware to the ESP8266 that owns the IP address mentioned, instead of trying to update the ESP8266 through USB. It will do this by sending out the special UDP package mentioned earlier that gets the ESP8266 to start retrieving firmware. In this process you computer will serve as the server the ESP8266 gets its firmware from. This means that you can’t have a firewall running at that time, or have to make an exception.

Conslusions:

  1. PlatformIO is a pretty awesome time-saver. Excellent stuff!
  2. Using platformIO it is fairly easy to get started building ESP8266 firmware. When the ESP8266 toolstack has not been installed yet, the entire toolstack is installed on-the-fly in less than a minute. On a Mac, you will have to find the right installation commands though (mentioned above), or it won’t install.
  3. The built-in OTA routines in the Arduino library for ESP8266 are quite awesome. They work as advertised and need a very limited amount of code to be added to your project.
  4. OTA firmware updating requires quite some memory on the ESP8266. the ESP01 versions with 512kB won’t play well with OTA because of memory limitations.
  5. If you want to do it nicely, video blogging and editing is a very tedious process. I guess I have a lot more respect for those vloggers now that seem to churn out vlogs on an almost daily basis.
Published inTechnoblog

37 Comments

  1. Andrew Andrew

    Amazing. was up and running in literally 10 seconds! worked exactly as expected. Well done!

    Got stuck trying to compile a project with libs… placed them all in the lib folder, but arduinojson causes it to stop as there:

    .pioenvs/nodemcu/ArduinoJson/ArduinoJson.h:13:36: fatal error: ../include/ArduinoJson.h: No such file or directory
    #include “../include/ArduinoJson.h”

    any ideas?

    • Wow, you were quick Ivan. I didn’t even have time to catch up with my own blog and you have responded already. Thanks so much for helping Andrew out 🙂

      Cheers,
      Jan

      • Andrew Andrew

        Yes. These guys were straight on it. amazing.. just came back here to post that.. and boom… beaten me too it!

  2. Dear Jan,

    Firstly, thanks a lot for the great article and video review! I added it to :

    * Reddit (https://www.reddit.com/r/esp8266/comments/3xts1e/overtheair_esp8266_programming_using_platformio/)
    * Articles about PlatformIO (http://docs.platformio.org/en/latest/articles.html)
    * Demo (https://github.com/esp8266/Arduino/blob/master/doc/platformio.md)

    Secondly, I have a few questions for this review:

    1. You use Atom Text editor for editing. Do you know about “platomformio” (https://atom.io/packages/platomformio) Atom package? I hoped that after console steps you will show users how to make uploading from Atom 🙂

    2. 12:30 (time) you use screen utility. Have you used platformio serialports monitor? Did you have any problems with it?

    P.S: We will wait for the new articles! 🙂

    Happy Holidays and Merry Christmas!

    Regards, Ivan Kravets
    – Ph.D, Researcher and Software Architect
    – “Creativity comes from talent and never from knowledge” (c)
    ________________________
    http://www.ikravets.com

    • Hi Ivan,

      Thanks so much for posting the links to those various places. I really appreciate that.

      As for your questions: I have done most firmware for the esp8266 using Espressif’s native SDK from Eclipse on Windows, without PlatformIO. I’m afraid I’m still a rookie on the topic of PlatformIO, and what I described in the video were just my first encounters. I’m very impressed with how smooth this went and how easy it is to get started. It definitely opens up possibilities for new parts of PlatformIO to explore and hopefully blog about.

      So, the reason why I didn’t include IDE integration yet, is definitely not because of any issues, but simply because I didn’t try them yet. But I’ll do so really soon! 🙂

      Cheers & Happy New Year!

      Jan

  3. Igor Igor

    Hi great video Blog, really helped me start using platformIO. I have a problem with OTA, I can upload the program via serial, but when I try OTA the program compiles OK but does not start to upload. I get this message:

    BeforeUpload([“upload”], [“.pioenvs\nodemcu\firmware.bin”])
    “C:\Users\UserName\.platformio\packages\framework-arduinoespressif\tools\espota.py” –debug –progress -i 192.168.1.40 -f .pioenvs\nodemcu\firmware.bin
    ============ [SUCCESS] Took 18.89 seconds ============

    Any idea what is causing this?

    • Hi Igor,

      I’m not sure what’s causing it, but a “Success” message on the console always sounds encouraging 🙂
      Do you see any activity on your ESP when you’re trying to start the update?

      On of the things I needed to do on my Mac, was to switch off the firewall. Is your Windows firewall perhaps in the way?

      Best regards,
      Jan

      • Igor Igor

        Hi thank you for your response. I made some progress with my problem but still no success. The original problem was that my .py files were asociated with pycharm . So when the OTA script got called it wouldn’t open in python.

        The second problem I encounterd was that The OTA script didn’t seem to recognize my input IP. I was getting:
        14:21:38 [CRITICAL]: Not enough arguments.

        It seems on windows you must specify upload port in ” ” like this.
        upload_port = “192.168.1.40”

        My next problem it seems I can’t connect to my ESP board. I get this output when I do an upload:
        esptool v0.4.6 – (c) 2014 Ch. Klippel
        setting board to nodemcu
        setting baudrate from 115200 to 115200
        setting port from COM1 to 192.168.1.40
        espcomm_upload_file
        stat .pioenvs\nodemcu\firmware.bin success
        error: Failed to open 192.168.1.40
        error: espcomm_open failed
        scons: *** [upload] Error -1
        [ ERROR ]

        Any Idea? I disabled the firewall and the IP is definitely OK. Also there is no activity on the serial monitor when I try OTA.

      • Jan Jan

        I’m happy to see that Ivan was able to help you out with this and that it has been resolved in version 2.7.

        Cheers,
        Jan

  4. Hi Jan,

    PlatformIO 2.7 has been released and doesn’t required SCons to be installed in system. Could you remove this line sudo pip install --egg scons --install-option="--no-install-man" from instruction?

    Thanks!

    Regards Ivan.

    • Thanks Ivan. I’ve made the necessary change.

      Cheers,
      Jan

  5. Jim Stewart Jim Stewart

    Great community. I’m impressed with all of you

    • Thanks a lot, Jim!

      Regards, Ivan from PlatformIO.Org

    • Jan Penninkhof Jan Penninkhof

      Thanks. And welcome! 😉

  6. Hey Jan,

    I see that you know how to make excellent articles and video reviews.

    How about as for PlatformIO IDE + ESP8266 without any steps with command line? 🙂 It should VERY INTERESTING for the beginners.

    Regards, Ivan.

  7. Randy Lust Randy Lust

    This ia a great video thanks for the help!
    All went well till I try to use OTA update,

    [Tue Mar 1 17:25:30 2016] Processing nodemcu (platform: espressif, upload_port: 192.168.1.124, board: nodemcu, framework: arduino)
    ——————————————————————————————————————————————————————————-
    ([“upload”], [“.pioenvs/nodemcu/firmware.bin”])
    “/usr/bin/python” “/Users/rlust/.platformio/packages/framework-arduinoespressif/tools/espota.py” –debug –progress -i 192.168.1.124 -f .pioenvs/nodemcu/firmware.bin
    17:25:31 [DEBUG]: Options: {‘esp_ip’: ‘192.168.1.124’, ‘host_port’: 23561, ‘image’: ‘.pioenvs/nodemcu/firmware.bin’, ‘host_ip’: ‘0.0.0.0’, ‘auth’: ”, ‘esp_port’: 8266, ‘spiffs’: False, ‘debug’: True, ‘progress’: True}
    17:25:31 [INFO]: Starting on 0.0.0.0:23561
    17:25:31 [INFO]: Upload size: 249168
    17:25:31 [INFO]: Sending invitation to: 192.168.1.124
    17:25:41 [ERROR]: No Answer
    scons: *** [upload] Error 1

    • Hey Randy,

      Did you upload “BasicOTA.ino” to the target board?

      Regards, Ivan

  8. Randy Lust Randy Lust

    I used platformio and installed the main.io file referenced in this post.

    Thanks, Randy

    • What’s the problem Pablo? Are you stuck somewhere in the process?

      • Pablo Pablo

        Yes, first download ota code on esp, i have platformio 2.8.6 fixed “Use current Python interpreter for Python-based tools // Resolve #417 ”
        but is the same problem ?

        01:01:18 [DEBUG]: Options: {‘esp_ip’: ‘192.168.43.57’, ‘host_port’: 20724, ‘image’: ‘.pioenvs/esp01_1m/firmware.bin’, ‘host_ip’: ‘0.0.0.0’, ‘auth’: ”, ‘esp_port’: 8266, ‘spiffs’: False, ‘debug’: True, ‘progress’: True}
        01:01:18 [INFO]: Starting on 0.0.0.0:20724
        01:01:18 [INFO]: Sending invitation to: 192.168.43.57
        01:01:18 [INFO]: Waiting for device…
        01:01:28 [ERROR]: No response from device
        scons: *** [upload] Error 1

        Regards…

        • It sounds as if there is a firewall active somewhere. Could it be that you haven’t deactivated the firewall on your PC?

  9. Pablo Pablo

    Maybe, i use RPI and samsung galaxy with ANDROID AP (DNS)

    RPI (192.168.43.52) —> (samsung galaxy AndroidAP)
    esp-01 (192.168.43.57) —>
    server DHT port tcp/80 work
    OTA UDP dont work

    I dont have trouble with TCP maybe deny UDP ?
    Regards
    regards..

  10. Pablo Pablo

    Maybe, i use RPI and samsung galaxy with ANDROID AP (DNS)

    RPI (192.168.43.52) —> (samsung galaxy AndroidAP)
    esp-01 (192.168.43.57) —>
    server DHT port tcp/80 work
    OTA UDP dont work

    I dont have trouble with TCP maybe deny UDP ?
    Regards

    • You’re right, I have noticed that not all routers support UDP that well. I’ve seen a few in The Netherlands, shipped by Ziggo that are quite problematic when it comes to transporting UDP packages right. Also notice that the ESP needs to establish a connection with your PC. This means that not only outbound connections from your PC should be allowed, but your firewall should also allow these inbound UDP packages. You could try to switch off your firewall temporarily on your PC to see if that works. If that still doesn’t work, I would see if I could build up a little test-network with different network equipment.

  11. Pablo Pablo

    Jan thank for answer, I use nmap for view port on esp and AndroidAP but i don’t see udp port open, change AndroidAp by a router d-link and disble de firewall in a router and see the port udp on esp but isnt´t 8266/UDP i see 5353/UDP ?

    RPI (192.168.43.52) —> (router d-link disable firewall)
    esp-01 (192.168.43.57) —>
    OTA_BASIC 5353/UDP

    I must see firewall in RPI, but i dont understand why change port number with ArduinoOTA.ino –>5353/UDP and espota.py try to download by port 8266/UDP
    something i am doing wrong lol

    Regards.

    • Then that might be the issue Pablo. You’ve got to make that UDP working first before you can install your firmware over the air.

  12. jas jas

    “upload_port = ”
    How do you find out the ip address of the esp8266?

    Thanks

    • The ESP will tell you if you keep it connected to your serial port after you flashed it the first time (through USB). Line 45 in the BasicOTA.ino sketch will take care of that: Serial.println(WiFi.localIP());.
      Alternatively, you can also have a look at your router or run a network scan.

  13. sunil sudevan sunil sudevan

    Hi Jan,

    thank you so much for posting this tut..,
    i have successfully managed to do OTA on my node mcu.

    I want to read the sensor data from the wifi port as i used to do while on usb.
    How can i do this over wifi ?

  14. Dank voor de heldere uitleg. Nog even een opmerking. Op 8:50 is je wachtwoord in plain text zichtbaar.

  15. Jindřich Jindřich

    It really works on Sublime3 🙂
    I had hard time to find out, that I have connected my laptop to another WIFi than the ESP 😀 so “NO answer” error was showing 🙂

    thx for great job! J

Leave a Reply

Your email address will not be published. Required fields are marked *

20 − 4 =