Over the Air (OTA) Programming of ESP8266 - Part II

OTA Firmware Updates using an HTTP server

In my previous Blog:Over the Air (OTA) Programming of ESP8266 - Part I , I described how to update ESP8266 firmware OTA, using the Arduino IDE to connect to the ESP8266 via a "network port". I pointed out that this is of no value when needing to update a device which spends the vast majority of its time in Deep Sleep (conserving battery power).

The HTTP server approach has the existing firmware on the device capable of updating itself from a named binary file (compiled by the Arduino IDE) which is in a known location on a server.

A simplistic approach would be to include an update sequence in the firmware, activated every time the device "wakes up" from deep sleep. However, most of the time the firmware would be updated with an unchanged copy. Since the update process takes a significant time and is using WiFi, it is easy to see that this would not help the conservation of battery power!!

A far better approach is for the device to determine whether or not there is a new version available and then to act accordingly. The guidance given in the ESP8266 package documentation:      https://arduino-esp8266.readthedocs.io/en/latest/ota_updates/readme.html    didn't inspire me, so I had a good google and found this:

       https://www.bakke.online/index.php/2017/06/02/self-updating-ota-firmware-for-esp8266/   by Erik Bakke.

Erik's approach is as follows:

1. The firmware includes a defined constant which is the firmware version number.
2. The Server holds two files - the firmware binary file and a "version" file which contains the latest version number of the firmware.
3. The firmware binary file and the version file have a name which is derived from the MAC address of the ESP8266 device
4. The device reads the contents of the version file, compares the version number from the file against its own embedded version number and (simplistically), if they don't match, then the firmware updates itself from the binary file. Ideally, the embedded version number ought to be less than the version number in the file, but in a "hobby" environment I think it's reasonable to assume that it's safe to say !=.
5. When the update is complete, the device reboots and, hopefully, the magic has worked!

I read through Erik's article several times and studied the code to try and understand it. I decided to start with a very simple sketch to test whether I could read the contents of a "version" file using the device MAC address to form the filename.

First of all, here is Erik's code to create a String containing the base filename from the device MAC address

String getMAC()
{
  uint8_t mac[6];
  char result[14];
  WiFi.macAddress(mac);
  snprintf( result, sizeof( result ), "%02x%02x%02x%02x%02x%02x", mac[ 0 ], mac[ 1 ], mac[ 2 ], mac[ 3 ], mac[ 4 ], mac[ 5 ] );
  return String( result );
}

I have to confess that I didn't understand what on earth the snprintf command was doing. (Note - I do understand it now, having had to look deeper into snprintf for other reasons) I don't like using code that I don't understand and, rather than spend time getting to grips with it, I decided to go around it and use the ESP.getChipId() command instead. This returns the lower portion of the MAC address (so not unique, but certainly unique on my private network!)The value is in decimal form. It was easy to convert that to a useable HEX string that is familiar from the labeling of IDE network ports in my previous blog

String getDevID()
{
  unsigned long chipID = ESP.getChipId();
  String devID = String(chipID, HEX);
  return devID;
}

I am currently working with two Wemos d1 mini devices, and they have DevIDs of 239ae3 and 583fd5. Conveniently very different!

The next thing to do is to build the full URL of the "version" file:

We need to be connected to the network - the following are relevant:

#include <ESP8266WiFi.h>

#include <ESP8266HTTPClient.h>
#include <ESP8266httpUpdate.h>

IPAddress ip(192, 168, 1, 40);              //Needed for fixed IP Address
IPAddress gateway(192, 168, 1, 1);
IPAddress subnet(255, 255, 255, 0);
IPAddress DNS(192, 168, 1, 1);

const char* ssid     = "Your Network SSID";         // WiFi network name
const char* password = "Your Network Password";     // Your WiFi network password


WiFiClient client;                          // Use WiFiClient class to create TCP connections
const int httpPort = 80;

I use a fixed IP Address; it's quicker than waiting for an allocation from the network DHCP server. The shorter the sketch execution time, the better the battery conservation

A few constants need defining:

const char* VERSION = "1.0.1";
const char* MODEL = "1";

const char* host = "192.168.1.30";
const char* fwURLLoc = "/website/otaSTORE/";

Note I that  have included  a MODEL number as well as a VERSION number. I have some vague ideas about why this might be useful in the future, but I won't try and explain them now. I have also split the definition of the host IP address from the website location of the file(s). This is relevant later.

Finally, connect to the network:

void setup()
{
  Serial.begin(9600);
  delay(500);
  Serial.println(" ");

  // Connect to WiFi Network
  WiFi.config(ip, gateway, subnet, DNS);  // Needed for fixed IP Address
  Serial.print("ESP8266 IP address: ");
  Serial.println(WiFi.localIP());

  Serial.print("Connecting to ");
  Serial.println(ssid);
  WiFi.mode(WIFI_STA);
  WiFi.begin(ssid, password);
  while (WiFi.status() != WL_CONNECTED) // Wait for connection
  {
    delay(500);
    Serial.print(".");
  }

  // WiFi connected
  Serial.println("");
  Serial.println("WiFi connected");

  Serial.print("Current Version Number: ");
  Serial.println(VERSION);
}

void loop()
{
// No code here
}

Here is the foundation of the checkForUpdates() routine, which is the heart of the OTA process

void checkForUpdates()
{
  String devID = getDevID();
  String fwURL = String ("http://") + host + fwURLLoc + devID;
  String fwVersionURL = fwURL;
  fwVersionURL.concat( ".ver" );

  Serial.println( "Checking for firmware updates." );
  Serial.print( "chipID: " );
  Serial.println( devID );
  Serial.print( "Firmware version URL:      " );
  Serial.println( fwVersionURL );

  HTTPClient httpClient;
  httpClient.begin( fwVersionURL );

  int httpCode = httpClient.GET();
  Serial.print("httpCode ");
  Serial.println(httpCode);
  if ( httpCode == 200 )
  {
    Serial.print( "Current Model Number: ");
    Serial.println( MODEL );
    Serial.print( "Current firmware version: " );
    Serial.println( VERSION );

    String verFileContents = httpClient.getString();
    Serial.print( "Available model/firmware version from File: " );
    Serial.println( verFileContents);
  }
}

Firstly we construct the full URL of the "version" file, This has the form: http://192.168.1.30/website/otaSTORE/239ae3.ver

The next few lines of code initialise the httpClient, perform a GET command on the version file and eventually return a String variable holding the WHOLE contents of the version file.

In reality, the version file only contains a single line of text of the form 1,1.0.1  where 1 is the MODEL number and 1.0.1 is the VERSION number. However, as I discovered to my great annoyance, if the line of text is terminated by CR/LF, then the CR and LF characters are included in the string and must be detected and stripped out, otherwise problems will occur.

Here's the section of code which deals with that issue:

    int lengthData = verFileContents.length();
    char testIt = verFileContents.charAt(lengthData - 2); 
    if (testIt == 13)
    {
      lengthData = lengthData - 2;
    }    
    int comma = verFileContents.indexOf(",");
    String newModel = verFileContents.substring(0, comma);
    String newVersion = verFileContents.substring(comma + 1, lengthData);
    
    Serial.print( "New Model Number: ");
    Serial.println( newModel );    
    Serial.print( "New Version: ");
    Serial.println( newVersion );

The partially completed checkForUpdates routine is called after the WiFi is connected, still within the setup() part of the sketch.

Here's the output from the simple sketch to construct the filename and read the contents of the version file:

And, if we change the Version Number to 1.0.2, in the version file, and run the sketch again, we get:

Now I was confident enough to move on to actually try an OTA Update. At the point where the demonstration sketch leaves off, we need to insert the code to actually carry out the update:

    Serial.print( "New Version: ");
    Serial.println( newVersion );

    if ( ! newVersion.equals( VERSION ))
    {
      Serial.println( "Preparing to update" );
      
// Constuct URL for new firmware
      String fwImageURL = fwURL;
      fwImageURL.concat( ".bin" );
      Serial.print( "Firmware Image File URL: ");
      Serial.println(fwImageURL);

// Update the firmware
      t_httpUpdate_return ret = ESPhttpUpdate.update( fwImageURL);
    
// Error handling
     switch(ret)
     {
       case HTTP_UPDATE_FAILED:
         Serial.printf("HTTP_UPDATE_FAILED Error (%d): %s", ESPhttpUpdate.getLastError(), ESPhttpUpdate.getLastErrorString().c_str());
         Serial.println(" ");
         break;

       case HTTP_UPDATE_NO_UPDATES:
          Serial.println("HTTP_UPDATE_NO_UPDATES");
          break;
     } // end of switch
   } // end of: if ( ! newVersion.equals( VERSION ))
   
   else    // newVersion.equals ( VERSION )
   {
     Serial.println( "Already on latest version" );
   }
  } //end of:  if ( httpCode == 200 )
  else  // httpCode !== 200
  {
    Serial.print( "Firmware version check failed, HTTP response code: " );
    Serial.println( httpCode );
  }
  httpClient.end();
  
} //end of void checkForUpdates()

And, in order to be able really see that an update has taken place, we need a sketch that actually does something visible, which we can change and apply the change OTA.

The following code is now inserted into the sketch:

  Serial.println(VERSION);

  int counter = 0;
  int counterLimit = 10;
  int flash = 500;
  Serial.print("\nFlashing LED at interval of ");
  Serial.print(flash);
  Serial.print(" mSeconds, ");
  Serial.print(counterLimit);
  Serial.println(" times");
  for (counter = 0; counter < counterLimit; counter++)
  {
    Serial.print("#");
    digitalWrite(LED_BUILTIN, LOW);
    delay(flash);
    digitalWrite(LED_BUILTIN, HIGH);
    delay(flash);
  }
  Serial.println(" ");

  checkForUpdates();
 

Published in error - incomplete. Part III will follow!!!!!!!!!!!!!!!!!!!!!