WHAT DID I DO?
Wednesday 27 November 2019 was cold, raining and generally miserable, so I decided to try and get back to some project work. I hadn't done any coding with my Arduino IDE since about May this year, so I thought I'd better check for upgrades before doing anything else........this turned out to be a bad mistake because I didn't make a note of the current version numbers of the IDE, the libraries I had installed and the ESP8266 package, so when it all seemed to go wrong, I didn't know where to roll back to!
So, the first thing I did was to update the Arduino IDE to the latest version - 1.8.10 - without first noting which version I was using!
Secondly, I went to the Arduino/ESP8266 website: https://arduino-esp8266.readthedocs.io/en/latest/index.html to remind myself how to update the ESP8266 package. This is done from the URL which is entered into the Arduino IDE Preferences window - it was already there, of course.
Then I went to Tools/Boards/Boards Manager in the Arduino IDE and updated the ESP8266 package to the latest version, again without first noting which version I was using!!
Are you counting? That's 2 substantial changes! Without testing anything!!
My ESP8266 based Temperature monitor needs the Dallas Temperature and OneWire libraries, to support the DS18B20 digital temperature probes. So I updated both libraries!!
The hole I have dug for myself is getting deeper!!
What Happened Next?
The development version of the Temperature Monitor sketch failed to compile, throwing errors which appeared to say that I had multiple copies of the ESP8266 WiFi, Dallas Temperature and OneWire libraries, and these could not be resolved.
What follows is not a detailed "blow-by-blow" account of everything I did to solve the problem(s); rather it is the relevant highlights. A colleague once described this sort of account as "The way backwards in reverse" - ie eliminating all the irrelevant side alleys that I went up!
I checked and found that there was definitely only one copy of the ESP8266 WiFi library - it's part of the ESP8266 package and I only had the latest version available - and there appeared to be only one copy of each, of the the other two, so I tried commenting out the lines which included the Dallas Temperature and OneWire libraries and tried to compile again. This time the compilation failed at the first reference to a OneWire function - makes sense.
I decided to delve deeper into the libraries and became even more confused - the latest version of the Dallas Temperature library appeared to include the relevant OneWire files. Was this the problem? I decided to try deleting the OneWire library, commenting out the line which loaded the OneWire library expecting the necessary file(s) to get loaded automatically from the Dallas Temperature location.
Recompilation failed because there was no OneWire library loaded!. I put the load line for the OneWire library back in, and tried again, expecting to load the files that were in the Dallas Temperature library. Again compilation failed because there was no OneWire library available!!
Perhaps the presence of the OneWire files within the Dallas Temperature library was confusing the loader. I deleted the OneWire files from the Dallas Temperature library folder, reinstalled the OneWire Library and tried to compile. Yet again the compilation failed but this time, because of multiple library references, including ESP8266 WiFi.
By this time my head was spinning so I decided to take a break. I shut down the IDE and my PC and went and watched a mindless TV program!
The next day I decided to........
Return to Something You Know Should Definitely Work!!
Thursday 28 November 2019: I started the Arduino IDE, and loaded up the trusty Blink sketch into the Arduino IDE, alongside the Temperature Monitor sketch which had loaded automatically as the default sketch. The Blink sketch failed to compile - the failure reason seemed to relate to the compiler's inability to rename a file because the new name already existed.
I unloaded the Temperature Monitor sketch and tried to compile the Blink sketch again. it failed.! I restarted the IDE and this time, of course, the Blink sketch loaded as the default sketch. It compiled successfully. I connected my Wemos d1 mini board and uploaded. The sketch uploaded and ran.
My conclusion is that if the last sketch that is present in the IDE, before the IDE is shutdown, has failed to compile, maybe some rubbish gets left behind in the file system that can interfere with the next compilation.
However, Blink will compile and upload, so it would appear that the IDE and basic components of the ESP8266 package are compatible and working.
Tackle the Problem Again
My next experiment was to delete the Dallas Temperature and OneWire libraries and reinstall earlier versions. The Dallas Temperature library latest version was 3.8.0 and I opted arbitrarily to re-install version 3.7.5 (which does not include its own OneWire files!!). The OneWire library latest version was 2.3.5 and I opted arbitrarily to re-install version 2.3.4.
I restarted the IDE, in order for the IDE to recognise the libraries, the Temperature Monitor sketch was default so I tried to compile it ......it failed again, with multiple library issues!!!
I went through the process of loading Blink, unloading Temperature Monitor, restarting the IDE so Blink was default and compiling Blink successfully. I then loaded Temperature Monitor and tried to compile - it still failed!!!!!!!!!!!!
Try a Distraction
At this point I needed another success, but perhaps one which stretched the ESP8266 a little further:
Because my Temperature Monitor is situated remotely, I would like to be able to update the firmware over WiFi ( OTA). But, the device spends the vast majority of its time in Deep Sleep, so it's not possible to to do an OTA update from the IDE. Instead, it's necessary for the device, when awake, to read the contents of a configuration file located on the server, in order to determine whether or not it needs to do an OTA update itself
I developed a sketch which connects to my WiFi network, issues a GET command, specifying the full URL of the configuration file. The contents of the configuration file are then returned at the end of a long response from the the server, which is captured in a String variable. All I needed to do was to include something very recogniseable at the start of the file - I chose ??? - and then search the String for ???. this then gave me the start of the file contents.
I developed the sketch stage by stage, compiling and testing at each stage. Eventually I had the sketch working. At this point I had 2 sketches in the IDE - Blink and my new Server Read sketch. I closed down the Server Read sketch, leaving Blink as the default sketch, and then closed down the IDE, powered down my PC and went to bed!
The Next Day
29 November 2019: when I had time, I started up the IDE and compiled the default sketch from yesterday - Blink - without a board connected. Connected the board and uploaded Blink. As expected, it worked perfectly.
I then loaded the Server Read sketch alongside the Blink sketch, compiled and uploaded. It worked perfectly.
I took the bull by the horns and loaded the Temperature Monitor sketch alongside the other two. Compiled the sketch - and it compiled!! - so I uploaded it and it ran perfectly!!
Confused?!? - You should be!!!
I unloaded the IDE so that Blink would be the default sketch on the next startup and shut the IDE down. I then pressed the reset button on the board and the sketch ran perfectly.
That's when I decided to write this post!
And having reached this point in my writing, I repeated the sequence again and it all worked again!
I haven't done anything that I could point to and say "That fixed it". It just started working. Perhaps the overnight reboot of my PC did something to the IDE file system - it didn't yesterday, maybe because the IDE file system was "dirty" after all the compilation failures the day before.
I hope someone who knows about the inner workings of the Arduino IDE reads this blog and can give me some clues.
In future I will certainly make sure I document my development environment before upgrading!