I’m using openHAB on my personal server to control various parts of my “intelligent” home, and while I can use the Android app just fine, I thought it would be interesting to reproduce its user interface with a very resource limited platform, namely an ESP32-WROOM-32D, connected to the ME817EV evaluation board received for the BT817 roadtest.
The goal of this experiment is thus to do the following:
- Retrieve the interface definition. This is a JSON file, representing a sitemap in openHAB terms
- Interpret that sitemap into elements to be displayed on the screen
- Send the appropriate commands to the BT817 chip
- Retrieve additional images to illustrate the status of various items in the sitemap
- Handle touch interaction to allow controlling sitemap items
I’m used to work with ESP32 under VSCode with PlatformIO Arduino mode, so I went looking on Bridgetek website which led me to this page: https://brtchip.com/SoftwareExamples-eve/#MCU%20Specific%20Examples
The ESP32 seems to be supported, but there are two issues here:
- The web page says “The code currently supports FT81x and FT80x and is being updated to support BT81x” which is annoying because the roadtest is about the BT817.
- The download links for the BRT_AN_025 download links are broken.
This is why I broadened my searches to the Internet at large and found this repository at GitHub: https://github.com/RudolphRiedel/FT800-FT813
Don’t let its name fool you, it supports the BT817 just fine and does so in a very elegant way, even allowing for faster communication between the ESP32 and the BT817 by using DMA transfers.
I thus decided to use it in a personal project which sources I have published at Github: https://github.com/obones/EVEopenHAB
Here is a little video showing how it behaves:
Interpreting the data
Once the wifi connection is up, the sitemap is retrieved. After it has arrived, a series of objects are created from it, representing its elements.
The root is called the homepage, and it contains widgets that each refer to an item that provides a state and may allow for some control, here is an example of the received JSON:
Modern development rules mandate that there is a clear distinction between the data part (called model) and the rendering part (called view) but doing so usually requires doubling the number of objects instantiated, thus doubling the RAM requirements.
As we are using a very limited platform, we cannot afford to waste memory which means that only one instance per element is created, and that instance is both holding the data and in charge of the rendering.
The homepage starts the “display list” places the common items and then loops over its widgets to tell them do to their own rendering. To do so, they place one or more instructions into the display list, such as “Write this text”, “Draw this line”, “Create a button”…
Once the widgets are done, the homepage finalizes the display list and sends it to the BT817 chip which does all the rendering.
This is one very nice feature here, because this allows the ESP32 not to care about the display itself, completely avoiding all the complications of compositing the final image, driving the RGB lines, and timing the appropriate VSYNC and HSYNC signals. This also has the added benefit of only requiring 4 pins on the ESP32, leaving all the others as free GPIOs to be used for whatever need you may have.
However, its has one drawback which is that you have to learn the EVE instructions, their usage and sometime work with what is not “natural” in other GUI environments where everything is lines, circles and pixels.
For instance, the rounded outline around the widgets is not simply done with a “rectangle” order because with EVE, they are always filled. To achieve this, there are various tricks, the easiest one being to draw a background-colored rectangle inside the outline one, effectively cutting through it.
EVE also only support ASCII characters by default, which is fine for the X and up arrow on the buttons, but impossible to use for the down arrow. The BT817 provides for loading UTF-8 characters from the associated Flash chip, but this consumes a lot of RAM inside the chip itself, so I decided to use the rotation matrix provided by EVE. This way, I use the ^ symbol that I rotate 180°.
What was the most challenging is the display of icons next to each widget. They are available from the openHAB server as SVG or PNG files and luckily EVE provides the CMD_LOADIMAGE instruction that supports loading PNG images.
Its first parameter is the address in RAM_G where the image data will be placed. This means that one has to manually manage the organization of that RAM segment, making sure that no two images overwrite each other’s data. This is much easier to do if all images have the same layout, that is if they are using the same PNG format. In my case, I decided that PNG4444 was good for me, that is 4 bits per channel, with an alpha channel. As this is two bytes per pixel, it is easy to calculate the required size for the 64x64 images sent by openHAB: 64*64*2 = 8192
This way, we can easily place images one after the other in the RAM_G section, keeping track of the next available area. The BT817 does provide a way to find what is the next available address after a call to LOAD_IMAGE but doing a “same size for all” approach allows to initialize the allocated space with a known value, giving the desired effect while waiting for the actual data to arrive. In the video above, I applied a green value for illustration purposes, but a production release would most likely make sure the area is initialized with all zeros, leaving the background to show through.
The programming guide states that PNG is supported but with this restriction: PNG with bit-depth 8 only and no interlace. While that last one not used anymore these days, the openHAB team used all the options they could to create the smallest PNG files possible. This means that most of what is offered does not respect the 8 bit requirement, and even if it does, it most often is not color_type 6 but rather value 3, leading to a paletted4444 image, which does not have a fixed size layout as required above.
After a bit of discussion with the openHAB folks, I found a way to make sure it only sends PNG32 files to my ESP32 and that’s how I get the rendering you have seen above.
Now that the display is good, we must implement a bit of interactivity via the capacitive touch screen support of the BT817. Working with raw sensor data can be quite time consuming to get right, as it needs proper timings, calibration, and associating coordinates with what is displayed on the screen.
Fortunately, the BT817 does all the hard work for us, once again getting off the ESP32’s plate which could have gotten quite overloaded with all this.
We can use the direct coordinates for up to 5 touch points, but the easiest way is to use TAG management, in where the BT817 associates a “tag” to display elements and when the touch happens in the display area of these elements, a simple register indicates which tag has been touched. Tag is a value between 1 and 255, but it is the programmer’s responsibility to manage the tag values. If the rendered screen is static, it is easy to create a set of constants for it, but in my case, the sitemap can change over time.
This is why I created a TagManager class that gives the next available one when asked to do so and associates it with a callback method that gets run when the appropriate touch is done.
This way, we can react automatically to a press on a given button, without worrying about its location or even the tag value changing following an update on the sitemap itself.
Getting all this right was a matter of trial and error on my side, going back to the Programming guide and/or the datasheet and you can see this in the commit history of the EVEopenHAB repository. However, on three occasions, I was stuck and that’s where the community forums have proven handy. Do not hesitate to go over there to ask your questions: http://www.brtcommunity.com/index.php?board=7.0
There are a few issues left, but they are not related to EVE, rather to connectivity issues in the download part of my application.
In the end, I have learned very easily how to use the commands from EVE and the resulting display is quickly very nice. The documentation may be confusing at times but using the EVE Screen Designer to test various options is a very robust way to achieve a desired result.