Is there a way to incorporate Linux hardware drivers into Python projects on SBC's?

Hi Colin,

Most of these drivers you indicate are kernel-space, and will be accessed using the file system (with paths that look like normal files), so there's nothing different that needs to be done to use Python compared to (say) C/C++, provided you can access the path (/dev/...) or use ioctl calls.

However, nine times out of ten, if high performance is not required, then people write user-space drivers which employ i2c-dev or spi-dev. This is because most software developers are not familiar with writing kernel-level device drivers, it's not insurmountable, it's just a more niche skill (there's not a lot of documentation/books on the topic). A firm with resources will be able to leave a developer to learn device drivers and create a custom one if needed. Other firms may just create their user-space driver, that makes use of i2c-dev etc.

Anyway, once you're done with finding or creating a device driver, since they are all usually accessed using the file system, you've got fairly similar performance whether you access from Python or any other language.

Since the Pi is different to a normal PC in that it has I2C and SPI etc exposed on a connector, whereas a normal PC it's more inaccessible, then you may find some existing Python code is only going to work on the Pi, and not on a PC.

If you're working on a PC with Python, then interfacing typically is done with USB, perhaps USB-UART, so then you could use the Python serial module (for instance).

If you're working on a Pi with Python, then you would use one of the modules for it - such as GPIOzero as Sean mentions. There's a summary here: medium.com/.../raspberry-pi-python-libraries-for-i2c-spi-uart-3df092aeda42 or seek out a Python module for the actual chip you're using (or create your own). Usually it is very easy (and one learns a lot about Python by doing it as an exercise), but for some devices it can be a bit more challenging. Many I2C or SPI devices are so simplistic, it's not a lot of overhead to do it in user-space with i2c-dev or spi-dev. I use MicroPython on Pico more than Python on Pi, but it's essentially the same thing. If I'm connecting to an I2C or SPI device, then if there isn't an existing module, or if I don't like the existing module (e.g. some are bloated, I'd rather just create a smaller simpler one) then I'll just directly use I2C/SPI etc. 

One other (excellent) technique is to write the high-performance code in C/C++, and then just call it from Python. I often do this with different languages, since it's desirable to do high-level stuff in (say) Python or JavaScript, and all the low-level machine interfacing in C/C++. The system becomes bigger, so then you'd need code to start up any processes (for instance to accept data asynchronously), and to manage restarting tasks if things fail. In any case this also makes use of the strengths of Linux, that large applications can be split up into separate processes easily, upgraded separately, debugged separately, etc.

shabaz said:

If you're working on a Pi with Python, then you would use one of the modules for it - such as GPIOzero as Sean mentions. There's a summary here: medium.com/.../raspberry-pi-python-libraries-for-i2c-spi-uart-3df092aeda42 or seek out a Python module for the actual chip you're using (or create your own). Usually it is very easy (and one learns a lot about Python by doing it as an exercise), but for some devices it can be a bit more challenging. Many I2C or SPI devices are so simplistic, it's not a lot of overhead to do it in user-space with i2c-dev or spi-dev.

I would agree with this point, for sure. This is why I am perplexed as to the purpose (or more specifically the value) behind creating all these specific device tree overlay files for specific sensors when i2c-dev and spi-dev do the job perfectly, as far as I can make out. I can understand the need to specify say which I2C bus or SPI bus is required but as to why the Linux kernel also needs to know which chip select (for SPI) is assigned is a bit strange. There certainly seems to be scope creep here. For example, what value does it bring to the software system controlling an application by having a compiled overlay file (dtbo) for say ST7735R TFT driver (https://github.com/raspberrypi/firmware/blob/master/boot/overlays/adafruit-st7735r.dtbo).

It seems to be a Linux approach as I noticed with the Zephyr Project, which is targeted at microcontrollers, is now also using this device tree architecture (https://docs.zephyrproject.org/latest/build/dts/index.html).

So there must be a reason for this over complication.

For interest, here is a video that delves into this mystery a bit more: https://www.youtube.com/watch?v=w8GgP3h0M8M

Hence my open question as I just prefer simplicity above all things.

The devices you mention such as TFT displays, ideally need to have a proper driver (even though you could use i2c-dev or spi-dev as you say), because then there is a standard API for Linux and for other apps to write to it, i.e. you can use any software that works with a HDMI display, on the small TFT too, seamlessly. The same applies for the other device drivers normally installed along with Linux, they present a standard interface, and not all may necessarily be used by user programs, they may be used by Linux kernel or utilities (an example could be a board temperature sensor).

As far as I'm aware, the original device tree reason for existence was to allow for different cards to be inserted in the computer, and allow the system to configure itself. This is useful for PCs. For embedded systems, it's not a lot of use to adhere to this, because you already know what hardware will be connected in an embedded application, so you may as well make a decision based on required performance, and the required interworking. Purists may want it, but I think it's a bit a lot excessive to write a proper driver if the performance requirement is low and if it's fairly certain that no other applications need to work with the device. 

For instance, if the embedded device's purpose is to monitor the temperature of something and report it to the cloud periodically, then there's no need for a device driver, since only your application will make use of that sensor. Then, i2c-dev could be used, if it is a simple sensor. On the other hand, if the embedded device has a TFT screen, then it's very useful to have a driver, so that you could (say) create the user interface using any preferred technology, for instance using HTML and JavaScript, and direct the browser engine to send to the normal video framebuffer, and it will then work with the TFT screen, and even if the TFT hardware is not yet ready, the developers can direct to a HDMI screen and so on.

shabaz said:

The devices you mention such as TFT displays, ideally need to have a proper driver (even though you could use i2c-dev or spi-dev as you say), because then there is a standard API for Linux and for other apps to write to it, i.e. you can use any software that works with a HDMI display, on the small TFT too, seamlessly.

This is the point of my original question because, when using the above TFT example, and I search for Python libraries for this TFT display, the official Adafruit CircuitPython library for example uses busio, which in turn uses spidev.

Hence I am wondering what is the point of the Adafruit overlay file, other than just being there as a just-in-case efficient implementation method, if it is never used other than say in very niche cases like low level c applications.

As such, I am now speculating whether this is just simply classic "silo mentality" where Python developers are not utilising the full tools at their disposal and have gone for messy convenience instead of developing proper efficient code libraries. Little wonder there is pervasive code bloat. Just a thought.

shabaz said:

As far as I'm aware, the original device tree reason for existence was to allow for different cards to be inserted in the computer, and allow the system to configure itself. This is useful for PCs. For embedded systems, it's not a lot of use to adhere to this, because you already know what hardware will be connected in an embedded application, so you may as well make a decision based on required performance, and the required interworking. Purists may want it, but I think it's a bit a lot excessive to write a proper driver if the performance requirement is low and if it's fairly certain that no other applications need to work with the device. 

I agree with you on this point. Hence, I am wondering what the logic is behind ZephyrOS's approach to using the device tree model.

Anyway I would also love to hear from others on this topic as it does impact how SBC's interact with hardware peripherals etc.

shabaz said:

The devices you mention such as TFT displays, ideally need to have a proper driver (even though you could use i2c-dev or spi-dev as you say), because then there is a standard API for Linux and for other apps to write to it, i.e. you can use any software that works with a HDMI display, on the small TFT too, seamlessly.

This is the point of my original question because, when using the above TFT example, and I search for Python libraries for this TFT display, the official Adafruit CircuitPython library for example uses busio, which in turn uses spidev.

Hence I am wondering what is the point of the Adafruit overlay file, other than just being there as a just-in-case efficient implementation method, if it is never used other than say in very niche cases like low level c applications.

As such, I am now speculating whether this is just simply classic "silo mentality" where Python developers are not utilising the full tools at their disposal and have gone for messy convenience instead of developing proper efficient code libraries. Little wonder there is pervasive code bloat. Just a thought.

shabaz said:

As far as I'm aware, the original device tree reason for existence was to allow for different cards to be inserted in the computer, and allow the system to configure itself. This is useful for PCs. For embedded systems, it's not a lot of use to adhere to this, because you already know what hardware will be connected in an embedded application, so you may as well make a decision based on required performance, and the required interworking. Purists may want it, but I think it's a bit a lot excessive to write a proper driver if the performance requirement is low and if it's fairly certain that no other applications need to work with the device. 

I agree with you on this point. Hence, I am wondering what the logic is behind ZephyrOS's approach to using the device tree model.

Anyway I would also love to hear from others on this topic as it does impact how SBC's interact with hardware peripherals etc.

"I search for Python libraries for this TFT display, the official Adafruit CircuitPython library "

CircuitPython is an abberation : )

On microcontrollers it makes some sense, but on a Raspberry Pi running Linux, by using CircuitPython drivers, you're locking yourself out of using all the nice features of other Linux apps, and the nice Python libraries for graphics-related tasks, such as the super-powerful matplotlib if you needed charts on the TFT screen.

If one was directly using Python on the Pi (and not CircuitPython), then there is no need for a Python driver library for the display, since it would work with the Linux device driver for the display automatically. You could either stick graphics into the framebuffer (cairo is pretty good, and simple to use, I've not used the Python version called Pycairo, but I expect it to work the same) or investigate any number of apps that could run as another process in Linux, e.g. the browser engine as mentioned, and then all button/text box styles are perfectly available with HTML, CSS, etc. Cairo will allow all manner of graphics to be displayed on the screen, as well as images, in a simple way (it has no concept of buttons or text boxes, just arbitrary graphics).

Actually even on microcontrollers CircuitPython is not that great, because it's so buggy. MicroPython on microcontrollers is more stable (admittedly CircuitPython has more device support however).

shabaz said:

If one was directly using Python on the Pi (and not CircuitPython), then there is no need for a Python driver library for the display, since it would work with the Linux device driver for the display automatically. You could either stick graphics into the framebuffer (cairo is pretty good, and simple to use, I've not used the Python version called Pycairo, but I expect it to work the same) or investigate any number of apps that could run as another process in Linux, e.g. the browser engine as mentioned, and then all button/text box styles are perfectly available with HTML, CSS, etc. Cairo will allow all manner of graphics to be displayed on the screen, as well as images, in a simple way (it has no concept of buttons or text boxes, just arbitrary graphics).

Good tip, thanks Shabaz. I will certainly try this out. LOL re CircuitPython.