I've been playing around with some SBCs and some touchscreens, trying to learn how to make a touchscreen UI (i.e. Fully touch driven, scrolly lists, single or multi finger gestures, etc.)
I think touchscreen UI development is maybe one of the weakest links in these fun little experimenter boards, unless I've missed something obvious. There's so many programming resources out there for the standard desktop UIs, internet of things type stuff, and other things like that, but as the world gets more smartphones and tablets and high tech car consoles, touchscreen UI stuff should be more accessible so kids (both young and old) can get their hands on it. I played around with android and iOS SDKs and they're nice and all, but they teach me nothing about how the bits in between the user's finger movement and the final animation rendering work, nor how I'd go about making a similar experience from the ground up and not tied to a phone OS.
I've got a Pi2, BBB, and a few kinds of touchscreens - one is a 7" multitouch expansion board designed/attached to the BBB, others are just HDMI+USB multitouch displays, 7-10", pretty standard stuff. The USB ones surprisingly pretty much work out of the box on either board, and the one designed for the BBB came with an image that contains the right drivers built in (I think it's made by ChipSee).
I'm a bit of a car tech geek, so a touchscreen car UI is one of the things I wanted to play around with making.
On the Pi2 I ran Kodi and had so much trouble getting the touches to be recognized in the right spot that I just pretty much gave up on it. I ran Kodi on my Windows touchscreen tablet and it was halfway decent, but couldn't figure out the problem with it on the touchscreens on Linux
So I grabbed the Qt car IVI demo from https://wiki.qt.io/In-vehicle_Infotainment_Demo and compiled Qt 4 on the SBCs and built it. It's written for Qt 4.7 and is pretty old, but it's as good a start as I can think of at the moment. I get a half decent looking touch-driven UI and some idea of what I could put together. But there's some noticeable performance blips (stutters), and scrolling lists lags behind my finger by a little or a lot. Now what? Where do I look for how to optimize this? is it mostly in the software because of all the translating between touch to mouse to Qt, or is there some hardware digging I gotta do?
Framerate Stutters
- guessing that's a result of the Qt application not being optimized well enough? I take it I'd need to do some profiling in the Qt IDE to figure out what might be causing this.
- not sure if I have hardware acceleration on? I tried to google all over the place for how to tell whether hardware acceleration is on, but couldn't find it (found plenty of info on how to do it for Qt5 tho, but this project doesn't build in 5)
Scrolling lag
- Is it because touch events are being translated to emulate mouse events on the desktop, and then those mouse events are themselves filtered or buffered by Qt before being translated to UI movement? Is it as simple as changing the Qt application to use touch events somewhere instead of using mouse events? It looks like the application in question just uses a regular ListView, I can't find where it tells it how to scroll it like a touchscreen is expected to scroll.
- What way to measure exactly how much latency there is between the physical touch on the screen, to the low level driver event, to whatever else is between that and the signal to the application that the user touched/dragged?
- Are there any touchscreen displays/controllers that are particularly known to be super low latency, or does that come at the cost of accuracy?
- Touchscreen connection methods... does USB, I2C or SPI work faster? I think the touchscreen attached to the BBB is connected to it via I2C or SPI, but it lags behind my finger even further than the HDMI+USB I have attached to the Pi2 right now (1280x800 7" open frame) so that kinda goes against my intuition...unless the other above factors play a greater part in it on the BBB
Hope this is the right place to ask...of course there's a Qt forum that probably has a better shot on the software side, but I'm not sure if the software is my only problem.
Thanks in advance <3
