Group Actions
Engagement
  • Author Author: misaz
  • Date Created: Date Created
  • Views 550 views
  • Likes 8 likes
  • Comments 4 comments
Related
Recommended

Blog #13: Improved Time-driven Swipe Gesture Detection Library

misaz

Hello Element14 community.

I welcome you to my next blog post as part of Experimenting with Gesture Sensors competition. This is 13th blog post as part of my journey with Gesture Sensors. In previous blogs I described my plans as part of this event, my first experiences with software and hardware, my stand which I made for experiments, some troubleshooting techniques which I used, my Library, my first project which I completed as part of competition and recently I described my own gesture detection algorithm. For summary there are link to all my previous blogs:

In this blog I will continue my previous blog and today I will describe my second algorithm for detecting swipe gestures.

Previous Algorithm Failure Analysis

As you may know from my previous blog, my first algorithm worked but not very well. I was thinking what is wrong with this algorithm. I realized two aspects:

  • My previous algorithm has fixed-size buffer (with 50 items) which contained data from 50 previous screens. For example, in the situation when gesture started 10 screens ago, then algorithm processed 40 screens of noise and 10 of gesture. Frequently 40 screens of noise resulted to the wrong gesture provided. Even more, there is effect of misleading gesture detection at the beginning and ending of gesture when user is moving hand to and from field of view. Thus, I thought that in this case algorithm detect gesture based on 40 screens of noise and then it process highly misleading data on remaining 10 screens.
  • In previous blog post I described algorithm of computation score for each direction. I designed this formula naturally without any rational background and I of course was not sure that it is correct and provide any valuable information.

I implemented my second algorithm from scratch, cleaned code and organized my code better than in previous case. But in many functionalities are very similar to previous algorithm or work exactly the same.

Memorization Improvement

As I stated processing always exactly 50 screens are simple to implement but it is not good idea. Instead, I implemented dynamic array with data from up to 50 screens. I did not change digital filtering stage very much and I deployed the same digital filter as in previous case. Consequently, I process data highlighting fast transitions (=gestures) and data are almost free from offsets. It is easy to apply threshold for deciding that gesture is or is not in progress. I add samples to array of data for gesture detection if and only if there is gesture in progress. Otherwise, my algorithm is idle. This is significant benefit over previous algorithm which processed data permanently including screens without any gesture which resulted to spurious gesture detection (which was resolved by output glitches filtering). My second algorithm has no output glitch filter because it is designed in a way that it does not need it. Gestures are not detected continuously but they are detected at the end of gesture (which can be easily detected since I can detect situation when I stop adding samples to the buffer for some period of time). This has benefit of much higher accuracy but also it is disadvantage because there is latency. My previous algorithm provided information about gesure faster. Often it provided information about gesture even it was still in progress and did not complete yet. This new algorithm has not this benefit. It always waits util gesture completes.

Direction score computation improvement

Except architectural changes in algorithm operation, I also changed scoring of movements as part of gesture. I removed squaring distance. Now step score is calculated as multiplication of distance and intensities at edge points of the step. Direction computation is the same and principal of incrementing scores for each direction and then selecting maximum score works exactly the same as in case of previous algorithm.

Result

This algorithm is much better and pacman is playable with this algorithm. The algorithm has much less false detections but there are still some. The biggest disadvantage is latency of detecting gesture because of waiting until gesture completes. In next blog I will show playing pacman with deployed this algorithm in background. Now you can look “foreground” to the debugging screen of this algorithm. Dot in right corner indicates if gesture is in progress (green in case of gesture in progress, red when no gesture is detected). Dots indicates collected center of mass.

Source codes

Source codes of this algorithm will be shared as part of next blog post in which I describe my final project utilizing this algorithm.

Conclusion

This is all from this blog and from part of developing custom gesture detection algorithm. In this and previous blog post I described evolution of my own algorithms. In this stage of MAX25405 competition I learned the most. I did not learn very much about MAX25405 and hardware, but I learnt that software processing data from this sensor is very important and quality of the result highly depends on it. At the end I have algorithm which of course is not as good as algorithm from Maxim, but it is acceptable for playing Pacman as you will see in next blog. Stay tuned for my latest blogs as part of this competition. Thank you for reading my blogs.

Next blog:  Blog #14: Gesture Controlled Pacman

Parents

  •   well done at getting your Pacman program working.

    One aspect I was grappling with is the MAX25405 sampling rates. As you well know you can change the SDLY register value to increase/decrease the End of Conversion Delay. This can as short as <2ms or it can be made to delay > 1.5s before next sample. Thus the selection of End of Conversion Delay value can make a big difference to how well the algorithms work in determining gestures. It also impacts the performance of the algorithms and the power usage.

    Did you stick with the default sampling rate?

    Did you test with any slower sampling rates?

Comment

  •   well done at getting your Pacman program working.

    One aspect I was grappling with is the MAX25405 sampling rates. As you well know you can change the SDLY register value to increase/decrease the End of Conversion Delay. This can as short as <2ms or it can be made to delay > 1.5s before next sample. Thus the selection of End of Conversion Delay value can make a big difference to how well the algorithms work in determining gestures. It also impacts the performance of the algorithms and the power usage.

    Did you stick with the default sampling rate?

    Did you test with any slower sampling rates?

Children
  • in reply to BigG

    Hi . Thank you for feedback.

    Most of the time I used the same parameters and sample rate as it is used by Maxim's GUI program which contains settings recommend by Maxim. End of conversion Delay is not the only parameter. Itegration Time and Number of Repeats also affects sample rate. Sample rate is about 76,8 FPS and algorithm work if you slightly reduce or increase it. Increasing sample rate does not make very sense. 76,8 FPS is good enough for even very fast gestures by hand. Very few people have hand so fast that sensor gest at least two points of movemement. Note that 2 is theoretical value. My algorithm has constant saying that at least 6 samples are needed for detecting gesture. but still even very fast gesture are captured by more than six samples when using "default" 76,8 FPS. Decreasing sample rate is possible and algorithm work well. The only limitation is speed of gesture. If you decrease FPS then you will not be able to catch fast gestures.

    I faced other situation when I needed to increase End of Conversion Delay: when using two sensors. I will blog about reasons today If the site lags allows me to post blog.

  • in reply to misaz

    Also note that changing some sequencing parameters like integration time or number of repeats affects not only sample rate, but also output values. If you for example increase integration time to maixmum posible you will receive very high values which may reach the boudary of 16-bit data type. Similarly If you try push FPS high as much as possible you will receive lower values because pixels face to the light less time.

  • in reply to misaz

    I used the 10FPS setting for my tests. Based on that sample rate I could not take as many samples otherwise I got poor results. Thus I reasoned the firmware needs to be aware of the sample rate in order to adjust number of samples taken when evaluating gesture movement.