Lava ball is a fairly simple game where you try to keep the ball out of the lava. You move the ball around by tilting the controller and then every so often, the white square moves and you have to make it back to safety before the ground gets too hot. Here's a quick video demonstration of the game:
Prerequisites
STM32F4 DiscoverySTM32F4 Discovery
STM32F4 Discovery WiFi BoardSTM32F4 Discovery WiFi Board
UART8000-U CableUART8000-U Cable
Raspberry Pi 2Raspberry Pi 2
Introduction
We are going to use the accelerometer in the STM32 Discovery board to be the controller for the lava ball game. This way, when we tilt the board, the accelerometer will pick up the motion. We can then use the STM32F4 Discovery WiFi module to transmit this information over the network and to our Raspberry Pi 2. On the Raspberry Pi 2, we will be running the main logic for the game. This will take all of the accelerometer inputs and translate them into a movement for the ball. The Raspberry Pi 2 will also be in charge of keeping track of all of the game state to make sure that the black ball is within the white rectangle at the required times.
Design Decision - UDP
There were a couple of big design decisions that needed to be made up front before any code was written. The first one was around using UDP, rather than TCP to do the communication over the network.
Most applications use TCP rather than UDP. That's because TCP provides a reliable transport mechanism. That means that if I attempt to send some information over TCP and the delivery fails, that the protocol will automatically try to send the message again. If it does ultimately fail because of a disconnected network cable or something like that, then the sender is notified of the failure. UDP on the other hand, has no such guarantees. It is more of a fire and forget mentality. A message is sent and I don't care whether it makes it there or not.
So, why would anyone choose UDP over TCP? Well, in an application like this, I'm going to be sending lots of update information very quickly. The updates come in as fast as the hardware can generate it and send it over the wire. Therefore missing one isn't super critical, since the next message will be coming in almost immediately. A much more important feature is that I can get the information quickly. With TCP, if the packet is lost, it must be re transmitted. This takes time, and by the time that the message actually arrives at the destination, it will be much too late to apply its update. The black ball will have already moved away from where it was, making the update useless. Therefore, since missing pieces of information isn't critical and since I need to get the information as quickly as possible, UDP is the right choice for this game. This way I can create a responsive application for the user.
Design Decision - Universal Windows Application
The other big design decision is that I wanted to write the main logic of the game as a Universal Windows Application. The reason is that I could do the majority of my testing on my desktop where it is familiar and comfortable and then only do some final trouble shooting and debugging on the Raspberry Pi 2. Also, I really like the C# programming language. It is extremely powerful and let's you do some pretty incredible things very quickly.
Using the Accelerometer and WiFi module
There is a great article on how to get the STM32F4 Discovery board's accelerometer working with the WiFi module:
It is a bit involved, so I'm not going to repeat any of it here. There are a couple of changes that we need to make to the program; the biggest one being that we need to change it over to using UDP instead TCP. Fortunately, the WiFi library and demo code comes with the ability to use either protocol.
Here's what the main.c file should look like:
Most of the code is very similar to the code in the article mentioned above. The main function contains some initialization logic and then it jumps into an infinite loop. The infinite loop is fairly simple. It starts by reading the data from the accelerometer on the STM32F4 Discovery board. Then in the next line, it sends the data over the network using UDP. That's really all that needs to be in the loop: grab the data from the accelerometer and send it over the network. However, the additional logic also lights up the LEDs on the board to give the user some feedback as to which direction the board was tilted. This turned out to be a great debugging tool!
Creating the Game
Now that we have the controller working and sending data about how to move the ball over the network, it is time to create a program that will consume that data.
The program is attached to this post, so if you want to skip ahead and just see the final solution, you can. However the next section provides some background on how I arrived at the solution that I did.
Reading Messages - Part 1
If you haven't created a Universal Windows Application for the Raspberry Pi 2 yet, here's a good example of how to create a simple “Hello World” application using Windows 10 and C#:
Once you've created a new Universal Windows Application, it is time to start adding some logic to it. The first thing that we want to make sure that we can do is consume the information that is being sent over the network. So, in order to do that, we'll start by creating a very simple program that just receives the information and then writes it to a debug log. Even though this is a very simple start to our project, it represents the most challenging aspect of it: communicating between the two boards.
So, in order to receive the messages, modify MainPage.xaml.cs to look like this:
And the modify the Package.appxmanifest to include the following section:
The code in MainPage.xaml.cs should be fairly straight-forward. When the page loads, we want to create a new datagram socket. The datagram socket allows us to receive UDP packets. Then we set up an event handler to handle when a new packet arrives, and finally we bind the socket to a given port. In this case, I picked port 1554 after the New Belgium beer.
When a new packet is received, we read the information out of the packet. We know that the information coming across is three short integers, because that is what we sent from our STM32F4 Dicscovery board. We also need to switch the endianness of the data that we are sent in order for it to be interpreted correctly. Finally, we write that information to the debug console, so we can see the messages as they come across.
The other change that we needed to make to the Package.appxmanifest was to enable server capabilities within our application. This is to open up the permissions to make this communication possible. More details about this can be found here:
http://ms-iot.github.io/content/en-US/win10/samples/BlinkyWebServer.htm
Updating Ball Location
Now that we can receive the information from socket, we can use that information to move a ball around the screen. In order to do this, make your MainPage.xaml.cs look like this:
And your MainPage.xaml look like this:
We'll start with the MainPage.xaml because that is easier. We just added a canvas and a black circle on to the canvas. Now, I know the tag is called an ellipse, but if you have an ellipse where the height and the width are the same, it is really a circle.
Now, the MainPage.xaml.cs has a lot more changes, but it should still keep the same basic shape that it did before. We now added in an event that occurs on rendering. Every time the scene is rendered, we want to move our circle to be at the new location. Now, we could do this every time a new packet arrives, but that would be overkill. The only time that we actually need to set the circle's location is when it is being drawn on the screen.
Also, you'll notice that we store the circle's location in a property and don't directly access the circle's location. This is because the message received event fires on another thread. Since all UI components must be updated from the UI thread, this would cause an exception if we tried to set it directly. Therefore, we save off the location into a property and then use that property later from within the rendering event (which does happen on the UI thread) to set the circle's location. So, even though it isn't completely obvious at first glance of the code, there is quite a bit going on as far as keeping track of what thread is doing which logic.
The other comment about threading is that there is a lock around updating the circle's location. This might be a bit of paranoia, but those operations need to happen in one atomic unit and can't be broken up without potentially getting into trouble. So, I added a lock around them to make sure that the circle's location never runs into weird threading issues. (The message received event is called in very rapid succession for the entire lifetime of the application!)
Now, if you run the application and move the ball around for awhile, eventually the ball will stop and now longer respond to any of your movements. Oh no! Even worse, no exception occurs or any indication that something went wrong. The problem appears to be the frequency in which messages arrive. The application is just getting hammered with update requests.
Reading Messages - Part 2
Microsoft has written a neat library called Reactive Extensions that makes handling events much, much easier. If you haven't seen it and you do a lot with events and C#, it is definitely worth checking out. Using Reactive Extensions, we can do a lot of interesting things to the events as they come in but the one that we are most interested in is to slow down the rate at which they arrive. This can be achieved using a function called Sample, and here is what the code looks like when you use Reactive Extensions:
Note: you need to install the Rx-Linq NuGet package for this code to work.
What is truly amazing about this is how little code needed to change in order to go from using events to using Reactive Extensions. We basically just wrap the event up in an Observable and then we are good to go. Once we have the Observable, we can sample it every ten milliseconds, which is still plenty fast and gives the PC a chance to keep up with the amount of data that is coming in. Now, the application should no longer freeze and you can move the ball around for hours, or at least until you get bored.
Final Touches
That's pretty much it for the hard parts, now we just need to put in a reason to move the ball around the screen. To do that, we are going to create a white square of safety and move it around the screen occasionally. Here's the code to put in MainPage.xaml.cs:
And here is the final version of MainPage.xaml:
The new addition to the program is a DispatcherTimer that counts down and eventually checks to see if you are in a safe spot (within the white square) or not. If so, then it moves the safe spot and starts counting down again. As the program counts down, it changes the background from green to yellow to red, indicating that you have less and less time to make it to safety.
Deploying On the Raspberry Pi 2
Once it is running and working on the PC, getting it to run and work on the Raspberry Pi 2 is a breeze. Just change the output target in Visual Studio (and the IP Address on the STM32F4 Discovery board to send the data to) and you are up and running! Here's a quick video describing the set up and me playing the game:
Conclusion
By combining the power of the STM32F4 Discovery board and the Raspberry Pi 2 we were able to create a fairly simple game that is still pretty exciting to play! The naturalness of being able to tilt the STM32F4 Discovery board and watch the ball respond to this movement is really what sells the experience. Well, that and playing with all of these fun single board computers and watching them work together.