This guide provides step-by-step instructions for connecting a unity client to your MATRIX Creator. This connection will be used to demonstrate how unity can read data from every sensor the MATRIX Creator has.
Required Hardware
Before you get started, let's review what you'll need.
- Raspberry Pi 3 (Recommended) or Pi 2 Model B (Supported) - Buy on Element14 - Pi 3 or Pi 2.
- MATRIX Creator - The Raspberry Pi does not have a built-in microphone, the MATRIX Creator has an 8 mic array perfect for Alexa - Buy MATRIX Creator on Element14.
- Micro-USB power supply for Raspberry Pi - 2.5A 5V power supply recommended
- Micro SD Card (Minimum 8 GB) - You need an operating system to get started. NOOBS (New Out of the Box Software) is an easy-to-use operating system install manager for Raspberry Pi. The simplest way to get NOOBS is to buy an SD card with NOOBS pre-installed - Raspberry Pi 16GB Preloaded (NOOBS) Micro SD Card. Alternatively, you can download and install it on your SD card.
- A USB Keyboard & Mouse, and an external HDMI Monitor - we also recommend having a USB keyboard and mouse as well as an HDMI monitor handy if you're unable to remote(SSH)into your Pi.
- Internet connection (Ethernet or WiFi)
- (Optional) WiFi Wireless Adapter for Pi 2 (Buy on Element14). Note: Pi 3 has built-in WiFi.
For extra credit, enable remote(SSH) into your device, eliminating the need for a monitor, keyboard, and mouse - and learn how to tail logs for troubleshooting.
Let's get started
We will be using MATRIX OS (MOS), to easily program the Raspberry Pi and MATRIX Creator in Javascript, and the Unity Engine.
Step 1: Setting up MOS
Download and configure MOS and its CLI tool for your computer using the following installation guide in the MATRIX Docs: Installation Guide
Step 2: Create a Unity-Sensor-Utility app
To create your own Unity-Sensor-Utility app on your local computer, use the command "matrix create Unity-Sensor-Utility". Then you will be directed to enter a description and keywords for your app. A new folder will be created for the app with five new files. The one you will be editing is the app.js file. From here you can clone the Unity-Sensor-Utility GitHub repo with the code or follow the guide below for an overview of the code.
Step 3: Start Socket Server
In the app.js file, you will need to require socket.io and create a server for the unity client to connect to. 6001 is the port left by default, but it can be changed to whatever you want.
///Start Socket Server
var io = require('socket.io')(6001);
console.log('server started');
Step 4: Configure & Start MATRIX Sensors
To read data from the MATRIX’s sensors, each sensor has to be initialized and configured with a refresh and timeout option. The options object will be used as a default for all the sensors. To save the data from each sensor, an empty JSON object is created and overwritten each time there’s a new sensor value. Each sensor has its own object.
////////////////////////////////////
// Config & Start MATRIX Sensors
///////////////////////////////////
var options = {
refresh: 100,
timeout: 15000
}
//Gyroscope
var gyroscopeData = {};
matrix.init('gyroscope', options).then(function(data){
gyroscopeData = data;
});
//UV
var uvData = {};
matrix.init('uv', options).then(function(data){
uvData = data;
});
//Temperature
var temperatureData = {};
matrix.init('temperature', options).then(function(data){
temperatureData = data;
});
//Humidity
var humidityData = {};
matrix.init('humidity', options).then(function(data){
humidityData = data;
});
//Pressure
var pressureData = {};
matrix.init('pressure', options).then(function(data){
pressureData = data;
});
//Accelerometer
var accelerometerData = {};
matrix.init('accelerometer', options).then(function(data){
accelerometerData = data;
});
//Magnetometer
var magnetometerData = {};
matrix.init('magnetometer', options).then(function(data){
magnetometerData = data;
});
Step 5: Event Listeners
With the MATRIX Creator now reading and storing sensor data, it’s time to handle how to send that data when requested. Event Listeners are created here to listen to any events that call the sensor name. Once that event is received, The MATRIX will respond by emitting another event back, but this event will contain the corresponding JSON object of the sensor requested. Sensor data will only be sent when requested because it is unlikely every sensor will be used at once. However they can all be sent if you choose.
////////////////////////////
//Event Listeners
///////////////////////////
io.on('connection', function (socket) {
console.log('Client Connected\n Sending Data...');
//Send gyroscope data on request
socket.on('gyroscope', function () {
socket.emit('gyroscopeData', gyroscopeData);
});
//Send uv data on request
socket.on('uv', function () {
socket.emit('uvData', uvData);
});
//Send uv data on request
socket.on('temperature', function () {
socket.emit('temperatureData', temperatureData);
});
//Send humidity data on request
socket.on('humidity', function () {
socket.emit('humidityData', humidityData);
});
//Send humidity data on request
socket.on('pressure', function () {
socket.emit('pressureData', pressureData);
});
//Send accelerometer data on request
socket.on('accelerometer', function () {
socket.emit('accelerometerData', accelerometerData);
});
//Send magnetometer data on request
socket.on('magnetometer', function () {
socket.emit('magnetometerData', magnetometerData);
});
//Client has left or lost connection
socket.on('disconnect', function () {
console.log('Client Disconnected');
});
});
Step 6: Unity Setup
If you haven’t already, download the latest version of unity here:
Unity will act as the client to the socket.io server running on the MATRIX Creator. Once you have unity up and running, you’ll need to install a socket.io plugin from the asset store
In the “SocketIO” folder, from the newly downloaded asset, navigate to the “Prefabs” folder and then drag&drop the prefab located inside onto the current scene.The SocketIO game object added will require you to input your Raspberry Pi’s IP address and the server port defined in the MOS app we made.
- ws://YOUR_PI_IP:6001/socket.io/?EIO=4&transport=websocket
Step 7: Creating MATRIX.cs
Moving onto the last steps, you’ll need to create a new c# file called MATRIX.cs inside your Unity Assets. Under the libraries you’ll need are the public booleans that will be determining which sensors we want from the MATRIX Creator. Below that is where the SocketIO object will be defined.
using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using SocketIO;
public class MATRIX : MonoBehaviour {
//Pick Desired Sensors
public bool useGyroscope = false;
public bool useUV = false;
public bool useTemperature = false;
public bool useHumidity = false;
public bool usePressure = false;
public bool useAccelerometer = false;
public bool useMagnetometer = false;
private SocketIOComponent socket;
Step 8: On Scene Start
Everything defined in this function will be executed once the moment the current scene runs. The first thing that needs to be done here is to locate the socket.io game object we created from the prefab at the end of step 6. After that, we can include an event listener, similar to what was done in the MOS app, for each sensor to handle its values. How we handle the data will be described in a later step. The last part is to begin a Coroutine that contains an infinite loop.
//On Scene Start
public void Start()
{
//locate socket.io prefab
GameObject go = GameObject.Find("SocketIO");
socket = go.GetComponent();
//Set Event Listeners
socket.On("open", Open);//connection made
socket.On("error", Error);//socket.io error
socket.On("close", Close);//connection lost
//Set MATRIX Sensor Event Listeners
socket.On("gyroscopeData", gyroscopeData);
socket.On("uvData", uvData);
socket.On("temperatureData", temperatureData);
socket.On("humidityData", humidityData);
socket.On("pressureData", pressureData);
socket.On("accelerometerData", accelerometerData);
socket.On("magnetometerData", magnetometerData);
//start non-blocking loop
StartCoroutine("eventLoop");
}
Step 9: Requesting Sensor Data
This eventLoop() Coroutine is essential because it allows us to write non-blocking code while we are requesting sensor data. A while(true) loop, that will never end, is defined here to request sensor data based on which booleans are set to true from step 7. If true, the loop will emit a sensor event to the MATRIX Creator that will then respond by sending us an event back with the sensor data.
////////////////////////////////////////////
// Requesting Device Data
////////////////////////////////////////////
private IEnumerator eventLoop()
{
//delay to properly initialize
yield return new WaitForSecondsRealtime(0.1f);
//loop forever
while (true)
{
yield return new WaitForSecondsRealtime(0f);//no delay
//Use sensors if requested\\
if (useGyroscope)
socket.Emit("gyroscope");
if (useUV)
socket.Emit("uv");
if (useTemperature)
socket.Emit("temperature");
if (useHumidity)
socket.Emit("humidity");
if (usePressure)
socket.Emit("pressure");
if (useAccelerometer)
socket.Emit("accelerometer");
if (useMagnetometer)
socket.Emit("magnetometer");
}
}
Step 10: Handling Sensor Data
Here is where we define the functions that the event listeners in step 8 call on. The first 3 are functions for logging connection, disconnection, and errors from the socket.io when connecting to the server running in MOS. The rest of the functions are for each sensor the MATRIX Creator has. Similar to our MOS app, each function reads any data put into it and stores it into a static class that can be read by other scripts.
////////////////////////////////////////////
// Event Listener Functions
////////////////////////////////////////////
//////////////////////////
// On Connection
public void Open(SocketIOEvent e)
{
Debug.Log("[SocketIO] Open received: " + e.name + " " + e.data);
}
//////////////////////////
// Socket.io Error
public void Error(SocketIOEvent e)
{
Debug.Log("[SocketIO] Error received: " + e.name + " " + e.data);
}
//////////////////////////
// Lost Connection To Server
public void Close(SocketIOEvent e)
{
Debug.Log("[SocketIO] Close received: " + e.name + " " + e.data);
}
//////////////////////////
// Gyroscope
public static class Gyroscope
{
public static float yaw = 0f;
public static float pitch = 0f;
public static float roll = 0f;
public static float x = 0f;
public static float y = 0f;
public static float z = 0f;
}
public void gyroscopeData(SocketIOEvent e)
{
Gyroscope.yaw = float.Parse(e.data["yaw"].ToString());
Gyroscope.pitch = float.Parse(e.data["pitch"].ToString());
Gyroscope.roll = float.Parse(e.data["roll"].ToString());
Gyroscope.x = float.Parse(e.data["x"].ToString());
Gyroscope.y = float.Parse(e.data["y"].ToString());
Gyroscope.z = float.Parse(e.data["z"].ToString());
Debug.Log(e.data);
}
//////////////////////////
// UV
public static class UV
{
public static float value = 0f;
public static string risk = "";
}
public void uvData(SocketIOEvent e)
{
UV.value = float.Parse(e.data["value"].ToString());
UV.risk = e.data["risk"].ToString();
Debug.Log(e.data);
}
//////////////////////////
// Temperature
public static class Temperature
{
public static float value = 0f;
public static string risk = "";
}
public void temperatureData(SocketIOEvent e)
{
Temperature.value = float.Parse(e.data["value"].ToString());
Debug.Log(e.data);
}
//////////////////////////
// Humidity
public static class Humidity
{
public static float value = 0f;
public static string risk = "";
}
public void humidityData(SocketIOEvent e)
{
Humidity.value = float.Parse(e.data["value"].ToString());
Debug.Log(e.data);
}
//////////////////////////
// Pressure
public static class Pressure
{
public static float value = 0f;
public static string risk = "";
}
public void pressureData(SocketIOEvent e)
{
Pressure.value = float.Parse(e.data["value"].ToString());
Debug.Log(e.data);
}
//////////////////////////
// Accelerometer
public static class Accelerometer
{
public static float x = 0f;
public static float y = 0f;
public static float z = 0f;
}
public void accelerometerData(SocketIOEvent e)
{
Accelerometer.x = float.Parse(e.data["x"].ToString());
Accelerometer.y = float.Parse(e.data["y"].ToString());
Accelerometer.z = float.Parse(e.data["z"].ToString());
Debug.Log(e.data);
}
//////////////////////////
// Magnetometer
public static class Magnetometer
{
public static float x = 0f;
public static float y = 0f;
public static float z = 0f;
}
public void magnetometerData(SocketIOEvent e)
{
Magnetometer.x = float.Parse(e.data["x"].ToString());
Magnetometer.y = float.Parse(e.data["y"].ToString());
Magnetometer.z = float.Parse(e.data["z"].ToString());
Debug.Log(e.data);
}
}
Step 11: Reading Data
With MATRIX.cs done all that’s left is to attach the script onto our SocketIO object in our scene. Once attached there will be boxes you can check that will let you pick which sensors you want to read. Each sensor chosen will log its value in the Unity console. If you see the values of the sensor you choose then you’re good to go! Usage for reading each sensor in Unity can be found here: https://github.com/matrix-io/MATRIX-Unity-Sensor-Utility-App/tree/master/Unity-Assets#usage
Github
- The entire repository for this guide can be found here: https://github.com/matrix-io/MATRIX-Unity-Sensor-Utility-App
- Example scene is included