Ever needed a way to protect your sweets or plans on world domination from eyes of others? Or maybe You have wondered how smart things can detect your person without fancy biometric scanners or ever-present pinpads? Take a look below to learn how you can apply accelerometer readings and gesture recognizing tech to make Your storage chest know You!
What is this project about.
This project aims to experiment on gestures and other deliberate actions to recognize a person. Whole modern security field seems to be based on actually two things that can be applied for people recognition. Something the person has (like a token or biometric feature) or something the persons knows (like pincode or a secret password). In this project we will focus on something the person know and it will not be oldschool password but rather a magic ritual. The prototype implementation of this idea will involve three main actors: AzureSphere starter kit, MGC3130 dev board and Azure as a cloud service. If you are curious please see the demo and overview diagram below and read on to know the details.
Prototype details.
General idea for the prototype is to use azure sphere starter kit (AZS) as a brain for the whole operation. AZS will communicate with onboard LSM6DSO accelerator to recognize the orientation of the setup and also tapping of the prototype. Additionally MGC3130 based Gesture recognition shield for RaspberryPi will be used to add bigger variety of gestures that can be observed by the prototype. Everything will be put into wooden box that will act as a smart storage box and a home for AZS. The box will be equipped with servo operated bolt lock to allow the box to lock itself and open when it sees fit. For improved security of one's belongings stashed away AZS will also monitor box movement and report everything to cloud.
Feature list.
- Servo operated bolt lock to lock and unlock lockbox
- Servo high side switch for servo disable
- Flick gesture recognition using MGC3130 rpi shield
- Accelerometer based lockbox orientation, movement and tapping recognition
- Unlocking the lock when aimed set of gestures (recipe) is recognized as user input
- Button activated locking with delay to close the box lid
- Lock state and unlocking recipe stored in mutable application file on the device
- Iot Central connectivity to report lockbox state, update unlocking recipe, operate lock remotely
- Possibility to still work without cloud connectivity with basic functions: gesture recognition, lock operating
Hardware list.
Electronic bits and pieces that were used to build the project:
- Azure Sphere starter kit (http://cloudconnectkits.org/product/azure-sphere-starter-kit)
- 3D Gesture & Tracking Shield for Raspberry Pi (https://www.seeedstudio.com/3D-Gesture-Tracking-Shield-for-Raspberry-Pi-MGC3130-p-4073.html)
- SG90 servo with wire (http://www.towerpro.com.tw/product/sg90-analog/)
- Proto board wide enough (10 columns) to cover Azure Sphere MikroE Click Boards interface
- Female to male precrimped dupont cable with 6 wires
- Dupont wire 6-pin headers x2
- Grove I2C cable
- NPN (2N3904) transistor driving PNP (2N5401) transistor powering up the servo
- 1k and 33 Ohm resistors
- Pin headers: 1x8, 1x6, 1x2, 1x3
- Soldering iron, solder, some wire
- Power bank to power the prototype
- USB micro cable
Mechanical parts that were used to house the prototype and implement locking mechanism (note that these depend heavily on chest shape and size so be creative when building Your own):
- Wooden chest (mine was 24x16x8,5 cm)
- Regular door bolt lock
- 6x metal 90degree brackets
- 4x plastic spacers (came with gesture tracking shield)
- some wooden screws of various length
- paper clip
- 1mm thick rubber sheet
Some pictures to show all the stuff:
| {gallery} MagickLockbox components |
|---|
 |
 |
 |
Needed software.
- AzureSphere SDK ver. 19.10
- Code editor of choice (I went with VS2019 as I started when it was mandatory)
- A7 application (so called High level app) source code located in GitHub (https://github.com/J-Wrobel/magiclockbox)
- basic knowledge around AzureSphere - if not present see the links at the end for tutorials
Project started to evolve from demo project posted by Avnet team (links at the end). As I gradually learned my ways around AzureSphere I started adding features and correct some minor issues. I also ported library provided by Seeed Studios for the MGC3130 shield so it can operate with AzureSphere.
Detailed operation description.
Basic scenario of usage would be to place something precious in the box. Then clicking A button will give You 5 seconds to close the lid before the lock will move the bolt. After that the list of currently recognized gestures is emptied. Now board sits and waits for gestures. When it recognizes one it will wait for one second before saving it to the list. If another gesture is recognized before one second expires the previous one is discarded. It will allow You to skip some gestures which have to be performed to get to the desired ones like for example rotating the box from x facing bottom to x facing up (it requires going through z facing up). Now, if gesture list will overflow without matching the recipe or 20 seconds will expire with no new gestures the list will be emptied. If the recipe will match the list of currently observed gestures unlocking will be performed in 5 sec (same drill as locking, might be good to change that a bit in the future).
By adding cloud connectivity, lockbox is reporting its state to cloud, allows for remote lock manipulation and change of unlocking recipe.
Build details.
Electronics.
The electronic side of the prototype is fairly simple. AZS is the base plus there is a DIY-MikroE Click extension board. It houses servo power switching circuit and some connectors for MGC3130 shield and servo.
I decided to make a power switching circuit for the servo because it can still draw some small power even if no signal is fed into it and it is not desired in battery powered devices. Beside that I never tried this kind of switching circuit so I thought it will be fun. Just as a tip for You it is very nice to simulate Your circuit even if You know what You are doing. Sites like this one http://tinyurl.com/y46rdga9 can be used for it.
To connect MGC3130 shield two cables are used. Grove I2C connector connects directly to AZS and carries 3v3, GND, SDA and SCL signals. Additional connection is made between the extension board and the shield. It carries another 3v3 signal routed to different pin than the Grove connector (learned the hard way that both need to be connected) and Transfer and Reset signals. Transfer is used to control data transfer on I2C bus and reset is used to reset the device. For convenience dupont cable got its single headers replaced with 1x6 header. This way you can have clearly defined wire order and You do not have to think about it when reconnecting the boards. Additional advantage is that You can rearrange it later if You need some small change i.e. wire swap. Just gently pull up plastic tooth that hold cable in place with fine tip tools an pull the cable out of header.
To speed up hand soldering on protoboard I roughly planned the routing of signals before hand in Fritizing using protoboard object and some wire connections.
Schematics for the electronics can be found below and in Git repo.
| {gallery} My Gallery Title |
|---|
 |
|
 |
 |
 |
 |
There is not much beside that, only connecting power bank to AZS with USB cable to give life to the prototype.
Mechanics.
The mechanical part of the prototype consists of three things: the box, the lock and mounting parts.
The box is wooden chest that is commonly bought for being decorated. Not much to add here - it is just a box. Maybe some suggestions are in place, next time I would buy one with a bit thicker walls and a bit deeper lid for easier installation of lock and boards.
The lock is based on regular door bolt lock bought in hardware store. I took the bolting part and attached servo to. Attachment is made with paper clip that has been straightened and one end made into a loop. The loop end was tightened around bolt handle. The other one was bent in Z shape inside servo arm. Bolt has been screwed into the box wall marking the place with a pencil before hand. Servo has been pre configured to move around 90dgrees as this was the distance it needed to slide bolt into lock position. Once the lock and servo were in 'locked' position the place to screw in servo could be marked. Servo has been attached to wall using two metal brackets with two pieces of 1mm rubber sheet sandwiched with servo (see pictures). One of 90 degree mounting brackets served as a piece that hang from lid and is held by bolt.
Board have been mounted using plastic spacers and 90 degree brackets. Each board had two brackets, two spacers and 4 plastic screws (2,5M I believe). Each bracket was equipped with one spacer and two screws. Each board received such bracket into two mounting holes on one diagonal.
Power bank was mounted similar to servo - it was pushed down to the bottom of the chest with one mounting bracket and two pieces of rubber sheet to discourage it from moving.
| {gallery} My Gallery Title |
|---|
|
 |
 |
 |
 |
 |
 |
 |
 |
 |
Cloud setup.
Cloud setup is pretty simple. First You should create Your account in Microsoft services and get to know the Iot Central a bit. Tutorials on element14 (links at the bottom) describe it nicely so I will skip to the device template setup.
This project uses cloud in three ways: to upload lockbox state, to allow remote lock switching and to setup unlocking gesture recipe. States which are reported are:
- system:initialize/ready - used to indicate system restart
- motion:active/inactive - used to notify when lockbox is being moved
- lock:unlocked/locked - used to notify lock state
Beside these functionalities the board can be operated without cloud connection. Setting gesture recipe is done through so called device twin - set of settings which are synchronized between real object and cloud object representing the real one. Recipe is encoded as a string of characters each representing one gesture. So for example 'lrx' would mean swipe left, swipe right and tap on x axis of box. For more details I encourage You tot take a look into source code. Changing lock state remotely is done by running direct method triggered by cloud on real device (tutorial for this down below).
Here is small demo of what cloud is capable of:
Setup of device template can be found in the pictures below (tip: you can use windows problem reporting tool to make screenshots in bulk).
| {gallery} Cloud setup |
|---|
|
 |
 |
Credits, notes and links.
Many thanks to Avnet team for tutorials and demo projects which helped kick-start this project:
- AzureSphere tutorials including Cloud setup guide (https://www.element14.com/community/groups/azuresphere/blog/2019/04/24/avnets-azure-sphere-starter-kit-out-of-box-demo-part-1-of-3)
- Demo project (https://github.com/CloudConnectKits/Azure_Sphere_SK_ADC_RTApp)
- Direct Method tutorial (https://www.element14.com/community/groups/azuresphere/blog/2019/09/12/implementing-direct-methods-in-azure-sphere)
Many thanks to Seeed Studios for providing the library for the shield (https://github.com/Seeed-Studio/Seeed_mgc3x30)
MGC3130 library datasheet http://ww1.microchip.com/downloads/en/DeviceDoc/40001718E.pdf
Source code for the project located in GitHub (https://github.com/J-Wrobel/magiclockbox)
Notes with thoughts on improvements:
- replace current gestures fixed array with circular buffer
- remove delay when unlocking
- move whole I2C communication or even whole MagicLockbox specific app to m4 core for better real time operating
- add more gestures like double tap, free fall, hand making circles etc. when above is done
- add sensing of closed chest with magnetic reed switch
- design and 3d print fancier bolt lock
Top Comments
Hello Dubbie, glad You like it. The delay You noticed is actually purely implementation quirk. It can be easily removed with small change in code. The cloud is purely for data acquisition and board configuration…
Hi Kevin, nice to hear that. I have not considered this project to be long term involvement. Rather short and fun way of unconventional usage of tech. But since there seems to be some interest in this…
I imagine twin boxes in which there is a gesture with which one opens the other remotely, for brothers or remote friends...