Group Actions
Engagement
  • Author Author: javagoza
  • Date Created: Date Created
  • Views 2583 views
  • Likes 9 likes
  • Comments 12 comments
Related
Recommended

I was there! ... sensing citizens recycling.

javagoza

I was there ... sensing citizens recycling.

Introduction

 

This is an extended version of the project I Was There that I presented at Hackster.io. The version presented in Hackster.io is focused on a solution to issue tokens with a date and time signed by the device. As an example of this solution, I implemented a recycling container that issues tokens to reward citizens who use the recycling container. This extended version is focused on monitoring the uses of recycling bins.

 

The hackster.io version has a reed switch that allows you to monitor the uses of the container, this new version also incorporates a VCNL4040 proximity sensor with a maximum distance of 20 cm to detect if the container is already full or to detect misuse of it. Likewise, the module's ambient light sensor can detect other anomalies in the container, such as the presence of a flame.

"I was there" creates secure temporary QR tokens (JSON Web Token) signed by an Azure Sphere device for later verification.

 

The problem

Remotely monitor the uses of recycling bins

 

The solution

 

The container has:

  • a Reed Switch that detects the door opening and closing.
  • a Proximity and Ambient Light Sensor with Infrared Emitter

How does it work

 

 

  • The user approaches the IWT device.

  • Then, the user opens the containers door.

  • If the device has no message programmed it shows estimate capacity level.

  • When the door closes the device generates offline a new QR code that contains a signed JWT token, JSON Web Token, including an unique identifier and the date and time of the token issuance.

  • Then the device displays the QR code on a low power consumption E-Paper display.

  • Asynchronously, the device sends an event to the Azure cloud in order to:

    • record and monitor new QR requests

    • record times the container is used
    • record the internal ambient light in the container
    • record the estimate capacity remaining.

  • The user can capture the image with any camera for later validation or can use a specific application that validates the token at the moment.

  • After a configurable time the token disappears from the screen and the device is ready to generate the next token.

  • For loyalty programs the IWT device can issue programmed reward messages from the IOT Central/IOT Hub platform.

 

image

 

 

Prototype implementation of IWT for a program of awards to citizens who recycle.

 

Alternatively, the device may generate an inaudible sound signal that can be recorded with any sound recording device (implementation pending).

 

 

My first decision was to try to incorporate a crypto-processor module to perform the encryption, signatures and store the cryptographic keys. Unfortunately, I still did not receive the cryptographic module on time and had to opt for a software solution.

  • For cryptography I decided to use the wolfCrypt library. Fortunately wolfSSL released a Visual Studio solution that contains a cryptographic algorithm test for the wolfCrypt library.
  • JWT tokens are being formed directly using string functions without using any ad hoc library.
  • I have made an adaptation of the STM32 EPD driver for the waveshare 1.54 inch e-Paper Module display.
  • I have made an adaption of SparkFun VCNL4040 distance sensor library.

Design

The Hardware

Bill of Materials

 

Product NameManufacturerQuantityBuy KitBuy Kit
AES-MS-MT3620-SK-G -  Development Kit, Azure Sphere Starter KitAVNET1Buy NowBuy Now
29A-W -  Reed Switch, Standard Gap, 29 Series, 1NO, Surface Mount, 400 mA, 160 VDC, 10 WGEORGE RISK INDUSTRIES1Buy NowBuy Now
MCMF0W8FF4751A20MCMF0W8FF4751A20 -  Through Hole Resistor, 4.75 kohmMULTICOMP PRO1Buy NowBuy Now

 

Additional Parts

 

Product NameManufacturerQuantity
Waveshare 1.54inch e-Paper ModuleWaveshare1
SparkFun Proximity Sensor Breakout - 20cm, VCNL4040 (Qwiic)Spark Fun1

 

Schematics

  • EPaper display conected in Click 1 (2,3,4,6,7,8).
  • Reed Switch conected to PIN GPIO_42, Click 1-1.
  • SparkFun Proximity Sensor Breakout - 20cm, VCNL4040 (Qwiic)

 

 

image

 

The hardware consists of an Avnet Azure Sphere MT320 Starter Kit development kit and a Waveshare 1.54inch e-Paper Module electronic ink display.

 

The advantages of Microsoft Azure Sphere solution

 

image.png

Avnet Azure Sphere MT320 Starter Kit

Product LinkProduct Link

  • The Azure Sphere MT3620 Starter Kit supports rapid prototyping using Avnet’s certified Azure Sphere module, based on the Microsoft MT3620AN device.
  • The MT3620 is the first Azure Sphere certified "micro-controller", a SoC IoT device that features “end-to-end security”.
  • User applications can target it's 500 MHz ARM Cortex-A7 core as well as two general purpose 200 MHz ARM Cortex-M4F I/O subsystem cores designed to support real-time requirements. The on-chip peripherals (GPIO, UART, I2C, SPI, I2S, PWM and ADC) can be mapped to any of these three user-accessible cores.

image.png

Waveshare 1.54 inch e-Paper Module electronic ink display

  • The electronic ink screen has been chosen for its good qualities to present QR codes and its reduced power consumption in a project of these characteristics.
  • It is an E-Ink display module, 1.54 inch, 200 x 200 resolution, with embedded controller, communicating via SPI interface, and supports partial refresh.
  • Characteristics: low power consumption, wide viewing angle, clear display without electricity.

 

image.png
image.png

 

1.54inch e-Paper V2 is an Active Matrix Electrophoretic Display (AMEPD), with interface and a reference system design. The 1.54” active area contains 200×200 pixels, and has 1-bit B/W full display capabilities. An integrated circuit contains gate buffer, source buffer, interface, timing control logic, oscillator, DC-DC. SRAM.LUT, VCOM and border are supplied with each panel.

 

https://www.waveshare.com/wiki/1.54inch_e-Paper_Module

 

PART NUMBERCOLORSGRAY SCALERESOLUTIONDISPLAY SIZE (MM)OUTLINE DIMENSION (MM)FULL REFRESH TIME (S)PARTIAL REFRESHPI HEADER1INTERFACE
1.54inch e-Paper Moduleblack, white2200x20027.60 × 27.6048.0 × 33.02√×SPI

Interface

 

SYMBOLDESCRIPTION
VCC3.3V/5V
GNDGround
DINSPI MOSI pin
CLKSPI SCK pin
CSSPI chip selection, low active
DCData/Command selection (high for data, low for command)
RSTExternal reset, low active
BUSYBusy status output, high active

 

  • Different from the traditional SPI protocol, the data line from the slave to the master is hidden since the device only has a display requirement.
  • CS is slave chip select, when CS is low, the chip is enabled. (CS#) is the chip select input connecting to the MCU. The chip is enabled for MCU communication: only when CS# is pulled LOW.
  • DC is data/command control pin, when DC = 0, write command, when DC = 1, write data. (D/C#) is Data/Command control pin connecting to the MCU. When the pin is pulled HIGH, the data will be interpreted as data. When the pin is pulled LOW, the data will be interpreted as command.
  • SCLK is the SPI communication clock.
  • SDIN is the data line from the master to the slave in SPI communication.
  • (RES#) is reset signal input. The Reset is active low. Note 1.5-4: This pin (BUSY) is Busy state output pin. When Busy is High the operation of chip should not be interrupted and any commands should not be issued to the module. The driver IC will put Busy pin High when the driver IC is working such as: - Outputting display waveform; or - Communicating with digital temperature.
  • (BS1) is for 3-line SPI or 4-line SPI selection. When it is “Low”, 4-line SPI is selected. When it is “High”, 3-line SPI (9 bits SPI) is selected.

SPI communication has data transfer timing, which is combined by CPHA and CPOL.

  • 1. CPOL determines the level of the serial synchronous clock at idle state. When CPOL = 0, the level is Low. However, CPOL has little effect to the transmission.
  • 2. CPHA determines whether data is collected at the first clock edge or at the second clock edge of serial synchronous clock; when CPHL = 0, data is collected at the first clock edge.

There are 4 SPI communication modes. SPI0 is commonly used, in which CPHL = 0, CPOL = 0.

Data transmission starts at the first falling edge of SCLK, and 8 bits of data are transferred in one clock cycle. In here, SPI0 is in used, and data is transferred by bits, MSB first.

 

image

Click 1 Pinout diagram

This table shows how the pinout on eINK click corresponds to the pinout on the mikroBUSTm socket (the latter shown in the two middle columns).

NotesAVNET KITPinMikrobus logo.pngPinAVNET KITNotes
Reed SwitchGPIO_42AN1ANPWM16D/CGPIO_0Data/Config
ResetGPIO_16RST2RSTINT15BSYGPIO_2Busy indicator
SPI chip selectGPIO_34CS3CSTX14NC
SPI clockGPIO_31SCK4SCKRX13NC
NC5MISOSCL12NC
SPI data inputGPIO_32SDI6MOSISDA11NC
Power supply3V3+3.3V73.3V5V10NC
GroundGNDGND8GNDGND9GNDGround

image

 

SparkFun Proximity Sensor Breakout - 20cm, VCNL4040 (Qwiic)

 

https://www.vishay.com/docs/84307/designingvcnl4040.pdf

The VCNL4040 is a fully integrated proximity and ambient light sensor. It combines an infrared emitter and photodiode for proximity measurement, ambient light sensor, and signal processing IC in a single package with a 16-bit ADC. The device provides ambient light sensing to support conventional backlight and display brightness auto-adjustment, and proximity sensing to minimize accidental touch input that can lead to call drops and camera launch.

With a range of up to 20 cm (7.9"), this stand-alone component greatly simplifies the use and design-in of a proximity sensor in consumer and industrial applications, because no mechanical barriers are required to optically isolate the emitter from the detector.

 

  • Qualitative Detection Range: 20cm
  • Integrated Modules: Infrared Emitter (IRED), Ambient Light Sensor (ALS), Proximity Sensor (PS)
  • Operates ALS and PS in Parallel Structure
  • No Dead Zone
  • Operating Voltage: 3.3V
  • I2C Address: 0x60
  • 2x Qwiic Connectors

image

Click 2 Pinout diagram

This table shows how the pinout for the VCNL4040 Proximity Sensor corresponds to the pinout on the mikroBUSTm socket (the latter shown in the two middle columns).

 

NotesPinMikrobus logo.pngPinNotes
NC1ANPWM16NC

Interrupt output

GPIO17

RST2RSTINT15INT
NC3CSRX14NC
NC4SCKTX13NC
NC5MISOSCL12SCL

I2C Clock

SCL2

NC6MOSISDA11SDA

I2C Data

SDA2

Power Supply3.3V73.3V5V10NC
        GroundGND8GNDGND9GNDGround

 

image

 

Reed Switch and Magnet

 

The magnet is behind the door.

The circuit is closed when a magnet is near the switch, indicating the opening is shut. Opening the door breaks the connection.

 

Sensing the user

In addition to launching the new QR code when the button is pressed, we can activate it when it detects when the user using the recycling container using a proximity sensor.

 

  • The reed switch we'll be using is normally-open. We have two alternatives to detect when the door is fully closed o when the door is fully opened. For this project we will be using the first alternative:
  • Detect door fully closed: The magnet is close the Reed Switch when the door is closed and will be in that state until the door is opened. We connect the Reed Switch to ground and we will pull up the GPIO pin 42 via 4.7K. In closed stated will read Low. The event for a new QR will be sent when passing from High to Low, that is when the door is fully closed again, rewarding the user for closing the door image after use.

 

GPIO42_INPUT.png

 

20191124_220617.jpg

 

Sensing the container capacity

 

Inside the container a Proximity sensor allow us to estimate the remaining capacity of the container.

 

 

Software Design

State Diagram

  • Idle show the "Press A"
  • Rewards show reward associated to the QR if any
  • JWT_QR issue and show new QR Token
  • Clock show time clock
  • Settings show settings
  • Synch Settings sync settings with Azure cloud
  • Azure Cloud Async Tasks send/receive/synch cloud data

image

User Interface

 

Using the Reed Switch to detect door opening and closing:

When the door is open Cloud Programmed message  is displayed.

When the door is closed a new JWT QR is issued and displayed during a programable delay.

 

image

 

Using the A Button:

  • When the A Button is pressed the Cloud Programmed message  is displayed.
  • When the  A Button is released a new JWT QR is issued and displayed during a programable delay.

 

 

 

{gallery} User Interface

image

Idle Screen: During the Idle state the screen shows the indication of pressing the A button to obtain a new QR

image

Rewards Screen: Once button A has been pressed if a new prize status was received for the next QR, the prize associated to the QR is displayed. Option for customer loyalty programs.

image

QR Screen: Then the new QR code containing a signed JWT token. A time set from the Azure cloud will be in this state.

image

Clock Screen & Settings: When the B button is pressed, a clock with the current time of the device is displayed and if pressed again the application settings are displayed.

 

 

Source Code

 

Github Repository

I-Was-There  https://github.com/javagoza/I-Was-There

C Code for Azure Sphere Platform. Use Git or checkout with SVN using the web URL. wolfssl library it is not included.

WolfSSL

You will also need to clone WolfSSL library under wolfssl folder

https://github.com/wolfSSL/wolfssl

 

wolfSSL (formerly CyaSSL) is a small, fast, portable implementation of TLS/SSL for embedded devices to the cloud. wolfSSL supports up to TLS 1.3! https://www.wolfssl.com

 

Configuring your hardware and libraries

 

WolfSSL Settings

 

Build Options

Epaper PIN definition

Reed Switch

 

#define AVT_MODULE_GPIO42_ADC1   MT3620_GPIO42 
#define AVT_SK_CM1_AN AVT_MODULE_GPIO42_ADC1 
#define PROXIMITY_GPIO AVT_SK_CM1_AN

 

PIN Defines

#define AVT_MODULE_GPIO31_SCLK1_TXD1 (GPIO_Id)31
#define AVT_MODULE_GPIO32_MOSI1_RTS1_CLK1 (GPIO_Id)32
#define AVT_MODULE_GPIO33_MISO1_RXD1_DATA1 (GPIO_Id)33
#define AVT_MODULE_GPIO34_CSA1_CTS1 (GPIO_Id)34
#define EPAPER_DC (GPIO_Id)0
#define EPAPER_BUSY (GPIO_Id)2
#define EPAPER_RESET (GPIO_Id)16
#define EPAPER_SCLK AVT_MODULE_GPIO31_SCLK1_TXD1
#define EPAPER_MOSI AVT_MODULE_GPIO32_MOSI1_RTS1_CLK1
#define EPAPER_CS AVT_MODULE_GPIO34_CSA1_CTS1

 

Samples that demonstrate SPI functionality on the Azure Sphere platform:

 

https://github.com/Azure/azure-sphere-samples/tree/master/Samples/SPI

 

Creating the JWT Token QR

Anatomy of the JWT Token QR

The device will use QR codes like the one in the nest image:

 

IWTQR.png

 

 

The QR code contains this string:

 

 

eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9
.eyJqdGkiOiJlMmRhNzc3My1hN2NkLTQ0YTAtODVkNy00M2ZlMDY1NGNiMjItN2RjNTQ4MTEtNWRjYjBjNzAiLCJpYXQiOjE1NzM1ODgwODB9
.Dm8qKzv5n0EHMCcIECn08Ev_eppzPNc-hGtuICCNGK0

 

 

It is a JSON Web Token in its compact form. A JWT typically looks like the following: xxxxx.yyyyy.zzzzz

 

  • JSON Web Token (JWT) is a compact, URL-safe means of representing claims to be transferred between two parties. JSON Web Token (JWT) is an open standard (RFC 7519) .
  • The claims in a JWT are encoded as a JSON object that is used as the payload of a JSON Web Signature (JWS) structure or as the plaintext of a JSON Web Encryption (JWE) structure.
  • This information can be verified and trusted because it is digitally signed.
  • JWTs can be signed using a secret (with the HMAC algorithm) or a public/private key pair using RSA or ECDSA. https://tools.ietf.org/html/rfc7519

 

It consists of three parts separated by dots (.), which are:

  • Header: contains the metadata for the token and it minimally contains the type of signature and the encryption algorithm.
  • Payload/Claim: contains the information we want to transmit,
  • Signature: calculated using the header and the payload, you can also verify that the content hasn't been tampered with.

 

Contents of the three parts are Base64Url encoded. Decoding the three parts of the above token:

image.png

 

The Header: declares that the encoded object is a JSON Web Token, and that it is signed using the HS256 algorithm.
{ "alg": "HS256", "typ": "JWT" }
The Payload /Claim: we are using two registered claim names. In the context of JWT, a claim can be defined as a statement about an entity, as well as additional metadata about the token itself. The claim and that the server can use to properly handle JSON Web Token authentication. There are multiple claims we can provide; these include registered claim names, public claim names and private claim names.
  • iat: “Issued at” time, in Unix time, at which the token was issued
  • jti: JWT ID claim provides a unique identifier for the JWT
{ 
"jti": "e2da7773-a7cd-44a0-85d7-43fe0654cb22-7dc54811-5dcb0c70",
 "iat": 1573588080
 }
Signature: digital signature or Message Authentication Codes (MACs).
HMACSHA256( base64UrlEncode(header) + "." + base64UrlEncode(payload), [your-256-bit-secret] )

Securing the JWT Token

Azure sphere does not provide APIs to get hardware support/acceleration for cryptographic function calls from the A7 and/or M4 subsystems. So we are using wolfSSL embedded SSL library (formerly CyaSSL)

 

wolfSSL

The wolfSSL embedded SSL library (formerly CyaSSL) is a lightweight SSL/TLS library written in ANSI C and targeted for embedded, RTOS, and resource-constrained environments - primarily because of its small size, speed, and feature set. It is commonly used in standard operating environments as well because of its royalty-free pricing and excellent cross platform support. wolfSSL supports industry standards up to the current TLS 1.3 and DTLS 1.2 levels, is up to 20 times smaller than OpenSSL, wolfSSL is powered by the wolfCrypt library.

 

Keyed hash hmac

 

wolfCrypt currently provides HMAC for message digest needs. The structure Hmac is found in the header "wolfssl/wolfcrypt/hmac.h".
HMAC initialization is done with wc_HmacSetKey(). For the project we are using HMAC SHA-256

 

Generating the QR Code

 

We are using Richard Moore QR Code library, a simple library for generating QR codes in C, optimised for processing and memory constrained systems.

 

Features:
  • Stack-based (no heap necessary; but you can use heap if you want)
  • Low-memory footprint (relatively)
  • Compile-time stripping of unnecessary logic
  • MIT License;

https://github.com/ricmoo/QRCode

Going to Cloud

Implementation steps:

  • Create a custom IoT Central application
  • Create a device template
  • Add QR Request events to the Measurements Tab
  • Add Toggle controls to the Settings Tab: RGB LED controls, WWAN and APP LED controls
  • Add Application information to the Properties Tab
  • Application Version
  • Current SSID
  • BSSID
  • Current Wi-Fi Radio Frequency

 

Resources

 

 

Monitoring Container

 

  • Whenever a new weekly QR code is issued asynchronously an event to the Azure cloud, either to Azure IoT Central or to Azure IoT Hub as the application was built.

 

  • The points seen in the next image correspond to events of QR code generation after pressing the A button or when the reed switch detects a step from open to closed.

 

image.png

 

image.png

 

  • We can see a detailed list view of the events with the generated QR ID.
Programming new messages to show
  1. New award or information messages can be programmed from IoT Central or completely disabled.

     

 

image.png

 

 

Development tools

 


Fritzing parts

 

 

 

Work attribution:

 

 

 

 

 

 

This my device: Fullcarga1

 

image

  • in reply to Fred27

    Thank you very much for all the time you have spent reviewing my project.

     

    Again you are absolutely right about the origin of JWT. I have also found this blog of one of the people who proposed the first draft of JWT.

     

    http://www.thread-safe.com/2015/01/jwt-5-years-in-making-history.html

     

    Regarding the expiration date, it is not included in the token because the tokens generated by the device follow the OTP paradigm (One Time Password)

     

    //PAYLOAD:DATA
//{
//  "jti": "e2da7773-a7cd-44a0-85d7-43fe0654cb22-2c0e818c-5dbda386",
//  "iat" : 1572709254
//}

// jti: JWT ID claim provides a unique identifier for the JWT
// Jti: device_id-random-unix_time
// iat: “Issued at” time, in Unix time, at which the token was issued

    device_id

    device_id

     

    The Jti is formed by a unique device identifier concatenated with a randomly generated number and with the date and time in unix_time format, the device is not able to create more than one token every two seconds.

     

    "jti": "e2da7773-a7cd-44a0-85d7-43fe0654cb22-2c0e818c-5dbda386"

    • device-id: e2da7773-a7cd-44a0-85d7-43fe0654cb22
    • random: 2c0e818c
    • unix-time: 5dbda386

     

    So that in the case that you propose in which the token remains fixed on the display it would only be valid for the first citizen to claim it. At least that was my initial idea.

     

    But I really appreciate your suggestions, this community is great thanks to people like you.

     

    Good luck with your project!

  • in reply to javagoza

    I think you missed my point on expiry. JWTs are supposed to be invalid after a short period of time. This mitigates against a maliciously obtained token from being misused.

     

    For instance, let's say your device had a power failure and a JWT remains displayed on the ePaper screen. Anyone will then be able to use this token to "prove" they recycled an item. With a correctly expiring JWT this "problem" will only be a problem for the short lifetime of the token. In your scenario the security problem does not go away.

     

    Auth0 are definitely not the creators of JWT. JWT is an open industry standard. It's most extensively used for OpenID Connect (also an open standard) and Auth0 provide a commercial implementation.

  • in reply to Fred27

    You are right about security implementation. In the current implementation, a symmetric key that is passed as a parameter is being used, relying on the Azure Sphere security implementation, so you can update the key by the same mechanism of remotely updating the app. The reason for using a symmetric key instead of a public / private key scheme is because of the size of the display, although the tests I generated with RSA were readable by many mobiles, sometimes it was difficult to scan them. Regarding using JWT, it may seem somewhat strange to use it for secure information exchange instead of using it as an authentication token, but it is a use recommended even by Auth0, the creators of JWT

  • I like the concept, but a JWT seems a bit of a mismatch with the idea that the QR code can be verified later. A JWT is designed to expire. It has a specific expiry time (often an hour after an issue) and a properly secure implementation will also expire and rotate the signing keys on a regular basis.