Introduction
Diesel vehicles are getting a bad rap these days. According to carmagazine.co.uk (February 2022):
“The UK is falling out of love with diesel fast and official new car sales figures show the collapse in public trust continues, with registrations of oil-burners plummeting to just 4.8% market-share in 2021, down from 11.9% in 2020 and a quarter of all car sales in 2019. How quickly motorists' behaviour has changed.”
So was the infamous Volkswagen emissions scandal in 2015 the trigger? Who knows and I’m not one to speculate.
But one thing is for sure, we can now monitor our own car exhaust emissions, whether it's diesel or petrol, especially if we have a Particulate Matter sensor and an Arduino to hand together with a WiFi router and access to the world wide web.
Let me show you how.
| {gallery}My Emissions Monitor |
|---|
|
|
|
|
Arduino MKR1000 Setup
The microcontroller (MCU) I’m using for this application is an Arduino MKR1000, which uses an Arm® Cortex®-M0 32-bit SAMD21 processor and an ATSAMW25 (ATWINC1500) SoC Wi-Fi module.
I’m using two particulate matter sensors, both of which provide a UART serial interface.
I attached the Panasonic SN-GCJA5 TX channel to pin 13 (Serial1 RX) on the Arduino MKR1000. The Panasonic SN-GCJA5 uses 9600 baud rate to send 8 bits of data with Even Parity and a single stop bit (8E1). For details on how to use the sensor’s UART TTL port, please refer to my getting started guide:
The Sharp GP2Y1026 sensor also uses UART TTL but in this case it uses 2400 baud with the default 8N1 configuration. To connect the GP2Y1026 sensor to the Arduino MKR1000, I needed to create an additional UART port.
However, the standard SoftwareSerial library does not work with SAMD21 processors as this processor allows you to create your own configurable UART hardware interface using something called a SERCOM peripheral. There is a tutorial available on Arduino explaining how to do it:
https://docs.arduino.cc/tutorials/communication/SamdSercom
Once I had my additional UART instance I could then parse the GP2Y1026 sensor data. It follows a similar structure to the Panasonic SN-GCJA5, by using defined start and stop byte values to frame the data.
I found this handy online reference very useful in explaining how the GP2Y1026 sensor works:
To use the WiFi module an additional library is required for the MKR1000, namely the WiFi101 library. This can be downloaded using the Arduino IDE’s Manage Libraries menu option.
Finally, I wanted to send data to the cloud, or to Google Sheets in particular, so as to allow me to analyse the data remotely. But Google Sheets uses https and I was not able to connect to the Google Sheets server using the standard WiFi101 library examples in order to post data.
I did find this tutorial for an ESP8266 on the Arduino Projects hub, which used a separate library called httpsredirect.h to support the process, but I did not try it:
https://create.arduino.cc/projecthub/24Ishan/log-temperature-data-to-google-sheets-0b189b
Then there was this project for an MKR1000, which used an online middleman to handle the process for you:
https://create.arduino.cc/projecthub/detox/send-mkr1000-data-to-google-sheets-1175ca
But this simply used an insecure http connection instead.
So I decided to go with PubNub.com, a real-time communications or messaging platform, as this allowed me much better control with minimal latency through a more secure connection, if and when I required it. What I find appealing about PubNub is that it provides serverless functions as a standard feature so it's much more powerful than a MQTT broker service, and it’s simpler and cheaper to use than say Amazon AWS, Microsoft Azure or Google Cloud Platform, for example. It’s perfect, in my opinion, for field testing (I am sure it’s also perfect for production purposes at scale but as I have not done this personally, I could not comment).
There is also a handy PubNub library available through the Arduino IDE Library Manager to make things easy to get started with.
All I had to do now was combine all these elements into one Arduino sketch… and here it is:
/*
This example for the Arduino MKR1000 board connects to local WiFi router
using the SSID and WPA encryption using the WiFi password provided.
The original WiFi client sketch (as part of the WiFi101 library) was created on
13 July 2010 by dlf (Metodo2 srl) and modified 31 May 2012 by Tom Igoe
This sketch also uses the PubNub sample client using the Pubnub library, which will
publish raw (JSON pre-encoded) PubNub messages. In addition this sketch adds in the UUID
for unique device identification through the published channel. It can also be used for
presence detection.
Attached to the Arduino MKR100 are two dust sensors. Both use UART for communication.
The Panasonic dust sensor uses Serial1 and the Sharp dust sensor uses a custom UART channel
created with the SERCOM peripheral driver.
When the Panasonic dust sensor is first powered on, the spec sheet says there's an 8 second delay before 1st reading
Then we will have 20 seconds before stable readings occur. For the purposes of this sketch
we assume the first 20 readings are always unstable. This sketch does not take into account
the sensor being unplugged while the sketch is running and plugged back in while MCU running.
This application sketch created 30 March 2022
by C Gerrish (BigG)
Permission to use, copy, modify, and/or distribute this software for any
purpose with or without fee is hereby granted.
This code is in the public domain.
*/
#include "wiring_private.h"
#include <SPI.h>
#include <WiFi101.h>
#include "arduino_secrets.h"
#define PubNub_BASE_CLIENT WiFiClient
#include <PubNub.h>
#include <Adafruit_NeoPixel.h>
// Custom UART for Sharp GP2Y sensor
#define GP2Y_RX_PIN 1
#define GP2Y_DUMMYTX_PIN 0
// Pin connected to the NeoPixels
#define LED_PIN 4
// Number of NeoPixels
#define NUMPIXELS 5
///////please enter your sensitive data in the Secret tab/arduino_secrets.h
const static char ssid[] = SECRET_SSID; // your network SSID (name)
const static char pass[] = SECRET_PASS; // your network password (use for WPA, or use as key for WEP)
const static char pubkey[] = PUBNUB_PUBLISHKEY; // stored in arduino_secrets.h (not shown)
const static char subkey[] = PUBNUB_SUBSCRIBEKEY; // stored in arduino_secrets.h (not shown)
const static char uuidkey[] = PUBNUB_UUID; // stored in arduino_secrets.h (not shown)
const static char channel[] = "mkr_partmatter";
const int STARTUPDELAY = 8000; // This is specified in the Panasonic sensor spec
const byte PLSIZE = 30;
const byte STABLE = 20;
uint32_t PM1_RegTotal = 0;
uint32_t PM25_RegTotal = 0;
uint32_t PM10_RegTotal = 0;
uint8_t PM_Cntr = 1;
uint32_t t_byte = 0;
byte Payload[PLSIZE] = {0};
byte cntr = 0;
byte StableCntr = 0;
bool Header = false;
bool StableFlag = false;
bool readDat = false;
uint8_t Vdat[4] = {0};
double Ugm3MTOT = 0;
uint8_t Ugm3Cntr = 1;
bool UgmRead = false; // Assume readings are repeated due to high frequency of output
uint8_t Vcsum = 0;
uint8_t i = 0;
Uart myGP2Y (&sercom3, GP2Y_DUMMYTX_PIN, GP2Y_RX_PIN, SERCOM_RX_PAD_1, UART_TX_PAD_0); // Create the new UART instance assigning it to pin 1 and 0
// Declare our NeoPixel strip object:
Adafruit_NeoPixel NeoPix(NUMPIXELS, LED_PIN, NEO_GRB + NEO_KHZ800);
int status = WL_IDLE_STATUS;
String http_postData = "";
// Attach the interrupt handler to the SERCOM
void SERCOM3_Handler() {
myGP2Y.IrqHandler();
}
void setup() {
http_postData.reserve(160);
// put your setup code here, to run once:
pinPeripheral(GP2Y_RX_PIN, PIO_SERCOM); //Assign RX function to pin 0
pinPeripheral(GP2Y_DUMMYTX_PIN, PIO_SERCOM); //Assign TX function to pin 1
Serial.begin(115200); // Initialize serial port for debug
Serial1.begin(9600, SERIAL_8E1); // Initialize UART connected to the Panasonic SN-GCJA5 PM Sensor
myGP2Y.begin(2400); // Initialize UART connected to the Sharp GP2Y1026 PM Sensor
delay(1000);
uint32_t t_now = millis();
// check for the presence of the shield:
if (WiFi.status() == WL_NO_SHIELD) {
Serial.println("WiFi shield not present");
// don't continue:
while (true);
}
// attempt to connect to WiFi network:
while (status != WL_CONNECTED) {
Serial.print("Attempting to connect to SSID: ");
Serial.println(ssid);
// Connect to WPA/WPA2 network. Change this line if using open or WEP network:
status = WiFi.begin(ssid, pass);
// wait 10 seconds for connection if not connected:
if (status != WL_CONNECTED) delay(10000);
}
Serial.println("Connected to wifi");
printWiFiStatus();
// PubNub.begin(pubkey, subkey, uuidkey); // I made a library modification to include UUID - not essential
PubNub.begin(pubkey, subkey);
Serial.println("PubNub set up");
NeoPix.begin(); // Initialize NeoPixel strip object (REQUIRED)
delay(500);
NeoPix.clear(); // Set all pixel colors to 'off'
NeoPix.setBrightness(50);
NeoPix.show(); // Turn OFF all pixels ASAP
delay(500);
for(int i=0; i< NUMPIXELS; i++) { // For each pixel...
// NeoPix.Color() takes RGB values, from 0,0,0 up to 255,255,255
NeoPix.setPixelColor(i, NeoPix.Color(125, 0, 0));
NeoPix.show(); // Send the updated pixel colors to the hardware.
delay(500);
}
while ((millis() - t_now) < STARTUPDELAY) {;;}
NeoPix.clear(); // Set all pixel colors to 'off'
NeoPix.show(); // Turn OFF all pixels ASAP
}
void loop() {
// put your main code here, to run repeatedly:
if (Serial1.available()) {
byte c = Serial1.read();
//Serial.println(c, HEX); // read it and send it out Serial (USB)
if (Header) {
if (cntr < PLSIZE) {
Payload[cntr] = c;
cntr++;
}
else {
// Lookout for the trailer byte which is 0x03
if (c == 0x03) {
Header = false;
// Wait for stable readings and only extract data if there is no error
if (StableFlag) {
if (Payload[28]) {
Serial.print("Fault: ");
Serial.println(Payload[28], BIN);
}
// Extract all the Particle Mass Density values as per spec
uint32_t PM_Reg = (Payload[0] | Payload[1]<< 8 | Payload[2]<< 16 | Payload[3]<< 24);
PM1_RegTotal += PM_Reg;
PM_Reg = (Payload[4] | Payload[5]<< 8 | Payload[6]<< 16 | Payload[7]<< 24);
PM25_RegTotal += PM_Reg;
PM_Reg = (Payload[8] | Payload[9]<< 8 | Payload[10]<< 16 | Payload[11]<< 24);
PM10_RegTotal += PM_Reg;
UgmRead = true;
if (PM_Cntr < 10) PM_Cntr++; // increment counter
else {
// We take an average from 10 readings
PM1_RegTotal /= 10; // Determine the average over 10 samples
PM25_RegTotal /= 10; // Determine the average over 10 samples
PM10_RegTotal /= 10; // Determine the average over 10 samples
http_postData = "\"DataType=202&PM01="+String(PM1_RegTotal,DEC)+"&PM25="+String(PM25_RegTotal,DEC)+"&PM10="+String(PM10_RegTotal,DEC)+"\"";
WiFiClient* client = PubNub.publish(channel, http_postData.c_str());
if (0 == client) {
Serial.println("publishing error");
//delay(1000);
//return;
}
// Don't care about the outcome
else client->stop();
Serial.print("SN-GCJA5 (μg/m3): ");
Serial.print("PM1.0 = "); Serial.print(PM1_RegTotal, DEC);
Serial.print(" | PM2.5 = "); Serial.print(PM25_RegTotal, DEC);
Serial.print(" | PM10 = "); Serial.println(PM10_RegTotal, DEC);
// Update the NeoPixels to indicate PM concentration levels (using pixels 1, 2 & 3)
if (PM1_RegTotal <= 24) NeoPix.setPixelColor(0, NeoPix.Color(0, 125, 0));
else if (PM1_RegTotal > 24 && PM1_RegTotal <= 96) NeoPix.setPixelColor(0, NeoPix.Color(125, 75, 0));
else NeoPix.setPixelColor(0, NeoPix.Color(125, 0, 0));
if (PM25_RegTotal <= 24) NeoPix.setPixelColor(1, NeoPix.Color(0, 125, 0));
else if (PM25_RegTotal > 24 && PM25_RegTotal <= 96) NeoPix.setPixelColor(1, NeoPix.Color(125, 75, 0));
else NeoPix.setPixelColor(1, NeoPix.Color(125, 0, 0));
if (PM10_RegTotal <= 24) NeoPix.setPixelColor(2, NeoPix.Color(0, 125, 0));
else if (PM10_RegTotal > 24 && PM10_RegTotal <= 96) NeoPix.setPixelColor(2, NeoPix.Color(125, 75, 0));
else NeoPix.setPixelColor(2, NeoPix.Color(125, 0, 0));
NeoPix.setPixelColor(3, NeoPix.Color(0, 0, 0));
NeoPix.show(); // Send the updated pixel colors to the hardware.
PM_Cntr = 1;
PM1_RegTotal = 0;
PM25_RegTotal = 0;
PM10_RegTotal = 0;
}
}
}
}
}
else {
// Lookout for the header byte which is 0x02
if (c == 0x02) {
//Serial.println(c, HEX);
Header = true;
cntr = 0;
if (StableCntr < STABLE) StableCntr++;
else StableFlag = true;
}
}
}
if (myGP2Y.available()) {
byte b = myGP2Y.read();
// Syncing in with the Panasonic sensor
if (StableFlag && UgmRead) {
if (b == 0xaa) readDat = true;
else if (b == 0xff) {
if (i == 4) {
if (Vcsum == (Vdat[0] + Vdat[1] + Vdat[2] + Vdat[3])) {
//Serial.print("\r\nChecksum match: ");
//Serial.println(Vcsum, HEX);
double Vout = (((Vdat[0]*256.0) + Vdat[1])/1024.0)* 5000.0;
double Ugm3 = Vout / 3.5;
UgmRead = false;
Ugm3MTOT += Ugm3;
// update the sample counter
if (Ugm3Cntr < 10) Ugm3Cntr++;
else {
Ugm3MTOT /= 10; // Determine the average over 10 samples
Ugm3Cntr = 1;
http_postData = "\"DataType=101&PM="+String(uint16_t(Ugm3MTOT+0.5),DEC)+"\"";
WiFiClient* client = PubNub.publish(channel, http_postData.c_str());
if (0 == client) {
Serial.println("publishing error");
//delay(1000);
//return;
}
// Don't care about the outcome
else client->stop();
Serial.print("GP2Y PM Average: "); Serial.print(uint16_t(Ugm3MTOT+0.5),DEC); Serial.println(" ug/m3");
// Update the NeoPixels to indicate PM concentration levels (using pixel 5)
if (Ugm3MTOT <= 24.0) NeoPix.setPixelColor(4, NeoPix.Color(0, 125, 0));
else if (Ugm3MTOT > 24.0 && Ugm3MTOT <= 96) NeoPix.setPixelColor(4, NeoPix.Color(125, 75, 0));
else NeoPix.setPixelColor(4, NeoPix.Color(125, 0, 0));
Ugm3MTOT = 0;
}
}
else {
//Serial.println("\r\nChecksum ERROR");
}
}
readDat = false;
i = 0;
}
else {
if (readDat) {
if (i < 4) {
Vdat[i] = b;
i++;
}
else Vcsum = b;
}
}
}
}
}
void printWiFiStatus() {
// print the SSID of the network you're attached to:
Serial.print("SSID: ");
Serial.println(WiFi.SSID());
// print your WiFi shield's IP address:
IPAddress ip = WiFi.localIP();
Serial.print("IP Address: ");
Serial.println(ip);
// print the received signal strength:
long rssi = WiFi.RSSI();
Serial.print("signal strength (RSSI):");
Serial.print(rssi);
Serial.println(" dBm");
}
PubNub Serverless Functions and Google Apps Script Setup
There are two core elements to my cloud application. The first element utilises PubNub’s serverless functions to form the bridge between the microcontroller and Google Sheets. The second element uses a Google Apps Script to process https POST request, parse this data and append new data into the designated spreadsheets for permanent storage.
For those who are unfamiliar with PubNub’s serverless functions here’s a handy link to their website to help you get started:
https://www.pubnub.com/products/functions/
https://www.pubnub.com/docs/functions/overview
PubNub functions are written in Javascript. The first thing to decide is the trigger point, that is when to trigger the function itself. If I wanted to manipulate the data before it is published, for example, I would need to choose the “Before Publish or Fire” event trigger. In my case, I had no need for that and so I used the asynchronous trigger event “After Publish or Fire”.
The next thing that is required is to provide the “topic” or “channel” which will be monitored and acted upon through the serverless function.
Then it’s a case of writing the script… and here it is. All it is doing is forwarding the message received onto the Google Apps Script URL using a https POST request:
export default (request) => {
console.log(request.message);
const xhr = require('xhr');
const http_options = {
'method': 'POST', // or PUT
'mode': 'cors', // no-cors, *cors, same-origin
'cache': 'no-cache', // *default, no-cache, reload, force-cache, only-if-cached
'credentials': 'same-origin', // include, *same-origin, omit
'redirect': 'follow', // manual, *follow, error
'referrerPolicy': 'no-referrer', // no-referrer, *no-referrer-when-downgrade, origin, origin-when-cross-origin, same-origin, strict-origin, strict-origin-when-cross-origin, unsafe-url
'connection': 'close',
'headers': {
'Content-Type': 'application/x-www-form-urlencoded'
},
'body': `${request.message}`
};
console.log("Posting data..."); //This is the message being sent
const url = 'https://script.google.com/macros/s/#### - unique identifier - #####';
return xhr.fetch(url, http_options)
.then((res) => {
//console.log(res);
return request.ok('Request succeeded.');
})
.catch((err) => {
return request.abort(`Request failed: ${err}`);
});
};
My Google Apps Script is just as straightforward. This too is written in Javascript. Here I simply extract the data and append it into the appropriate table depending on whether the data is from the Sharp GP2Y1026 sensor or the Panasonic SN-GCJA5 sensor.
function doPost(e) {
var ss = SpreadsheetApp.openByUrl("https://docs.google.com/spreadsheets/d/#### - unique identifier - #####");
var sheet1 = ss.getSheetByName("SharpGP2Y");
var sheet2 = ss.getSheetByName("PanasonicSNGCJA5");
addEmissionsData(e,sheet1,sheet2);
}
function addEmissionsData(e,sheet1,sheet2) {
var TimeStamp = new Date();
var DataType = e.parameter.DataType;
if (DataType == 101) {
var PM = e.parameter.PM;
sheet1.appendRow([TimeStamp,PM]);
}
else if (DataType == 202) {
var PM01 = e.parameter.PM01;
var PM25 = e.parameter.PM25;
var PM10 = e.parameter.PM10;
sheet2.appendRow([TimeStamp,PM01,PM25,PM10]);
}
}
Application Demo's
Now onto the demos. The first demo is just a walk through the setup and how the system works.
The next two demos show the device in action. The first shows the emissions monitor perched on a ladder, approximately 1 meter above the surface. The second shows the emissions monitor on the ground, very close the car's exhaust outline.
Here are charts for the two sensors for the first test. It is very interesting to see just how dusty the environment was on the day I carried out this test as there is a large housing construction nearby where the vegetation has been removed exposing the dry soil. As such the sensor readings were all above 20. Still, it was possible to detect minor changes to readings due to exhaust emissions. This was when I had my foot on the accelerator to generate more emissions.
And here are charts for the two sensors for the second test. Similarly I was just as surprised with the test results. At first I thought there was something wrong with the sensors as the initial readings were very low, but then I realised that it had been raining and so would have cleaned the air. Amazing how good rain is. Then when I turned the car on, it became very clear that sensor location is very important. In this location the Sharp sensor was much better at detecting the particulate matter mass density changes than the Panasonic sensor, as the Panasonic sensor inlet was below the exhaust.
Overall, I am rather pleased with this setup. The remote uploading of data to Google sheets worked brilliantly and makes field testing that much easier.
And the best thing is... I think I won't be losing any sleep over the results. My diesel vehicle should be good for another 100,000 kilometers (hopefully).