Idea
I live in one family house in Poland which will be subject of hacking in this tutorial. This house is huge, old and has big energy demand.
The problem was we were using only gas furnace for heating and domestic hot water. This gas furnace was a bit old already, not really energy efficient and required frequent maintenance like cleaning from water deposit and carbon.
That is why we have installed photovoltaic panels for solar water heating, this solution is very energy efficient, maintanance free and very durable. Also at this moment we are waiting for PV panels with inverter for electrical energy generation.
Electricall current is being produced in six 280W polycrystalline photovoltaic panels each connected in series and is transported through cables to immersion heater placed in 250 liters tank.
This tank is constantly heated by solar energy through PV but also when there is little or no sun and demand for hot water is big the gas furnance needs to be turned on. This gas furnance is being turned on manually and also when it reaches certain temperature another water pump needs to be turned manually so hot water from gas furnance can be transported using pipes through water tank. In this way water in the tank is heated up from hot water running in pipe using convection. On the other side pump needs to be switched off when water temperature in gas furnance is lower then water in tank to avoid cooling down efect.
The photovoltaic controler is also very simple, its function is barely condition energy applied to heater. It is pulsing when there is little sun and switching transistor constantly on where voltage and thus current is close to nominal.
My project would involve fully automating process of water heating using photovoltac energy on one side and gas on the other. I will also collect telemetry like current produced by PV, energy, and temperature of heated water. My goal is to provide most confortable environment to live in and also most energy efficient. As switching device for pump I will use Idead Smart Switch - Sonoff S26 Hacked to work in offline mode (LAN).
Architecture of the system i came up with:
In this architecture Azure Sphere serves as telemetry broker and secure gateway to the cloud. Telemetry from sensors is being provided to Sphere OS by local server (Paspberry Pi) and by Real Time capable Application which is using ADC to sample data.
Connection between Cloud, Edge (Azure Sphere), Local Server (Pi3) and Sonoff Smart Switch is possibile thanks to WiFi.
Azure Sphere
Azure Sphere has been little modified. I have soldered USB type A socket to I2C header near edge (OLED Header) where other micro USB is. This way I can connect my OLED display using cable with usb TypeA on one side and 8 pin header on the other. Im using OLED configured to work in I2C mode so 4 lines from USB are perfect to communicate ( Vcc, GND, SDA, SCL).
I have also used current sensor ACS711EX wchih has output proportionall to measured current. Sensor is powered using 5V Vcc form Azure Sphere, its output voltage when current is 0 is at Vcc/2 wchih is equal to 2.5V. This output has been connected to Channel 1 of ADC controller. (Channel 0 is used by built in light sensor). This way when current increases, voltage decreases proportionall to sensitivity and it is measured by Real Time Core.
Like you can see on above photo, white cable goes directly to AN header on Socket 1 click, Red and black goes to 5V and GND respectively on Socket 1 click. This thick black and grey cables are transferring current to immersion heater. Sensor is connected in series on plus side.
High Level application is using:
- IoT Hub API to connect to IoT Central and send telemetry data or receive events - Secure Gateway Function.
- I2C bus to connect to OLED display - oled shows networking statuses, sensors values, generated energy and other diagnostic messages.
- Network socket (TCP) to connect to Local Server (RPi3) and send commands while receiving sensor data and statuses from Sonoff devices.
- Socket communication to RTApp to trigger ADC measurements and receive sensors data.
- Non volatile memory used to store total amound of produced energy.
Real Time Capable application (RTApp) is using:
- ADC converters to read output voltage of Current Sensor ACS711EX (-15A to +15A).
- Send ADC value to HighLevel App for data processig.
- It was enough for me to use bare metal application and synchronize with High level app using socket. I know that it is possible to use simple OS like FreeRTOS and i will try that in the future since real time M4 core of Azure Sphere has some power.
Local Server ( Raspberry Pi 3):
- Protocol Converter: converting data received from Azure Sphere on TCP port to WebSocket TCP data sent to actuators ( "Hacked" Sonoff S26)
- Python library pysonofflan to send commands to actuators working in LAN mode.
- Sensor Integrator: any exisitng sensor can be used in home automation like wireless WiFi sensors, blootooth sensors, I2C and so on. Also any application layer protocol can be used - HTTP, RESTApi, based on JSON or simple URL. Library pysonofflan uses WebSocket on TCP and sends data using JSON.
- Database Server: In case of internet connection lose I might be able to use SQL or non SQL database on Raspberry Pi to store data and send to IoT Central through Azure Sphere when internet is back on.
- In the future I would be able to use voice recognition to control my actuators like above Sonoff S26.
- Raspbian and python offers endless posibilities thanks to many available libraries for IoT and Smart Home.
"Hacked" Itead Sonoff S26
I was finally able to safely use my Sonoff and control it from the cloud !. This device has ORIGINAL firmware V.2.6.1. To connect it to original cloud and control it, you need to install App on your phone and configure your device using it. When I bought it few years ago it was fine for me but after some time I have noticed that Android Application is using a lot of battery of my phone and also it is sending some weird data to the cloud even if im not using it! It was the reason I had to uninstall it and loose posibility to control my Sonoff devices in the same time. Later on i have discovered a Hack that enables You to controll Sonoff devices using original firmware in local mode. Original firmware has built in mechanism which enables WebSocket interface to the device when it looses connection to Sonoff cloud, I have disabled internet connection on my router for all Sonoff devices and started to controll them using open source tool pysonofflan. Then I have been looking for some way to control them in more autonomus and smart way locally and also using safe cloud... and i have found Azure Sphere 
.
Some newer Sonoff devices like Basic R3，RFR3 and Mini has official LAN mode support (still need to use eWeLink app), others dont and you need to disable internet connectivity for them and you can start controlling them using WebSocket or RESTFull API (in older versions it depends on firmware they are running). You can find more info here https://github.com/itead/Sonoff_Devices_DIY_Tools or here https://github.com/beveradb/sonoff-lan-mode-homeassistant .
Itead Sonoff devices are very popular in custom home automation projects, Itead offers devices from smart wall plugs, internet controlled relays to motion sensors, light sensors, wall switches and light sockets. Examples below:
The biggest problem for some customers is that You need to use eWeLink App and chinese Cloud to control, some Itead devices offers official LAN mode using WebSockets or RESTfull API but still you need to use eWeLink App.
Most people are flashing custom firmware to their Itead Sonoff so they can use safe (mostly open source) cloud and control safely (I can highly recomend Supla Project). Using my approach you can use Sonoff with original firmware.
So first thing i have made to hack my Home was to force my Sonoff S26 to LAN mode. Then i have configured pysonofflan on Raspberry Pi to control it. After that i have connected Azure Sphere to Raspberry Pi using TCP connection over WiFi to send commands to sensors from Azure Sphere. Every time Raspberry Pi receives request from Azure Sphere it forwards it to appropriate device using pysonofflan library or other protocol wchih this IoT device is using.
On Azure Sphere side I have created retry mechanizm when connection with Raspberry Server is lost and also there is echo confirmation when request has been forwarded to device.
Video presenting early testing of my protocol can be seen below. OLED displays connection status to Raspberry Pi server as well as WiFi connection information.
Sonoff S26 smart switch is controlling little pump used to circulate water. Water in the pipes acts as heat exhanger between gas furnace and water tank.
Temperature on the gas furnace side (T1) and in the water tank (T2) is measured by two temperature sensors.
Algorithm for switching pump is simple, when T1 > T2 pump is switched on, otherwise pump is turned off to avoid cooling down.
Below algorithm is implemented on Azure Sphere. I have also added some protective mechanisms.
/// <summary>
/// Turn On or Off heat exhanging pump when needed
/// </summary>
static int HandlePumpControl()
{
if (TempSensor1 > TempSensor2 && PumpIsOff)
{
OnDelayCounter++;
OffDelayCounter = 0;
if (OnDelayCounter >= PUMP_DELAY_COUNT)
{
NewledState = !ledState;
OnDelayCounter = 0;
PumpIsOff = 0;
}
}
else if (TempSensor1 < TempSensor2 && !PumpIsOff)
{
OnDelayCounter = 0;
OffDelayCounter++;
if (OffDelayCounter >= PUMP_DELAY_COUNT)
{
NewledState = !ledState;
OffDelayCounter = 0;
PumpIsOff = 1;
}
}
}
Im using NewledState and ledState to trigger toggling on and off my Sonoff It will be refactored. The reason is im turning on red LED on Azure Sphere Avnet Starter kit when Sonoff is turned on.
Power from photovoltaic panes is constantly transfered to heater inside water tank. Value of current flowing to heater is dependent on amount of sun photovoltaic panels are getting and so on.
Current actual value is measured by ACS711EX connected to ADC 1 of Azure Sphere. ADC on the other hand is acceses by Real Time Core.
ACS711EX sensor is based on Hall efect, it converts current to proportional voltage. It looks as follows:
Thanks to holes on left side thick cables can be connected, this is needed in case of high currents. In my case current up to 9 Amps will be measured.
When current is 0 A output voltage is equal to sensor supply voltage divided by 2. (Vcc/2). In my case it is 5V/2 = 2.5V. When DC current starts flowing in either way ( from IP+ to IP- or reverse) the output voltage increases or decreases proportionally. In this way this sensor can measure current in both directions. It also has FAULT output which indicates that current has exceded maximal rated current.
Oled display serves few purposes. One of them is to see diagnostic information regarding connectivity. It would be WiFi network SSID, signal strength and status of connection to Raspberry Pi 3 server.
When connection to server is not present display shows actual error strerror(errno) (string.h) from socket. It works great and description is self explanatory.
Hardware connection to Azure Sphere is also very interesting. On display side it is connected using 8 pin female header, on other side is is connected using USB type A. This way I can anytime unplug my OLED and Azure Sphere from installation and take it at my desk leaving connection cable fixed to water tank.
Other important feature is displaying total Energy generated by photovoltaic panels.
Display is placed on water tank and connected to nearby Azure Sphere. Despite the fact that this place is far from WiFi router, connectivity is good.
It might be thanks to built in WiFi antena on MT3620 chip.
Also displayed screen is changed every 4 seconds so every parameter can be reviewed.
Raspberry Pi 3
Raspberry Pi3 has been used as central server for remote sensors. It has been set up to be able to integrate existing Home Automation devices and make them available to Azure Sphere.
I have used Pi3 running Raspbian Buster light (headless). I have also created little Proof of concept server or protocol converter in python. To run this script I'm using Python 3.7.3.
#socket_echo_server.py
import socket
import sys
import os
from pysonofflan import SonoffSwitch
from TempSensors import Sensor
#TODO: Decoding, proper feedback
# Create a TCP/IP socket
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
# Bind the socket to the port
server_address = ('192.168.1.122', 9999)
print('starting up on {} port {}'.format(*server_address))
sock.bind(server_address)
# Listen for incoming connections
sock.listen(1)
while True:
# Wait for a connection
print('waiting for a connection')
connection, client_address = sock.accept()
try:
print('connection from', client_address)
# Receive requests and echo for commands or resp for data requests
while True:
data = connection.recv(16)
msg = data.decode("utf-8").strip()
print('received' , msg, ' of type ', type(msg))
print('received {!r}'.format(data))
if b'on\x00\x00' == data:
#device = SonoffSwitch("192.168.1.169")
res = os.system('python3 pysonofflan --host 192.168.1.169 on')
print('sending data back to the client')
connection.sendall(data)
elif b'off\x00' == data:
print('off request received from: ', client_address)
res = os.system('python3 pysonofflan --host 192.168.1.169 off')
connection.sendall(data)
elif b'T1\x00\x00' == data:
s1 = Sensor(1)
Temp1S = s1.getTemp()
resp = '{0:04b}'.format(Temp1S)
connection.sendall(resp)
elif b'T2\x00\x00' == data:
s2 = Sensor(2)
Temp2S = s2.getTemp()
resp = '{0:04b}'.format(Temp2S)
connection.sendall(resp)
else:
print('invalid data received, bye')
connection.sendall(b'ERR\x00')
break;
finally:
# Clean up the connection
connection.close()
You can use it like this: python3 socket_echo_server.py & to run it in the backgroung. Like you can see im not good at python and i could not figure out how to convert binary to string and compare them inside so i were comparing requests in binary. Also i wasn't able to run pysonfflan as library so I have used os.system calls.
Raspberry Pi has endless posiblilities regarding home automation integration thanks to massive amounts of python libraries, tools and open source code. You can even make voice asistant like Alexa or Google Home of Raspberry Pi. Im very happy i could use Raspberry Pi and python in this project because I would like to improve my knowledge in this field
Normally I think someone should implemement better protocol between Azure Sphere and Raspberry Pi3 so that it would send IP address and command/request in one message and then forward it to appropriate sensor based on IP. For this purpose json files would be enough in my opinion. Above code is just working proof of concept that you can integrate many home automation modules using Raspberry Pi and control them using other devices or cloud.
Cloud
Azure IoT Central is used to store and visualize current (power) value through entire day. It is also collecting total energy value sent from Azure Sphere. Temperatures are sent also and visualied as bar charts.
I have created simple dashboard to monitor current value.
My approach was to just forward scaled and conditioned telemetry value to IoT Central. Rest of processing will be done there. For example I can view custom timeframe of my measurements, change displayed samples interval time despite actual interval with samples are sent to cloud. Drop markers on specific values or time or view data on table instead on chart.
IoT Central allows to analyze energy usage patterns to forecast generation and demand, extreamely usefull when we receive more photovoltaic panes with inverter for power generation. Also it is possible to download data locally as .csv or export data to Azure Blob storage.
On Azure IoT Central I have created Dashboard containing:
On Overview:
- Line chart showing average current over time. This way i can see how early sun went up or wheter it was cloudy day. It is also very important for diagnostic purposes. If I see that sun is up and it is strong but current produced is not as expected it could mean that panels are damaged or dirty. Ploting current value is also usefull for planing further modernizations or optimization. Panels on my house are placed on the roof at 40 degree angle. This angle can be modified manually. It is extremely important in summer or winter when sun is on different maximal height on the sky depending on season.
- Maximal Current value. Azure dashboard provides boxes showing KPI. I have choosen max value indicating max current value from 12 hours.
- Average current value from 12 hours. I have choosen this KPI to see how efective this day is so far or was because average current is proportional to power produced by system.
- Total Energy value. Cumulated total energy is sent from Azure Sphere every 4 minutes, It is stored in non volatile memory of MT3620, read on every power up and saved every 4 minutes. I think it is one of most important KPIs because it indicates how many energy my panels have poduced overall.
- Heat map showing Power produced over time. This way i can see which timeframe of the day was most efficient in energy production.
- Temperatures of water in heat exchanger heated by gas furnance and water in water tank. Bar chart representing this temperatures side by side is extremely efficient.
Like You can see on above screen this was very cloudy day, sun was at its max power at noon, temperature in water tank was higher (gray bar) than this one in gas furnance so pump was not activated.
Power generated was so far 2.19 kW.
Code
My code can be found here -> https://github.com/dbo01/HomeEnergyHandling. At the time of writing this it is far from ideal but it is working I have taken some ideas from Peter Fenn advanced tutorial source code and also from official microsoft azure sphere samples. In my free time I will cleanup and refactor this code so that I would be able to use it on other projects.
Some things i would do differently now:
- I would use certificate instead connection string to connect to IoT Central.
- I would use some existing protocol on top of socket instead of sending raw strings when communicating Azure Sphere with Pi 3. Or at least use json format because both Azure Sphere and raspberry pi can decode it out of the box. I had an idea of using Google Protocol Buffers and i will test it later on.
- I would configure ( and probably I will) power management for Azure Sphere so it could run on battery. Battery would be charged at day when energy is produced and Azure would go to sleep at night.
To sum up, while creating my project i was following changes in:
Advanced Tutorial from Peter Fenn (Avnet) -> https://github.com/Avnet/Azure_Sphere_SK_ADC_RTApp
Azure Sphere Samples my Microsoft -> https://github.com/Azure/azure-sphere-samples/tree/master/Samples
Bill of material
While creating my project i have used following devices or materials:
- Azure Sphere MT3620 Avnet Starter Kit
- Raspberry Pi 3
- Oled 0.96" Display
- Sonoff S26
- ACS711EX current sensor
- Sonoff TH16 x2 (Custom made using ESP12E + DS18B20)
- USB type A female header soldered to Azure Sphere I2C header
- 8 pin header connected to OLED header and on the other side to USB type A male
- Breadboard cables
This was very interesting project, most important for me was that i had full control over data sent to cloud and I was not restricted by cloud capabilities like in other commercial solutions made for smart home. Im aware that IoT Hub has more posibilities than IoT Central but i have choosen this for begining. I was able to hack and use Sonoff device with its original firmware, now every Sonoff can be used out of the box. Pysonofflan has feature of detecting connected Sonoffs (by its MAC addreses I believe) so only forwarding this to Azure Sphere and later to Azure IoT Central is left to implement to fully use any Sonoff out of the box!
Top Comments