Hello
After playing a little with Adafruit MQTT python client and feeds, I decided to change the data transmission protocol between nodes, from HTTP requests to MQTT, RaspberryPi acting as broker+client and ESP8266 nodes being clients.
Looks like Mosquitto is a very popular MQTT broker among RaspberryPi users (and not only), installation on RPi side is pretty straightforward, since Mosquitto is included in Raspbian Jessie repositories:
sudo apt-get install mosquitto mosquitto-clients
Mosquitto parameters and behavior are imported from a configuration file (/etc/mosquitto/mosquitto.conf) - more details on it's configuration can be found on Mosquitto site.
Mosquitto broker can be started with sudo /usr/sbin/mosquitto -c etc/mosquitto/mosquitto.conf or after reboot it is automatically started.
On RaspberryPi3 client side I used Eclipse Paho MQTT Python Client library, installation package, installation instructions, examples and documentation can be found on Paho project site and Github and on PyPi website. Install can be as simple like this:
pip install paho-mqtt
or using the full code from Github repository and setup instructions from PyPi.
On ESP8266 side, I used PubSubClient for Arduino by Nick O'Leary, install package and documentation can be found on his Github page and website.
How it works
For those without a previous experience on MQTT, MQTT is a Client/Server publish/subscribe messaging transport protocol designed to be lightweight and easy to implement. The server (broker) manage topics on which clients publish data or subscribe to topics to retrieve data. A few key points of this system are:
- publishers and subscribers are decoupled from each other, they have to know only the identity of the broker which manages and eventually filter messages.
- publisher and subscriber don't have to run at the same time and publishing/subscribing don't have to be synchronized in any way.
- operation of publishers and subscribers is not halted during publishing/subscribing process.
As I said before, on my project RaspberryPi play the role of central node, by running the Mosquitto broker and a MQTT Python client which subscribe to sensors feed topic in order to collect data sent by WiFi nodes and store it locally for further processing. The subscribe client is implemented in pilot_mqtt.py available on Github repository. Code is pretty much commented so I detail only a couple of points:
Data received by subscribing client is stored in two folders:
DIR_BASEINST = "/var/www/html/datainst/" - here are stored instantaneous values read by sensors and these values are later published on sensor feeds for Adafruit dashboard
DIR_BASEHIST = "/var/www/html/datahist/" - here are stored all values received from wireless nodes and put in individual files for each node
mqttc = mqtt.Client()
mqttc.on_message = on_message - this is executed when a message arrive on "sensorfeed" topic without a predefined target (temperature, humidity, etc)
# client follow and process only messages published on following topics, by calling corresponding functions
mqttc.message_callback_add("sensorsfeed/temperature/#", on_message_temperature) - on_message_temperature is called when a message arrive on "sensorsfeed/temperature/#" topic
mqttc.message_callback_add("sensorsfeed/humidity/#", on_message_humidity)
mqttc.message_callback_add("sensorsfeed/light/#", on_message_light)
mqttc.message_callback_add("sensorsfeed/detection/#", on_message_detection)
Character "#" is used as wildcard for all levels under "sensorsfeed/temperature/" for example.
Next part is the code which deal with received temperature values:
def on_message_temperature(mqttc, obj, msg):
global receivedvaltemp
receivedvaltemp = str(msg.payload) - this is the payload of message published by wireless nodes
recsplit = receivedvaltemp.split("=") - payload is split in temperature value and MAC address of the node
tempval = recsplit[1]
nodemac = recsplit[2]
print "Node MAC from temperature feed = ", nodemac
tempdatahist = DIR_BASEHIST + nodemac + "_temphist" - this is the name of the file which store logged data
tempdatainst = DIR_BASEINST + nodemac + "_tempinst" - this is the name of the file which store instant values
# fhtdh = file handler temp data hist - this file store a log temperature values sent by nodes
# if a log file for temperature data of a node did not exist, it is created - nodes can be added on the fly without previous configuration
if not exists(tempdatahist): - if the log file did not exist it is created and then values are write inside it. This is useful because log files for new nodes are added on the fly without any previous configuration
fhtdh = open(tempdatahist,'w')
fhtdh.close()
fhtdh = open(tempdatahist,'a')
fhtdh.write(time.strftime("%D-%H:%M:%S ") + tempval + '\n')
fhtdh.close()
# fhtdi = file handler temp data inst - this file store only the last temperature value sent by nodes
# being an instantaneous value, the content of the file is overwritten every time
fhtdi = open(tempdatainst,'w')
fhtdi.write(tempval + " " + nodemac)
fhtdi.close()
print "Temperature ", tempval
Next is the publisher\subscriber client implemented on wireless ESP8266 modules with the help of PubSubClient for Arduino. Code for ESP modules is on Github repository - esp_mqtt.ino
On the publish side, the ESP node works as follow:
- connect to wireless router through WiFi
- once wireless connection is established, attempt to connect to MQTT broker
- on the subscribing part, ESP node is subscribing to "sensorsfeed/commands/espMAC" topic. espMAC is the node own MAC address and is read on the run time. In this way each node subscribe to own commands topic for receiving commands.
- on the publishing part, ESP node is publishing to "sensorsfeed/temperature" topic, on which pilot_mqtt.py client is listening. The message payload contain temperature value and MAC address of publishing node. In this way the subscribing python client can identify the source of temperature values and put these values in corresponding log files.
As in HTTP version, ada_mqtt.py client is used to publish instant temperature values to Adafruit dashboard and is located in /home/pi/pilot along with node_config.txt file.
Hardware
On the hardware side, I needed more ESP nodes in order to collect environment values from multiple points around the house. For these new nodes I decided to use ESP-01 modules, they are smaller and I hope I'll be able to embed them more easily in common things around the house. They have only two GPIOs available but I think it will be enough for now.
One thing I noticed, the DS18B20 temperature sensor don't like to be nearby live electronics, especially in the vicinity of the RF part of ESP modules. Temperature readings were at least 5 deg. Celsius higher that in reality. So I moved sensor from onboard socket on a 10cm twisted wire and adverse effects disappeared.
Github repository for Mosquitto version.
Top Comments