[Forget me Knot Week 1] BASHing my head against the wall

So first of all. Not a heavy picture/video based blog this week. I did video capture some of my work, but most of that went out of the window as I had to re image the pi a couple of time after bashing my head against the wall a few times.

So... This week I received the EnOcean Pi, Sensors and the Raspberry Pi Model B+. As the MicroSD cards I have to hand are either too slow or are in use I've been using my old Raspberry Pi Model B until I set myself a decent MicroSD for the Pi.

I started work on trying to prep openHAB to use the EnOcean sensors. The documentation for getting the EnOcean Pi up and running with openHAB is out there, but is a bit sparse. Anyways if you are trying to set up openHAB to talk to your EnOcean PI the first thing you need to do is define the serial port in openhab.cfg Not a big deal, from a fresh install you need to copy the sample config file openhab_default.cfg to openhab.cfg in the configuration folder and then edit it so the line enocean:serialPort=/dev/ttyAMA0 has been added. The default config file contains a reference to the EnOcean binding around the 950 line mark which you can edit.

Sounds good right, you have configured openHAB to talk to the EnOcean PI. Nope... You need to also tell openHAB that you wish to use non standard serial ports too.

I spend a good amount of time bashing my head against the wall trying to figure out why openHAB would load the EnOcean plugin but refused to listen to the seasons. Well a good while of Googling later I stumbled across the Serial Binding page for openHAB (https://github.com/openhab/openhab/wiki/Serial-Binding) which states that you also need to add '-Dgnu.io.rxtx.SerialPorts=/dev/ttyAMA0' as a parameter when launching the java app. You can easily add that to start.sh or start_debug.sh launching scripts for openHAB but its something that can snag people. Keep a note of it.

So we are good tight? NOPE. So If you are lucky and openHAB connects to the EnOcean Pi after doing the above it doesn't mean that it will always connect. If you run openHAB and it connects and you decide that you are going to mess around with the setup so you need to restart openHAB, well openHAB doesn't cleanly tidy up after it self after sending a CTRL-C. It will disconnect from the EnOcean Pi but it will leave the lock file for ttyAMA0 in /run/lock/ in place. Well if that file is there openHAB will not connect to ttyAMA0 as it thinks it is in use by something else. If you issue the shutdown command in the openHAB console it will drop the connection and remove the lock file but you may still get error messages from the bash script until you kill the script.

OK so if you are manually starting and stopping and you issue a shutdown command and then you kill the bash script and you have made sure that there isn't a lock file in /run/lock/ you should be fine to restart the script and reconnect right? NOPE. If openHAB can not connect for any reason than the connection fails silently. I'm going to have a look at the source for the binding over the weekend and see if I can see why it fails silently. But enough of bashing my head against the wall. Lets get something working.

So as I stated above before starting openHAB there are a few things that need to be configured:-

• You will need to make sure that the Raspberry Pi isn't using the serial port on the GPIO header as a console.
  To do this manually you need to edit 2 files, /boot/cmdline.txt and /etc/inittab - In /boot/cmdline.txt you need to remove any references to ttyAMA0, in the latest version of Raspbian you just need to remove "console=ttyAMA0,115200" but you may also need to remove "kgdboc=ttyAMA0,115200". In /etc/inittab you need to remove or comment out the line "T0:23:respawn:/sbin/getty -L ttyAMA0 115200 vt100" in my build of Raspbian it was the last night of the file, to comment it out just put a # at the start of the line.
  
  To do this using a script then Andrew Scheller provides a nice script that can be used to disable/enable the serial terminal, his github page for the script (https://github.com/lurch/rpi-serial-console) provides all the info needed to getting this script running.
• Download and unzip the openHAB run time into a folder of your choice.
• Download and unzip the openHAB Add-on zip into the addons folder
  Doing the above will give you a basic openHAB setup, I did play around with the demo config but I wanted to build on top of a "clean" setup.
• Copy or rename openhab_default.cfg to openhab.cfg in the 'configurations' folder as openHAB will look for this file when it starts.
• Edit openhab.cfg to include the line enocean:serialPort=/dev/ttyAMA0
• You need to add the parameter '-Dgnu.io.rxtx.SerialPorts=/dev/ttyAMA0' to what ever method you use to start openHAB. openHAB does provide two bash scripts to start it start.sh and start_debug.sh which do what they say on the tin, they start openHAB either with normal logging or with debug logging enabled. I would recommend adding the parameter to both files. Don't add the parameter to the end of the line starting with "java" but put it somewhere after the word java. For example my line in start.sh reads :-
  

  java -Dosgi.clean=true -Declipse.ignoreApp=true -Dosgi.noShutdown=true -Dgnu.io.rxtx.SerialPorts=/dev/ttyAMA0 -Djetty.port=$HTTP_PORT -Djetty.port.ssl=$HTTPS_PORT -Djetty.home=. -Dlogback.configurationFile=configurations/logback.xml -Dfelix.fileinstall.dir=addons -Djava.library.path=lib -Djava.security.auth.login.config=./etc/login.conf -Dorg.quartz.properties=./etc/quartz.properties -Dequinox.ds.block_timeout=240000 -Dequinox.scr.waitTimeOnBlock=60000 -Dfelix.fileinstall.active.level=4 -Djava.awt.headless=true -jar $cp $* -console
• I also wanted some extra logging of the EnOcean system which allowed me to get the id's of the sensors I have. To do this I added the following lines to both logback.xml and logback_debug.xml files located in the 'configurations' folder:-
  

  <logger name="org.openhab.binding.enocean" level="DEBUG"/>

  <logger name="org.enocean" level="DEBUG"/>

  This will enable debug logging of the EnOcean addon which will let us capture the packets the EnOcean Pi receives and also get the id's for each of the sensors. When the sensors transmits a message we will now get a log entry like:-
  

  DEBUG o.o.b.e.i.bus.EnoceanBinding[:270]- Received new value Pressed for device at EnoceanParameter: {id="00:29:49:A3", channel="B", parameter="O"}

  which will give us the all important id for the sensor.
• I also created the files 'home.items' in the items folder inside the configurations folder and 'home.sitemap' in the sitemaps folder also in the configurations folder. For my first run of openHAB I only knew the id of one of my sensors (the rocker switch - it was printed on the back) so I was only able to set up that sensor. In home.items Following the details on the EnOcean bindings page on the openHAB github page (https://github.com/openhab/openhab/wiki/EnOcean-Binding) I added the following 2 lines:-
  

  Switch EnOcean_switch_002949A3_A {enocean="{id=00:29:49:A3, eep=F6:02:01, channel=A}"} 
  Switch EnOcean_switch_002949A3_B {enocean="{id=00:29:49:A3, eep=F6:02:01, channel=B}"}

  00:29:49:A3 being the id of my rocker switch. I will have to change my naming system as when I add more switches I'm going to start forgetting which id matches which switch, but I could just add the location of the switch to the name.
  in my home.sitemap file I added the following in order to visualize the status of the switch:-
  

  sitemap demo label="Main Menu" {
      Frame label="Sensors" {
          Switch item=EnOcean_switch_002949A3_A label="Switch"
          Switch item=EnOcean_switch_002949A3_B label="Switch"
      }
  }

  This well give us a simple iOS 7 style webpage that will show us the status of the switches just to make sure everything is working
• Before we can easily run the start scripts we need to give them executable permission, in the openHAB root folder just run the following command:-
  

  chmod +x *.sh
• We can now simply run the script to start openHAB, but before we do lets do one more thing, in a different terminal I'm going to run tail and grep for "enocean" on openHAB's log file. This will let me easily see the enocean log entries as I am having some issues with openHAB connecting to the EnOcean Pi but it will also let me easily see the id's for the other sensors. To do this in another terminal run the following command:-
  

  tail -f /your/path/to/openhab/logs/openhab.log | grep "enocean"

  Change /your/path/to/openhab/logs/openhab.log to where ever you have openHAB - for me that is /opt/openhab
  This will watch the log file but only return entries with "enocean" in them.
• I now ran ./start_debug.sh as I want as much data as I can get my hands on and let the script do its thing. If nothing went wrong after a while in the terminal running tail you should see:-
  

  INFO  o.o.b.e.i.bus.EnoceanBinding[:290]- Connecting to Enocean [serialPort='/dev/ttyAMA0' ].

  Nice, openHAB is doing its thing and once we have hit this line the webserver should be running and the EnOcean items should of also been loaded. We should now be able to navigate to http://<ADDRESS OF PI>:8080/openhab.app?sitemap=<SITEMAP NAME> for me using the name "home" for my items and sitemap this will be:-
  http://192.168.1.248:8080/openhab.app?sitemap=home and we should be presented with the default webpage style with two switches (one for each channel on the one switch)
  
  Well the items I defined in home.sitemap are being displayed, its a good start now lets see what happens if I press a button on the switch.
  
  Woo Hoo... The page auto updated and changed to reflect the button press. The tails|grep logging also showed data
  
  openHAB is receiving data from the sensors. So now lets press the buttons on the other two sensors I have. First the contact sensor (note: I cleared the terminal screen between these screenshots)
  
  Where we can see "sender=01:80:FE:68" that is the ID for the sensor, bingo. note that down and push the button on the temp sensor
  
  And there we have the id for the temp sensor too.
• I can now edit both home.items and home.sitemap which this information. In home.items again referring to the EnOcean binding page I included the following two lines:-
  

  Number EnOcean_sensor_TEMP "Temperature [%.1f C]" <temperature> (enocean) {enocean="{id=01:81:98:1C, eep=A5:02:05, parameter=TEMPERATURE}"}  
  Contact EnOcean_sensor_CONTACT "Contact [MAP(en.map):%s]" <contact> (enocean) {enocean="{id=01:80:FE:68, eep=D5:00:01, parameter=CONTACT_STATE"}

  and in home.sitemap I added the following:-
  

  Text item=EnOcean_sensor_TEMP
  Text item=EnOcean_sensor_CONTACT

  Just underneath the switch lines and saved both files. After a few seconds openHAB detected the changes in the files and refresh of the webpage now shows the status of the temperature and the contact sensors.
  
  So now I have the sensors talking to openHAB I can start playing around with setting rules and getting the data talking to other applications and logging the data.
• For logging the data I decided to use MySQL as its something I have experience in using and there is a simple to install package for it. Calling "sudo apt-get install mysql-server mysql-common -y' was all is needed to install the mysql server on the Raspberry Pi. I also installed lighttpd and php5-cgi because a) I'm feeling lazy and wanted to use phpMyAdmin to help manage MySQL and b) I will be using some simple php scripts for displaying my logged data. I won't go into details about setting up lighttpd in this blog post, if people ask for those details I will happily right them up in another post. Once I had MySQL/PHP/lighttpd/phpMyAdmin setup I created a user for openHAB which a matching database and created a created another user for reading the database and linked it to the same database.
  ,k
  With the details for MySQL in hand I edited openhab.cfg. There is one change you need to make as well giving the config file the details to log on and store data in the database. On line 35 of the default config file you need to change
  

  persistence:default=rrd4j

  to
  

  persistence:default=mysql

  This tells openHAB that you wish to use MySQL as your persistence storage. We also need to add the details too connect to the MySQL server, if you search though the config file for "mysql" you will find some example config options, edit these to connect to your MySQL server, you will need something like this:-
  

  mysql:url=jdbc:mysql://localhost:3306/openhab
  # the database user
  mysql:user=<USERNAME>
  # the database password
  mysql:password=<PASSWORD>
  mysql:sqltype.string=VARCHAR(20000)
  mysql:reconnectCnt=1 
  mysql:waitTimeout=60

  I'm using localhost in the connection url as the MySQL server is running on the Pi but you can connect to any MySQL server that allows remote connections. The refence to openhab at the end of the line tells openHAB the database to select.
  You also need to create a 'mysql.persist' file in the 'persistence' folder. To get myself up and running quickly I used a slightly modified example as given in the openHAB persistence github page (https://github.com/openhab/openhab/wiki/Persistence).
  

  // persistence strategies have a name and a definition and are referred to in the "Items" section
  Strategies {
      everyHour : "0 0 * * * ?"
      everyDay  : "0 0 0 * * ?"
      every15Mins : "0 0,15,30,45 * * * ?"
      // if no strategy is specified for an item entry below, the default list will be used
      default = everyChange
  }
  Items {
      * : strategy = everyChange, every15Mins
  }

  This will save every change into the database. But also update the DB every 15 mins. The reason I have it on every 15 mins is because if the connection to the server goes stale and openHAB hasn't noticed the connection to the server resets but the data sent from the sensor isn't saved (something else I need to look at over the weekend, I messed up on the above cron timings, I ended up writing to the DB every 5 seconds. Doh, tweaked it to 15 mins (hopefully)).
  
  I've not yet looked into the whole configurations for persistence so openHAB is just adding tables for the items as it see them with names such as "item1, item2, item3, etc". For a small number of devices its isn't an issue as I just told PHP to select which item table my temp sensor is, but its something I need to look into and see if I can choose the table name.
  
  Once I had saved these details in the config file I restart openHAB and restarted it again because it didn't connect to the EnOcean Pi (bangs head against the wall again - in fact if you look at my simple logging page later in the post you will see it took me a while to get it logging again. Must look at the source when I can.) Anyways with openHAB restarted we can see that it is connecting to the MySQL server by looking though the logs and looking for the string "to database"
  

  DEBUG o.o.p.m.i.MysqlPersistenceService[:361]- mySQL: Connected to database jdbc:mysql://localhost:3306/openhab

  I connected to the database using phpMyAdmin and I could start seeing data being inserted.
  
  From this point I can create PHP report pages however I like to display the data. This is also away from the openHAB server so I can publicly link to it and it doesn't create a bridge back to the openHAB server. I created a very simple PHP page to just print out this data https://pi.fmkapp.uk/temp.php This page is hosted on the Raspberry Pi using lighttpd and PHP and is pulling data from the MySQL DB running on the Pi which openHAB is injecting data into. The temperature drop from 1:20 AM to Midday was because I wanted to see if I could log the data from the temperature sensor whist it was sitting in my fridge, it turns out that a) I can :-) b) I need to adjust the temp in my fridge a little, its a little bit too warm in there.

Here is to next week. Hope with less head banging.

EDIT: Someone messaged me wanting to see the PHP Script. Its so basic I never even gave it a thought to include it, but here it is (with the MySQL password omitted)

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"
"http://www.w3.org/TR/html4/loose.dtd">
<html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=utf-8">
<meta http-equiv="refresh" content="60">
<meta name="viewport" content="width=device-width, user-scalable=no">
<title>Simple EnOcean Temp Logging</title>
</head>
<body>
<?PHP
// Simple print out of the mysql records for the temp sensor

// Connect to the database.
$db_db        = "openhab";
$db_user      = "openhab-php";
$db_pass      = "";

// Create connection
$db_con=mysqli_connect("localhost",$db_user,$db_pass,$db_db);

// Check connection
if (mysqli_connect_errno()) {
    exit("Failed to connect to MySQL: " . mysqli_connect_error());
}

// SQL for all the records for the temp sensor
$sql = "SELECT * FROM `Item2` ORDER BY `Item2`.`Time` DESC;";
$result = mysqli_query($db_con,$sql) or exit("Failed to query DB");

// Print output
while($row = mysqli_fetch_array($result)) {
    // openHAB is currently storing data in the DB in UTC so I'm cheating and adding an hour to the record
    echo("<DIV>".date("d/M/y - H:i:s",strtotime($row['Time'])+3600)." : ".number_format($row['Value'], 1, '.', '')." °C</DIV>\n");
}

// Close the DB
mysqli_close($db_con);
?>
</body>
</html>