I finally have my cc3200 Launchpad posting data to Plot.ly using Energia (www.energia.nu) compiler / Wiring framework. It took awhile to sort out but feels great to finally get all my charts initialised in the correct format and my data streaming to Plot.ly.
Energia is very well suited for my purposes of linking up to the WiFi network and posting data. It is quite straightforward to amend the included examples for WiFi to connect to your home network. I had no problems at all (unlike its blue coloured cousins which can be rather temperamental) as the handling of the WiFi routines are all taken care of . I found that the Energia compiler for CC3200 is nice and simple and pretty fast too. Smiles all round! However, Energia has some limitations compared to CCS. CCS provides ready made examples for multi-threading using RTOS or Free-RTOS. It also allows you far better user control through a web browser interface and you can download professional looking web pages onto the cc3200 using the uniflash utility program. I also have yet to work out what sleep options can be triggered using the Energia coding framework.
Regarding Plot.ly.
As far as I am aware Plot.ly is the only online platform that allows streaming of multiple data sets onto the same chart. Popular platforms such as Xively only stream one data set per chart in time series. By manipulating the data prior to posting to Plot.ly you get near real time data analysis, dynamically. Plot.ly also allows you to format your charts to make them more readable, which is particularly important when analysing complex data relationships. Plot.ly is also flexible in that you can split out data to a different y-axis on the same chart or can actually separate out data to a sub-chart (can stack or make insert), which reduces data overload on a chart,
In my case I am not too interested in the absolute Lux values. What I want to see is the relative values, on the basis that the top sensor should generally get proportionally more light than the other two sensors, which would get proportionally less depending on grass length and density (well that is the theory anyway). The chart below is simply test data while my wireless node is sitting on my desk talking to the cc3200 with an RF device attached. Also the attached picture is missing the Y-Axis reference which is a relative value per 1000 (so if all sensors had identical values you would see all three values at 333 - this is derived by the formula: actualVal / total_of_3_sensorvals * 1000).
The way I have it set up at present is a data upload every 4 minutes which is a little over the top but this is helpful for testing purposes.
Here is another update chart. This is still pretty raw, hence some odd behaviour at beginning as was amending chart through Plot.ly, but thought to add as it illustrates what is possible with Plot.ly. This is streaming data so you get "live" analysis of your data as it happens.
Once you have some data (initialised your chart) you can go in and manually adjust the chart layout to what you want and this will be used to show streaming data. So if, for example, you want to display two different types of sensors on the same chart, you can simply select the trace of choice and change the y-axis (a bit like you would do with MS Excel) - I have illustrated this below with my moisture sensor data. You can also stack charts to show different sensors tracking the same x-axis. So in my case I have two moisture sensors streaming data to top chart and the 3 lux sensors streaming data to the bottom chart, with both using the same x-axis data. Another very important feature, and where most automated graphing systems fail, is the ability to set your y-axis. With auto-scaling you can very quickly catch yourself out by misinterpreting your data simply because the scale automatically changed.
As you can see, the data is starting to show some insight. For example I covered the bottom light sensor slightly and the values dropped relatively. I placed the soil moisture sensor in some water and this climbed.... Not sure why my rain sensor is giving readings but that is just a calibration issue for later.
Over the next couple of days I should now be able to get real field data. <<BLOG UPDATE>> But here are some earlier test results, which I started today (while it is not raining :-)). At the moment I have my CC3200 being powered through USB as still changing / tweaking code. Attached to CC3200 is my H12 RF device with quarter-wave (not half-wave) wire antenna which is talking to my grass node some 30 metres away outside. If wondering what else is showing in picture, it is some external EEPROM which I may use for later (handle web-server config info etc.).
Then my new node device is now enclosed (as now need to cope with the rain) inside a $3 transparent case from SeeedStudio (Buy PS(Poly Styrene) Transparent Case - 130x60x25 mm [322060002] | Seeedstudio). Moisture and rain/water sensor also from SeeedStudio. Everything is now on one protoboard with batteries enclosed within own waterproof enclosure. I decided to leave out the IR LEDs for now. We will see how well this works over next coming days.
Here are two formatted Plot.ly charts showing some positive results from my grass sensor node being placed in three different locations. First location has longish grass, second location had much shorter grass height and third location it was rather overgrown. By the time I moved node to third location some interesting data values were spotted, which will require troubleshooting. Notes have been added to 3rd chart highlighting probable issues.
It is now the end of the day and the field trials have worked out quite well in my opinion, with my data streaming ok through to plot.ly and my RF devices proving quite reliable without any resilience added to the code. I thought to add this chart as it highlights how real time analysis works "on the fly". This was picked up while monitoring the streaming data through my smartphone. I was then able to quickly check and took some pictures for recording keeping and to note the time. This chart captures the issue of when light fades....
Here ends my first field test with blog updates as it happened.
--------------------------------------------------------------------------------------------
So how does one get this working with Energia.
First thing is to understand the plot.ly API process. There is a 3-step process...
[1] Initialise routine for your chart. This essentially describes the type of chart that will be created, how many data streams it will have, the styling and the layout. What is also included in the initialisation routine is your authentication credentials, what you want to name the chart as and whether it will be a public accessible chart or private.
[2] Open up a streaming connection.
[3] Post your streaming data
If you look at the Arduino library for Plot.ly and the initialisation routine, it is rather messy. This is because there is plenty of formatted text to be posted and you need to manually work out the content length. It works fine for simpler charts but as soon as you want to create more complicated charts mistakes can happen (all it takes is a missing comma or bracket and it won't be accepted). Hence I found (thanks to the suggestion from plot.ly) that a better way is to manually create the charts using (http://www.hurl.it/). This way if you restart your cc3200 or upload amended code to cc3200 you are not overwriting previous data.... (BLOG UPDATE) Note this works fine if you are not using timestamps. If you are using timestamps you will need to initialise the chart using the MCU as the API requires "millis()" to keep track of the time. I have now added the initialisation routine to the code example below.
Here is a screen dump of what you can send through hurl.it to initialise your charts:
You will get your response to tell you whether it has been accepted or otherwise. If your requested has been accepted you will see this in the response body:
- "stream-status": "All Streams Go!"
- "url": "https://plot.ly/~USERNAMEHERE/0" (the /0 might be another number depending on what was requested - also this format is for public charts.)
- "message": "High five! You successfuly sent some data to your account on plotly....
Then for Energia I created a couple of functions based on the Arduino library (this is a cut and paste so hopefully no errors):
// This code is open to all, is "as-is" and acknowledges the following:
// This code is based on examples provided within Energia
// This code is based on plot.ly Arduino libraries... see https://plot.ly/learn/ for examples.
#include <WiFi.h>
#define SSID_ "INSERT HERE" // your network name also called SSID
#define PASSWORD "WIFI PASSWORD HERE" // your network password
#define PLTYURL "plot.ly"
#define PLTYURL_STR "stream.plot.ly"
#define PLTYTIMEZONE "Europe/Dublin" // make sure you use a recognised time zone
const unsigned int
NUM_TRACES = 5, // See your Plot.ly settings (adjust number of traces to match your requirements and number of streaming tokens. Min is 1 trace.
PLTY_MAXPOINTS = 30,
RFTIMEOUT = 2000,
BUFFsize = 32;
const String
PLTYUSERNAME = "YOURPLOTLYUSERNAME", // user agent is the project name
PLTYFILENAME = "\"Plotly_File_Name_Reference\"", // filename reference for plot.ly
PLTYAPIKEY = "YOURPLOTLYAPIKEY", // replace with your api key here
PLTYFILEOPT = "\"overwrite\"", // need to have the additional quotations (options here can be: "overwrite", "extend", etc.... refer to plot.ly for instruction
PLTY_INI = "version=2.3&origin=plot&platform=arduino&un=",
PLTY_KEY = "&key=",
PLTY_ARG = "&args=[",
PLTY_XYTYP = "{\"x\": [], \"y\": [], \"type\": \"scatter\"", // Change graph type here
PLTY_TOKE = ", \"stream\": {\"token\": ",
PLTY_FOPT = "]&kwargs={\"fileopt\": ",
PLTY_TRAC = ", \"traces\": [0,1,2,3,4]", // change this according to here many traces you want plotted (can be less or equal to NUM_TRACES)
PLTY_STYL = ", \"style\": [{\"name\": \"bottomSensor\"}, {\"name\": \"middleSensor\"}, {\"name\": \"topSensor\"}, {\"name\": \"rainSensor\"}, {\"name\": \"soilSensor\"}]", // Name your traces here.
PLTY_LAYO = ", \"layout\": {\"title\": \"Moisture and Lux Sensor Data from: Node 1\"}",
PLTY_FNME = ", \"filename\": ",
PLTY_WLDR = ", \"world_readable\": ";
char
*streaming_tokens[NUM_TRACES] = {"\"INSERTTOKEN_ID1\"", "\"TOKEN_ID2\"", "\"TOKEN\"", "\"TOKEN\"", "\"TOKEN\""}; // Token IDs here. Don't forget to include \" either side of token id. Number of tokens must match NUM_TRACES
String
charString = "",
pltyStrMaxPnts = ", \"maxpoints\": " + String(PLTY_MAXPOINTS) + "}}";
boolean
Plty_Public = true, // Otherwise referred to as world_readable
Plty_Timestamping = true, // if you want your x axis to be timestamped
Plotly_Response = false;
unsigned long
t_init = 0L,
t_start = 0L,
t_now = 0L;
WiFiClient client;
void setup()
{
// DO WHAT YOU NEED TO DO SETTING UP CONNECTIONS, GPIO, etc.
Serial.begin(9600);
// REFER TO WIFI EXAMPLE FOR WIFI SET UP CODE
// TO INITIALISE A CHART IN PLOT.LY.
Serial.print("Connecting to plot.ly...");
init_Plotly();
Serial.println("");
delay(500);
Serial.println("... Connecting to plotly streaming server");
openStream_Plotly();
delay(500);
}
void loop()
{
unsigned long t_now = millis();
sendIntData_Plotly(t_now, <<INSERT INTEGER VALUE HERE>>, streaming_tokens[0]);
sendIntData_Plotly(t_now, <<INSERT INTEGER VALUE HERE>>, streaming_tokens[1]);
sendIntData_Plotly(t_now, <<INSERT INTEGER VALUE HERE>>, streaming_tokens[2]);
sendIntData_Plotly(t_now, <<INSERT INTEGER VALUE HERE>>, streaming_tokens[3]);
sendIntData_Plotly(t_now, <<INSERT INTEGER VALUE HERE>>, streaming_tokens[4]);
// PLACE YOUR OTHER CODE
// NOTE THE UPLOAD RESTRICTIONS ON HOW FREQUENT DATA CAN BE UPLOADED TO PLOT.LY -- NEED TO CREATE SOME DELAY
}
/**************************************************************************/
/*
Clears the serial char buffer used for RF communication
*/
/**************************************************************************/
void clearBuff(char *buffName, int multi)
{
for (int i = 0; i<BUFFsize*multi; i++)
buffName[i] = NULL;
}
/**************************************************************************/
/*
Connect to Plot.ly and Initialise the chart
*/
/**************************************************************************/
void init_Plotly()
{
unsigned int
fibonacci_ = 1,
contentLength = PLTY_INI.length() + PLTYUSERNAME.length() + PLTY_KEY.length() + PLTYAPIKEY.length() + PLTY_ARG.length() +
NUM_TRACES * (PLTY_XYTYP.length() + PLTY_TOKE.length() + 12 + pltyStrMaxPnts.length()) + (NUM_TRACES-1)*2 + // 12 is token length, 2 represents ", " between parameters
PLTY_FOPT.length() + PLTYFILEOPT.length() + PLTY_TRAC.length() + PLTY_STYL.length() + PLTY_LAYO.length() +
PLTY_FNME.length() + PLTYFILENAME.length() + PLTY_WLDR.length() +1; // will add more to content length later (init_Plotly() function)
while(1) {
if (client.connect(PLTYURL, 80)) {
Serial.println("connected to Plot.ly server");
client.println("POST /clientresp HTTP/1.1");
client.println("Host: 107.21.214.199");
client.println("User-Agent: CC3200_Launchpad");
client.print("Content-Length: ");
if(Plty_Public){
contentLength += 4;
} else{
contentLength += 5;
}
client.print(contentLength);
// Terminate headers with new lines
client.println(""); client.println("");
// Add the querystring body, sending login credentials and to initialise type of chart wanted
client.print(PLTY_INI + PLTYUSERNAME + PLTY_KEY + PLTYAPIKEY + PLTY_ARG);
client.flush();
// print a trace for each token supplied
for(int i=0; i<NUM_TRACES; i++) {
client.print(PLTY_XYTYP + PLTY_TOKE);
client.print(streaming_tokens[i]);
client.print(pltyStrMaxPnts);
if(i < NUM_TRACES-1) client.print(", ");
}
client.print(PLTY_FOPT + PLTYFILEOPT);
client.print(PLTY_TRAC + PLTY_STYL + PLTY_LAYO);
client.flush();
client.print(PLTY_FNME + PLTYFILENAME + PLTY_WLDR);
if(Plty_Public){
client.print("true");
} else{
client.print("false");
}
client.print("}");
// final newline to terminate the POST
client.println("");
client.flush();
// THIS IS JUST FOR DEBUGGING TO CHECK THAT YOU HAVE NOT MAKE A MISTAKE
// COMMENT THIS SECTION OUT WHEN HAPPY THINGS ARE WORKING AS NOT NEEDED
// =====================================================================
Serial.println(contentLength);
Serial.println(""); Serial.println("");
// Add the querystring body, sending login credentials and to initialise type of chart wanted
Serial.print(PLTY_INI + PLTYUSERNAME + PLTY_KEY + PLTYAPIKEY + PLTY_ARG);
Serial.flush();
// print a trace for each token supplied
for(int i=0; i<NUM_TRACES; i++) {
//client.print(pltyString3);
//client.print(streaming_names[i]);
Serial.print(PLTY_XYTYP + PLTY_TOKE);
Serial.print(streaming_tokens[i]);
Serial.print(pltyStrMaxPnts);
if(i < NUM_TRACES-1) Serial.print(", ");
Serial.flush();
}
Serial.print(PLTY_FOPT + PLTYFILEOPT);
Serial.print(PLTY_TRAC + PLTY_STYL + PLTY_LAYO);
Serial.flush();
Serial.print(PLTY_FNME + PLTYFILENAME + PLTY_WLDR);
if(Plty_Public){
Serial.print("true");
} else{
Serial.print("false");
}
Serial.print("}");
// final newline to terminate the POST
Serial.println("");
Serial.flush();
// =====================================================================
// Wait for a response
// Parse the response to search for the "All Streams Go!" and proceed to streaming if we find it
t_now = millis();
while(client.connected() && (millis()-t_now) < RFTIMEOUT*5){
Serial.print("+");
if(client.available()){
Serial.print(".");
client.readBytesUntil('\n', pltyBuff, BUFFsize*8);
charString = &pltyBuff[0];
charString.trim();
clearBuff(pltyBuff, 8);
}
if (charString.length()) {
Serial.println(charString);
// Search string for "stream-status" -- note this string is enclosed within quotations
// Read the response
if (charString.indexOf("All Streams Go!") > -1) {
Serial.println("Ready to proceed");
Plotly_Response = true;
// Note there are also "warning", "error" and a "message" response which could also be checked.
// Now show the URL on screen to allow for public viewing of data stream
contentLength = charString.indexOf("url", contentLength);
if (contentLength > -1) {
Serial.print("... View streaming plot here: ");
Serial.println(charString.substring(contentLength+6, charString.indexOf(",", contentLength+6)));
} else Serial.println("... Problem extracting streaming plot URL");
}
}
delay(100);
}
client.stop();
if (!Plotly_Response) Serial.println("... Timeout");
break;
}
Serial.print(".");
delay(100 + fibonacci_);
fibonacci_++;
}
}
/**************************************************************************/
/*
Opens Plotly connection to stream data to
*/
/**************************************************************************/
void openStream_Plotly()
{
while(1) {
if (client.connect(PLTYURL_STR, 80)) {
Serial.println("connected to Plot.ly streaming");
client.println("POST / HTTP/1.1");
client.println("Host: arduino.plot.ly");
client.println("User-Agent: CC3200_Launchpad");
client.println("Transfer-Encoding: chunked");
client.println("Connection: close");
if(Plty_Timestamping){
client.print("plotly-convertTimestamp: \"");
client.print(PLTYTIMEZONE);
client.println("\"");
}
client.println("");
Serial.println("... Done, ready to stream!");
break;
}
Serial.print(".");
delay(200);
}
}
/**************************************************************************/
/*
Closes Plotly connection to stream data to
*/
/**************************************************************************/
void closeStream_Plotly()
{
client.println("0");
client.println("");
client.stop();
}
/**************************************************************************/
/*
Reconnects Plotly connection to stream data to
*/
/**************************************************************************/
void reconnectStream_Plotly()
{
while(!client.connected()) {
Serial.println("... Disconnected from streaming servers");
closeStream_Plotly();
openStream_Plotly();
}
}
/**************************************************************************/
/*
Send Integer Data to Plotly Streaming connection
*/
/**************************************************************************/
void sendIntData_Plotly(unsigned long x, int y, char *token)
{
reconnectStream_Plotly();
client.println(String(x,DEC).length()+String(y,DEC).length()+44, HEX);
client.print("{\"x\": ");
client.print(x, DEC);
client.print(", \"y\": ");
client.print(y, DEC);
client.print(", \"streamtoken\": ");
client.print(token);
client.print("}\n");
client.println("");
client.flush();
}