BLOG# 8.2 -Edible Algae System - (Growing Spirulina in space)

System Build

This is Part 2 of my 8th blog post in a series of blog post for the Design Challenge 1 Meter of Pi

version 1.2

INTRODUCTION

• In this blog, I will introduce you to the database that is used to persist the telemetry data generated by the EAS system
  • MongoDB will be used to persist the data.
  • For people that are not aware,  MongoDB is a NO-SQL database. Here's  a link to the Developer Page
  • The Mongo DB server is running in the MongoDB Cloud service, called Atlas
  • There is a Mongo Client library written in python called pymongo ythat you will need to install.
  • • on PyPi you can get a version of the client library by executing the following command:
  • • • $ python3 -m pip install pymongo
    • If you get an error that you need dns support,  you can install it with
      • $ python3 -m pip install dnspython
  • • I'll give some examples of how to connect to the database

• MONGO DB Server

  • As Blog #8 mentioned,, Mongo DB will be used to store the data . I will be using Atlas, , a Cloud based Mongo server. The Cool thing about using Atlas is that you don't need to have it installed on the Raspberry PI and It's FREE for an unlimited number of 512MB Sandboxes. What I found in my research , was that the latest version of mongo db does not run on a 32-Bit Arm OS like Raspberry PI OS.
  • Follow the following steps to setup your database on atlas
    • 1. Setup an Account
      • Register for free access to Atlas HERE
      • After Registering,
        • Follow the instructions on the welcome page.
        • on page 2.. pick a cloud provider.
        • on the next page.. Be sure to select the FREE shared cluster.
          • for Shared RAM 512 MB Storage
          • OR Recreate a paid plan for more features. I'm using the Free version to experiment with IOT . it does run out of space fast with IOT , but it's great to experiment with Proof of concepts.
        • Name your Cluster if you wish or keep the default
        • Press the "Create Cluster Button" on the the bottom right
  • The Project Dashboard will be displayed. You have several task to complete
  • Create a user
    • In order to allow a client to access this cluster you will need to create a user and assign cridentials to the user.
    • Navigate to the "Security" tab from the Clusters page.
      • Choose the "Database Access"  and select the "Add A new User" button
      • If not already selected, for "Authentication Method", choose Password
        • Enter a new username and password
      • Under Database User Privileges
        • Select" Read and write to any database" for this user
    • Press the "Add User" button to create the new user.
  • • Allowing Access to Your Cluster
    • In order to use Atlas with the application, you need to update your IP Whitelist so that your app can talk to the cluster:
    • Navigate to the "Security" tab from the Clusters page.
    • From the "IP Whitelist" tab, there should be an option to "Add IP Address" in the top right corner.
    • Choose "Allow Access from Anywhere" and then click "Confirm".
    • • Note that we do not generally recommend allowing access to your Atlas cluster from anywhere.  However, in this class it minimizes network issues and allows us to provide better support.

• Getting a MongoDB connection string

• Atlas has a neat way to construct a connection string in any language for use in the connection call in the client Driver.
• To get a connection string:
• Go to the Atlas Cluster page and  go to the connect page by
• Pressing the "CONNECT"  button
  • 
• Select the following Button
  • 
• This will bring up the following dialog box.
  • 
  • Select "PYTHON" from the "DRIVER" dropdown.
  • Now Select your python version from the VERSION dropdown
  • Copy the string for later use. You will need it later for a connection string. For now past it to a text editor for safe keeping..

• Creating a new Database on your cluster

  • Now, while still in Atlas you can create a database and collection in the Atlas dashboard by following these steps:
  • Press the COLLECTIONS Button
  • • 
  • The following  screen displays the create Database button, a list of collections if they exist, and a query window on the right hand side.
  • TIP: collections are the same concept as tables in SQL DB's
  • The query window displays the documents (records in SQL), for the selected collection. In

• • Hare's a screenshot of mine, yours will have no databases at first
    • 

• • Press the "+ Create Database" BUTTON
  • the following dialog box appears:
  • enter a database name
  • add a collection name. You can also create a collection after the database is created.
    • 

• Create a document in a collection

  • Select the newly created collection
  • on the query to the right, press the "Insert Document" button.
  • The following dialog box appears.
  • Documents are stored in tuples pairs. You will see that in the code later on.
  • add some more fields. note that a unique key is added for you.after finished editing, press the "Insert" BUTTON.
    • 
  • you will see the document in the query results window.
    • 

• Create another Collection in a database

  • to create collections on the left side navigation list expand the database if not alreay expanded and hover over the name and click on the "+" button
  • enter a collection name and create.
  • thats all there is to it

• MongoDB Compass

  • A MongoDB Tool that allows you to manipulate your data.
  • It does what Atlas does and more.
  • It is really cool and it's FREE.
  • Check it out

• Now your ready to connect to this cluster using the python driver described bellow.

• MongoDB Python Client Driver

  • At first I had a bit of trouble running the client driver package  'pymongo".to use it in code.
    • The Connection string "mongodb+srv://<user>-<password>@<server>/<dbname>"  would not connect.
    • After some research, I found that i needed  to install the package "dnspython" in order to give pymongo "mongodb+srv" support.
    • There is an excellent course on the Mongo University Site that teaches the use of the driver with python. It also describes how to setup a database and an outstanding app on Atlas. I highly recommend it and It's all FREE

• EXAMPLES

  • Here is some sample code showing how to connect to the database created above and run some commands

• • Code	Comment	Output
    from pymongo import MongoClient	Import the MongoClient object from the pymongo package	NONE
    uri ="mongodb+srv://<user>:<password>@cluster0.c2cwd.mongodb.net/<dbname>?retryWrites=true&w=majority"	copy the string from the above copy into a variable uri Substitute:<password><dbname>You should already have <user> filled in	variable
    client = MongoClient(uri)	The MongoClient constructor accepts many different arguments to configure how the driver connects to MongoDB and how many operations will be performed. We'll look at the most basic configuration first, which is passing the SRV string of our Atlas cluster to MongoClient.	creates instance of the MongoClient
    client.stats	Use the mongoclient object to get the status of the cconnection	Database(MongoClient(host=['cluster0-shard-00-00.c2cwd.mongodb.net:27017', 'cluster0-shard-00-01.c2cwd.mongodb.net:27017', 'cluster0-shard-00-02.c2cwd.mongodb.net:27017'], document_class=dict, tz_aware=False, connect=True, retrywrites=True, w='majority', authsource='admin', replicaset='atlas-jrchbn-shard-0', ssl=True), 'stats')[
    client.list_database_names()	get a list of databases	'eas', 'admin', 'local']
    eas = client.easeas.list_collection_names()	Let's use the EAS database. One useful property of a MongoClient object is we can use property accessors	['waterTemp', 'light']
    light = eas.light	Now that we have a database object and have listed available collections, let's create a collection object. As with the database object, we can use either property or dictionary accessors.	Instantiates a collection object called light.
    light.count_documents({})	And let's perform a query on our movies collection. We'll just get the count of documents in the collection.	1
    The MongoClient constructor also accepts many optional keyword parameters. We can set the maximum connection pool, default read and write concerns, whether to retry writes, configuring SSL, authentication, and much more. A full list and how to use MongoClient for more advanced use cases is available HERE	SummaryMongoClient accepts many optional keyword arguments to fine-tune your connection.After instantiating the client, databases handles can be created via property or dictionary accessors on the client object.Collections handles are referenced from the database object.Collection specific operations like querying or updating documents are performed on the collection object.

Conclusion and summary

• I split Blog 8 into 2 parts, because I felt that MongoDB was a central part of my EAS system.

• I have been using MongoDB for more than a year now.
• This is the first time however,  that I have used it to implement an IoT telemetry data repository.
  • It was so easy to create a data layer that I can change on the fly. All the insert statements in the driver create a collection (table) if it does not exist,
  • making it so easy to just throw telemetry data at it and it's persisted.
  • I'll have to keep track of my storage  because the FREE sandbox cluster is limited to 512 MB.
• I highly recommend it. If your a SQL person, then it takes a bit of getting used to. But once you get a feel for it, it's a great tool to have in your developer toolbox.
• If you are interested in learning more, I suggest you take the courses your interested in and jump right in.
• It's all FREE!! for the Sandbox, university and user forums.
• I'm also using Atlas Charts, to Chart the data for the system.