Blog#5: My Journey (a hands-on) with the AWS Cloud (Services).

2 Jun 2020

AWS Cloud (Services)

In this bloc I will be discussing my hands-on with AWS Cloud and its Services. At the end of this Blog we would have learnt to do the following

Create IAM User and Groups
Able to Connect our Pioneer Kit to AWS Cloud
Able to perform MQTT protocol service between Pioneer Kit and AWS Cloud.

With this being said we would have knowledge and requirements to host our Pioneer Kit LIVE onto Connected Cloud.

Please forgive me since I'm new to Cloud and I'm using this opportunity to understand & get hands-on with AWS Cloud Services. On my previous blogs I mention on the challenges I will encounter for my project, and AWS Cloud is one of them. The idea here is to lay the foundation (ground) work for building upon my Propose Solution hence the focus is on these listed AWS Cloud Services for a start.

1. IAM Users (Console)

2. AWS CLI

3. CodeCommit (Console)

4. IoT (Console)

5. FreeRTOS (Console)

6. MQTT (FreeRTOS\aws-demo)

We'll explore each one of them in some detail.

1. IAM Users (Console)

===========================

IAM Console (Link1)

IAM Console (Link2)

=> Lean more on IAM (User-Guide)

=> Learn more on Managing signing certificates

As the first step, we require to create a IAM User to delegate certain permissions later it would help us make programmatic request to AWS Cloud.

Root Access cannot (should not) be used to make programmatic access because its the root user and has complete access to your AWS Cloud.

Figure-1: IAM User

Best practice is to create an IAM User and allow restricted access to desired resources (services/policy) only.

for e.g.

AWSCodeCommitPowerUser

allow full access & permission for Code-Commit, i believe this is similar to GitHub

AmazonDynamoDBFullAccess

allow full access to database services (read, write, modify, etc)

AmazonConnectFullAccess

allows User access to Connect to AWS Cloud.

AmazonFreeRTOSFullAccess

allows your device to be able to connect and access AWS Cloud.

AWSIoTFullAccess

allow how your device operates with AWS Cloud.

AmazonMCSFullAccess

provide full access to Amazon Managed Apache Cassandra Service

We can assign individual policies to the User or Create a common Group and assign this group to Users. In my case I have created a group named "group1" and assigned it to a user i created by this i would not have to add the required policies individually to a user, I have done this coz I'm in development stage and have kept it KISS - Keep it Simple Stupid.

Figure-2: User Group

Setting up a User IAM account is important for use programmatic however before that we have to setup some credentials

a) Console sign-in setup

User can login from a web browser into his/her account.
Don't forget to create/assign a console password so user can login to his/her console.

Figure-3: Console Sign-in setup

b) (Optional) MFA

We'll ignore this for the moment.

c) Signing Certificates

Certificate to authenticate this user.

Figure-4: Signing Certificate

To create a signing certificate
- Install and configure OpenSSL.
- Create a private key using the openssl genrsa command and save the output to a .pem file. We recommend that you create a 2048- or 4096-bit RSA key.

openssl genrsa 2048 > private-key.pem

- Generate a certificate using the openssl req command.

openssl req -new -x509 -nodes -sha256 -days 365 -key private-key.pem -outform PEM -out certificate.pem

=> Learn more about Signing Certificates

d) Access Keys

Allows Rest or Http protocol request to AWS Service APIs.
Use the IAM Dashboard's to "Create Access Key".

Figure-5: Access keys

=> Learn more about Access Keys

e) SSH keys for AWS CodeCommit

SSH public keys to authenticate access to AWS CodeCommit repositories
Use this command to generate "SSH keys for AWS CodeCommit", careful not to overwrite your existing .ssh/id_rsa file

ssh-keygen -t rsa -b 2048 -f id_rsa_aws-magnet0

Figure-6: SSH Keys for AWS CodeCommit

=> Learn more SSH Keys for AWS CodeCommit

f) HTTPS Git credentials for AWS CodeCommit

Authenticate HTTPS connections to AWS CodeCommit repositories
Use the IAM Dashboard's to "Generate the Credentials" and make a note of the same as you will need it.

Figure-7: HTTP Git Credentials for AWS CodeCommit

=> Learn more HTTPS GIT Credentials for AWS CodeCommit

IAM User Configure (consolidation)

Figure-8: IAM User Configure (a) to (f)

2. AWS Command-Line

===========================

=> Learn more on "AWS CLI User Guide"

=> Learn more on "Creating IAM users and groups"

AWS Command Line Interface enable you to interact with AWS Cloud Services using a command shell.

AWS CLI is an alternative to accessing "console.aws.amazon.com" via web browser instead most of your activity can be done offline via AWS CLI.

for e.g.

You can create new users and groups and delegate policies and permission via CLI.
Create DynamoDB database and add records via CLI.

I will be highlighting below the required steps required to access AWS Cloud services via AWS CLI.

a) AWS CLI Installation

You can choose your installation from the link below, in my case its windows so I have downloaded "Installing the AWS CLI version 2 on Windows" and install it on my Desktop/Laptop. You can choose your installation here https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html

b) AWS CLI Configure

This setup is to configure IAM User for access to AWS Cloud Services.

Learn more on how to configure AWS CLI

Refer to the "$ aws configure" command in the below picture. first half of this picture has already been investigated.

Figure-9: aws configure

When you have finish with "$ aws configure" command, You will notice a .aws folder in your home directory with two files in it. Keep them safe and do not share these files.

Figure-10: .aws config files

c) Test - (if you have access to AWS Cloud Service)

For e.g. 1

aws CLI to create a new user, remember to delete this dummy user.

Note: IAM User must have "IAMFullAccess" policy attached only then will this command be successfully.

$ aws iam create-user --user-name DummyUser

Output:

{

"User": {

"UserName": "DummyUser",

"Path": "/",

"CreateDate": "2018-12-14T03:13:02.581Z",

"UserId": "AIDAJY2PE5XUZ4EXAMP",

"Arn": "arn:aws:iam::123456789012:user/DummyUser"

}

For e.g. 2

aws cli command to create a table in dynamodb

$ aws dynamodb create-table \

--table-name MusicCollection \

--attribute-definitions AttributeName=Artist,AttributeType=S AttributeName=SongTitle,AttributeType=S \

--key-schema AttributeName=Artist,KeyType=HASH AttributeName=SongTitle,KeyType=RANGE \

--provisioned-throughput ReadCapacityUnits=1,WriteCapacityUnits=1

For e.g. 3

aws cli command to put an item into a table we created earlier in dynamodb

$ aws dynamodb put-item \

--table-name MusicCollection \

--item '{

"Artist": {"S": "No One You Know"},

"SongTitle": {"S": "Call Me Today"} ,

"AlbumTitle": {"S": "Somewhat Famous"}

}' \

--return-consumed-capacity TOTAL

Output

{

"ConsumedCapacity":

{

"CapacityUnits": 1.0,

"TableName": "MusicCollection"

}

=> Learn more on AWS CLI

3. CodeCommit (Console)

==============================

a) Create SSH keys

Use the below command to "Create SSH keys". This command will create two files (containing public & private keys), Copy them into your home ~/.ssh folder and take care not to overwrite your existing id_rsa files. Use the -f option to write to a custom file.

ssh-keygen -t rsa -b 2048 -f id_rsa_aws-magnet0

b) Upload SSH Public Key

Upload the "id_rsa_aws-magnet0.pub" content (not the file) into the "SSH keys for AWS CodeCommit" section of the users account.

Figure-11: Upload SSH public key

c) Git Configure to access AWS CodeCommit repository

If you are an expert, then you will be able to handle multiple ssh keys into your home ~/.ssh folder. In my case

for e.g. ~/.ssh folder contains

id_rsa and id_rsa.pub = i use this for my github repository, Your ssh-config file should have this entry

id_rsa_aws-magnet0 and id_rsa_aws-magnet0.pub, I will use this to access my AWS CodeCommit repository

My ~/.ssh/ssh-config file content looks like this, contains 3 important fields

i) Host

ii) User

iii) IdentityFile

Figure-12: ssh-config file

d) Test if you have access

Run these commands to verify access to AWS repository

; us-east-2, aws region server

ssh -v git-codecommit.us-east-2.amazonaws.com

; asia\mumbai, aws region server

ssh -T git-codecommit.ap-south-1.amazonaws.com

4. IoT (Console)

===================

Click here to sign-in to IoT Dashboard

IoT helps Connect your device, a mobile or web app.
IoT is a managed Cloud service that helps Connected devices interact with each other on a common platform.
IoT helps Connected devices Securely and efficiently communicate with each other via a Gateway Device.

Learn more on AWS IoT (User Guide) details the Steps involve in AWS IoT Setup

You can click on each section and follow the instruction in the given link. Keep in mind that when you are registering a device (as a thing) you should choose a region (location in the server where your device is registered). Choose the region closest to where you are located (Live).

Figure-13: Choosing a Region in IoT Console

a) Sign in to the AWS IoT console

b) Create a thing

c) Register a device

Except (e) You can ignore the remaining steps for now, it will be done later.

d) Configure your device

e) View device MQTT messages with the AWS IoT MQTT client

f) Configure and test rules

g) Create and track an AWS IoT Core job

I have already done the above steps, so i will attach a few pictures

AWS IoT Console > Manage > Things

Figure-14: Thing

Figure-15: Thing (representing my Pioneer kit)

AWS IoT Console > Secure > Policies

Figure-16: Create Policies (which will allow secure connection to devices)

AWS IoT Console > Secure > Certificates

Figure-17: Certificates

5. FreeRTOS (Console)

============================

Click here for FreeRTOS (Console), This console link is useless since our Pioneer kit is not listed in Predefined configurations so you can ignore it.

Click here to Getting Started with Amazon FreeRTOS, - Suggest you to not refer to this link directly instead follow "Getting Started with Amazon FreeRTOS and PSoC62 + 43xxx.pdf"

Click here to Getting Started with Amazon FreeRTOS and PSoC62 + 43xxx.pdf, This document provides instructions for getting started with the following kits.

PSoC 6 Wi-Fi BT Prototyping Kit (CY8CPROTO-062-4343W)
PSoC 6 WiFi-BT Pioneer Kit (CY8CKIT-062-WIFI-BT)

Before You Begin (follow only this session from pdf "Getting Started with Amazon FreeRTOS and PSoC62 + 43xxx.pdf")

a. Click here for Pre-requisites (follow these steps before you being with FreeRTOS)

b. Github repository: Clone Amazon FreeRTOS from Cypress

You can download this source code from Github and do not use any other FreeRTOS source code from elsewhere.
This Github source is specific to the following kit

PSoC 6 Wi-Fi BT Prototyping Kit (CY8CPROTO-062-4343W)

PSoC 6 WiFi-BT Pioneer Kit (CY8CKIT-062-WIFI-BT)

https git clone command

git clone --recurse-submodules https://github.com/cypresssemiconductorco/amazon-freertos.git --branch 201910-MTBAFR1951

ssh git clone command

git clone --recurse-submodules git@github.com:cypresssemiconductorco/amazon-freertos.git --branch 201910-MTBAFR1951

follow these steps if you are not sure which branch (release) is the latest

git clone --recurse-submodules https://github.com/cypresssemiconductorco/amazon-freertos.git

git tag

201908-MTBAFR1941

201910-MTBAFR1951 (this is the latest at the time i am writing this blog)

git checkout 201910-MTBAFR1951

c) Wi-Fi settings (SSID, password) and MQTT endpoint in Source code amazon-freertos\demos\include\aws_clientcredential.h

Figure-18: Wifi Settings and Mqtt endpoint configure in FreeRTOS

instructions given in the pdf file suggest to manually edit the aws_clientcredential_key.h file

instead there is an alternate and easy way to edit aws_clientcredential_keys.h file.

Launch CertificateConfigurator.html open html file in IE or Chrome, This file is located in source code amazon-freertos/tools/certificate_configuration/CertificateConfigurator.html

Figure-19: aws_clientcredential_keys.h (modified).

6. MQTT Demo App (FreeRTOS aws-demo)

===============================================

Setting Up the Development Environment
Build and Run the Amazon FreeRTOS Demo Project
- Build the Amazon FreeRTOS Demo
- Run the Amazon FreeRTOS Demo Project
Monitoring MQTT Messages on the Cloud

I have successfully built aws-demo project using "ModusToolBox" and Flash the pioneer kit with the binary using the tool

Figure-20: aws-demo project build & flash using ModusToolBox tool

I will attach the putty logs of this demo app, refer to file "aws-demo-puttylog-mqtt.txt"

Monitoring MQTT Messages on the Cloud

Figure-21: Test for MQTT

######################################################################################################################################################

In my next Blog i will pick a sample code-example and customize it for my requirement, I think I have enough ground work done on the pioneer kit and AWS Cloud Services hence,

Next Milestone for me would be the following.