Hi! Welcome to our 11th blog post. In this post, we will talk about the Arduino MKR1000, its primary functions and the GET/POST requesting done by it.
First, let's talk about what all the Arduino has to do, It has to:
- Capture an image using the Camera
- Store the image in the SD Card
- Upload the image to the server
- Wait for the processing to end
- Retrieve the processed .wav file and store it in the SD card
- Play the .wav file using the speaker
All this logic is handled by 4 primary functions in the Arduino's sketch:
- cam_2_SD(): This function will capture an image using the ArduCAM Mini 2MP camera and store it in the SD Card.
- img_POST(): This function will send a POST request to the server, containing the image just snapped.
- wav_GET(): This function will retrieve the processed .wav file from the server, and store it in the SD Card
- play_WAV(): This function will supply the .wav file to the speaker, which will then play it.
We already talked about getting the camera images into the SD card in Blog #7. We have also talked about SD card audio playback in Blog #9 So, we'll talk about the middle two functions.
In order to send a image stored in the SD card to the server, we need to send a POST request to the server containing that image file.
This is a sample POST request:
POST / HTTP/1.1 HOST: flask_hostname Content-Type: multipart/form-data; boundary = AaB0x Content-Length: (length of content) --AaB03x Content-Type: image/jpeg Content-Disposition: form-data; name="file"; filename=image_name Content-Encoding: binary Binary data of image here --AaB03X--
In the Arduino sketch, it is written like so:
void img_POST() {
Serial.begin(115200);
while (!Serial) {
; // wait for serial port to connect. Needed for native USB port only
}
//Load Normal Picture
while (SD.begin(4));
String normalFilename = String(n)+".JPG";
File normPicture = SD.open(normalFilename);
// Get the size of the image (frame) taken
while (!normPicture.available());
long jpglen = normPicture.size();
Serial.println("Sending "+String(n)+" image of");
Serial.print(jpglen, DEC);
Serial.println(" bytes");
// Prepare request
String start_request = "";
String end_request = "";
start_request = start_request + "\n" + "--AaB03x" + "\n" + "Content-Type: image/jpeg" + "\n" + "Content-Disposition: form-data; name=\"file\"; filename=\"" + normalFilename + "\"" + "\n" + "Content-Transfer-Encoding: binary" + "\n" + "\n";
end_request = end_request + "\n" + "--AaB03x--" + "\n";
long extra_length;
extra_length = start_request.length() + end_request.length();
//Serial.println("Extra length:");
//Serial.println(extra_length);
long len = jpglen + extra_length;
//Serial.println(start_request);
Serial.println("Starting connection to server...");
char flask_hostname[]="192.168.1.109";
// Connect to the server, please change your IP address !
if (client.connect(flask_hostname, 5000)) {
Serial.println("Connected");
client.print("POST ");
client.print("/");
client.println(" HTTP/1.1");
client.println("Host: " + String(flask_hostname));
client.println("Content-Type: multipart/form-data; boundary=AaB03x");
client.print("Content-Length: ");
client.println(len);
client.print(start_request);
//Serial.println("Connected1");
if (normPicture) {
//Serial.println("Connected2");
int count = 0;
byte clientBuf[128];
int clientCount = 0;
while (normPicture.available()) {
clientBuf[clientCount] = normPicture.read();
clientCount++;
count++;
if (clientCount > 127) {
//Serial.println(".");
client.write(clientBuf, 128);
clientCount = 0;
}
}
if (clientCount > 0) client.write(clientBuf, clientCount);
Serial.println(String(count));
normPicture.close();
}
//Serial.println("Connected3");
client.print(end_request);
client.println();
Serial.println("Transmission over");
}
else {
Serial.println("Connection failed");
}
char sentence[100];
Serial.println("HEARING STARTS");
Serial.println("Sentence "+String(n)+" is ");
while (client.connected()) {
int flag=0;
int i=0;
while (client.available()) {
// Read answer
char c = client.read();
if(c=='$')
{
flag=flag+1;
continue;
}
if(flag==1)
{
Serial.print(c);
}
}
}
Serial.println("HEARING ENDS");
delay(100);
client.stop();
}
}
//Serial.print(c);
}
}
//Serial.println("Sentence "+String(n)+" is ");
//Serial.println(sentence);
Serial.println("HEARING ENDS");
delay(100);
client.stop();
//while(true);
}
//Insert POST code here
This request will be sent to the server over WiFi as the MKR1000 is WiFi-enabled.
When the server receives this POST request, it will:
- Retrieve the image file and save it on its storage
- Generate a caption for the image using the model
- Convert the caption (string) to an audio file using TTS
- Host the audio file, available for download
In the previous blog, we saw how .wav files were generated on the server side. We need those .wav files on our Arduino's SD card so that the speaker could access them. Since the MKR1000 is Wifi-enabled, connecting and retrieving the audio files is basically sending a GET request for the audio file to the server.
Sample GET request:
GET /output.wav HTTP/1.1 User-Agent: Mozilla/4.0 (compatible; MSIE5.01; Windows NT) Host: flask_hostname Accept-Language: en-us Accept-Encoding: gzip, deflate Connection: Keep-Alive
In the Arduino sketch, it is written like so:
void wav_GET()
{
Serial.println("Starting connection to server...");
char flask_hostname[]="192.168.1.109";
// Connect to the server, please change your IP address !
if (client.connect(flask_hostname, 5000)) {
Serial.println("hello");
client.println("GET /output"+String(n)+".wav HTTP/1.1"); // change resource to get here
client.println("Host: 192.168.1.109"); // change resource host here
client.println("Connection: close");
client.println();}
else{
Serial.println("Connection failed");
}
Serial.println("Creating file.");
theFile = SD.open("output"+String(n)+".wav", FILE_WRITE); // change file name to write to here
if (!theFile) {
Serial.println("Could not create file");
while (1);
}
Serial.println("Created");
int x=1;
Serial.println("Loop starts");
while(x!=0)
{
if (client.available()) {
char c = client.read();
//Serial.println("Available");
if (c == '\n' && currentLineIsBlank) {
// end of HTTP header, now save requested file
while (client.connected()) {
// stay in this loop until the file has been received
if (client.available()) {
c = client.read(); // get file byte
theFile.print(c); // save file byte
//Serial.println("reading");
}
}
}
// detect the end of the incoming HTTP header
if (c == '\n') {
// starting a new line
currentLineIsBlank = true;
}
else if (c != '\r') {
// got a character on the current line
currentLineIsBlank = false;
}
}
// if the server's disconnected, stop the client:
if (!client.connected()) {
Serial.println();
Serial.println("disconnecting.");
client.stop();
theFile.close();
Serial.println("Finished writing to file");
// do nothing forevermore:
//while (true);
x=0;
}
}
Serial.println("loop ends");
}
That's it for this blog. The next blog would be the final one, which will bring all this together working in synergy.
Thanks for having us. The next one will be coming soon!
