Stove Assistant - Introduction

Cooking is sometimes difficult. At least for me. Not so much for my wife or my mother. Especially for some meals like pancakes. When I cook them are always partly stiff but also doughy. I think this is because I work at the wrong temperature or let them too long in the pan.

This is where my stove assistant comes in. He will check the actual temperatures and have an eye on the clock and help me to cook pancakes as good as my wife.

The setup

Therefore a pi camera together with a temperature sensor is placed atop of the cooktop of the stove I will use the Panasonic Grid-Eye Infrared Array Sensor  AMG8834AMG8834 http://uk.farnell.com/panasonic-electronic-components/amg8834/temperature-sensor-ir-3deg-c-module/dp/2576420) as temperature sensor.

Here is a mock-up of the stove assistant installation: (Sorry, I am not so good at photoshop. You get a better picture at the end of the project)

The camera and the temperature sensor will be aligned so that there is temperature information for every section of the image. The camera will be used to identify the used part of the cooktop (where the pan is placed). I want to implement this with OpenCV and maybe the use of deep neural networks if applicable.

The HMI can be done by a display with touch next to the Raspberry Pi. But the Raspberry Pi has to be mounted atop of the cooktop so this could be difficult to reach and read. So i will check if it is more practicable to do a remote HMI on a smartphone via Wifi.

Description of the cooking process:

1. The user starts the system
2. The user chooses to make pancakes on the HMI of the Raspberry Pi
3. The user places the pan on the cooktop and heats it up.
4. The Raspberry Pi automatically checks where the pan is placed and checks its temperature
5. The Raspberry Pi gives notice to the user when the right startup temperature is reached.
6. The user places the paste in the pan.
7. The Raspberry Pi checks the temperature of the pancake and gives notice to the user when it is the right time to turn the pancake (by monitoring its temperature and the baking time)
8. The Raspberry Pi checks the temperature of the now turned pancake and gives notice to the user when it is finally baked (by monitoring its temperature and the baking time)
9. The user can go to step 5 and bake a new pancake or finish the cooking.

The proceeding of the steps will be recognized by the Raspberry Pi by image detection.

During the whole process the temperature is monitored and advice is given to the user by the system to turn the heat of the cooktop up or down. Unfortunately the Raspberry Pi can't directly control the heat of a cooktop without modification of the cooktop. So the stove assistant is limited to tell the user to change the heat.

As a side effect the temperature profile of the whole cooking process is monitored and can be compared with an optimal profile to give the user hints how to improve his cooking.

Project plan

I plan to go through the project according to the following plan. There will be one or two blog posts for each step. If new challenges arrise during the process of the project this plan can be adopted.

• Unboxing, bring up Rapsberry Pi and camera with OpenCV
• connect infrared sensor to Raspberry Pi and read out temperature information
• check alignment of temperature sensor and camera
• implement a graphic algorithm with OpenCV to identify the active part of the cooktop (where the pan is placed)
• implementation of the HMI
• do  some test cookings and record the temperature profile
• write the main program to guide the user through the cooking
• do more testing and tasting

additional steps (if accidentally time is left)

• get the user feedback on the taste of the meal and optimize the process automatically
• adopt the project to other meals.

Risks

As with every project there are some risks that could delay the project or prevent the realisation. If you identify most of them in advance you have a good chance to avoid them.

• pollution of the sensors by smoke and steam of the cooking process could impact the measurements.
• reflections prevent correct temperature measurements
• low WAF

Future development

When the project works out well it could be adopted to other uses or expand its functionality

• Identify more meals where this sensor could help
• connect with an online database to exchange recipes and temperature profiles
• prepare several meals simultaneously
• find a cooktop where the heat can be influenced directly so that automatic cooking is possible
• connect to home automation