In this two day before the deadline, I will show you the progress of my project. I think the progress is around 80-90%. So, hopefully, I can finish the remaining 10-20% before the deadline ( maybe in the last minutes). This post shows my progress in finishing the program 2 (the hardest problem), finishing the block set (master and slave) since I made modification and changes in this component, and then the weight estimation based on OpenCV.
Modifying and Finishing Block Set
I added the LED light in this block-set to face the condition if the light of the kitchen is not so bright. Moreover, since this system employ the computer visison-image processing, the light condition will affect significantly to the accuracy and precision. Then, I also add some parts to make it neater than before. The figure below shows the final version of block-set both for master and slave machines,
Fig 1. Modification and changes of block-set
Finishing Program 2 (Check and Order)
After many problems in some installation of library and framework, the program2 has been finished and works and now I am still in the bug fixing and refining the code structure Below is the program codes,
Fig 2. Code progress of Program 2
Blob Detection/Weight Estimation
The most challenging part of this project is estimating the weight using computer vision. At this stage, I will employ blob detection to extract the remaining material area. Then I estimate the weight using the correlation between blob area-weight/mass of each object. This correlation was made experimentally depend on the observed data. However, for demonstration purpose now I will make a dummy correlation of two materials (material A and B). Below is the flowchart of the program of computer vision in raspberry pi,
Fig 2. Flowchart of Program 1
The estimation of the weight based on blob area assumes that the depth of the material is uniform. Simply, I can say that the uniformity of the dept is used in this approach. Below is the illustration of correlation between the blob area and the weight/mass/volume I used for this project:
Fig 3. How the correlation between blob area and weight/mass used in this project
Then for the object, as a first step, I will use 2D colored object printed on a paper for test the method,
Fig 4. Checking the method of weight estimation with the dummy printed object (inside bottle): 80%, 30%
Fig 4. Checking the method of weight estimation with the dummy printed object (green vegetable) 80%, 30%
Of course, this approach has some limitations such as:
- The detected blob is limited to the specific object
- The distance between the object and camera must be constant.
- The assumption that the depth of the object is uniform.
However, for this stage of development, I think it is enough to show the system works. I will extend this approach so that can be more reliable facing unstable environment.
Closing Remarks
We have described the recent progress of this project. The next step is bug fixing, refining code/program structure, and algorithm, installation and testing. Hope that there is no any further problems so that we can finish this project before the deadline.
