We plan to make a detector of the aperture of sliding windows for our VenTTracker Project that monitors Natural Ventilation Habits in Classrooms.
There we implemented two different methods to know the aperture of the window, one using an accelerometer module and another with a quadrature encoder made with reed magnetic switches.
The problem for both approaches is that you need a device attached to the window.
In this project we plan to use the Raspberry Pi 4 Model BRaspberry Pi 4 Model B and a camera to monitor the aperture using OpenCV and ArUco Markers positioned on the fixed and moving parts of the window.
This will allow us to monitor the aperture of several windows at the same time using only a Raspberry and a camera.
In this blog we present the process to install the necessary software on the Raspberry Pi and we do a first proof of concept to see if we are able to recognize the codes at a reasonable distance from the windows.
ArUco codes
AugmentedReality uco Codes These are the special augmented reality markers which have a unique identification number inside them.
These markers are decoded in binary serialization and can be decoded manually as well as by computer also.
ArUco marker is a 5×5 grid that is black and white in color.
Reference : https://docs.opencv.org/3.4/d5/dae/tutorial_aruco_detection.html
Testing with a casement window to determine the distance at which the codes are detectable with a small webcam and the Rapberry Pi 4 model B
ArUco codes from id 1 to id 4 using dictionary DICT_6X6_250
Our environment
Our configuration is as headless server. Headless server is a computer without a monitor, keyboard or mouse. They're managed by a console that has access through either SSH or telnet.
- Raspberry Pi 4 Model BRaspberry Pi 4 Model B
- Plastic enclosure with Four passive heat sinks and a fan
- microSD card 16 GBmicroSD card 16 GB
- USB Power adapterUSB Power adapter
- USB webcam Logitech c170
Installing OpenCV on the Raspberry PI 4B
Preparation
Install Operating System - Raspberry Pi OS (32-bit)
Install Raspberry Pi OS using Raspberry Pi Image Tool https://www.raspberrypi.org/software/
Raspberry Pi OS version check
https://en.wikipedia.org/wiki/Raspberry_Pi_OS
pi@raspberrypi:~ $ uname -a Linux raspberrypi 5.10.17-v7l+ #1421 SMP Thu May 27 14:00:13 BST 2021 armv7l GNU/Linux
Check EEPROM software version.
pi@raspberrypi:~ $ sudo rpi-eeprom-update
BOOTLOADER: up to date
CURRENT: Thu 29 Apr 2021 04:11:25 PM UTC (1619712685)
LATEST: Thu 29 Apr 2021 04:11:25 PM UTC (1619712685)
RELEASE: default (/lib/firmware/raspberrypi/bootloader/default)
Use raspi-config to change the release.
VL805_FW: Dedicated VL805 EEPROM
VL805: up to date
CURRENT: 000138a1
LATEST: 000138a1
If needed, to update the firmware
pi@raspberrypi:~ $ sudo rpi-eeprom-update -a pi@raspberrypi:~ $ sudo reboot
VNC Server and Viewer
We are using VNC with a HDMI virtual monitor.
Setup Real VNC Server in Raspberry PI
Setup Real VNC Viewer in Microsoft Surface
OpenCV
OpenCV is an Open source library of computer vision and machine learning software.
Installation of OpenCV 4 from pre-built binaries may be a problem since apt repositories may not contain the latest version of OpenCV always .
Building OpenCV from source
Install dependencies
pi@raspberrypi:~ $ sudo apt-get update pi@raspberrypi:~ $ sudo apt-get upgrade pi@raspberrypi:~ $ sudo apt-get install cmake gfortran pi@raspberrypi:~ $ sudo apt-get install libjpeg-dev libtiff-dev libgif-dev pi@raspberrypi:~ $ sudo apt-get install libavcodec-dev libavformat-dev libswscale-dev pi@raspberrypi:~ $ sudo apt-get install libgtk2.0-dev libcanberra-gtk* pi@raspberrypi:~ $ sudo apt-get install libxvidcore-dev libx264-dev libgtk-3-dev pi@raspberrypi:~ $ sudo apt-get install libtbb2 libtbb-dev libdc1394-22-dev libv4l-dev pi@raspberrypi:~ $ sudo apt-get install libopenblas-dev libatlas-base-dev libblas-dev pi@raspberrypi:~ $ sudo apt-get install libjasper-dev liblapack-dev libhdf5-dev pi@raspberrypi:~ $ sudo apt-get install protobuf-compiler
Install Qt5
The software can be used to beautify OpenCV windows and other user interfaces, such as sliders and checkboxes. This is not necessary but we will use it for demostration.
Then we'll have to set the -D WITH_QT flag during the build.
pi@raspberrypi:~ $ sudo apt-get install qt5-default
Download OpenCV
There are two packages needed; the basic version and the additional contributions.
Later release is 4.5.2
https://github.com/opencv/opencv/releases/tag/4.5.2
Basic version
pi@raspberrypi:~ $ cd ~ pi@raspberrypi:~ $ wget -O opencv.zip https://github.com/opencv/opencv/archive/4.5.2.zip
Additional contributions
Later release is 4.5.2
https://github.com/opencv/opencv_contrib/releases/tag/4.5.2
pi@raspberrypi:~ $ wget -O opencv_contrib.zip https://github.com/opencv/opencv_contrib/archive/4.5.2.zip
Unzip the archives
pi@raspberrypi:~ $ unzip opencv.zip pi@raspberrypi:~ $ unzip opencv_contrib.zip
And rename the folders
pi@raspberrypi:~ $ mv opencv-4.5.2 opencv pi@raspberrypi:~ $ mv opencv_contrib-4.5.2 opencv_contrib
Create virtual environment
Check python version
pi@raspberrypi:~ $ python3 --version Python 3.7.3
Get location
pi@raspberrypi:~ $ which python 3.7 /usr/bin/python
Merge VIRTUALENVWRAPPER_PYTHON=location/version
pi@raspberrypi:~ $ echo "export VIRTUALENVWRAPPER_PYTHON=/usr/bin/python3.7" >> ~/.bashrc
and then reload profile
pi@raspberrypi:~ $ source ~/.bashrc
Install the virtual environment software
pi@raspberrypi:~ $ sudo pip3 install virtualenv pi@raspberrypi:~ $ sudo pip3 install virtualenvwrapper
Then edit bashrc
pi@raspberrypi:~ $ echo "export WORKON_HOME=$HOME/.virtualenvs" >> ~/.bashrc pi@raspberrypi:~ $ echo "source /usr/local/bin/virtualenvwrapper.sh" >> ~/.bashrc pi@raspberrypi:~ $ source ~/.bashrc
With the command mkvirtualenv, a virtual environment is set up for the OpenCV installation.
pi@raspberrypi:~ $ mkvirtualenv cv452
created virtual environment CPython3.7.3.final.0-32 in 1281ms
creator CPython3Posix(dest=/home/pi/.virtualenvs/cv452, clear=False, no_vcs_ignore=False, global=False)
seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/home/pi/.local/share/virtualenv)
added seed packages: pip==21.1.2, setuptools==57.0.0, wheel==0.36.2
activators BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator,XonshActivator
virtualenvwrapper.user_scripts creating /home/pi/.virtualenvs/cv452/bin/predeactivate
virtualenvwrapper.user_scripts creating /home/pi/.virtualenvs/cv452/bin/postdeactivate
virtualenvwrapper.user_scripts creating /home/pi/.virtualenvs/cv452/bin/preactivate
virtualenvwrapper.user_scripts creating /home/pi/.virtualenvs/cv452/bin/postactivate
virtualenvwrapper.user_scripts creating /home/pi/.virtualenvs/cv452/bin/get_env_details
(cv452) pi@raspberrypi:~ $
OpenCV depends on Numpy, we need to install in our new virtual environment
(cv452) pi@raspberrypi:~ $ pip3 install numpy
Build make
Lets create new folder for the build
(cv452) pi@raspberrypi:~ $ cd ~/opencv/ (cv452) pi@raspberrypi:~/opencv $ mkdir build (cv452) pi@raspberrypi:~/opencv $ cd build
We save space by excluding any (Python) examples or tests.
(cv452) pi@raspberrypi:~/opencv/build $ cmake -D CMAKE_BUILD_TYPE=RELEASE \ -D CMAKE_INSTALL_PREFIX=/usr/local \ -D OPENCV_EXTRA_MODULES_PATH=~/opencv_contrib/modules \ -D ENABLE_NEON=ON \ -D ENABLE_VFPV3=ON \ -D WITH_OPENMP=ON \ -D WITH_OPENCL=OFF \ -D BUILD_TIFF=ON \ -D WITH_FFMPEG=ON \ -D WITH_TBB=ON \ -D BUILD_TBB=ON \ -D BUILD_TESTS=OFF \ -D WITH_EIGEN=OFF \ -D WITH_GSTREAMER=OFF \ -D WITH_V4L=ON \ -D WITH_LIBV4L=ON \ -D WITH_VTK=OFF \ -D WITH_QT=ON \ -D OPENCV_ENABLE_NONFREE=ON \ -D INSTALL_C_EXAMPLES=OFF \ -D INSTALL_PYTHON_EXAMPLES=OFF \ -D BUILD_opencv_python3=TRUE \ -D OPENCV_GENERATE_PKGCONFIG=ON \ -D BUILD_EXAMPLES=OFF ..
And this is the generated report. Look for python3 section
-- General configuration for OpenCV 4.5.2 ===================================== -- Version control: unknown -- -- Extra modules: -- Location (extra): /home/pi/opencv_contrib/modules -- Version control (extra): unknown -- -- Platform: -- Timestamp: 2021-06-20T09:21:03Z -- Host: Linux 5.10.17-v7l+ armv7l -- CMake: 3.16.3 -- CMake generator: Unix Makefiles -- CMake build tool: /usr/bin/make -- Configuration: RELEASE -- -- CPU/HW features: -- Baseline: VFPV3 NEON -- requested: DETECT -- required: VFPV3 NEON -- -- C/C++: -- Built as dynamic libs?: YES -- C++ standard: 11 -- C++ Compiler: /usr/bin/c++ (ver 8.3.0) -- C++ flags (Release): -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Wsuggest-override -Wno-delete-non-virtual-dtor -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections -mfpu=neon -fvisibility=hidden -fvisibility-inlines-hidden -fopenmp -O3 -DNDEBUG -DNDEBUG -- C++ flags (Debug): -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Wsuggest-override -Wno-delete-non-virtual-dtor -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections -mfpu=neon -fvisibility=hidden -fvisibility-inlines-hidden -fopenmp -g -O0 -DDEBUG -D_DEBUG -- C Compiler: /usr/bin/cc -- C flags (Release): -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wuninitialized -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections -mfpu=neon -fvisibility=hidden -fopenmp -O3 -DNDEBUG -DNDEBUG -- C flags (Debug): -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wuninitialized -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections -mfpu=neon -fvisibility=hidden -fopenmp -g -O0 -DDEBUG -D_DEBUG -- Linker flags (Release): -Wl,--gc-sections -Wl,--as-needed -- Linker flags (Debug): -Wl,--gc-sections -Wl,--as-needed -- ccache: NO -- Precompiled headers: NO -- Extra dependencies: dl m pthread rt -- 3rdparty dependencies: -- -- OpenCV modules: -- To be built: aruco bgsegm bioinspired calib3d ccalib core cvv datasets dnn dnn_objdetect dnn_superres dpm face features2d flann freetype fuzzy gapi hdf hfs highgui img_hash imgcodecs imgproc intensity_transform line_descriptor mcc ml objdetect optflow phase_unwrapping photo plot python2 python3 quality rapid reg rgbd saliency shape stereo stitching structured_light superres surface_matching text tracking ts video videoio videostab wechat_qrcode xfeatures2d ximgproc xobjdetect xphoto -- Disabled: world -- Disabled by dependency: - -- Unavailable: alphamat cnn_3dobj cudaarithm cudabgsegm cudacodec cudafeatures2d cudafilters cudaimgproc cudalegacy cudaobjdetect cudaoptflow cudastereo cudawarping cudev java julia matlab ovis sfm viz -- Applications: perf_tests apps -- Documentation: NO -- Non-free algorithms: YES -- -- GUI: -- QT: YES (ver 5.11.3) -- QT OpenGL support: NO -- GTK+: NO -- -- Media I/O: -- ZLib: /usr/lib/arm-linux-gnueabihf/libz.so (ver 1.2.11) -- JPEG: /usr/lib/arm-linux-gnueabihf/libjpeg.so (ver 62) -- WEBP: build (ver encoder: 0x020f) -- PNG: /usr/lib/arm-linux-gnueabihf/libpng.so (ver 1.6.36) -- TIFF: build (ver 42 - 4.2.0) -- JPEG 2000: build (ver 2.4.0) -- OpenEXR: build (ver 2.3.0) -- HDR: YES -- SUNRASTER: YES -- PXM: YES -- PFM: YES -- -- Video I/O: -- DC1394: YES (2.2.5) -- FFMPEG: YES -- avcodec: YES (58.35.100) -- avformat: YES (58.20.100) -- avutil: YES (56.22.100) -- swscale: YES (5.3.100) -- avresample: NO -- v4l/v4l2: YES (linux/videodev2.h) -- -- Parallel framework: TBB (ver 2020.2 interface 11102) -- -- Trace: YES (with Intel ITT) -- -- Other third-party libraries: -- Lapack: NO -- Custom HAL: YES (carotene (ver 0.0.1)) -- Protobuf: build (3.5.1) -- -- Python 2: -- Interpreter: /usr/bin/python2.7 (ver 2.7.16) -- Libraries: /usr/lib/arm-linux-gnueabihf/libpython2.7.so (ver 2.7.16) -- numpy: /usr/lib/python2.7/dist-packages/numpy/core/include (ver 1.16.2) -- install path: lib/python2.7/dist-packages/cv2/python-2.7 -- -- Python 3: -- Interpreter: /home/pi/.virtualenvs/cv452/bin/python3 (ver 3.7.3) -- Libraries: /usr/lib/arm-linux-gnueabihf/libpython3.7m.so (ver 3.7.3) -- numpy: /home/pi/.virtualenvs/cv452/lib/python3.7/site-packages/numpy/core/include (ver 1.20.3) -- install path: lib/python3.7/site-packages/cv2/python-3.7 -- -- Python (for build): /usr/bin/python2.7 -- -- Java: -- ant: NO -- JNI: NO -- Java wrappers: NO -- Java tests: NO -- -- Install to: /usr/local -- ----------------------------------------------------------------- -- -- Configuring done -- Generating done -- Build files have been written to: /home/pi/opencv/build
Enlarge memory swap space temporally
Enlarge memory swap space to 2048 MByte. We need that fo the build process. After the make we'll have to reset the swap space back to its original 100 Mbyte. Flash memory can only write a limited number of cycles. In the end, it will wear your SD card out. It is therefore wise to keep memory swapping to a minimum.
Edit /etc/dphys-swapfile with nano:
(cv452) pi@raspberrypi:~/opencv/build $ sudo nano /etc/dphys-swapfile
And change CONF_SWAPSIZE to 2048. <Ctrl-X>, then Y and ENTER
Activate the new swap size:
(cv452) pi@raspberrypi:~/opencv/build $ sudo /etc/init.d/dphys-swapfile stop [ ok ] Stopping dphys-swapfile (via systemctl): dphys-swapfile.service. (cv452) pi@raspberrypi:~/opencv/build $ sudo /etc/init.d/dphys-swapfile start [ ok ] Starting dphys-swapfile (via systemctl): dphys-swapfile.service.
Make
This takes a lot of time.
Start 11:37 finish 12:42, total time 1 hour and 5 minutes, with VNC.
Use 4 cores for the make
(cv452) pi@raspberrypi:~/opencv/build $ make -j4
After one hour
[100%] Linking CXX shared module ../../lib/cv2.so [100%] Linking CXX shared module ../../lib/python3/cv2.cpython-37m-arm-linux-gnueabihf.so [100%] Built target opencv_python2 [100%] Built target opencv_python3
Temperature during the build
pi@raspberrypi:~ $ /opt/vc/bin/vcgencmd measure_temp temp=51.6'C
External temperature 31'C
Install all the generated packages and update the database of the system:
(cv452) pi@raspberrypi:~/opencv/build $ sudo make install (cv452) pi@raspberrypi:~/opencv/build $ sudo ldconfig
And some cleaning
(cv452) pi@raspberrypi:~/opencv/build $ make clean (cv452) pi@raspberrypi:~/opencv/build $ sudo apt-get update
Resetting the swap space back to its original 100 Mbyte
(cv452) pi@raspberrypi:~/opencv/build $ sudo nano /etc/dphys-swapfile
<CTRL-X> [Y] [Enter]
Remove zip archives an reboot to change swap space size
(cv452) pi@raspberrypi:~/opencv/build $ cd ~ (cv452) pi@raspberrypi:~ $ rm opencv.zip (cv452) pi@raspberrypi:~ $ rm opencv_contrib.zip (cv452) pi@raspberrypi:~ $ sudo reboot
Make a symbolic link to the library
pi@raspberrypi:~ $ cd ~/.virtualenvs/cv452/lib/python3.7/site-packages pi@raspberrypi:~/.virtualenvs/cv452/lib/python3.7/site-packages $ ln -s /usr/local/lib/python3.7/site-packages/cv2/python-3.7/cv2.cpython-37m-arm-linux-gnueabihf.so pi@raspberrypi:~/.virtualenvs/cv452/lib/python3.7/site-packages $ cd ~ pi@raspberrypi:~ $
Check the installation
Lets check from python3
More cleaning
pi@raspberrypi:~ $ sudo rm -rf ~/opencv pi@raspberrypi:~ $ sudo rm -rf ~/opencv_contrib
Get Build Information
pi@raspberrypi:~ $ python
Python 2.7.16 (default, Oct 10 2019, 22:02:15)
[GCC 8.3.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import cv2
>>> cv2.__version__
'4.5.2'
>>> print(cv2.getBuildInformation())
General configuration for OpenCV 4.5.2 =====================================
Version control: unknown
Extra modules:
Location (extra): /home/pi/opencv_contrib/modules
Version control (extra): unknown
Platform:
Timestamp: 2021-06-20T09:21:03Z
Host: Linux 5.10.17-v7l+ armv7l
CMake: 3.16.3
CMake generator: Unix Makefiles
CMake build tool: /usr/bin/make
Configuration: RELEASE
CPU/HW features:
Baseline: VFPV3 NEON
requested: DETECT
required: VFPV3 NEON
C/C++:
Built as dynamic libs?: YES
C++ standard: 11
C++ Compiler: /usr/bin/c++ (ver 8.3.0)
C++ flags (Release): -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Wsuggest-override -Wno-delete-non-virtual-dtor -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections -mfpu=neon -fvisibility=hidden -fvisibility-inlines-hidden -fopenmp -O3 -DNDEBUG -DNDEBUG
C++ flags (Debug): -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Wsuggest-override -Wno-delete-non-virtual-dtor -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections -mfpu=neon -fvisibility=hidden -fvisibility-inlines-hidden -fopenmp -g -O0 -DDEBUG -D_DEBUG
C Compiler: /usr/bin/cc
C flags (Release): -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wuninitialized -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections -mfpu=neon -fvisibility=hidden -fopenmp -O3 -DNDEBUG -DNDEBUG
C flags (Debug): -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wuninitialized -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections -mfpu=neon -fvisibility=hidden -fopenmp -g -O0 -DDEBUG -D_DEBUG
Linker flags (Release): -Wl,--gc-sections -Wl,--as-needed
Linker flags (Debug): -Wl,--gc-sections -Wl,--as-needed
ccache: NO
Precompiled headers: NO
Extra dependencies: dl m pthread rt
3rdparty dependencies:
OpenCV modules:
To be built: aruco bgsegm bioinspired calib3d ccalib core cvv datasets dnn dnn_objdetect dnn_superres dpm face features2d flann freetype fuzzy gapi hdf hfs highgui img_hash imgcodecs imgproc intensity_transform line_descriptor mcc ml objdetect optflow phase_unwrapping photo plot python2 python3 quality rapid reg rgbd saliency shape stereo stitching structured_light superres surface_matching text tracking ts video videoio videostab wechat_qrcode xfeatures2d ximgproc xobjdetect xphoto
Disabled: world
Disabled by dependency: -
Unavailable: alphamat cnn_3dobj cudaarithm cudabgsegm cudacodec cudafeatures2d cudafilters cudaimgproc cudalegacy cudaobjdetect cudaoptflow cudastereo cudawarping cudev java julia matlab ovis sfm viz
Applications: perf_tests apps
Documentation: NO
Non-free algorithms: YES
GUI:
QT: YES (ver 5.11.3)
QT OpenGL support: NO
GTK+: NO
Media I/O:
ZLib: /usr/lib/arm-linux-gnueabihf/libz.so (ver 1.2.11)
JPEG: /usr/lib/arm-linux-gnueabihf/libjpeg.so (ver 62)
WEBP: build (ver encoder: 0x020f)
PNG: /usr/lib/arm-linux-gnueabihf/libpng.so (ver 1.6.36)
TIFF: build (ver 42 - 4.2.0)
JPEG 2000: build (ver 2.4.0)
OpenEXR: build (ver 2.3.0)
HDR: YES
SUNRASTER: YES
PXM: YES
PFM: YES
Video I/O:
DC1394: YES (2.2.5)
FFMPEG: YES
avcodec: YES (58.35.100)
avformat: YES (58.20.100)
avutil: YES (56.22.100)
swscale: YES (5.3.100)
avresample: NO
v4l/v4l2: YES (linux/videodev2.h)
Parallel framework: TBB (ver 2020.2 interface 11102)
Trace: YES (with Intel ITT)
Other third-party libraries:
Lapack: NO
Custom HAL: YES (carotene (ver 0.0.1))
Protobuf: build (3.5.1)
Python 2:
Interpreter: /usr/bin/python2.7 (ver 2.7.16)
Libraries: /usr/lib/arm-linux-gnueabihf/libpython2.7.so (ver 2.7.16)
numpy: /usr/lib/python2.7/dist-packages/numpy/core/include (ver 1.16.2)
install path: lib/python2.7/dist-packages/cv2/python-2.7
Python 3:
Interpreter: /home/pi/.virtualenvs/cv452/bin/python3 (ver 3.7.3)
Libraries: /usr/lib/arm-linux-gnueabihf/libpython3.7m.so (ver 3.7.3)
numpy: /home/pi/.virtualenvs/cv452/lib/python3.7/site-packages/numpy/core/include (ver 1.20.3)
install path: lib/python3.7/site-packages/cv2/python-3.7
Python (for build): /usr/bin/python2.7
Java:
ant: NO
JNI: NO
Java wrappers: NO
Java tests: NO
Install to: /usr/local
-----------------------------------------------------------------
>>>
Set Thonny
Interpreter: /home/pi/.virtualenvs/cv452/bin/python3 (ver 3.7.3)
Generate Aruco Tag
import numpy as np
import cv2 as cv
import sys
tag = np.zeros((300, 300, 1), dtype="uint8")
cv.aruco.drawMarker(cv.aruco.Dictionary_get(cv.aruco.DICT_4X4_50), 1, 300, tag,$
# write the generated ArUCo tag to disk and then display it
cv.imwrite("tags/DICT_5X5_100_id24.png" , tag)
cv.imshow("Aruco Tag", tag)
cv.waitKey(0)
Generate 12 markers
import numpy as np
import cv2, PIL
from cv2 import aruco
import matplotlib.pyplot as plt
import matplotlib as mpl
aruco_dict = aruco.Dictionary_get(aruco.DICT_6X6_250)
fig = plt.figure()
nx = 4
ny = 3
for i in range(1, nx*ny+1):
ax = fig.add_subplot(ny,nx, i)
img = aruco.drawMarker(aruco_dict,i, 700)
plt.imshow(img, cmap = mpl.cm.gray, interpolation = "nearest")
ax.axis("off")
plt.savefig("/home/pi/Pictures/markers.pdf")
plt.show()
cv2.waitKey(0)
Capturing image with the webcam
import cv2 as cv
# initialize the camera
cam = cv.VideoCapture(0)
ret, image = cam.read()
if ret:
cv.imshow('SnapshotTest',image)
cv.waitKey(0)
cv.destroyWindow('SnapshotTest')
cv.imwrite('/home/pi/book/output/SnapshotTest.jpg',image)
cam.release()
Processing the image
import numpy as np
import cv2 as cv, PIL
from cv2 import aruco
import matplotlib.pyplot as plt
import matplotlib as mpl
# initialize the camera
cam = cv.VideoCapture(0)
ret, image = cam.read()
cam.release()
if ret:
gray = cv.cvtColor(image, cv.COLOR_BGR2GRAY)
aruco_dict = aruco.Dictionary_get(aruco.DICT_6X6_250)
parameters = aruco.DetectorParameters_create()
corners, ids, rejectedImgPoints = aruco.detectMarkers(gray, aruco_dict, parameters=parameters)
frame_markers = aruco.drawDetectedMarkers(image.copy(), corners, ids)
plt.figure()
plt.imshow(frame_markers)
for i in range(len(ids)):
c = corners[i][0]
plt.plot([c[:, 0].mean()], [c[:, 1].mean()], "o", label = "id={0}".format(ids[i]))
plt.legend()
plt.show()
Reading the codes remotely with a webcam
Next steps
- Determine what are the best codes for our project.
- Develop routines to accurately locate the position of the moving parts of the windows.
- Find a better camera for the Pi and select an appropriate lens.
- Add remote monitoring.
Next Blog: Window Opening Monitor with ArUco - Tracking window movements >>