M5Stack UnitV2 - Jupyter Notebooks

I've found Jupyter Notebooks to be very useful for interactive Python development.  I first started using them with the PYNQ-Z2 and most recently running machine learning models in Google Colab.

I had hoped to see a few AI example notebooks on the UnitV2, but it turns out there was only a single example notebook covering basic interface functions - check Python version (3.8), control LEDs via GPIO, communication over Grove UART, camera image capture and display.

It's funny that the notebook was last modified 51 years ago .  I had updated the system time (it hadn't been set, so was running from the beginning of Linux time - 1970).

Introduction to M5Stack UnitV2.ipynb

# Hi, Welcome to M5Stack UnitV2!


## This introduction will help you go through all the basic function of the M5 UnitV2.
### We will have more documents and examples in this notebooks in the following software updates!


------
### Section 1: Introduction to python


UnitV2 along with this Jupyter Notebook is running the Python 3.8 which supports all cool features of the python!


import sys
print("This is Python {}.{}.".format(sys.version_info.major, sys.version_info.minor))


You can try the fancy prints like this:


hello = 'hello'
print(f'{hello} %s' % 'world')


or some new gramma candy like this:


a = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11]
if (n := len(a)) > 10:
    print(f"List is too long ({n} elements, expected <= 10)")


For all the new features, please refer to https://docs.python.org/3/whatsnew/3.8.html and check out the python tutorial https://docs.python.org/3.8/tutorial/index.html !


--------
### Section 2: Control LED
Inside the M5Stack UnitV2 there is two LED.


A white led on the GPIO0 and a red led on the GPIO1. You can control them by export the GPIO and set the value.


def control_white_led(value):
    open('/sys/class/gpio/export', 'w').write('0') # Export the GPIO0
    open('/sys/class/gpio/gpio0/direction', 'w').write('out') # Set the direction of the GPIO
    open('/sys/class/gpio/gpio0/value', 'w').write(str(value)) # Set the calute, '1' or '0'
    
def control_red_led(value):
    open('/sys/class/gpio/export', 'w').write('1') # Export the GPIO0
    open('/sys/class/gpio/gpio1/direction', 'w').write('out') # Set the direction of the GPIO
    open('/sys/class/gpio/gpio1/value', 'w').write(str(value)) # Set the calute, '1' or '0'


The LED are connected between the VCC and the GPIO,so we need to set it to zero to let it shine!


control_white_led(0)


We can also let blink the led!


import time


for i in range(10):
    control_white_led(0)
    time.sleep(0.5)
    control_white_led(1)
    time.sleep(0.5)


-------------------------
### Section 3: UART Communication on Grove


The grove port is connect to the device only through the on-board UART.
The UART device is located at /dev/ttyS1.


You can operate the port like this:


import serial




# Open the serial port, and set the baudrate to 115200
uart_grove = serial.Serial('/dev/ttyS1', 115200, timeout=0.5)


# Send Something
uart_grove.write('hello'.encode('utf-8')) 
# encode is for converting the string to bytes, or you can directly send bytes
uart_grove.write(b'hello') 


# Receive soemthing
x = uart_grove.read()          # read one byte
s = uart_grove.read(10)        # read up to ten bytes (timeout)
line = uart_grove.readline()   # read a '\n' terminated line


You can read more on the serial package at here: https://pyserial.readthedocs.io/en/latest/index.html


-----------------------------
### Section 4: Camera and Image Processing


M5Stack UnitV2 has a built-in camera located at /dev/video0.


You can try to capture a still image like this:


import cv2




camera = cv2.VideoCapture(0)
ret, frame = camera.read()


if ret:
    print('Capture Successful!')
else:
    print('OOps, we get some trouble!')


If you see 'Capture Successful', that means we captured a frame, congratulations!
Let try to show it here!


The first thing to do is import a libiary called matplotlib. this can help us show image and plots!


from matplotlib import pyplot as plt


# Magic to tell matplotlib display the image in jupyter notebook
%matplotlib inline 


# Let show the image! 
plt.imshow(frame)


Hmmmm, the color looks weired, doesn't it? What is going wrong?


OpenCV , for historical reason,is using BGR instead of RGB, but matplotlib do not know it. We need to help matplotlib show the correct image!


# Convert from BGR to RGB
frame_rgb = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)


plt.imshow(frame_rgb)


Wow, it looks cool! We can also convert it to grey.


# Convert from BGR to RGB
frame_gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)


# We need to tell matplotlib it is a grey image. You can try to change to other cmap, refer here: https://matplotlib.org/stable/tutorials/colors/colormaps.html
plt.imshow(frame_gray, cmap='gray')

The Jupyter environment opens terminals with root privilege.  The notebook directory is adjacent to the m5stack directory.

The Linux file structure is familiar.

The Intro Notebook says that more examples will be available with software updates, but no indication of when that will be.  In the meantime, I'll find some video AI models to try out.  From the OOB examples, I don't expect great performance but it does have a Neon accelerator.

I decided that I needed a simple desktop stand for the UnitV2 while I'm playing with programming.  Using the tripod mount with a short tripod takes up too much space.  Good news is that the unit also comes with a Lego Mount Clip, so I just printed a short stand that works great .

Here's the output from a quick Test Notebook that I generated:

So, an older version of OpenCV.  I may try updating that later.

The camera is connected to the processor via USB, so it looks to the system like a USB webcam.  Here's a picture of my messy desktop:

One thing I discovered is that I wasn't able to set the camera to FHD (1920x1080).  As a matter of fact, I was only able to set 640x480@15fps and 320x240@30fps.  I couldn't set any resolution higher or lower.  I'm not sure if this is a function of the camera driver.  I've posted a question on the M5Stack forum.

Another issue I encountered is that a number of the utilities that I use with OpenCV are not installed, so I'll need to fix that also.

Full list of installed modules is below:

Package              Version  
-------------------- ---------
aioblescan           0.2.6    
aiocoap              0.3      
aioconsole           0.1.15   
aiodns               2.0.0    
aiohttp              3.6.2    
aiohttp-cors         0.7.0    
aiohttp-debugtoolbar 0.6.0    
aiohttp-jinja2       1.2.0    
aiohttp-mako         0.4.0    
aiohttp-remotes      0.1.2    
aiohttp-security     0.4.0    
aiohttp-session      2.9.0    
aiohttp-sse          2.0.0    
aiojobs              0.2.2    
aiologstash          2.0.0    
aiomonitor           0.4.5    
aiorwlock            0.6.0    
aiosignal            1.0.0    
aiozipkin            0.6.0    
argon2-cffi          20.1.0   
arrow                0.15.2   
asgiref              3.2.3    
asn1crypto           1.2.0    
async-generator      1.10     
async-timeout        3.0.1    
attrs                19.3.0   
Babel                2.7.0    
backcall             0.1.0    
bcrypt               3.1.7    
bleach               3.3.0    
cached-property      1.5.1    
cchardet             2.1.5    
certifi              2019.9.11
cffi                 1.13.1   
chardet              3.0.4    
Click                7.0      
cryptography         2.8      
cycler               0.10.0   
decorator            4.4.1    
defusedxml           0.6.0    
docker               4.1.0    
dockerpty            0.4.1    
docopt               0.6.2    
ecdsa                0.13.3   
entrypoints          0.3      
esptool              2.8      
Flask                1.1.2    
Flask-Babel          0.12.2   
Flask-Cors           3.0.8    
Flask-JSONRPC        0.3.1    
Flask-Login          0.4.1    
Flask-SQLAlchemy     2.4.1    
frozenlist           1.0.0    
idna                 2.8      
ipykernel            5.4.3    
ipython              7.9.0    
ipython-genutils     0.2.0    
ipywidgets           7.6.3    
iso8601              0.1.12   
itsdangerous         1.1.0    
jaraco.functools     2.0      
jedi                 0.15.1   
Jinja2               2.10.3   
jsonschema           3.2.0    
jupyter              1.0.0    
jupyter-client       6.1.11   
jupyter-console      6.2.0    
jupyter-core         4.7.1    
jupyterlab-pygments  0.1.2    
jupyterlab-widgets   1.0.0    
keyring              10.5.0   
kiwisolver           1.1.0    
libusb1              1.7.1    
Mako                 1.1.0    
Markdown             3.1.1    
MarkupSafe           1.1.1    
matplotlib           3.0.3    
mistune              0.8.4    
more-itertools       7.2.0    
multidict            4.7.5    
nbclient             0.5.1    
nbconvert            6.0.7    
nbformat             5.1.2    
nest-asyncio         1.5.1    
networkx             1.11     
notebook             6.2.0    
numpy                1.16.4   
oauthlib             3.1.0    
packaging            20.9     
paho-mqtt            1.4.0    
pandocfilters        1.4.3    
parso                0.5.1    
pathtools            0.1.2    
pexpect              4.7.0    
pickleshare          0.7.5    
Pillow               7.0.0    
pip                  19.3.1   
portend              2.5      
prometheus-client    0.9.0    
prompt-toolkit       2.0.10   
protobuf             3.11.0   
ptyprocess           0.6.0    
py                   1.8.0    
pyaes                1.6.1    
pyalsaaudio          0.8.4    
pycares              3.1.1    
pycparser            2.19     
Pygments             2.4.2    
pylibconfig          0.0.1    
pyparsing            2.4.2    
pyrsistent           0.17.3   
pyserial             3.4      
pyserial-asyncio     0.4      
python-dateutil      2.8.1    
python-logstash      0.4.6    
pytz                 2019.3   
PyYAML               5.3.1    
pyzmq                18.1.1   
qtconsole            5.0.2    
QtPy                 1.9.0    
requests             2.22.0   
SecretStorage        3.1.1    
Send2Trash           1.5.0    
setuptools           41.4.0   
six                  1.14.0   
SQLAlchemy           1.3.10   
tempora              1.14.1   
terminado            0.9.2    
terminaltables       3.1.0    
testpath             0.4.4    
texttable            1.6.3    
tornado              6.1      
tqdm                 4.40.1   
traitlets            4.3.3    
urllib3              1.25.9   
wcwidth              0.1.7    
webencodings         0.5.1    
websocket-client     0.57.0   
Werkzeug             0.16.0   
widgetsnbextension   3.5.1    
yarl                 1.4.2    
Note: you may need to restart the kernel to use updated packages.

New toys are fun, but a lot of work...

I did get a chance measure the outer case temperature after the unit had been running for hours at 24 C.  It was 40 C, which is warm.  I'm not sure what the thermal resistance from the die is, but I'm sure there is a reasonable temperature delta to the case.  The sensors command wasn't found and a pip install didn't work, so I'll need to figure out whether I can get a die temp reading.  The fan is quiet enough that I don't notice it over the ambient noise.