Track object in movie file and transform into data to be used with arduino servos

RE: Track object in movie file and transform into data to be used with arduino servos

Sean_Miller — Thu, 06 Feb 2020 15:29:31 GMT

If you solely need the coordinate data that you will later do some math on to get your servo values , here's how I'd do it:

Shoot your footage with a tripod. Be perpendicular to the action being video'd to avoid "into the plane" movement.
Download, install, and open Blender
Click the icon by the View Menu to change the Editor Typeto Movie Clip Editor
Click open to load in your movie.
Find something on what you want to track that stands out. Hold CTRL and left click it. This will add our tracker.
Hit the track ahead button or CTRL t so it will advance the movie and track your object.
You can now scrub the view with the icons on the bottom of the page and get the coordinates on the right. I'd record time, x, and y in a spreadsheet and then make a function to translate that to servo values.

Here is a picture where I just tracked my lawn mower as the camera moved. You can see a white box at its wheel, a red line showing where the box was, and a blue line where the box is going in time. The other red dotted sections the track forward button (left) and the coordinates (right).

[View:/resized-image/__size/620x336/__key/commentfiles/f7d226abd59f475c9d224a79e3f0ec07-537cdd2d-2fe5-4a80-a11f-6b1a8f71858d/6136.contentimage_5F00_205985.png:620:336]

Sounds like you have a fun project.

See ya',

Sean

RE: Track object in movie file and transform into data to be used with arduino servos

BigG — Thu, 06 Feb 2020 10:59:26 GMT

Before you even get to image recognition you need to consider how much memory resource is required just to capture and store one image. Then with a video you need to then recognise that a new image will arrive every 1/24th of a second (if 24 frames per sec) or even 1/30th of a second (if 30 frames per second). So MCU processing speed and run time memory need to match the camera and image resolution to start things off and then you need additional resource for the image processing to determine if the object you are looking for is present etc.

So, you may be quite limited on options with a standard Arduino - there are new Arduino models just out that should work (e.g https://www.arduino.cc/pro/hardware ). Adafruit's Metro M4 (or any decent ARM Cortex M4 with fpc) should also work too in simpler image recognition cases (https://www.adafruit.com/product/3382 ). SBC's like Raspberry Pi, Beaglebone etc. are probably the better choice here if minimal latency is a requirement for servo movement.

https://www.tensorflow.org/ or https://www.tensorflow.org/lite are popular choices for SBC's.

Then there are some popular Python based options, which could work on an M4: https://opencv.org/ and SimpleCV

Others picked up from online search include: https://github.com/jesolem/PCV and a c++ option https://github.com/dtschump/CImg

Otherwise you may be better off using the cloud for the image recognition grunt work. All the major players, like Google, AWS, Azure, IBM offer image recognition api's.

PS. Another consideration/idea is to use a mobile phone to handle the image capture and processing (using tensor flow) and then link with an arduino via BLE to push coordinates across for servo movement... so, basically there are other ways to skin this cat...

RE: Track object in movie file and transform into data to be used with arduino servos

dubbie — Thu, 06 Feb 2020 09:46:45 GMT

An interesting idea. Maybe you could re-use some ideas from the PixyCam device as it is open source and replace the live input video stream with the recorded video stream.

Dubbie