Group Actions
Engagement
  • Author Author: rictrajano
  • Date Created: Date Created
  • Views 815 views
  • Likes 2 likes
  • Comments 3 comments
Related
Recommended

Track object in movie file and transform into data to be used with arduino servos

rictrajano

Hello,

 

After doing some research I haven’t found exactly what i’m looking for. Basically I looking for a way to translate movement of an object in a video file (like a recording of a ball jumping up a down) to values that I can use to move servos. I’m more comfortable with arduino so that’s the microcontroller that I’m aiming to use with this project.

I’d like to either be able to move the servos in realtime in sync while playing the video or saving the values and using them later on by themselves.

I imagine something like playing a video, defining what in the video I want to track and having it saving the values of the object position in the screen.

 

Thanks for your help in advance.

  • If you solely need the coordinate data that you will later do some math on to get your servo values , here's how I'd do it:

     

    1. Shoot your footage with a tripod.  Be perpendicular to the action being video'd to avoid "into the plane" movement.
    2. Download, install, and open Blender
    3. Click the icon by the View Menu to change the Editor Typeto Movie Clip Editor
    4. Click open to load in your movie.
    5. Find something on what you want to track that stands out.  Hold CTRL and left click it.  This will add our tracker.
    6. Hit the track ahead button or CTRL t so it will advance the movie and track your object.
    7. You can now scrub the view with the icons on the bottom of the page and get the coordinates on the right.  I'd record time, x, and y in a spreadsheet and then make a function to translate that to servo values.

     

    Here is a picture where I just tracked my lawn mower as the camera moved.  You can see a white box at its wheel, a red line showing where the box was, and a blue line where the box is going in time.  The other red dotted sections the track forward button (left) and the coordinates (right).

    image

     

    Sounds like you have a fun project.

     

    See ya',

    Sean

  • Before you even get to image recognition you need to consider how much memory resource is required just to capture and store one image. Then with a video you need to then recognise that a new image will arrive every 1/24th of a second (if 24 frames per sec) or even 1/30th of a second (if 30 frames per second). So MCU processing speed and run time memory need to match the camera and image resolution to start things off and then you need additional resource for the image processing to determine if the object you are looking for is present etc.

     

    So, you may be quite limited on options with a standard Arduino - there are new Arduino models just out that should work (e.g https://www.arduino.cc/pro/hardware ). Adafruit's Metro M4 (or any decent ARM Cortex M4 with fpc) should also work too in simpler image recognition cases (https://www.adafruit.com/product/3382 ). SBC's like Raspberry Pi, Beaglebone etc. are probably the better choice here if minimal latency is a requirement for servo movement.

     

    https://www.tensorflow.org/  or https://www.tensorflow.org/lite  are popular choices for SBC's.

     

    Then there are some popular Python based options, which could work on an M4: https://opencv.org/  and SimpleCV

     

    Others picked up from online search include: https://github.com/jesolem/PCV  and a c++ option https://github.com/dtschump/CImg

     

    Otherwise you may be better off using the cloud for the image recognition grunt work. All the major players, like Google, AWS, Azure, IBM offer image recognition api's.

     

    PS. Another consideration/idea is to use a mobile phone to handle the image capture and processing (using tensor flow) and then link with an arduino via BLE to push coordinates across for servo movement... so, basically there are other ways to skin this cat...

  • An interesting idea. Maybe you could re-use some ideas from the PixyCam device as it is open source and replace the live input video stream with the recorded video stream.

     

    Dubbie