element14 Community
element14 Community
    Register Log In
  • Site
  • Search
  • Log In Register
  • Community Hub
    Community Hub
    • What's New on element14
    • Feedback and Support
    • Benefits of Membership
    • Personal Blogs
    • Members Area
    • Achievement Levels
  • Learn
    Learn
    • Ask an Expert
    • eBooks
    • element14 presents
    • Learning Center
    • Tech Spotlight
    • STEM Academy
    • Webinars, Training and Events
    • Learning Groups
  • Technologies
    Technologies
    • 3D Printing
    • FPGA
    • Industrial Automation
    • Internet of Things
    • Power & Energy
    • Sensors
    • Technology Groups
  • Challenges & Projects
    Challenges & Projects
    • Design Challenges
    • element14 presents Projects
    • Project14
    • Arduino Projects
    • Raspberry Pi Projects
    • Project Groups
  • Products
    Products
    • Arduino
    • Avnet Boards Community
    • Dev Tools
    • Manufacturers
    • Multicomp Pro
    • Product Groups
    • Raspberry Pi
    • RoadTests & Reviews
  • Store
    Store
    • Visit Your Store
    • Choose another store...
      • Europe
      •  Austria (German)
      •  Belgium (Dutch, French)
      •  Bulgaria (Bulgarian)
      •  Czech Republic (Czech)
      •  Denmark (Danish)
      •  Estonia (Estonian)
      •  Finland (Finnish)
      •  France (French)
      •  Germany (German)
      •  Hungary (Hungarian)
      •  Ireland
      •  Israel
      •  Italy (Italian)
      •  Latvia (Latvian)
      •  
      •  Lithuania (Lithuanian)
      •  Netherlands (Dutch)
      •  Norway (Norwegian)
      •  Poland (Polish)
      •  Portugal (Portuguese)
      •  Romania (Romanian)
      •  Russia (Russian)
      •  Slovakia (Slovak)
      •  Slovenia (Slovenian)
      •  Spain (Spanish)
      •  Sweden (Swedish)
      •  Switzerland(German, French)
      •  Turkey (Turkish)
      •  United Kingdom
      • Asia Pacific
      •  Australia
      •  China
      •  Hong Kong
      •  India
      •  Korea (Korean)
      •  Malaysia
      •  New Zealand
      •  Philippines
      •  Singapore
      •  Taiwan
      •  Thailand (Thai)
      • Americas
      •  Brazil (Portuguese)
      •  Canada
      •  Mexico (Spanish)
      •  United States
      Can't find the country/region you're looking for? Visit our export site or find a local distributor.
  • Translate
  • Profile
  • Settings
Digital Fever
  • Challenges & Projects
  • Project14
  • Digital Fever
  • More
  • Cancel
Digital Fever
Blog SecuCam #1: Experimenting with the MKR Vidor 4000
  • Blog
  • Forum
  • Documents
  • Events
  • Polls
  • Files
  • Members
  • Mentions
  • Sub-Groups
  • Tags
  • More
  • Cancel
  • New
Join Digital Fever to participate - click to join for free!
  • Share
  • More
  • Cancel
Group Actions
  • Group RSS
  • More
  • Cancel
Engagement
  • Author Author: Attila Tőkés
  • Date Created: 21 Mar 2021 10:46 AM Date Created
  • Views 5224 views
  • Likes 8 likes
  • Comments 2 comments
  • arduino mkr vidor 4000
  • mkr vidor 4000
  • digitalfeverch
Related
Recommended

SecuCam #1: Experimenting with the MKR Vidor 4000

Attila Tőkés
Attila Tőkés
21 Mar 2021

This is the first blog post of my "SecuCam" project.

 

The goal of the project is build a basic Security Camera while also experimenting with hardware-accelerated video processing features like Motion Detection.

 

1. Arduino MKR Vidor 4000

 

So, after I proposed an idea, tariq.ahmadkindly sent me an Arduino MKR Vidor 4000 board. Thanks for that! image

 

The Arduino MRK Vidor 4000 board is a pretty interesting Arduino board from the MKR Family.

 

image

 

It is based on a Atmel / Microchip SAMD21 controller, but what makes it special is the Altera / Intel Cyclone 10 FPGA.

 

Along the standard stuff, the board has a MIPI CSI Camera Input, a Micro-HDMI Ouput and a WiFi module.

 

1.1 Getting Started

 

Getting started with Arduino MKR Vidor 4000 is fairly simple.

 

We need an Arduino IDE, with Arduino SAMD Boards support installed:

image

 

At this point we should be able to run the Blink sketch on the board, the same way as on any other Arduino board.

 

Note:
if the upload to the MKR Vidor 4000 does not works try "double-clicking" the Reset button. It should bring the board into boot-loader mode.

 

1.1 Camera to HDMI Example

Blink is boring! So, I jumped right into searching how to use the Camera and HDMI features of the board image

 

image

 

For this we need the following external components:

  • Raspberry Pi Camera Module v1.3 with the OV5647 sensor
  • HDMI Display
  • MicroHDMI to HDMI cable (or adapter + HDMI cable)

 

We need to connect them like this:

image

 

On software side we need to install the VidorGraphics library:

image

 

Now we can have some fun by running the VidorEnableCam example:

 

#include "VidorGraphics.h"
#include "VidorCamera.h"


VidorCamera vcam;


void setup() {
  Serial.begin(9600);
  
  // wait for the serial monitor to open,
  // if you are powering the board from a USB charger remove the next line
  while (!Serial) {}


  if (!FPGA.begin()) {
    Serial.println("Initialization failed!");
    while (1) {}
  }


  // begin() enables the I2C communication and initializes the camera


  if (!vcam.begin()) {
    Serial.println("Camera begin failed");
    while (1) {}
  }


  delay(4000);
  Serial.println("Power On");


  // The camera should be on now, streaming to the HDMI output
}


void loop()
{
  // Wait for the user input to stop the camera


  String res = Serial.readStringUntil('\n');
  if (res.indexOf("STOP") > 0) {
    vcam.end();
  }
}

 

Once we upload this example to the Vidor 4000 and open a Serial Monitor, the board should start streaming the RaspiCam's image to the HDMI Display.

 

image

 

The VidorGraphics library comes with pre-built FPGA bitstream image, capable of handling MIPI CSI-2 Camera inputs, HDMI Display outputs and some miscellaneous functionality.

 

The library also offers a Vidor_GFX graphics API, similar to the one implemented in the Adafruit_GFX library. We will use this later. image

 

2. Re-Compiling the VidorGraphics FPGA Bitstream?

 

The VidorGraphics library comes with a pre-built FPGA image which contains the implementation the GFX API and some miscellanies functionality.

 

My initial idea was to try to rebuild this FPGA image, and then try to implement some image processing on the Camera video stream.

 

The FPGA project for VidorBitstream library, along with some other ones, are available at:

https://github.com/vidor-libraries/VidorBitstream

 

MKR Vidor 4000 FPGA Development

 

Although Arduino made the FPGA project available, FPGA development on the MKR Vidor 4000 does not seems to be officially supported.

 

To compile the projects either the Intel Quartus II 18.1 Lite (free) or Standard (paid) versions are needed. The official image is build with the Standard (paid) version, and it is supposed to have significantly better performance over a version built with the Lite (free) version.

 

I attempted to build open the project with Quartus II 18.1 Lite, but unfortunately it fails because apparently some custom components are missing.

image

There are some patches provided to fix this, but it does not seems to work.

 

At this point I decided to give this up custom FPGA development on the MKR VIdor 4000, at least for now.

 

Anyways, I was still interested in what the Cyclone 10 FPGA is used for in the MKR Vidor 4000.

 

From a brief on look on the FPGA project and a the VidorGraphics library implementation, I think the FPGA project may be structured something like this:

  • MIPI CSI RX subsystem is captured the camera image
  • a HDMI TX subsystem drives the MicroHDMI output
  • the GFX API seems to have its own Buffer (probably in the SDRAM)
  • some Video Mixer component may be used to combine the Camera and GFX images
  • there is a NIOS II Soft-Core Processor, which handles commands received from SAMD21 over UART
  • the FPGA is programmed by the SAMD21 via JTAG
  • there are some GPIO, UART, SPI modules for GPIO and communication with the WiFi module

 

image

Disclaimer:

the above diagram is more like an educated guess on what is implemented in the VidorGraphic's FPGA image. I did not fully checked what is actually in the project.

 

As I can't easily do FPGA development on the MKR Vidor 4000, the device will be used basically as a MIPI CSI to HDMI converter. Along with the RaspiCam V1.3 it forms a fancy HDMI Camera.

 

But, I came up with a new plan image. The actual hardware accelerated video processing will be done on PYNQ Z2 using Pynq!

 

3. Customizing the HDMI Output

 

At this point the MKR Vidor 4000 was still was just a MIPI CSI to HDMI converter. So, I thought we need some features! image

 

I decided on adding a Project14 "logo" and a Timestamp as overlays to the HDMI Output.

 

To draw on the HDMI output we can use the VidorGraphics library. Fortunately, a separate image buffer is used for VidorGraphics. The means we can draw or logos and timestamps as slow as we want, and the camera image still remains responsive. image

 

Adding the "Logo" was easy. I just printed the "Project14" text with some hack to make the text bold. I was too lazy covert an actual Project 14 logo into a bitmap and try to actually display that on the output. image

 

Adding the "Timestamp" caused some headaches, as I wanted to get the current time from the Internet and for this I needed to get the Wifi Module to work.

 

The Wifi Module can be used with the WiFiNINA library from Arduino, but with the latest versions the MKR Vidor 1000 is not officially supported. So, to get it work took some browsing on the Arduino forum, building a firmware and flashing it on the Wifi module.

 

Next, I used a Rest API form WorldClockApi.com to get the Current Time (UTC). The time than is displayed on the screen.

 

This is what I ended up with: image

image

 

Note:

The source code is attached to this post. It is pretty hackish. Do not take it too seriously! image

 

4. Fixing Camera Settings

 

The VidorGraphics library has support for the OV5647 Camera Sensor. This is the same sensor as used in the RaspiCam v1.3 camera modules.

 

The OV5647 has a maximum resolution of 5 MegaPixels. With the MKR Vidor 4000 works with the 640x480 resolution.

 

Now, in the VidorGraphics library the OV5647 is configured with somewhat non-optimal settings.

 

Namely, the just the top-left corner of the sensor is used, which especially on a wide angle camera leads to a weird view-port.

 
(Red is what is used, Blue is the optimal for 640x480 resolution, Green is what the camera is capable of)imageimage

 

We can improve this by using and area in the middle of the sensor. This can be done by setting the X and Y offset registers. On the top of that we can use 2x2 sub-sampling (this was already enabled), which leads to an area of 1280x960 pixels used in the middle of the sensor.

 

Here is what the Original and Fixed images look like:

imageimage

 

To achieve this in the VidorCamera.h file complete cam_640x480 list with the following values:

  { 0x5003, 0x09 },
  { 0x5025, 0x11 },
  { 0x504B, 0x30 },
  { 0x3814, 0x31 },
  { 0x3815, 0x31 },
    
  { 0x3800, 0x00 },
  { 0x3801, 0x00 },
  { 0x3802, 0x00 },
  { 0x3803, 0x20 },
  { 0x3804, 0x0A },
  { 0x3805, 0x1F },
  { 0x3806, 0x07 },
  { 0x3807, 0x8F },


  { 0x3811, 0x08 },
  { 0x3813, 0x02 },


  { 0x5006, 0xF0 },
  { 0x5007, 0xC0 },
  { 0x5018, 0x31 },
  { 0x5019, 0x31 },
  { 0x5046, 0x09 },


  { 0x3800, 0x02 },
  { 0x3801, 0x98 },
  { 0x3802, 0x01 },
  { 0x3803, 0xF0 },
  
  { 0x380c, 0x07 },
  { 0x380d, 0xCC },

 

Note:

It would be even better if we could run the camera in a higher resolution, but this needs a new FPGA image and we may not have enough memory for this.

 

In the next blog post I will experiment with hardware accelerated video processing on a PYNQ Z2 board.

 

Hope You enjoyed this post! image

Attachments:
SecuCam.ino.txt.zip
  • Sign in to reply

Top Comments

  • DAB
    DAB over 4 years ago +1
    Nice start. DAB
  • dubbie
    dubbie over 4 years ago

    Very interesting and informative. I know abit about MKR and a bit about FPGA but almost nothing about how to use them to process video images. Should I ever get a MKR 4000 this would be something to try out. Could be useful for mobile robots.

     

    Dubbie

    • Cancel
    • Vote Up 0 Vote Down
    • Sign in to reply
    • More
    • Cancel
  • DAB
    DAB over 4 years ago

    Nice start.

     

    DAB

    • Cancel
    • Vote Up +1 Vote Down
    • Sign in to reply
    • More
    • Cancel
element14 Community

element14 is the first online community specifically for engineers. Connect with your peers and get expert answers to your questions.

  • Members
  • Learn
  • Technologies
  • Challenges & Projects
  • Products
  • Store
  • About Us
  • Feedback & Support
  • FAQs
  • Terms of Use
  • Privacy Policy
  • Legal and Copyright Notices
  • Sitemap
  • Cookies

An Avnet Company © 2025 Premier Farnell Limited. All Rights Reserved.

Premier Farnell Ltd, registered in England and Wales (no 00876412), registered office: Farnell House, Forge Lane, Leeds LS12 2NE.

ICP 备案号 10220084.

Follow element14

  • X
  • Facebook
  • linkedin
  • YouTube