element14 Community
element14 Community
    Register Log In
  • Site
  • Search
  • Log In Register
  • Members
    Members
    • Benefits of Membership
    • Achievement Levels
    • Members Area
    • Personal Blogs
    • Feedback and Support
    • What's New on element14
  • Learn
    Learn
    • Learning Center
    • eBooks
    • STEM Academy
    • Webinars, Training and Events
    • Learning Groups
  • Technologies
    Technologies
    • 3D Printing
    • Experts & Guidance
    • FPGA
    • Industrial Automation
    • Internet of Things
    • Power & Energy
    • Sensors
    • Technology Groups
  • Challenges & Projects
    Challenges & Projects
    • Design Challenges
    • element14 presents
    • Project14
    • Arduino Projects
    • Raspberry Pi Projects
    • Project Groups
  • Products
    Products
    • Arduino
    • Dev Tools
    • Manufacturers
    • Raspberry Pi
    • RoadTests & Reviews
    • Avnet Boards Community
    • Product Groups
  • Store
    Store
    • Visit Your Store
    • 'Choose another store...'
      • Europe
      •  Austria (German)
      •  Belgium (Dutch, French)
      •  Bulgaria (Bulgarian)
      •  Czech Republic (Czech)
      •  Denmark (Danish)
      •  Estonia (Estonian)
      •  Finland (Finnish)
      •  France (French)
      •  Germany (German)
      •  Hungary (Hungarian)
      •  Ireland
      •  Israel
      •  Italy (Italian)
      •  Latvia (Latvian)
      •  
      •  Lithuania (Lithuanian)
      •  Netherlands (Dutch)
      •  Norway (Norwegian)
      •  Poland (Polish)
      •  Portugal (Portuguese)
      •  Romania (Romanian)
      •  Russia (Russian)
      •  Slovakia (Slovak)
      •  Slovenia (Slovenian)
      •  Spain (Spanish)
      •  Sweden (Swedish)
      •  Switzerland(German, French)
      •  Turkey (Turkish)
      •  United Kingdom
      • Asia Pacific
      •  Australia
      •  China
      •  Hong Kong
      •  India
      •  Korea (Korean)
      •  Malaysia
      •  New Zealand
      •  Philippines
      •  Singapore
      •  Taiwan
      •  Thailand (Thai)
      • Americas
      •  Brazil (Portuguese)
      •  Canada
      •  Mexico (Spanish)
      •  United States
      Can't find the country/region you're looking for? Visit our export site or find a local distributor.
  • Translate
  • Profile
RoadTests & Reviews
  • Products
  • More
RoadTests & Reviews
Blog PYNQ-Z2 Dev Kit - CIFAR-10 Convolutional Neural Network
  • Blog
  • RoadTest Forum
  • Documents
  • RoadTests
  • Reviews
  • Polls
  • Files
  • Members
  • Mentions
  • Sub-Groups
  • Tags
  • More
  • Cancel
  • New
RoadTests & Reviews requires membership for participation - click to join
  • RoadTest
  • pynqworkshpch
  • pynq-z2
  • machine learning
  • Subscribe by email
  • More
  • Cancel
  • Share
  • Subscribe by email
  • More
  • Cancel
Related
Recommended

PYNQ-Z2 Dev Kit - CIFAR-10 Convolutional Neural Network

ralphjy
ralphjy
13 Aug 2019

The first neural network implementation that I'm going to look at is for CIFAR-10 (Canadian Institute For Advanced Research).  CIFAR-10 is a computer vision dataset used to train and test neural networks for object recognition.  The CIFAR-10 data consists of 60,000 32x32 color images in 10 classes, with 6000 images per class. There are 50,000 training images and 10,000 test images.

 

Labeled Image Classes

  • airplane
  • automobile
  • bird
  • cat
  • deer
  • dog
  • frog
  • horse
  • ship
  • truck

 

The CIFAR-10 examples are from the BNN-PYNQ GitHub repository https://github.com/Xilinx/BNN-PYNQ.

 

First example: Deer image

The first example is in the Jupyter notebook CNV-BNN_Cifar10.  It uses trained parameters for the CNV network using 1 bit for weights and activation.  The CNV is described in the FINN paper https://arxiv.org/abs/1612.07119 .  "CNV is a convolutional network topology that contains a succession of (3x3 convolution, 3x3 convolution, 2x2 maxpool) layers repeated three times with 64-128-256 channels, followed by two fully connected layers of 512 neurons each."

 

This example instantiates both a hardware accelerated and pure software inference classifier.  It classifies an image from the CIFAR-10 test set using both hardware and software classifiers to demonstrate the speed advantage of the hardware implementation.  The image in this example is of a deer.  I added a cell to show the detailed classification ranking of the different CIFAR-10 classes for this image.

 

Video showing the step-by-step execution of the notebook.

You don't have permission to edit metadata of this video.
Edit media
x
image
Upload Preview
image

 

Here are the classifier rankings copied from the notebook output:

image

You can see that for this image the classification is unambiguous.  The deer ranking is 1.7x the next nearest class.

 

Second example: Dog images

Next I created a notebook to classify images in the 'dog' class.  For this example I used one image from the CIFAR10 test set and 4 images that I downloaded from the internet.  The images are shown below.

image

 

Videos are nice to see the execution sequence but since the flow is the same as the previous example - I've just extracted the notebook data for easier reading.

 

When I first ran the classification, the classifier had difficulty with dog2 (differentiating it from a cat) and puppy2 was mis-classified as a 'ship'.  Looking at the puppy2 image you can see that the puppy is sitting on a rug that is a distinct horizontal shape in the image.  As an experiment I edited the image to remove the rug and ran the classifier again.

 

Here is the modified image:

image

 

And here are the full classifier results:

image

image

You can see that removing the rug (horizontal shape) reduced the 'ship' ranking dramatically.  This demonstrates the difficulty of using a network trained on such a small set of images.  There will be extreme sensitivity to any artifacts in the image relative to the subject you are trying to classify.

 

And here is a plot of the data:

image

 

Last example: Webcam images

As a final example of the BNN-CIFAR10 network, I'm going to try classifying images captured by a webcam.

 

The hardware inference (classification) will be performed withe different precision for weights and activation.

 

Here are the 3 cases:

  1. W1A1 - 1 bit weights and 1 activation, this is the BNN we were using previously
  2. W1A2 - 1 bit weight and 2 activation
  3. W2A2 - 2 bit weights and 2 activation

 

As the complexity of the network increases the execution time will increase but hopefully so will the inference accuracy.

 

I am using a Logitech C525 USB webcam which is HD 720P with autofocus.  It is plugged directly into the USB Host port on the PYNQ-Z2.

 

The code to capture the image from the webcam is straightforward using the OpenCV-Python (cv2) and the Python Imaging Library (PIL).  The code also allows for brightness enhancement as webcam brightness is very sensitive to ambient lighting.

import cv2
from PIL import Image as PIL_Image
from PIL import ImageEnhance
from PIL import ImageOps


# says we capture an image from a webcam
cap = cv2.VideoCapture(0) 
_ , cv2_im = cap.read()
cv2_im = cv2.cvtColor(cv2_im,cv2.COLOR_BGR2RGB)
img = PIL_Image.fromarray(cv2_im)
img

# The enhancement values (contrast and brightness) depend on backgroud, external lights etc
bright = ImageEnhance.Brightness(img)                                     
img = bright.enhance(0.95)

 

W1A1 Classifier

hw_classifier = bnn.CnvClassifier(bnn.NETWORK_CNVW1A1,"cifar10",bnn.RUNTIME_HW)
class_ranksW1A1=hw_classifier.classify_image_details(img)
inferred_class=class_ranksW1A1.argmax()
print("Inferred class: {0}".format(inferred_class))
print("Class name: {0}".format(hw_classifier.class_name(inferred_class)))

 

W1A2 Classifier

hw_classifier = bnn.CnvClassifier(bnn.NETWORK_CNVW1A2,"cifar10",bnn.RUNTIME_HW)
class_ranksW1A2=hw_classifier.classify_image_details(img)
inferred_class=class_ranksW1A2.argmax()
print("Inferred class: {0}".format(inferred_class))
print("Class name: {0}".format(hw_classifier.class_name(inferred_class)))

 

W2A2 Classifier

hw_classifier = bnn.CnvClassifier(bnn.NETWORK_CNVW2A2,"cifar10",bnn.RUNTIME_HW)
class_ranksW2A2=hw_classifier.classify_image_details(img)
inferred_class=class_ranksW2A2.argmax()
print("Inferred class: {0}".format(inferred_class))
print("Class name: {0}".format(hw_classifier.class_name(inferred_class)))

 

 

For my first test of the 3 different network precisions, I am going to use the deer test image to generate a comparison.  That will be my performance baseline before using the webcam captured images.

 

Case 1:

W1A1 - 1 bit weights and 1 activation

Inference took 1582.00 microseconds

Classification rate: 632.11 images per second

Inferred class: 4

Class name: Deer

 

Case 2:

W1A2 - 1 bit weight and 2 activation

Inference took 1627.00 microseconds

Classification rate: 614.63 images per second

Inferred class: 4

Class name: Deer

 

Case 3:

W2A2 - 2 bit weights and 2 activation

Inference took 4867.00 microseconds

Classification rate: 205.47 images per second

Inferred class: 4

Class name: Deer

 

All 3 networks correctly inferred the 'deer' class.  You can see the increase in inference time with increased complexity.  So what did the complexity buy us?

 

Here is a comparison of the classification rankings:

 

W1A1:

image

W1A2:

image

W2A2:

image

For this image W1A2 and W2A2 are clearly better than W1A1, but there isn't much difference between W1A2 and W2A2.  W2A2 is 3x slower than W1A2.

 

To be continued.....

 

I had intended to put all the CIFAR-10 stuff in a single blog, but the webcam results surprised me a bit so I'll finish in another post.

  • Sign in to reply

Top Comments

  • dubbie
    dubbie over 4 years ago +1
    This was a very interesting blog. Having done some work with Artificial Neural Networks (but only a very little) I could understand the issues involved. It was also fun to see that this example would recognise…
  • mohitajais
    mohitajais over 2 years ago +1
    This is really very interesting. I am interested in doing its real time implementation using cifar10 dataset. I want to know how should I integrate base overlay and cnv overlay using vivado IP integrator…
  • mohitajais
    mohitajais over 2 years ago

    This is really very interesting. I am interested in doing its real time implementation using cifar10 dataset. I want to know how should I  integrate base overlay and cnv overlay using vivado IP integrator for real time implementation using PYNQ board

    • Cancel
    • Vote Up +1 Vote Down
    • Sign in to reply
    • More
    • Cancel
  • dubbie
    dubbie over 4 years ago

    This was a very interesting blog. Having done some work with Artificial Neural Networks (but only a very little) I could understand the issues involved. It was also fun to see that this example would recognise cats. For my CatDogFox CatDogFoxBot #6 : Trying out an Artificial Neural Network  Project 14 Remote Monitoring and Control activity I have used a much simpler ANN and set of images to try and recognise a cat using data from a GridEye thermopile array sensor. It works, sort of, but as you indicated, more training and more data should produce a better result.

     

    Dubbie

    • Cancel
    • Vote Up +1 Vote Down
    • Sign in to reply
    • More
    • Cancel
element14 Community

element14 is the first online community specifically for engineers. Connect with your peers and get expert answers to your questions.

  • Members
  • Learn
  • Technologies
  • Challenges & Projects
  • Products
  • Store
  • About Us
  • Feedback & Support
  • FAQs
  • Terms of Use
  • Privacy Policy
  • Legal and Copyright Notices
  • Sitemap
  • Cookies

An Avnet Company © 2023 Premier Farnell Limited. All Rights Reserved.

Premier Farnell Ltd, registered in England and Wales (no 00876412), registered office: Farnell House, Forge Lane, Leeds LS12 2NE.

ICP 备案号 10220084.

Follow element14

  • X
  • Facebook
  • linkedin
  • YouTube