element14 Community
element14 Community
    Register Log In
  • Site
  • Search
  • Log In Register
  • Members
    Members
    • Benefits of Membership
    • Achievement Levels
    • Members Area
    • Personal Blogs
    • Feedback and Support
    • What's New on element14
  • Learn
    Learn
    • Learning Center
    • eBooks
    • STEM Academy
    • Webinars, Training and Events
    • Learning Groups
  • Technologies
    Technologies
    • 3D Printing
    • Experts & Guidance
    • FPGA
    • Industrial Automation
    • Internet of Things
    • Power & Energy
    • Sensors
    • Technology Groups
  • Challenges & Projects
    Challenges & Projects
    • Design Challenges
    • element14 presents
    • Project14
    • Arduino Projects
    • Raspberry Pi Projects
    • Project Groups
  • Products
    Products
    • Arduino
    • Dev Tools
    • Manufacturers
    • Raspberry Pi
    • RoadTests & Reviews
    • Avnet Boards Community
    • Product Groups
  • Store
    Store
    • Visit Your Store
    • 'Choose another store...'
      • Europe
      •  Austria (German)
      •  Belgium (Dutch, French)
      •  Bulgaria (Bulgarian)
      •  Czech Republic (Czech)
      •  Denmark (Danish)
      •  Estonia (Estonian)
      •  Finland (Finnish)
      •  France (French)
      •  Germany (German)
      •  Hungary (Hungarian)
      •  Ireland
      •  Israel
      •  Italy (Italian)
      •  Latvia (Latvian)
      •  
      •  Lithuania (Lithuanian)
      •  Netherlands (Dutch)
      •  Norway (Norwegian)
      •  Poland (Polish)
      •  Portugal (Portuguese)
      •  Romania (Romanian)
      •  Russia (Russian)
      •  Slovakia (Slovak)
      •  Slovenia (Slovenian)
      •  Spain (Spanish)
      •  Sweden (Swedish)
      •  Switzerland(German, French)
      •  Turkey (Turkish)
      •  United Kingdom
      • Asia Pacific
      •  Australia
      •  China
      •  Hong Kong
      •  India
      •  Korea (Korean)
      •  Malaysia
      •  New Zealand
      •  Philippines
      •  Singapore
      •  Taiwan
      •  Thailand (Thai)
      • Americas
      •  Brazil (Portuguese)
      •  Canada
      •  Mexico (Spanish)
      •  United States
      Can't find the country/region you're looking for? Visit our export site or find a local distributor.
  • Translate
  • Profile
RoadTests & Reviews
  • Products
  • More
RoadTests & Reviews
Blog PYNQ-Z2 Dev Kit - CIFAR-10 Webcam continued....
  • Blog
  • RoadTest Forum
  • Documents
  • RoadTests
  • Reviews
  • Polls
  • Files
  • Members
  • Mentions
  • Sub-Groups
  • Tags
  • More
  • Cancel
  • New
RoadTests & Reviews requires membership for participation - click to join
  • RoadTest
  • pynqworkshpch
  • pynq-z2
  • cifar10
  • machine learning
  • Subscribe by email
  • More
  • Cancel
  • Share
  • Subscribe by email
  • More
  • Cancel
Related
Recommended

PYNQ-Z2 Dev Kit - CIFAR-10 Webcam continued....

ralphjy
ralphjy
14 Aug 2019

In the previous blog PYNQ-Z2 Dev Kit - CIFAR-10 Convolutional Neural Network , I verified the 3 hardware classifiers against the reference "deer" test image.  Now I'm going to see how the classifiers perform with captured webcam images.  I expect the performance will be degraded because the webcam will produce lower quality images due to issues like image brightness and focus.  CIFAR-10 has a small training set (5000 images per class), so I'm going to use a solid background to help keep the image simple.  Intuitively, I would expect that with higher quantization level the classification accuracy would improve.

 

I decided that since CIFAR-10 has transportation classes (automobile, truck, ship, airplane) that I would try to classify a few Matchbox vehicles that had escaped being recycled.

 

Here is my test setup.  I was a bit disappointed in the auto-focus and auto-exposure of the webcam, but I haven't figured out whether I can manually control this particular camera with OpenCV.

image

 

 

First try is the firetruck.  I expected this one to be easy, but only the W2A2 classifier got it right.

image

 

Second try is the convertible.  All the classifiers thought this was an airplane.  I'm assuming that the odd colors may have been confusing.

image

 

Third try is a car that I thought should be easy but interestingly the classifiers with higher quantization very clearly classified it as an airplane and the binary classifier thought it was a ship.

image

 

 

As a head check I grabbed one of the CIFAR-10 test images and it correctly classified with increasing accuracy as the quantization increased.

image

 

At this point I'm confused as to the primary cause of the poor classification of the webcam images.  Is it the image quality or the actual image.  I suspect it is both.

 

I noticed that the example notebook includes a sample webcam capture of an elk figurine which should classify as a "deer".

image

 

I would consider this a much harder image to classify because of the background.  It's interesting that in two of the classifiers that "airplane" and "deer" are closely ranked.  I'm getting the impression that CIFAR-10 trained networks are a good educational tool, but would not work well in the real world.  I'm going to move on to look at other neural network examples.

  • Sign in to reply

Top Comments

  • beacon_dave
    beacon_dave over 4 years ago in reply to genebren +2
    Just wondering if the solid background and the loose framing isn't helping matters here ? Perhaps try adding some whitespace around the sports car training image and see if it too starts to fly...
  • ralphjy
    ralphjy over 4 years ago in reply to beacon_dave +2
    Okay, I did as you suggested and now it indeed does fly. Something tells me you've done this before . It does suggest that this network would not have worked as I would have tried to use it.
  • dubbie
    dubbie over 4 years ago +2
    It is one of the strange (?) things about using Artificial Neural Networks (ANN), particularly when using image systems or other data that is viewed by humans, is that they use all the data. When a human…
  • dubbie
    dubbie over 4 years ago

    It is one of the strange (?) things about using Artificial Neural Networks (ANN), particularly when using image systems or other data that is viewed by humans, is that they use all the data. When a human looks at the car picture they just see the car and use that part of the image for recognition. an ANN isn't intelligent and uses all the image, including the background so when recognising the car, or plane or whatever, it is actually just recognising the array of pixel values and working out the nearest trained image of pixels values it has been given.  So by making all the backgrounds similar (blank or anything else really) then it becomes the differences such as the car or plane and so on, that the ANN uses to make a match. The ANN will also use the position of the different pixels as well, so either the car or plane has to be precisely centred in every image, or, the training data must contain images of the car located in as many alternative locations within the image as possible. It can lead to a very large training data set, with the corresponding problem that the larger the training data set then the longer the training period, along with a greater possibility of not converging onto a working solution. Another  problem with ANNs is, depending on the type of ANN and training used, that because training starts with some randomised numbers that training is not repeatable. So you could train, get a working solution, decide to retrain without changing anything and then when retraining, it will just not converge. My approach was that if a working system was obtained, always make sure to keep a copy of it before retraining.

     

    Dubbie

    • Cancel
    • Vote Up +2 Vote Down
    • Sign in to reply
    • More
    • Cancel
  • ralphjy
    ralphjy over 4 years ago in reply to beacon_dave

    Okay, I did as you suggested and now it indeed does fly.  Something tells me you've done this before image.   It does suggest that this network would not have worked as I would have tried to use it.

    image

    • Cancel
    • Vote Up +2 Vote Down
    • Sign in to reply
    • More
    • Cancel
  • clem57
    clem57 over 4 years ago

    Training is more art than scienceimage

    • Cancel
    • Vote Up 0 Vote Down
    • Sign in to reply
    • More
    • Cancel
  • ralphjy
    ralphjy over 4 years ago in reply to beacon_dave

    An easy enough experiment to try.  Did not yield though.  Still favors airplane although now the binary classifier thinks it's a cat image.

     

    image

    Sorry misinterpreted your comment.  Let me give it another go with the training image....

    • Cancel
    • Vote Up +1 Vote Down
    • Sign in to reply
    • More
    • Cancel
  • beacon_dave
    beacon_dave over 4 years ago in reply to genebren

    Just wondering if the solid background and the loose framing isn't helping matters here ?

     

    Perhaps try adding some whitespace around the sports car training image and see if it too starts to fly...

    • Cancel
    • Vote Up +2 Vote Down
    • Sign in to reply
    • More
    • Cancel
>
element14 Community

element14 is the first online community specifically for engineers. Connect with your peers and get expert answers to your questions.

  • Members
  • Learn
  • Technologies
  • Challenges & Projects
  • Products
  • Store
  • About Us
  • Feedback & Support
  • FAQs
  • Terms of Use
  • Privacy Policy
  • Legal and Copyright Notices
  • Sitemap
  • Cookies

An Avnet Company © 2023 Premier Farnell Limited. All Rights Reserved.

Premier Farnell Ltd, registered in England and Wales (no 00876412), registered office: Farnell House, Forge Lane, Leeds LS12 2NE.

ICP 备案号 10220084.

Follow element14

  • X
  • Facebook
  • linkedin
  • YouTube