Group Actions
Engagement
  • Author Author: dubbie
  • Date Created: Date Created
  • Views 1926 views
  • Likes 11 likes
  • Comments 7 comments
Related
Recommended

ARBot #1 : Testing the Concept of Augmented Reality (for Electronic Toys)

dubbie
image

Electronic Toys

Enter Your Electronics & Design Project for a chance to win a Grand Prize of a SNES Classic and $200 Shopping Cart of Product!

Back to The Project14 homepage image

Project14 Home
Monthly Themes
Monthly Theme Poll

 

Having spent some time thinking about a possible activity for the Project14 Electronic Toys competition I have decided to try and make some sort of augmented reality game. Augmented Reality, as far as this project is concerned, is taking a video stream  and processing it in some way and adding additional information on screen as an overlay. I thought about Virtual Reality but I think that is just too far out of my reach.

 

Although I have done some simple image processing on static images using MATLAB I do not have any experience of using live video streams. Nor am I sufficiently good at programming to take an existing image processing package and use it to achieve augmented reality in the time scale available. Instead I am aiming to take a number of low cost, existing systems and modules and attempt to bodge them all together to achieve some sort of working augmented reality system. The system will be designed to recognise specific objects within an environment. A small mobile robot containing a video camera will be moving about in the environment and transferring the augmented video stream to a head-mounted display. The orientation of the video camera will be synchronised to the head-set orientation using some sort of 3D orientation sensor. The mobile robot movements will be controlled by the user, probably with some sort of joystick or similar manual controller and the aim will be to move through an environment, identifying the pre-selected targets, and then if possible, having some method of 'shooting' at them, using the headset to aim. A bit like a video game, except with a 'real' mobile robot moving over a module landscape.

 

To make sure that this all seems at least remotely possible, I have connected together a PixyCam to my laptop and run the supplied PixyMon programme. I have trained the Pixycam to recognise yellow objects, in this example they are yellow Lego men. I will not be able to stick a complete laptop on the headset so I have used an LCD display connected via a HDMI cable. The trial LCD display is quite large but a much smaller one, probably 3.5 inches or maybe 5 inches, looks likely to work just as well. Below is a video showing the trial system actually working (yes, I was surprised as well). It appears a bit blurry as the Pixycam doesn't have a very good lense and it is a video of a display, but it does seem to demonstrate that the concept might work.

 

 

There is a white line box drawn around each of the two yellow Lego men. It is a bit faint but it is there and a label is given to these, top left. The Pixy cam also outputs the size and type of object that it has identified, using a serial communication port (either I2C or SPI - I cannot remember at present) which can be connected to an Arduino for further processing. It is possible to train the PixyCam to recognise several different colour objects (it actually recognises the colour and not the object shape) so some could be friendly and others enemies to shoot at. The headset will then be used to move the PixyCam so that an objected to be shot at is within the centre of the image and then either a light can be shone at the object, or possibly even something physical fired at the object. At present I am not sure how to register a hit if using light, maybe a light sensor in the object. If I can get a gun that fires some sort of physical object, maybe rubber disks, then that could be used to knock over the object.

 

Well, it might all work, you never know, but it should be fun trying.

 

Dubbie

Parents

  • Dubbie,

     

    It should be fun indeed,  The Pixy cams are pretty cool.  I remember looking into some time a go.  They seem to do a pretty good job of recognizing objects once properly trained.

     

    In thinking about you targeting objects, what if you used a low power laser pointer.  Then you might be able to sense if the object is hit by using the Pixy cam, assuming that you can train it to recognized a laser pointer 'painted' object as well as an unmarked object.

     

    Good luck!

     

    Gene

  • in reply to genebren

    Gene,

     

    Thanks for the helpful comments. Hopefully it will not be too challenging! I have thought about using a laser pointer and I really wanted to use one of the green laser beam ones but they seem quite expensive, and possibly quite dangerous. I used lasers when I was studying at University and it gave me the willies - one slip or mistake and you were blind, so I would want to stay with Class I or II lasers, which are of course, not that bright! I tried using a simple torch to create a bright spot on the yellow Lego man but although I could see it, the PixyCam didn't, part of the image processing it does I expect. Also, I wasn't sure what colour a green laser would create on a yellow Lego man, as the PixyCam detects colours. I'm sure there would be some colour combination that would be detectable, if I can get a bright enough laser pointer.

     

    Dubbie

Comment
  • in reply to genebren

    Gene,

     

    Thanks for the helpful comments. Hopefully it will not be too challenging! I have thought about using a laser pointer and I really wanted to use one of the green laser beam ones but they seem quite expensive, and possibly quite dangerous. I used lasers when I was studying at University and it gave me the willies - one slip or mistake and you were blind, so I would want to stay with Class I or II lasers, which are of course, not that bright! I tried using a simple torch to create a bright spot on the yellow Lego man but although I could see it, the PixyCam didn't, part of the image processing it does I expect. Also, I wasn't sure what colour a green laser would create on a yellow Lego man, as the PixyCam detects colours. I'm sure there would be some colour combination that would be detectable, if I can get a bright enough laser pointer.

     

    Dubbie

Children
  • in reply to dubbie

    "so I would want to stay with Class I or II lasers, which are of course, not that bright!"

    The class II green laser pointer I have in some MicroCue presentation clickers are pretty bright.

     

    I recall that there was an article published a few years back about a danger of some low brightness green laser pointers. Apparently the energy was ending up in the infrared band and often exceeded class II levels and the eye's blink response was unlikely to protect you from injury. I recall that they demonstrated a low cost test rig to help people check their pointers.

     

    "Also, I wasn't sure what colour a green laser would create on a yellow Lego man"

    The above green laser pointer shows up as very green when pointed at a yellow plastic storage bin.

     

    The Ansmann Future T50F LED torches are surprisingly bright and if the lens is put into spot beam mode may give you a tight enough and bright enough light source. It will produce a visible 150mm dia spot at 1m in bright office conditions.

    Product LinkProduct Link

  • in reply to beacon_dave

    Dave,

     

    Thanks for the link, the document was interesting and helpful. I might think about trying a green laser pointer to see if it works. Maybe not one of the higher powered ones though.

     

    I tried a smaller, cheaper version of this torch (free in a Christmas cracker I think), although I could see the brighter spot the PixyCam didn't as it levelled out all the different brightnesses of the yellow to make it's image processing easier. A different colour torch might work so I might have to substitute a coloured LED instead of a white LED. Or maybe just buy some lenses and make my own spot LED torch (which sounds like a project all by itself).

     

    Dubbie