Recently, NVIDIA revealed their researchers have developed a new system using unique deep learning that allows a robot to learn through human function/action. The idea here is to make it an interesting spin for the robot to learn by having it develop some sort of communication between robot and human - ideally, the robot would have the ability to observe and copy the human in every move and action a human makes. The team is also hoping this will be beneficial in the future when robots and humans can work together continuously.
An image being processed by the robot to identify objects using synthetic data and recognizing the shape. (Image Credit: Nvidia) 
The researchers used NVIDIA TITAN X GPUs to train a series of neural networks to perform specific duties involving perception, program generation and execution.
The way it works is, the robot observes what is happening in relation to a task then it comes up with a list of steps required to complete the task. Afterward, a human would verify the validity of the list to confirm the correctness of it before the robot implements the steps it created. Researchers were able to correctly establish and test this out with cubes - which the robot was required to stack in the correct order. This can be seen in the video below:
A benefit of this is that it doesn't require an overly large amount of training data to be inputted into the system. What happens instead is the system produces the data synthetically - which allows the robot to come up with the training data it needs to come up with, learn and complete the task. It requires very little effort to achieve something like this, as a result. The team also uses an image-centric domain randomization approach - which is the first time this has ever been done on robots - and that allows the system to use synthetic data. The data has a large amount of variety for what it can read via image perception neural networks and doesn't need to rely on the environment it's in or a camera to perform its duties.
The perception network being used and identified with these objects is applied to any real-world object, and that can be used by its 3D bounding cuboids. The robot never observes a real image under training, but it can detect cuboids of objects in real images even where it may be difficult to see it.
The researchers will present their findings this week at the International Conference of Robotics and Automation. They will use this opportunity use their data in other areas to improve their research - specifically for synthetic data to use these methods in many different scenarios.
Have a story tip? Message me at: cabe(at)element14(dot)com
Top Comments