Cassie taught itself how to walk using reinforcement learning. After some accidents and spills, Cassie taught itself how to walk around. (Image credit: University of Berkley)
If you're afraid of robots taking over the world, then you won't be happy to hear this. A pair of robot legs dubbed Cassie recently taught itself how to walk using reinforcement learning. Even more surprising, Boston Dynamics aren't the makers behind it. A team from the University of California, Berkley, is working with Cassie, who moves and stumbles around like a toddler.
So, what is reinforcement learning? It's a training technique that teaches AI complex behavior by trial and error. In other words, the robot tries to walk and learns from its mistakes whenever it fails, similar to how babies learn how to walk. But before Cassie could stumble around in the real world, the team started with a simulation of the robot in a virtual world. Using a training environment called MuJoCo, the simulated robot referenced a large library of possible movements and learned how to apply them. The team then ran a second simulation, called Matlab SimMechanics, to test the robot in simulated real-world conditions.
After the simulations, the research was transferred to Cassie, which used it to teach itself how to walk. Over time, Cassie could not only successfully walk forwards and sideways. It could even carry unexpected loads and managed to compensate when it damaged two of its motors.
"The learned policies enable Cassie to perform a set of diverse and dynamic behaviors, while also being more robust than traditional controllers and prior learning-based methods that use residual control," read the study. "We demonstrate this on versatile walking behaviors such as tracking a target walking velocity, walking height, and turning yaw."
The Berkley team will continue to work on Cassie in hopes of improving its movements and trying out "more dynamics and agile behaviors." Meanwhile, other tech companies are using reinforcement learning for their own projects. Last year, Google used the method to train a four-legged robot, and OpenAI has used it to train robotic arms. The approach is fairly new, but researchers believe it's promising. Edward Johns, head of Imperial College London's Robot Learning Lab, told MIT Technology Review, "This is one of the most successful examples I have seen."
Have a story tip? Message me at: http://twitter.com/Cabe_Atwell