Artificial intelligence, and neural network-based machine learning in particular, has been in the headlines a lot over the past several years. Some are so sophisticated that they almost seem like magic, or that they truly possess intelligence. The reality, of course, is that there is nothing magic about it, and no machine has ever actually been intelligent by human standards. Even the most impressive machine learning systems are simply the result of very clever programming, and that means they’re susceptible to error. To test how well they’ll work in the real world, researchers at MIT have developed a method for assessing machine learning models.
Machine learning, at its core, is a method for automatically creating analytical models based on data sets. That’s something humans and other animals do naturally. For instance, you learn early on that some objects will float in water and others will sink. With enough experience (data), you can eventually infer whether a new object is likely to float or not, even if you’ve never seen that object in the water. What you’ve done is build an analytical model in your mind based on known conditions and their results, and used that model to predict the outcome of a new scenario that you haven’t experienced before.
Simple classifiers can determine if an image contains a cat or dog (Photo Credit: Tech Xplore)
That’s the same principle that machine learning systems follow. If you wanted a machine learning system to identify pictures of dogs, for example, you could show it thousands of pictures of dogs and thousands of pictures of things that aren’t dogs. With enough data, the system will find similarities between the pictures that indicate the presence of a dog. With that machine learning model, the system can then look at an entirely new photo and determine if there is a dog in it.
The problem is that a machine learning model is only as good as the data it has been given, and, because it doesn’t possess any true intelligence, it can’t improvise when an unexpected input is received. Imagine that, as a child, you never saw a single object float in the water. Everything you saw go into the water was a dense object like a rock, and would immediately sink. Unless someone told you otherwise, you would logically conclude that everything sinks in water. That’s the same thing that happens when a machine learning system isn’t given enough data or the proper kind of data.
“Adversarial examples” are inputs that have some modification that fools a convolutional neural network into misclassifying the input. An image-classifying machine learning convolutional neural network, for example, can be thrown off by just a couple of pixels that look “wrong.” That’s because a convolutional neural network doesn’t see an image like a person does, it just relies on its training to recognize the values of groups of pixels and what those most often correspond to.
When developing machine learning convolutional neural networks, researchers will often modify an image very slightly to see if it becomes an adversarial example for their model. But that is a very time-consuming task when thousands of test images are needed. The method developed by the researchers from MIT automates that process so that every possible variation of an image, within certain parameters, can be tested quickly. If any adversarial examples are found, they know the convolutional neural network needs more work, and they can attempt to determine why it was fooled.
Their testing method starts with an image that the convolutional neural network is completely capable of correctly classifying. For example, if the system is trained to recognize certain kinds of foods, they might use a simple picture of a hamburger. When it’s operating properly, the neural network will work through all of the possible food choices, and eventually decide that it is most likely a hamburger. But that’s just the choice with the highest probability, and the system would say that there is a chance it’s a sandwich, and an even smaller chance that it’s a hotdog.
The goal of the testing is to see if the neural network can be made to decide that the hamburger is actually a hot dog—or anything else. That would be an adversarial example that proves the neural network may not be ready for the real world. But they don’t want to make any modifications to the image that would also confuse a human, only changes that people could easily process.
They do that by systematically changing the values of pixels, and testing the results. Those value changes can be as simple as lightening or darkening a single pixel, but that’s often enough to fool a neural network. Because that process is automated, they can quickly test thousands of variations of an image. As they’re doing so, they can track how much the neural network’s classification certainty changes. If at any point the neural network classifies the hamburger as a sandwich, hot dog, or spaghetti, they know they’ve found an adversarial example.
In the case of classifying food, that hardly seems important. But the same principles can be applied to any kind of machine learning convolutional neural network, such as the one that tells a self-driving car that there is a stop sign ahead. Or the one that tells the bank what numbers are written on the check you signed. Thoroughly testing neural networks ensures that they’re ready for the real world, and could save a lot of money—and even lives.