Researchers developed a Smartwatch app that can predict the mood of a conversation, and could have a promising future for autistic or socially anxious people. MIT’s new smartwatch app that detects user emotions used with Samsung’s Simband (via Jason Dorfman/MIT CSAIL)
I saw a loose mention of this device on the latest episode of "The Big Bang Theory," (Season 10, episode 14) The Emotion Detection Automation. But, it was not as cool as the real thing.
Social interaction can be very complicated, confusing, and distressing for some people, and although many people might be curious about this technology for its entertainment value, it could serve a more practical purpose for people that struggle in social situations. Researchers from the Computer Science and Artificial Intelligence Laboratory (CSAIL) and the Institute of Medical Engineering and Science (IMES) at MIT have developed a wearable system that can evaluate the tone of conversations relatively accurately using information from the user’s vitals and speech patterns. According to MIT graduate student Tuka Al Hanai, this research is leading the world toward the possibility of having an artificially intelligent social coach in their pocket.
This technology works by analyzing audio, text, and physiological information to ascertain the prevailing tone of the conversation, and differentiates between happy, sad, and neutral. According to MIT News writers, Adam Conner-Simons and Rachel Gordon, the algorithms this system employs are “deep-learning techniques” that enable it to provide a “sentiment score” for five-second intervals. When they were developing the algorithms, to gauge more natural emotions, they asked participants to tell ‘happy’ or ‘sad’ stories rather than try to act out those emotions. After 31 conversations, the researchers modeled two algorithms from the data: one that categorizes the overall tone of the conversation as either happy, sad, or neutral, and another that analyzes 5-second chunks and classifies them as positive, negative, or neutral. Finally, this work resulted in a system that, on average, could classify each five-second interval approximately 18% more accurately than chance, and 7.5% more than existing methods.
It’s accuracy in preliminary stages is promising, and while it currently only runs on Samsung’s Simband, the researchers hope to eventually be able to adapt their system to the Apple Watch so their technology can be more widely available. Al Hanai says that their next step is to, “...improve the algorithm’s emotional granularity so that it is more accurate at calling out boring, tense, and excited moments, rather than just labeling interactions as ‘positive’ or ‘negative’”, and although the technology isn’t reliable or accessible enough to be sold commercially for social coaching, that is the ultimate goal of their research. Based on the enormity of what they have accomplished thus far, it seems that their goal is just waiting to be met.
The video below is a demonstration of this smart technology in action:
Have a story tip? Message me at: cabe(at)element14(dot)com
