The wearable subtitles display text in the wearer’s field-of-view based on what a speaker is saying. (Image Credit: Google Research)
People who are deaf or hard of hearing have trouble understanding anyone’s speech. Some solutions to this problem involve using a smartphone. Google researchers have developed Wearable Subtitles, a device that displays captions of a speaker’s speech. These smart glasses are designed for people who are deaf and hard of hearing. Even though this new model is not yet available for commercial use, the researchers say it may find its niche.
However, this technology isn’t relatively new. Many people with a hearing impairment use other devices like smartphones to recognize the speech of a speaker. Using this approach isn’t always convenient. Google’s Wearable Subtitles solve a number of those problems. For instance, the wearer isn’t required to maintain eye contact with the speaker or constantly check the smartphone to respond on time.
The researchers say that these smart glasses, which use a low-power microcontroller architecture, can function for 15 hours a day before needing to be recharged. This prototype contains a very thin PCB with MediaTek MT2523D System-in-Package (SiP), which is a single chip Cortex M4F with Bluetooth 4.0 EDR and Bluetooth Low Energy (BLE) transceiver. It also consists of power management, MIPI-DSI display controller and memory subsystem. All of this is packaged onto the right temple of the 3D-printed frame, where it’s neatly hidden out of sight from others. The system communicates with the smartphone via BLE and utilizes the phone’s microphone and a communication protocol to write out the words and sounds.
The 8-layer PCB assembly contains wireless communication and display. (Image Credit: Google Research)
The team implemented the application layer using Protocol Buffers with message types for drawing graphics primitives, configuring sensors and peripherals, and changing device modes. There is another set of message types responsible for transmitting sensor data, compressed audio captured from the system’s microphones, and remote procedure call (RPC) send values back to the phone.
Additionally, the team implemented interpolative primitives run on the device to drive position and scale parameters of other primitives for the smooth-scrolling transcript text. Using this method, they were able to reduce the bandwidth needed to push these effects since the keyframes (initial position, animation target, and interpolation strategy) have been specified.
The Wearable Subtitles prototype uses a 1-lane MIPI-DSI compatible microdisplay engine with 30 FPS graphics rendering. For this proof-of-concept, the transcript is sent to a monocular display found on the right lens. The text is displayed in white to maximize contrast and visibility since the optical see-through display adds light.
Google also surveyed people who are deaf and hard of hearing to enhance text visibility, speed of transcription, connectivity, physical button ergonomics, and text placement. The volunteers compared smartphone speech recognition with this new technique. In every situation, the wearer’s comfort and awareness of ambient sounds were the same or improved with the glasses.
Have a story tip? Message me at: http://twitter.com/Cabe_Atwell