TypeAnywhere allows a user to type on any surface. (Image Credit: University of Washington)
HMIs are the most important feature, in my opinion. How else do we interface with the world and devices?
Researchers at the University of Washington developed TypeAnywhere, a QWERTY-based text entry device for everyday computing. TypeAnywhere features wearable sensors on two hands for finger-tap action detection purposes, a decoder to convert tap sequences into text, and a text editor interface. Finger-tap sequences are fed into a neural decoder altered from the BERT model that shows the text on the interface. In addition, the team developed a text corrector for the device, which doesn’t require cursor navigation.
It also uses two Tap Straps that contain five rings for the user to wear on each finger. These rings have accelerometer integration for tap-action detection, which has over 98% accuracy. TapStrap helps to ensure that a one-handed text entry with chording can be achieved.
More importantly, the text entry device does not require position information to decode the user’s typed letters. Instead, it relies on the finger’s index performing the tap along with the text’s context. Plus, the user can tap anywhere with the correct finger since the device uses the tapping finger’s index and doesn’t rely on the position information. This functionality provides a simpler design, which provides higher text decoding accuracy and generalizes the decoder for various finger-to-key mappings. A Python server handles the HTTP communication between the web interface, hardware, and decoder.
Compared to similar QWERTY-based text devices, the researchers trained the device without data collection from user studies. They curated training data by converting letters in the phrase set to its finger ID. This made it possible to train the model on an extensive text corpus with over 3.6 million samples and adapt finger-to-key mappings via finger ID training set modifications. The team also ran neural decoder computational evaluations, “achieving a 1.6% character error rate (CER) on the Cornell Movie-Dialogue Corpus.” Other gram-language models achieved a 5.3% CER.
The team recruited five participants to help determine the effectiveness of TypeAnywhere, which took five days to evaluate. Each participant used TypeAnywhere on a flat table surface for thirty minutes per day, achieving 70.6 WPM and 1.50% CER. The top performer managed to reach 91.4 WPM and 3.15% CER after using TypeAnywhere for five days. Participants were then asked to use the device on their laps, achieving 43.9 WPM and 1.37% CER. As a result, users can easily and quickly learn TypeAnywhere with very little practice and achieve high performance after practicing for 2.5 hours. The team says the device could have applications for computer environments.
The researchers want to implement special character support since TypeAnywhere supports spaces and letters. Punctuation, numbers, symbols, and modifier keys could be added to support everyday applications. They propose assigning various symbols to different chords, and “users would perform a mode gesture to switch between typing and symbol-entry modes.”
https://www.youtube.com/watch?v=WDIp7moK0wo
Have a story tip? Message me at: http://twitter.com/Cabe_Atwell