headerSearch form

Changing the World through Creative Research

Silent Speech Recognition with Strain Sensors and Deep-Learning Analysis of Directional Facial Muscle Movement

Journal
ACS Applied Materials & Interfaces
Date
2022.11.23
Abstract

Silent communication based on bio-signals from facial muscle requires accurate detection of its directional movement and thus optimally positioning minimum numbers of sensors for higher accuracy of speech recognition with a minimal person-to-person variation. So far previous approaches based on electromyogram or pressure sensors are ineffective in detecting directional movement of facial muscle. Therefore, in this study, high-performance strain sensors were used for separately detecting x- and y-axis strain. Directional strain distribution data of facial muscle was obtained by applying three-dimensional digital image correlation. Deep-learning analysis was utilized for identifying optimal positions of directional strain sensors. The recognition system with four directional strain sensors conformably attached to the face showed silent vowel recognition with 85.24% accuracy and even 76.95% for completely new subjects. These results show that detection of the directional strain distribution at the optimal facial points will be the key enabling technology for highly accurate silent speech recognition. 

Reference
ACS Appl. Mater. Interfaces 2022
DOI
http://dx.doi.org/10.1021/acsami.2c14918