Assistive Magnetic Skin System for Speech Reconstruction: An Empowering Technology for Aphonic Individuals

Individuals with a voice disorder or speech sound disorder (SSD) (i.e., aphonic or mute people) can have any combination of difficulties with perception, articulation/motor production, and phonotactics, which may impact their speech intelligibility and acceptability, thus finding challenging to communicate with the public. As a result, many individuals suffer from frustration, isolation, and depression. Natural verbal communication for SSD patients is now more feasible than ever thanks to advancements in wearable artificial skins and machine learning. This article presents an assistive magnetic skin system for speech reconstruction (AM2S-SR), which enables SSD-afflicted people to communicate with their mouths. Using magnetic field sensors integrated into Magnetphones, the system reads the movement of the mouth by tracking the movement of magnetic skin patches attached next to the bottom lip. The measured magnetic field signals data are then processed using a Fine K-Nearest Neighbor machine learning classifier. The classified data are then exported verbally on speakers, or visually on a display. AM2S-SR successfully identifies the full English alphabets with average success rate of 94.96%, thus enabling SSD people to talk using the mouth and have a more natural conversation with others. Assistive magnetic skin system for speech reconstruction enables natural communication for speech impaired people. The system utilizes a biocompatible, breathable, and highly elastic magnetic skin placed next to the bottom lip of the individual; the movements are tracked through a noninvasive, wearable device with embedded magnetic field sensors. The process of translation is achieved by a K-Nearest Neighbor classifier model.image (c) 2023 WILEY-VCH GmbH

Automated summary

Individuals with voice or speech disorders face difficulties in communicating with the public, leading to frustration, isolation, and depression. However, advancements in wearable artificial skins and machine learning have led to the development of an assistive magnetic skin system (AM2S-SR), allowing speech impaired individuals to communicate naturally using their mouths.