In a groundbreaking development, experts have successfully enabled a paralyzed woman to communicate once again using a technology commonly used in video games. The woman had been left paralyzed after suffering a stroke.
Researchers from UC San Francisco (UCSF) and UC Berkeley have created the world’s first brain-computer interface capable of generating electronic speech and facial expressions from brain signals. This innovation has the potential to revolutionize communication for individuals who have lost their natural means of speaking.
The technology uses the same software found in video games to translate brain waves into a talking digital avatar. Researchers decoded brain signals into three forms of communication: text, synthetic voice, and facial animation on a digital avatar. This included lip sync and emotional expressions.
Facial animation generated from brain waves was a significant achievement in this study. The woman’s brain signals, which would normally control muscles in her tongue, jaw, voice box, and face, were intercepted and connected to computers powered by artificial intelligence (AI). These computers were able to analyze and interpret brain activity, allowing the woman to communicate by controlling a digital avatar.
The woman was eventually able to speak and write based on recordings from before she became paralyzed, such as those from her wedding.
According to Michael Berger, CTO and co-founder of Speech Graphics company, the creation of a real-time digital avatar connected directly to the brain demonstrates the potential for AI-driven technology beyond video games. Berger emphasizes that human communication involves a combination of audio and visual cues, and restoring both voice and facial communication is essential for a natural sense of expression.
The research, published in the journal Nature, revealed that the woman could even make the avatar display specific emotions and move individual facial muscles. This achievement not only restores voice but also the crucial aspect of facial communication, providing a sense of embodiment and control to someone who has lost it due to paralysis.
While the technology is currently in the scientific proof-of-concept stage and not yet ready for everyday use, it marks a significant step forward in restoring rapid communication for individuals with paralysis. Lead author Frank Willett noted that this development has the potential to transform the lives of those who cannot speak due to paralysis.
