A Brain-Computer Interface (BCI) marries the brain to Artificial Intelligence (AI), using signals recorded from the brain to enable communication or to control a neuroprosthesis. This technology is now being widely used, however, there is vast room for improvement with key biological and engineering problems remaining to be resolved. These hurdles include low-quality recordings by home users, low translation speed, rudimentary accuracy of translation and adapting applications to the needs of the user. Now, a study from researchers at Columbia University develops a system capable of translating thought into intelligible, recognizable speech. The team states this breakthrough harnesses the power of speech synthesizers and artificial intelligence and could lead to new ways for computers to communicate directly with the brain. The opensource study is published in the journal Scientific Reports.
Previous studies show when people speak or imagine speaking, distinguishable patterns of activity appear in their brains. A distinct pattern of signals also emerges when listening to someone speak, or when a person imagines listening. Reconstructing speech from the human auditory cortex creates the possibility of a speech-based neuroprosthetic with the ability to establish direct communication with the brain. However, the low quality of reconstructed speech has severely limited the utility of this method for BCI applications. The current study combines recent advances in deep-learning with the latest innovations in speech synthesis to reconstruct closed-set intelligible speech from the human auditory cortex.
The current study utilizes a vocoder, a computer algorithm used by Amazon Echo and Apple Siri, to synthesize speech after being trained on recordings of people talking. Epilepsy patients, already undergoing brain surgery, were asked to listen to sentences and numbers spoken by different people, while their patterns of brain activity were recorded via invasive electrocorticography to train the vocoder. Results show the sound produced by the vocoder in response to the patient’s brain signals was analyzed and cleaned up by virtual neural networks, AI biomimicking the structure of neurons in the biological brain. Data findings show the output from this BCI is a robotic-sounding voice reciting an accurate sequence of numbers.
To test the accuracy of the recording, the group asked the participants to listen to the recording and report what they had heard. Results show the patients could understand and repeat the sounds approximately 75% of the time. The lab states the sensitive vocoder and virtual neural networks represented the sounds the patients had originally listened to with surprising accuracy. They go on to add by monitoring someone’s brain activity, their technology can reconstruct the words a person hears with unprecedented clarity.
The team surmises they have developed a BCI able to translate brain signals directly into speech. For the future, the researchers state they now plan to test more complicated words and sentences in the hope their system could one day be used as part of an implant translating the wearer’s thoughts directly into words.
Source: Columbia Engineering
Get Healthinnovations delivered to your inbox:
Michelle Petersen is the founder of Healthinnovations, having worked in the health and science industry for over 21 years, which includes tenure within the NHS and Oxford University. Healthinnovations is a publication that has reported on, influenced, and researched current and future innovations in health for the past decade.
Michelle has been picked up as an expert writer for Informa publisher’s Clinical Trials community, as well as being listed as a blog source by the world’s leading medical journals, including the acclaimed Nature-Springer journal series.
Healthinnovations is currently indexed by the trusted Altmetric and PlumX metrics systems, respectively, as a blog source for published research globally. Healthinnovations is also featured in the world-renowned BioPortfolio, BioPortfolio.com, the life science, pharmaceutical and healthcare portal.
Most recently the Texas A&M University covered The Top 10 Healthinnovations series on their site with distinguished Professor Stephen Maren calling the inclusion of himself and his team on the list a reflection of “the hard work and dedication of my students and trainees”.
Michelle Petersen’s copy was used in the highly successful marketing campaign for the mega-hit film ‘Jumanji: The Next Level, starring Jack Black, Karen Gilian, Kevin Hart and Dwayne ‘The Rock’ Johnson. Michelle Petersen’s copywriting was part of the film’s coverage by the Republic TV network. Republic TV is the most-watched English language TV channel in India since its inception in 2017.
An avid campaigner in the fight against child sex abuse and trafficking, Michelle is a passionate humanist striving for a better quality of life for all humans by helping to provide traction for new technologies and techniques within healthcare.