display
Virtual assistants have been able to decode our language with incredible precision for some time.
We tell Alexa or Siri which song we want to hear and, whoosh, the song is played.
However, developments in the field of artificial intelligence go much further.
Researchers at the University of California at San Francisco (UCSF) have been working for some time to convert thoughts into spoken language with the help of artificial intelligence.
And they are now one big step closer to their goal.
In an experiment, neurosurgeon Edward Chang from the UCSF's Chang Lab and his team succeeded in converting brain activity into language - without even speaking a word.
They used electrical impulses.
Source: Getty Images / Andriy Onufriyenko
display
The research team worked with four epilepsy patients for this purpose.
They were implanted with electrodes in the cerebral cortex.
Participants were then asked to say 30 to 50 simple sentences aloud several times.
The team tracked their neural activity as they spoke.
In this way, the researchers were able to measure brain activity and create an electrocorticogram, a so-called brain wave image.
This data was then fed into a machine learning algorithm, a type of artificial intelligence system that analyzed brain activity data for each sentence spoken.
The result was numbers that corresponded to certain language signatures such as vowels, consonants or mouth movements.
Then a second AI decoded these representations and converted the data into words.
At first the system only generated nonsensical sentences.
But then the AI compared each sequence of words with the sentences actually read out, improved and learned how the sequence of numbers related to words and which words typically follow one another.
display
The team tested the AI and only generated a written text from the brain activity while speaking - et voila:
Ideally, the system was correct 97 percent of the time, so it only had a word error rate of three percent.
In their investigation, the researchers naturally also provided examples in which the AI was wrong.
So she made small mistakes like in the sentence: "Part of the cake was eaten by the dog".
This became: “Part of the cake was the biscuit”.
But there were also major mistakes, such as the sentence: “She was wearing a warm, fluffy woolen suit”.
The AI translated this as follows: "The oasis was a mirage".
Source: Getty Images / Francesco Carta fotografo
Nevertheless, the researchers rate the results as a great success.
So far, the word error rate in AIs has been 60 percent, they write in their study, which was published in March 2020 in the journal "Nature Neuroscience".
Typical speech recognition programs would have an error rate of five percent.
In the case of the current experiment, the AI provided significantly better and more accurate results.
However, it must be emphasized that the current study only comprised around 250 words.
display
For the time being, only a few previously defined words can be recognized.
Nevertheless, the approach pursued by the US researchers could one day help people who are no longer able to articulate themselves.
According to the researchers, the AI is able to code large amounts of words and sentences quickly and accurately.
We're not finished yet, but we believe this could be the basis for a speech prosthesis.
Dr.
Joseph Makin, co-author of the study, University of California at San Francisco
And don't worry, no one will be able to read your mind just like that.
Of course, the whole thing only works if the test subject has consented to the implant of the brain electrode.
This is where you will find third-party content
In order to interact with or display content from third parties, we need your consent.
Activate external content
I consent to content from third parties being displayed to me.
This allows personal data to be transmitted to third party providers.
This may require the storage of cookies on your device.
More information can be found here.
This article was first published in April 2020.