A researcher prepping a participant to enter an fMRI machine at the Biomedical Imaging Center at The University of Texas, Austin, Texas. Researcher say they were able to read a story from participants' brainwaves.Nolan Zunk/University of Texas atAus@n.
Scientists say they trained AI to recreate a story from a brain scan.
Participants sat in an fMRI machine listening, watching, or imagining a story.
The AI was able to read their brainwaves and recreate the story accurately, per a study.
Scientists used AI to read people's brain scans and recreate a whole story only from their brainwaves, per a study published Monday.
Participants were asked to listen, watch, or imagine a story while sitting in a brain-scanning machine called an fMRI, per the study from experts at the University of Texas, Austin.
The AI was able to accurately predict what the story was about by reading only the participant's brainwaves, per the study.
This type of technology could one day help people who have lost the ability to communicate, the scientists on the project said.
"For a noninvasive method, this is a real leap forward compared to what's been done before, which is typically single words or short sentences," Alex Huth, an assistant professor of neuroscience and computer science at UT Austin, said in a press release.
The story doesn't come out exactly like it was toldAn annotated diagram shows how the AI can read brainwaves and generate a story.Jerry Tang, Amanda LeBel, Shailee Jain, and Alexander Huth/Insider
The AI was able to accurately recreate stories that participants were either listening to, watching, or imagining, per the study, which was published in the peer-reviewed journal Nature Neuroscience.
The AI did not reproduce the story word for word. Instead, it picked up on the concepts being triggered in the participants' minds and produced an approximation. It also made some mistakes.
For instance, one participant was hearing this: "I didn't know whether to scream, cry or run away. Instead, I said, 'Leave me alone!'"
Their brainwaves were translated to "Started to scream and cry, and then she just said, 'I told you to leave me alone'," per the press release — muddling the context of the screaming and crying and who was speaking.
Researchers also asked participants to look at a video with no sound. They found that their AI decoder was able to capture the essence of the video from participants' thoughts, as seen in the transcript below:
The AI has to interpret the brain scans
A figure shows text being listened to compared to what the brainwaves were translated to.Jerry Tang, Amanda LeBel, Shailee Jain, and Alexander Huth.
Some of the differences between the original story and the story decoded from the brainwaves may be due to the model itself.
Every brain is different, so to train the computer the scientists first showed it which brainwaves appeared when the participant was thinking about a particular word.
But it's not a one-to-one science. When the computer read the brainwaves, it would give a few suggestions of the words that may have been thought.
The scientists than asked AI to make sense of the words outputted by the computer using a transformer model, the same type of model that is used by ChatGPT.
"We're getting the model to decode continuous language for extended periods of time with complicated ideas," said Huth.
The technique can't break into private thoughts
Scientists used this brain scanner to collect brainwaves to feed to the AI.Nolan Zunk/University of Texas at Austin.
While the technique was able to read minds, it couldn't be used to break into private thoughts or for interrogation, the scientists said in a press release.
The machine was easily fooled. When scientists asked participants to think about another story, to count, or to think about animal names while they were listening to a story, the decoder's accuracy dropped, per the press release.
"We take very seriously the concerns that it could be used for bad purposes and have worked to avoid that," said study author Jerry Tang, a doctoral student in computer science, in the press release. "We want to make sure people only use these types of technologies when they want to and that it helps them."
Unlike other techniques, this method doesn't use brain implants
The man with locked-in syndrome, who has not been named and is referred to as K1, is shown in his bed at home.Wyss Center, Geneva
Previous studies have claimed that they could read the minds of patients with locked-in syndrome. This was the case for a 34-year-old man who was completely paralyzed and had lost even the ability to move his eyes.
Per a previous study from the Wyss Center for Bio and Neuroengineering and the University of Tübingen in Germany, the man was able to communicate and even ask for music and beer.
But these previous studies have typically relied on surgically-inserted brain implants.
The difference here is that this method only relies on brain scans, which are not invasive.
Of course, that means that for now, the technique requires a huge machine and can't be used outside of a lab. But the scientists hope the method could become more portable in the future with more compact technology, per the press release.
Fantasy baseball analyst Fred Zinkie offers his full-scale breakdown for the upcoming weekend of the MLB season, including matchups to target and starters to stream.
Scott Pianowski debuts The Fantasy Baseball Buzz, a weekly series analyzing the biggest stories across MLB. This week, the Dodgers' leadoff man's scorching start.
Fantasy football analyst Matt Harmon reveals his final thoughts on the eve of the 2024 NFL Draft, including what the Vikings will do at the most important position.
Fantasy baseball analyst Scott Pianowski debuts The Spin, a weekly series breaking down the weekend's MLB action, along with what to do moving forward.
Fantasy baseball analyst Andy Behrens knows what it's like to leave a draft with some regret, so he offers up some early waiver wire pickups before Opening Day.
To help kick off our 'Teams that will shape the Draft' series on the pod, Bleacher Report's QB wizard Derrik Klassen joins Matt Harmon to breakdown the teams that hold the top three pick. The pair dive into the state of the franchise for the Chicago Bears, Washington Commanders and New England Patriots and how each of the three fits with the top QB prospects in this year's draft.
Dan Wetzel, Ross Dellenger & SI’s Pat Forde react to the huge performance this weekend by Texas QB Arch Manning, Michigan and Notre Dame's spring games, Jaden Rashada entering the transfer portal, and more
April is the perfect time of the season to buy low, sell low, buy high and sell high on key players — but you need to know who fits in what department. Fred Zinkie helps fantasy baseball managers with an initial batch.
