The frontier of science written in an ancient Chinese character
If a person living in the future watched today's scientific video records, he would probably be amused by the hat in front of him.
It looks like a swimming cap, covered with metal pieces the size of a fingernail, with electrodes inlaid on it to collect EEG signals.
It takes a while to put on such an EEG collection cap. In order to make those metal products better understand the human brain through the bones, skin, and body fluids, the researchers have to find a way to narrow their distance-fill in the cream The shape of electrolyte gel makes the contact between the scalp and the metal electrode closer.
It sounds uncomfortable to wear this hat.
But right now, the magic it brings can definitely hedge against discomfort.
It shoulders the mission of spying on the secrets of the human brain. Since the end of the last century, people have transformed the captured human brain signals into instructions and manipulated external devices to achieve "mind control".
Before the Spring Festival of 2021, two post-95 college students from the neuroengineering team of Tianjin University wore EEG caps and imagined a Chinese character together. Their EEG signals were captured and converted into instructions. The mechanical arm holding a brush was placed on red paper. Write the word "Fu".
This is the first time in the world that a multi-person collaborative way of brain-controlled robotic arms is used to "write" Chinese characters with ideas.
A new world record is born
Luo Ruixin, a 23-year-old female graduate student, is one of the writers of the word "Fu".
For "computer friends" to understand themselves better, just like humans, they have to spend time getting along.
The way of making friends is direct and silent.
Luo Ruixin wears a hat with 64 electrodes, sits in front of the computer screen with the cursor blinking, and keeps her eyes on 216 points one by one for at least 5 seconds. The computer instantly records the different brain waves generated when she looks at different cursors. .
When she wants to try to write a horizontal line, just "draw" a horizontal line on the screen with her eyes, and the "computer friend" can analyze the changes in brain waves from the trajectory of her eyes at different points. , And then understand her intentions.
She also needs to take care of the reaction speed of her "computer friend".
The "write" of this stroke was too fast for the other party to react.
She has to find out the rhythm of her "friends", and at the same time make her focus more when she imagines the strokes. After about 15 minutes, the accuracy of information exchange between the two parties can reach more than 90%.
Han Jin, a 27-year-old biomedical engineering doctoral student, is the builder of this two-person brain-controlled robotic arm writing system. He explained that the prerequisite for achieving brain control is to allow the machine to understand human thinking and intentions.
Machine learning and mathematical modeling methods can establish a mapping model between different brain waves and different thinking tasks, and identify different thinking ideas, that is, "appointing" a communication method that both a person and a computer understand.
Each person's EEG characteristics are unique.
"The same person, at different times and in different states, produces different brain waves when completing the same task." Han Jin said that in order to ensure the accuracy of the task, human-computer training is required before each task. Get acquainted with each other."
Generally speaking, the subject needs to make a series of different actions first, let the computer read the data from the electrodes, and roughly figure out which action intention will trigger what kind of brain waves.
Next time, when the same brain waves appear, the computer will understand what to do.
In 2018, Han Jin tried to type letters with his mind for the first time in the laboratory. “I think here, one letter after another will pop up on the screen over there, it feels very sci-fi.” One year later, "Nezha" brain-controlled The smart arm robot system was born in that laboratory.
Writing Chinese characters is much harder than writing letters.
Luo Ruixin and his teammates disassembled the word "Fu" into multiple parts according to the strokes, and the two divided them equally. She was responsible for controlling the radical on the left and the spoken word on the right, and the other was responsible for the horizontal on the upper right of the word fortune and the Tian word below.
The two "writers" stared at each stroke on the screen. From the start to the end, the light beam hits the retina. The octopus-like neurons in the brain are stimulated to induce corresponding brain waves.
The brain is like the core of a Russian doll, wrapped in layers of tissues, and the skull has a poor electrical conductivity. The "writer" must be focused enough to minimize body movement in order to generate a "penetrating" EEG signal that penetrates the meninges. , Cerebrospinal fluid, skull...the electrodes that reach the scalp.
A computer "talks" to multiple people at the same time, which saves time for "talking" one by one.
Compared with single-person brain control, the problem to be solved by two-person cooperative control of a robotic arm is more complicated. It is necessary to design a reasonable task allocation strategy so that the two can complete the control task with the highest efficiency and cannot conflict with each other.
The idea commands issued by Luo Ruixin and teammates at the same time were received by the computer at the same time, and then instructed the robotic arm to complete the writing in the order of priority to efficiency.
The entire writing process takes 3 minutes, which is twice the speed of a single person.
"In human activities, there are a lot of complicated and delicate work that cannot be done by a single person. It requires multiple people to work together." Han Jin said that this time it is two people who control a single robotic arm. In the future, multiple people and multiple robotic arms can work together. Complete some more onerous or more refined tasks.
For example, astronauts complete mechanical repairs in outer space with brain-controlled robotic arms.
For a long time, a key issue facing brain control technology is how to accomplish more tasks while maintaining a higher speed.
There is a natural contradiction between "fast" and "good". Xu Minpeng, an associate professor at the Institute of Medical Engineering and Translational Medicine of Tianjin University, said that in previous studies, the largest BCI (brain-computer interface) command that can be completed is 84, but The recognition time of a single instruction exceeds 8 seconds; if the recognition time is reduced to 0.8 seconds, only 40 instructions can be completed.
The Tianda team integrated the two common brainwave paradigms of "Mind Typing"-P300 and SSVEP. Both have advantages in stability and speed. One can finely distinguish weak EEG characteristics, and the other can quickly decipher and decode.
This time, they set a world record of 216 instructions for scalp EEG BCI online control, and realized a major change in the way of BCI information output from "spelling" to "writing" and from single to multiple users in the world.
The old world record that was broken was 108 instructions, which was also set 2 years ago.
Peep into the small window of the "three-pound universe"
At this time, two to three million years have passed since the ancestors of mankind invented stone tools.
A smart brain helps humans unlock a wider world, and the brain itself has always been a secret of scientific exploration.
Scientists use the vast universe to describe the complexity of the brain, which is why it is called the "three-pound universe" (3 pounds is approximately 1.36 kg, which is close to the weight of an adult brain).
Ming Dong, director of the Tianjin Brain Science and Brain-like Research Center and executive deputy dean of the Tianjin University Research Institute, said that brain science is the "last frontier" for humans to understand nature.
The brain, weighing only 3 pounds, is the most complex object known in the universe. Neuroengineers describe it as "the most dense, structured, and most self-organized thing currently known."
It has a texture similar to jelly, and it runs at only 20 watts. It takes 24 million watts to start an equally powerful computer.
In a sense, what you see in the mirror is nothing but a carrier for the 3-pound "jelly."
In the American film "The Matrix", as long as a tube is inserted behind the head, people can escape from the body and reach the virtual world of the brain; the protagonist of "Ghost in the Shell" realizes a full-body mechanized prosthetic body, and only a human is retained brain.
Regardless of the form, the brain-computer interface (BCI) has become a window through which humans can peer into the "three-pound universe".
This idea comes from the principle of brain operation.
Xu Minpeng introduced that the brain is composed of about 100 billion neurons, which is roughly the same as the number of stars in the Milky Way.
They form an unimaginable huge neural network, which is the material basis for human thinking activities.
When we think, brain neurons fire synchronously. These signals are called brain waves.
The brain waves contain information about the consciousness of human thinking activities. When different thoughts emerge in the brain, the brain electrical signals will change accordingly.
Theoretically, as long as people can understand the direct relationship between brain waves and thinking activities, they can reverse their thoughts by identifying brain waves.
The brain-computer interface is a technology that interprets human thoughts by studying brain waves.
Xu Minpeng explained that the ideas in the human brain are currently expressed through the peripheral nervous system, muscle tissue, or language, while the brain-computer interface bypasses these traditional paths and directly establishes a connection between the brain and the computer.
"To put it more bluntly, what we think in our minds can be directly read by the computer through the brain-computer interface."
Break the boundaries between man and machine?
Two people concentrate all their attention and spend 3 minutes writing a Chinese character, which is obviously not satisfactory.
In the 1960s, NASA scientists first proposed the term "cyborg" (Cyborg), which is the unity of humans and electronic machinery.
The half-human, half-mechanical superman in Marvel movies and the mechanical prosthetic in "Robocop" are all manifesting a desire-that humans want to break the clear boundaries with machines.
The machine can compensate for the physical defects of the disabled and become a part of their body.
In 2014, a high-positioned paraplegic teenager controlled a mechanical exoskeleton to score the first kick for the Brazil World Cup.
That year, the neuroengineering team of Tianjin University independently developed the "ShenGong Series" artificial rehabilitation robot system, which was put into clinical use in many hospitals to help stroke hemiplegic patients recover their motor functions through brain control technology.
The body of a stroke patient does not listen to it, but the original movement "flow chart" stored in the central nervous system in the body may be preserved intact or in part.
In traditional rehabilitation treatment, the participation of the brain is low, and it is difficult to achieve functional reconstruction of damaged brain areas and surrounding brain areas, which is one of the important factors affecting the rehabilitation effect.
There is a therapy that allows the patient to repeatedly imagine the movement of the limbs in the brain, constantly awakening the related movement memory, so as to activate the relevant areas of the brain and improve the motor function.
But for a long time, this method lacks objective evaluation indicators, cannot monitor the brain activity status in real time, and it is difficult to provide timely effect feedback.
In order to improve this therapy, the "Shen Gong" brain-computer interface of the Tianda team detects the patient's movement intention when performing motor imagination, triggers electrical stimulation that simulates nerve impulse, promotes active muscle contraction, and drives the bones and joints to move autonomously.
This is equivalent to building a complete artificial nerve pathway outside the patient's body, forming a "brain-machine-human body" information loop.
At Tianjin People's Hospital, an 85-year-old man received treatment using brain-computer interface technology.
He was paralyzed on his right hand and foot due to a stroke.
Researchers who assisted in rehabilitation said that the new treatment has a significant impact on the legs of the elderly.
"He could only stand for a minute or two before, but now he can stand for five or six minutes, walking about 50 meters."
The machine deciphers and transmits the instructions of the brain, and then gives the instructions to the body for execution.
The principle seems simple, but it is extremely difficult to implement.
According to the assumption, the third generation of "Shen Gong" will be transformed into a smaller wearable portable device, trying to assist patients to complete more complex and delicate body movements, while monitoring the state of the brain, and adjusting the rehabilitation training mode in time.
In 2016, the Tianda team and the China Astronaut Center developed the world's first orbital brain-computer interface system for space stations, and successfully completed the first space brain-computer interaction experiment in human history.
Mingdong believes that this is the most ideal way of human-computer interaction in the space environment.
Brain control technology allows astronauts to issue operating instructions without speaking and movement, and can also monitor their mental load and other neurological function status.
This year, the team took the lead in realizing continuous real-time brain control of drones with "4 degrees of freedom" and 12 instructions.
Brain-controlled smart wheelchairs, brain-controlled typewriters, and brain-controlled smart prostheses are all developing.
Han Jin guessed that the brain-computer interface in the future will be so small that it can be buried under the hair, making it invisible to others.
It can connect the brain to a mobile phone to make calls; it can also connect to smart home appliances at home, turn on lights, draw curtains, and enter security door codes.
No matter how big the brain hole is, perhaps in the future it will be possible to realize the "brain-to-brain" spiritual communication.
All kinds of worries also followed. The most terrifying thing that sounds like, "What if our consciousness is kidnapped by machines or manipulated by humans".
Xu Minpeng said that the prerequisite for human-computer interaction is that both parties abide by the mutual agreement. If humans are unwilling to be read from the brain, then the connection cannot be made. "We have only taken a small step."
Just as fire brings civilization, it also brings danger.
Mingdong believes: "Human-machine technology has been evolving, and in the future, humans will definitely be effectively connected with machines." He particularly likes a sentence. The moment humans lift their hoe, the hoe has become an extended part of the human body.
China Youth Daily·China Youth Daily reporter Hu Chunyan