"Hello, Academician Jiang, what impressed you the most when you were studying at East China Normal University?"
"When I was studying at East China Normal University, I was most impressed by the late autumn of 1990, when Mr. Pan Daojun encouraged graduate students to attend academic conferences on their own. I attended the <>rd National Conference on Quantum Chemistry with Dai and met many top people in my professional field, and this experience ignited my new ambition. ”
In January 2024, in the hall of the School of Software Engineering on the second floor of the Science Building on the Putuo Campus of East China Normal University, a one-person-high electronic screen appeared with the image of Jiang Hualiang, the late academician of the Chinese Academy of Sciences, former director of the Shanghai Institute of Materia Medica, Chinese Academy of Sciences, and alumnus of East China Normal University. The initiator of the project is Qian Xuhong, president of East China Normal University and academician of the Chinese Academy of Engineering, and the leader is Jiang Hualiang's student and dean of the School of Pharmacy of East China Normal University. Homo sapiens can simulate Academician Jiang Hualiang's tone, Southern Mandarin accent and speech form, and have a natural dialogue with the questioner.
"In the future, we may build the world's first digital intelligent education platform - the Digital Intelligent Human University, which adopts the cross-span intersection of arts, science and engineering and the deep integration of virtual and reality, establishes the key technical standards of digital intelligent people, creates standardized applications and databases for digital intelligent people, and continuously improves the 'digital intelligence' evaluation and governance system. Qian Xuhong introduced that in the future, various digital intelligent people may appear in all corners of East China Normal University to interact with teachers and students.
"Reproduction" masters are far more difficult than you think
"Although my father is gone, it is also a kind of companionship for us to be able to 'reappear' in this form. Jiang Yuxi, the daughter of Academician Jiang Hualiang, said that she participated in the process of "reproducing the master" with new technology.
Starting from April 2023, Jiang Yuxi will communicate with Wang Jiangtao's team at least once a month, "The project team will update and iterate the digital Homo sapiens model every month based on the opinions of me, my mother and Li Honglin, and then let us give our opinions."
Jiang Yuxi's comments include "not quite consistent with his father's tone", "his father's southern Mandarin accent problem", "his expression is a little too stiff", "the frequency of blinking is too low", etc.
In order to create a more real "Jiang Hualiang", Wang Jiangtao's team has thought of many ways. For example, they collected about 450 pieces of data from Academician Jiang's relatives and friends in the early stage, and each piece of data had to be manually checked and some useless data was deleted. After the data is determined, they have to use a large model algorithm to let the digital intelligent person learn the data content - however, because the total amount of data is not large, the digital intelligent human training 2,5 times and <>,<> times, showing little difference.
"The data we have is very limited, which is one of the difficulties of the 'reproduction masters'. Wang Jiangtao introduced that the idea of the team's "reproduction master" was first proposed by the principal Qian Xuhong. As early as three years ago, in the interdisciplinary project of "Frontiers of International Research in Chinese Philosophy and Social Sciences", East China Normal University put forward the concept of digital intelligent people for the first time, and explored it in combination with the foundations of education, management, psychology, big data, computer science and other disciplines. Two years ago, Wang Jiangtao's team tried to "reproduce" Yuan Yunkai, the old president of East China Normal University, who passed away in 3.
At that time, it was the digital technology commonly used in live broadcasts today - a real person was required to speak and act in the background, and then the voice and image were synthesized by the computer - and it was not a digital intelligent person who really used large model algorithms. This kind of technology, on the one hand, has high requirements for the performance technology of "actors", which can simulate the words and deeds of the old principal, and on the other hand, it also has high requirements for the production of digital images at the front desk, and it is difficult to be "vivid", and the digital image can speak, but it cannot be able to talk to people.
Two years later, in the experiment of Academician Jiang Hualiang's digital sapiens, Wang Jiangtao's team wanted to achieve a higher goal - not only no longer needed backstage "actors", but also required the expressions of digital sapiens to be vivid and natural, and to be able to achieve "dialogue" with teachers, students, relatives and friends.
Using algorithms to "teach" digital sapiens to "speak"
In the early stage of research and development, there will always be such and such problems. The launch of a digital intelligent person, including pronunciation-driven and image-driven, even if it is just a subtle facial expression, is calculated by the background algorithm from moment to moment.
"In the beginning, when the digital sapiens spoke, their bodies were always shaking, and they didn't blink. Xu Gang, an engineer at the School of Software Engineering of East China Normal University, is responsible for the image driving part of Academician Jiang's digital intelligence, and he told reporters that because Academician Jiang's video materials are mostly public lectures, speeches, speeches, etc., and the expression is relatively formal, so when the machine learns its form, it often can't capture the action of "blinking". Later, on the one hand, the team members contacted the family members to collect videos of the academician's daily life, and on the other hand, modified the background algorithm to solve the problem of unnatural eyes and body when the digital intelligent person spoke.
Academician Jiang Hualiang is a native of Changzhou, Jiangsu, and he speaks with a heavy southern accent, which also requires special calculations and simulations in the background. According to the team members, the initial version was a bit blunt, and Academician Jiang's family and students were not satisfied. It's instantly recognizable as a "digital human" – neither realistic nor intelligent.
After the tone, expression, and demeanor are handled, the next difficulty is to let the digital intelligent people "say things accurately". To this end, the R&D team collected a large number of written materials from Academician Jiang's lifetime, covering the professional field of drugs he focused on, stories about his students, etc., as well as personal life and hobbies. Among them, in terms of personal life, East China Normal University held a memorial service for Academician Jiang, where his former friends, colleagues, students, and family members recalled his life and interesting stories one by one. "The more data is collected, the more intelligent people will be able to answer questions. Wang Jiangtao said.
The problem that arises is that after studying a large amount of literature, digital sapiens often "keep talking as soon as they answer". This is not consistent with Academician Jiang Hualiang's speech before his death, which was "concise and accurate".
For this reason, "setting boundaries" for the answer of digital sapiens has been put on the agenda by the research team.
Yao Junjie, an associate researcher at East China Normal University's School of Software Engineering, is responsible for the underlying data optimization of large models, and his work with his colleagues is to make digital intelligent people have cognitive abilities. Digital Homo sapiens should not only have a good grasp of Academician Jiang's life and deeds, but also have a good understanding of relevant knowledge in the fields of pharmacy and biology.
"In the beginning, we would use a lot of emotional words, ask a question, and he would pour out all the relevant content in one go. Yao Junjie said that Academician Jiang Hualiang's innovation of digital intelligent people is, to a certain extent, also a rehearsal of digital intelligent people for future teachers of East China Normal University, "no matter which field other teachers are replaced in the future, they will face the problem of setting boundaries and letting digital intelligent people speak accurately."
How big is the imagination and application space of digital Homo sapiens?
Artificial intelligence is not only a problem in the field of natural sciences and engineering technology, but also a concern in the field of humanities and social sciences.
Lei Huanjie, deputy director of the Center for Science, Technology and Society Research of the Chinese Academy of Social Sciences and assistant researcher at the Institute of Philosophy, predicts that artificial intelligence will definitely drive research in the humanities and social sciences in the future.
For example, in course teaching, AI can assist teachers in generating teaching materials such as course outlines and exercises, as well as answering questions, and in the process of talent training, AI can help students train dialectical, critical, innovative thinking, and form independent thinking and problem-solving skills. In the construction of disciplines, artificial intelligence can also promote the generation of new research methods and ideas, which is not only conducive to the incubation of some emerging disciplines and interdisciplinary disciplines, but also helps basic disciplines, key disciplines, and unpopular disciplines to find new growth points.
In fact, East China Normal University has gradually tried to "high-simulation digital intelligent person (video) teaching" on campus, and the high-simulation "professor" has produced a data analysis course of the Ministry of Education's industry-university cooperation and collaborative education project;
At present, East China Normal University has constructed the first large-scale model comprehensive educational ability evaluation framework in China, and developed the first large-language model "EduChat" in China specifically for the field of education.
Wang Jiangtao told reporters that whether it is President Yuan Yunkai's digital intelligence or Academician Jiang Hualiang's digital intelligence, they all belong to the early exploration. In the future, the application scenarios of digital intelligent people can be used in archives, museums, school history museums, etc., "historical figures can be reproduced, and maybe the public can write a poem with Li Bai and have an exchange with Lu Xun?"
For another example, the needs of students to learn English can also be met by digital intelligent people, "At present, an English teacher may only be able to fully dialogue with a few students and give small classes, but in the future, if there is a simulated English professor to teach children, he can achieve 'one-to-one' with more children."
Wang Jiangtao introduced that the team is currently in contact with professionals in Chinese as a foreign language, trying to design a digital intelligent person to teach Chinese to foreigners, "If there is a Chinese digital intelligent teacher to chat with you every day and practice Mandarin, will your Chinese level improve greatly?"
China Youth Daily, China Youth Network reporter Wang Yejie Source: China Youth Daily