At present, the popularity of large language models is unprecedented, such as Wen Xin Yiyan, ChatGPT, etc. have been able to interact with people, answer questions, assist in creation, and gradually applied to people's work and life, but also caused heated discussions in society. Recently, Wang Haifeng, chief technology officer of Baidu and director of the National Engineering Research Center for Deep Learning Technology and Application, once again visited CCTV-2 "China Economic Lecture Hall" to explain the product capabilities, technical principles and industrial value of such large language models as Wen Xin Yiyan.
Baidu CTO Wang Haifeng as a guest at "China Economic Lecture Hall"
Wen Xin's reading exceeded trillions, and the five major abilities were leading the industry
In the era of artificial intelligence, the IT technology stack can be divided into "chip layer, framework layer, model layer and application layer", and Baidu is one of the few artificial intelligence companies in the world that carries out full-stack layout. From Kunlun chip, Flypads deep learning platform, Wenxin large model to application, at each layer of the technology stack, there are leading key self-developed technologies, which realize layer-to-layer feedback, end-to-end optimization, and greatly improve efficiency. Wenxin Yiyan is the natural result of Baidu's years of artificial intelligence technology accumulation and industrial practice, especially the joint optimization of the flying propeller deep learning platform and Wenxin big model, which provides solid technical support for Wenxin Yiyan.
Wang Haifeng emphasized that Wenxin's words are completely a large language model independently developed by Baidu. Baidu released ERNIE2019.1 as early as 0, after nearly 4 years of research and development and iteration, Wenxin model has formed an industrial-level knowledge enhancement large model technology system, including natural language processing, vision, cross-modal, biocomputing, industry large models, and tool platforms to support large model applications, containing a large number of Baidu's independent innovation and large-scale industrial application of verified technology. Some of these key technologies have been patented or published papers, and some related technologies have also been open source.
Wen Xin Yiyan is a typical representative of the big language model. Wang Haifeng took "reading ten thousand volumes, writing like a god" as an example, interpreting the big language model to learn from massive data, which is equivalent to reading trillions of books, absorbing trillions of knowledge, and achieving understanding, on this basis, you can generate copywriting, answer questions, and complete summary analysis according to the needs of users.
Regarding the origin of the name "Wen Xin Yiyan", Wang Haifeng made a specific explanation: "Wen" is language and writing, "heart" is to understand with heart, "Wen Xin" refers to the natural language understanding model committed to understanding and using language and writing, and also echoes "Wen Xin Eagle Dragon", which means to use the fine effort of carving dragon patterns to study the connotation and charm of language and writing; "One word" not only has the meaning of "one word is determined, one word is nine dings", but also has the ardent expectation of smooth communication between man and machine "you say one word and I say one word". Wen Xin's "one word" can be "two lives, two lives, three lives, ten thousand words".
The scene also demonstrated Wen Xin's literary creation, commercial copywriting, reasoning calculation, Chinese comprehension and multimodal generation and other capabilities. In terms of literary creation, Wen Xin can not only write the film review copy of "The Wandering Earth 2", but also create the picture of the circle of friends, and continue the follow-up plot of "The Wandering Earth 3"; In terms of commercial copywriting, Wen Xin helped entrepreneurs who want to open stores to make preparations, such as investigating the taste preferences of office people within 3 kilometers of Zhongguancun, Beijing, designing store names, creating promotional slogans and Tibetan poems, and collecting and summarizing the government departments and related procedures involved in establishing restaurants; In terms of reasoning and calculation, Wen Xin Yiyan can accurately calculate complex mathematical problems, first understand the problem, then reason through the thinking chain, and finally generate the answer; In terms of Chinese understanding, Wen Xin's words can accurately answer the natural scene and the physical phenomenon behind the verse "purple smoke from the sunshine incense burner, looking at the waterfall hanging in front of the river"; In terms of multimodal generation, Wen Xin Yiyan can accurately answer questions about ancient Chinese poetry, and make tables, paint, and read aloud in dialects.
With the blessing of the six core technologies, Wen Xin came to fruition
Wen Xin Yiyan is a new member of the Wen Xin large model family, developed on the basis of Wen Xin knowledge enhancement large model ERNIE and dialogue large model PLATO, based on the training and deployment of flying paddle deep learning platform, its key technologies include, supervised fine tuning, human feedback reinforcement learning, prompt, knowledge enhancement, retrieval enhancement and dialogue enhancement. The first three are the technologies that will be used in such large language models, which have been applied and accumulated in ERNIE and PLATO models, and Wen Xin has been further strengthened and polished in a word, so as to better understand Chinese, Chinese culture, and Chinese use scenarios; The latter three are the re-innovation of Baidu's existing technical advantages, and they are also the increasingly strong technical foundation of Wen Xin's words.
Wang Haifeng took the example of teachers teaching students to explain the technical principles behind the big language model in simple terms. Pre-trained large models like well-read students, remember a lot of knowledge, but need the teacher to guide how to use, and supervised fine tuning is that the teacher is teaching the students, the extracted knowledge points, typical examples, etc. to the model, so that it knows how to conform to human norms, habits and values, to perform corresponding actions, generate corresponding content. Wen Xin Yiyan trained the reward model, scored the results of each output and gave feedback, and carried out reinforcement learning, with more and more feedback from real users, the effect of Wen Xin Yiyan will become better and better, the ability is getting stronger and stronger, and the progress is "thousands of miles a day". In addition, Wen Xin Yiyan also integrates different types of data and knowledge, and automatically constructs prompts, including examples, outlines, specifications, knowledge points and thinking chains, etc., providing rich reference information, stimulating model-related knowledge, and generating high-quality results.
For the re-innovation of Baidu's existing technical advantages, Wang Haifeng also made a further interpretation. In terms of knowledge enhancement, knowledge is the crystallization of human wisdom to understand and transform the world. Baidu has built a knowledge graph of 5500 billion facts. Wen Xin Yiyan based on the huge knowledge graph to do knowledge enhancement, from massive data and large-scale knowledge integration learning, can also directly call the knowledge graph to do knowledge reasoning, automatically build prompts, efficient to meet user needs. Under the guidance of knowledge, Wen Xin's words are like standing on the shoulders of giants, learning well and quickly, and the efficiency and effect of the model have been greatly improved.
In terms of search enhancement, Baidu has the world's largest Chinese search engine, Baidu search has developed to a new generation of search architecture based on semantic understanding and matching, deeply understand user needs and web content, perform semantic matching, and obtain more accurate search results, and then provide high accuracy and time-sensitive reference information for large models to better meet user needs.
In terms of dialogue enhancement, based on the accumulation of dialogue technology and application, Wen Xin Yiyan has the ability of memory mechanism, context understanding and dialogue planning to achieve better dialogue coherence, rationality and logic. Baidu has been deeply engaged in dialogue technology for many years, and has achieved internationally leading technical achievements, winning the China Patent Gold Award and Wu Wenjun Special Prize for Artificial Intelligence Science and Technology Progress, etc., laying the foundation for the successful research and development of Wen Xin Yiyan.
Wen Xin's words benefit thousands of industries and accelerate the intelligent transformation of the industry
The rapid development of large language models has caused heated discussions in society, and Wang Haifeng also gave answers to problems such as job replacement, education model change, and artificial intelligence security.
He said that artificial intelligence technology such as Wen Xinyan is essentially a tool to improve productivity, which can replace humans to complete some work, but it will also create more jobs so that humans can do more creative work. Just like any scientific and technological revolution and industrial transformation in human history, some jobs will be replaced and more new jobs will be created. In the transformation of the education model, "rote memorization" has fallen behind, and education will keep pace with the times and change in the direction of inspiring inspiration and cultivating creativity.
In terms of AI safety, Baidu firmly abides by relevant laws, regulations and ethical norms, and is specially equipped with corresponding supervision and management mechanisms to ensure safety issues from all aspects. In the development process of Wen Xin's words, from the initial data collection, processing, model training, to the final use process, five security lines were constructed. Baidu has also established a data management committee and cooperated with all sectors of society to continuously improve relevant policies and rules and strengthen AI security. Regarding whether artificial intelligence will control human beings, Wang Haifeng emphasized that just as artificial earth satellites will never be equated with natural satellites such as the moon, artificial intelligence will never be directly equated with human intelligence, and the study of artificial intelligence is in the study of using technical means to simulate, extend and expand human intelligence, and the ultimate goal is to bring more advanced technology to human beings and serve the development of human beings for a better life and society.
At present, artificial intelligence has become an important driving force for a new round of scientific and technological revolution and industrial transformation, and deep learning, as the core technology of artificial intelligence, has strong versatility, showing standardized, automated, modular industrial large-scale production characteristics, and promoting artificial intelligence into the industrial large-scale production stage. Large models have the characteristics of good effect, strong generalization and standardized R&D process, and are becoming a new foundation for artificial intelligence technology and application. However, at the same time, the R&D threshold of large models is high and difficult, relying on algorithms, computing power and data comprehensive support, industrialization faces challenges: the model is large in size and the training difficulty is high; Large scale of computing power and high performance requirements; The scale of the data is large and the quality of the data is uneven.
How to realize the industrialization of large models? Wang Haifeng said that similar to the chip foundry model, enterprises with algorithm, computing power and data comprehensive advantages can encapsulate the complex process of model production and provide large model services for thousands of industries through low-threshold, high-efficiency production platforms. In the future, big language models such as Wen Xin Yiyan will become a general empowerment platform, and all walks of life such as finance, energy, media, and government affairs can achieve intelligent transformation, improve efficiency and create huge business value based on Wen Xin Yiyan. It is expected that all sectors of society will actively embrace new technologies and work together to achieve high-level scientific and technological self-reliance and self-improvement, and bring more momentum to high-quality economic growth.