"Dolphin 1" is on the march! Look at the "art of navigating the sea" in the AI era

【Produced by Deep Pupil Studio】

Written by: Reporter Zhang Jiaxing and Li Liyun

Correspondent Huo Ping

Planner: Zhao Yingshu, Lin Lijun, Teng Jipu

Float! The peaceful voyage was interrupted by a warning sound. On the inaugural voyage of the "Dolphin 1", there were a number of floats, and below the floats were endless marine farms.

This made Zhang Zhi, the leader of the first voyage of China's first digital twin intelligent test ship "Dolphin 1" and a professor at Harbin Engineering University, sigh in his heart: "The chart does not indicate that there is a farm here, what should I do?" It was too late to make a temporary detour, and I had to pass through this confused place of 'reefs'. ”

"Can it do it around these floats with unknown positioning rules and unknown exact coordinates?"

This question flashed in Zhang Zhi's mind, however, the lidar, solid-state radar, panoramic infrared vision system mounted on the "Dolphin 1" and its data processing speed of <> megabits per second made Zhang Zhi think too much.

With ultra-high configuration and decision-making ability, the "Dolphin 1" only took a short time to solve the problems encountered in the first voyage and skillfully crossed the "dangerous beach".

At the World Navigation Equipment Conference held on October 10, the "Dolphin 12" exhibited in the exhibition hall of Harbin Engineering University attracted wide attention, and visitors couldn't help but praise when they learned about the intelligent level of the ship in three aspects: information fusion, situational awareness, and digital twin.

Led by the "Dolphin 1", China's intelligent ship development has taken a solid step forward.

Let the ship have the superpower of "knowing things like gods".

Since the International Maritime Organization (IMO) put forward the concept of e-navigation (electronic navigation) in 2006, many countries such as Europe, the United States, Japan and South Korea have launched large-scale plans to improve the level of ship intelligence.

"With the deepening application of artificial intelligence, digital twin and other technologies, ship intelligence has far exceeded the original definition of 'informatization'." Professor Xia Guihua, head of the digital and intelligent technology innovation team of Heilongjiang Engineering University and the head of the digital and intelligent technology innovation team of Heilongjiang Province, said in an interview with a reporter from Science and Technology Daily that the digital twin intelligent scientific research test ship developed by the team is named "dolphin" because dolphins are a symbol of intelligence.

"Dolphin 1" not only has the advantage of accurately detecting tiny targets 2.0 meters high on the water surface within 5 nautical miles, but also uniquely gives it "wisdom" due to AI technologies such as situational intelligent perception and advanced prediction.

"On land, intelligent networked vehicles are changing the safety of transportation, the efficiency of energy use, and are moving towards intelligent driving and even unmanned driving. In the ocean, the revolution of smart ships is also catching up. Xia Guihua said.

"Dolphin 1" has opened a precedent for future smart ships to create "smart eyes, smart ears, and smart brains", which will empower China's shipbuilding after further improvement, and lay a solid foundation for the formation of fully autonomous technical capabilities for the main systems and key equipment of future smart ships.

"To achieve such a leap, we must equip Dolphin 1 with a real 'super brain'." Professor Zhu Qidan, technical leader of the digital and intelligent technology innovation team of Harbin Engineering University, said. The team has accumulated a wealth of research results in intelligent ship design, manufacturing, modeling, and calculation, and can provide high-precision information for intelligent ships through real-scale ship dynamics modeling, navigation hydrodynamic performance simulation, power system simulation, wind, wave and flow field simulation, etc. "It is equivalent to being able to conduct pre-rehearsal in an environment close to the actual ship, giving 'Dolphin 1' the ability to anticipate, think, and judge." Zhu Qidan told reporters.

In order to make such a model, the team collected tens of thousands of pictures of real ships covering a variety of scenarios, accurately labeled, sorted and analyzed them one by one, and finally formed a dataset for artificial intelligence machine learning. With the support of the core module, "Dolphin 1" can collect and process external multi-source information data in real time at a speed of hundreds of megabits per second.

All these allow the "Dolphin 1" to independently avoid obstacles and plan the optimal route in real time, as if it is a "god operator" that can deal with the unknown world.

"Dolphin 1" has a "digital clone"

From the outside, the "Dolphin 1" is petite and ordinary. It is only 25 meters long and has a displacement of only 100 tons. But the biggest difference between it and other ships is that there is a "digital twin" in the virtual world that can predict for it.

You know, the biggest advantage of the virtual world is the low cost of trial and error. Imagine if there is a virtual person who interprets the shortcomings in your life in advance, will you still let the shortcomings happen in real life?

The R&D team tailored such a virtual boat for the "Dolphin 1". Not only can they sail synchronously, but they can also rehearse sailing.

"The physical ship stops, and the virtual ship can continue sailing." Zhang Zhi said that in this way, the real ship can predict its next sailing strategy, predict the possible dangers, and test how to operate when encountering these dangerous conditions to ensure the safety of the real ship's operation.

"We hope to 'copy' the wisdom of Dolphin 1 to thousands of ships. However, it cannot be taught one by one, and the role of the ship digital twin integrated software platform and its application system is to 'teach and teach' like a teacher. Xia Guihua told reporters that the platform of virtual and real interaction provides a more changeable and richer "training ground" for intelligent ship testing than the real environment.

In the future, the efficient testing, verification, operation, scheduling, and control platform on which "Dolphin 1" relies may be applied to the early research and development, mid-term testing, and later operation of autonomous intelligent ships, accelerating the application and deployment of China's independent intelligent ships in accessible sea areas.

The environment in the sea is changing rapidly, and there are many factors that affect the navigation of ships. How can we build an inference platform that maps the real world and has accurate simulation functions, such as virtual boats, virtual oceans, and virtual weather environments?

Xia Guihua gave the key secret of the interaction between the virtual and the real: "In the digital twin system, the virtual and the real are a dynamic dialectical relationship, which can be simply summarized as 'from the real to the virtual', 'from the virtual to the real', and finally to achieve 'virtual control of the real'. ”

Following this principle, the team constructed a digital twin ship and a digital "cyber" marine environment based on the ship in the real physical space and the marine environment in which it sailed, and used information and artificial intelligence technology to make real-time interaction between the virtual and the real.

In the cabin of the "Dolphin 1" and in the remote digital support control center of the ship in Huangdao and Harbin, you can see the "virtual boat" of real-time synchronization of the "Dolphin 1". It not only digitally reproduces the mechanical structure, host system, perception system, equipment components and pipeline circuit system of the body, but also completely connects with the perception system and control system of "Dolphin 1".

"The digital modeling in the virtual ship can remotely support and control the physical ship, and realize the autonomous navigation, automatic berthing and unberthing and automatic collision avoidance of the intelligent ship." Zhang Zhi said that the landing of "Dolphin 1" solves the key problem of realizing remote digital support for autonomous navigation of ships based on digital twin technology, and provides basic technical support for the autonomous navigation and operation of intelligent ships and marine engineering equipment based on remote support in the future.

Two original innovations have laid a solid foundation for technology

"It's not a completely new field." Professor Cai Chengtao, chief of the team's system, said that industrial digitalization is part of the new industrialization. Countries around the world have started the development of digital twin intelligent ships many years ago. For example, South Korea began to build intelligent ships in 2012 to promote the development of informatization, automation, intelligence, and green energy conservation of ships.

China's choice is to plan and then act.

In 2015, Harbin Engineering University and the 702 Institute of China Shipbuilding Group jointly led the research and development of the world's first numerical pool virtual test system V1.0 with a number of scientific research institutes and universities with technical advantages in marine science and sea, and passed the domestic and international double certification of China Classification Society and BV Classification Society.

This is the world's first large-scale efficient parallel computing of 100 million grids, which can provide extremely detailed hydrodynamic performance simulations for ship and offshore design, sea trials and operation.

"At that time, the trend was data-driven, just like an alpha dog that could play Go, with the support of certain algorithms, it was trained to produce 'intelligence' in fixed tasks through massive datasets." Zhang Zhi said, but Xia Guihua did not lead the team on this path.

"Data is important, but the model is the soul of performance. The two run in with each other to achieve high-precision computing on the basis of a certain amount of computing power. Xia Guihua said that the digital system to be applied in the shipbuilding industry must meet two conditions: first, it needs to establish the most accurate model possible, and secondly, it must be iteratively upgraded through data-driven models.

Needless to say, there is an iron law in the field of information technology: only mastery of code can determine the pace of iteration.

Because of this, when the research on intelligent ships and digital twin technology began to be implemented in 2019, Xia Guihua once again chose to innovate at the source.

"Teacher Xia and I used to have an argument. I think the architecture and models on the open source platform are polished by many AI enthusiasts, which is rich and solid enough. Mr. Xia insisted on source innovation. Zhang Zhi said that looking back, it was far-sighted to make such a decision, as the foundation of industrial digitalization, the underlying code must be firm in serving long-term development and serving the national strategy.

The decision to innovate at the source not only strengthens the foundation of the digital twin technology, but also makes the application of the technology "flourish".

"We are already a recognized 'leader' in the marine industry when it comes to digital twin technology for ships." Zhang Zhi said that if the team had done things with the idea of intersection, the team would not have been experienced, and now the code in the digital twin system is typed out by the team one by one, and the team has a very thorough understanding of the mechanism and laws of its underlying and core layers.

In the future, remote "substitute driving" will ride the wind and waves

A strong wind hits, and big waves rise, wave after wave. At sea level, the sky was clear and peaceful, but it turned into a turbulent wave and an undercurrent, and the hull of the ship swayed violently.

"The wind was relatively strong on the section from Weihai to Rushan, and the 'Dolphin 1' swayed violently, and almost all the researchers on the ship were seasick, lying down and not daring to move." Zhang Zhi recalled the experience of the first voyage: although he felt uncomfortable, he did not have to worry at all. Because, there is a digital doppelganger of "Dolphin 1" to assist the crew to "drive".

At present, the "Dolphin 1" is a "quadruplet", and its performance is most similar to the digital twin system in the cabin, which can directly obtain all the perception data of the "Dolphin 1" and become the "strongest assistance" for the crew to drive the ship.

In the remote digital support center of intelligent ships in Building 61 of Harbin Engineering University and Qingdao Innovation and Development Base of Harbin Engineering University, which is thousands of miles away from the sea, due to the limitation of satellite data transmission and network bandwidth data transmission capacity, only a small amount of data can be transmitted. In this regard, the team members are full of confidence.

"In the future, with the assistance of the intelligent ship situation perception system, we can accurately grasp the surrounding environment situation even if we are not on the scene, and easily drive the ship." Professor Liu Zhilin, a member of the "Dolphin 1" team, said. The team has developed a comprehensive information fusion software system, which can integrate the information obtained from hardware such as panoramic vision, lidar, and solid-state radar with the information obtained from electronic charts, Beidou navigation, and real-time satellite remote sensing, analyze and calculate, and simultaneously build a digital marine environment, realize intelligent perception of ship navigation situation, and guide the remote control of ship navigation.

The lidar on the Dolphin 1 carries the genes of the Chang'e lunar probe. The lidar developed by the Shanghai Institute of Technical Physics of the Chinese Academy of Sciences has been successfully applied to the Chang'e 3, 4 and 5 lunar probes. This time, they made a special lidar for the "Dolphin 1". Huang Genghua, director of the Shanghai Institute of Technical Physics of the Chinese Academy of Sciences, explained: "Compared with the aerospace lidar with a detection distance of about 100 meters, the ship's lidar detection requirement is 4 kilometers, which is equivalent to increasing the detection index by 40 times." ”

On the day of the maiden voyage of "Dolphin 1", the "maze" encountered in the farm relied on another device called "solid-state radar" to locate and mark and successfully break through. Beijing Hailanxin Data Technology Co., Ltd. developed a solid-state radar for the "Dolphin 1". Li Changwei, chief engineer of the company, introduced that the solid-state radar has a high ability to suppress miscellaneous baud, and can clearly mark the strings of floats in the breeding area.

The vision system of the "Dolphin 1" is also divided into two sets: infrared and visible. The two sets of 360-degree panoramic vision systems are composed of 12-channel spliced high-resolution cameras, which can quickly store the collected ship navigation visual information through the embedded system, and quickly calculate and synthesize a seamless 360-degree panorama, which is transmitted back to the remote tracking and control center and the shipboard navigation perception system.

High-precision sensing equipment, real-time information transmission, and advanced algorithm models will jointly depict a subversive change for future ships, and remote "driving" may be just one of the typical applications.

From Zheng He's seven voyages to the West to Columbus's round-the-world travels, mankind's conquest of the sea has never stopped. In the era of AI, humans are expected to achieve maximum operational efficiency while reducing the risk of going to sea and reducing energy consumption. Not only that, with the blessing of self-learning, self-adaptation, self-evolution and other algorithms, the ship's own systems can also be continuously iterated and optimized like smartphones, reproduce the state of the ship system at any time at any time, deduce or simulate the "what-if" scenarios at future moments, and generate predicted response strategies.

Open up the blue ocean of ship intelligent manufacturing

90% of the world's trade and transportation depends on the sea.

The design and manufacture of ships are highly innovative and high-tech, and they are the engine for the high-quality development of the shipbuilding industry.

In July, China's self-developed, designed and built pure battery-powered container ship N7 with independent intellectual property rights made its maiden voyage.

In September, China's first large-scale cruise ship with an integrated operating system and 9G signal coverage completed sea trials.

At present, green and intelligent have become the main theme of the transformation and upgrading of the ship design and manufacturing industry.

"Ships are different from land transportation such as automobiles, and their production and manufacturing have the characteristics of multiple varieties, small batches, and complex structural systems, so the degree of standardization and modularization is not high. Digital systems can compensate for this disadvantage. Cai Chengtao said that the use of digital twin technology can form some general software modules, such as perception, decision-making, control, environment, etc., which greatly promotes the modular "flexible customization" assembly of intelligent ships and accelerates the rapid development of ship intelligent technology.

How to achieve transformation and upgrading of the ship design and manufacturing industry in the AI era? How can new ships efficiently complete various performance verifications such as safety and stability?

"'Dolphin 1' is to build a mobile offshore laboratory for the development of China's ship intelligent technology, so as to accelerate the iterative process of China's ship intelligent technology." Xia Guihua said that the "Dolphin 1" can be equipped with a variety of intelligent systems, through data accumulation, model analysis, to provide efficient and optimized solutions for the design of future intelligent ships.

"The Dolphin 1 cabin is equipped with interfaces to connect to the experimental equipment, and there are the necessary equipment to make experimental tools." Liu Zhilin introduced that the ship is an open system, with a modular design, and all internal systems have interfaces to meet the experimental needs of ships and marine technology and equipment.

With the support of this "mobile laboratory", the team has successfully completed the demonstration and verification of a series of key technologies of intelligent ships, such as autonomous navigation of intelligent ships, remote control, multi-scale and multi-modal intelligent ship perception, and will provide real ship test and verification conditions for the independent research and development of shipbuilding industry software in the future.

Harbin Engineering University is uniting industry advantages to jointly establish an industry alliance for intelligent navigation equipment. In the future, the members of the alliance can use the "Dolphin 1" to conduct experiments and actively promote the development of the intelligent ship industry.

"Digitalization is the foundation of intelligence, industrial software is the means to achieve digitalization, and ship digital and intelligent control is the main direction of future ship development." Xia Guihua said that the ship design and upgrading of many key core technologies, including greening and intelligence, can not only open up a new blue ocean for the development of China's shipbuilding industry, but also show the style of green and intelligent high-end ship type "created in China" in the international shipbuilding market.