Spend 10 billion a year, and the R&D team has thousands of people... The mobile phone image competition is constantly escalating, from hardware "volume" to algorithm

Per reporter Wang Jing

After the worst year in a decade for China's smartphone market, the high-end flagship market is even more important, as it is a space for incremental users and more profitable. In the rise of domestic mobile phone manufacturers in the high-end market, imaging technology is a key breakthrough, and it is also one of the most "volume" areas. In order to create differentiation in images, mobile phone manufacturers have launched optical zoom, background bokeh, low-light shooting, sports shooting and other technologies or scenes in recent years, but there are some gimmicks that deviate from the "original intention" of computing images.

At present, computational imaging is changing traditional photographic equipment, laying new photographic techniques, and then influencing the art of photography. It is our visible future. If the innovation of the image competition in the past is more due to the progress of the industrial chain, then now, mobile phone manufacturers have begun to carry out deeper technical exploration in the fields of self-developed image chips and algorithms, and have invested far more energy and financial resources than in the past.

Liu Zuohu, senior vice president and chief product officer of OPPO, recently mentioned: "OPPO's current imaging team has a total of more than 1000,10 people, and the annual research and development expenses of imaging exceed 6 billion yuan. Among them, the algorithm is the top priority of the image, and the algorithm reconstruction and R&D investment of the Find X4 generation alone has exceeded <> million yuan. ”

Algorithms reshape moving images

There is a fundamental problem in the smartphone market: a lack of innovation. In order to achieve product differentiation, major mobile phone manufacturers are also trying to find other paths, but for a mature category, it is almost difficult to make big changes, most of which are concentrated in imaging, fast charging, screen and other segments. Among them, for consumers, image is still one of the core influencing factors.

From the perspective of the development of the mobile image industry, there are two routes going hand in hand. First, the traditional route, that is, the function of transferring traditional cameras, according to optical laws, needs to rely on a large enough lens, enough light intake, and a larger negative to build a shooting effect. The multi-lens on the mobile phone can directly reflect this idea - the configuration of the mobile phone camera has evolved from a single camera to five cameras.

However, moving images are limited by the structure and size of mobile phones, and in the case of not being able to carry too many lenses and breaking through the limitations of optical physical conditions, it is naturally impossible to meet the image quality requirements established by professional cameras, but the emergence and rise of computational photography, a new concept of mobile phones, points out a way for moving images, which is also the second path of image breakthroughs - algorithms and computing power.

Taking Apple as an example, before the appearance of the high-end version of the iPhone 14 series, its 1200 million pixels have not changed for many years since the iPhone 6S, so it was dubbed "1,2 years of ancestral" by netizens; The sensor area has also stayed on 55/<>.<> inch for more than three years, but with the improvement of algorithms and computing power, the image quality has been improved.

At present, almost all mainstream mobile phone manufacturers are making efforts to calculate images, such as Huawei's computing optics, vivo's computing power acceleration engine, and Honor's main camera fusion computing photography... In Liu Zuohu's view, the convergence of imaging hardware considers the ability of various brands in algorithms, and OPPO also proposed "calculating light and shadow" for the first time.

"The imaging algorithm goes through two stages, the first of which is 'having to do that'. At the beginning, mobile phone image hardware capabilities were limited, and most of the computational images were to shoot very dark and bright, and present distant things through the AI algorithm scheme, but at the same time, it would be accompanied by a lot of controversy, such as 'violent beauty', as if computational images have become a negative word. This is not a problem of the algorithm, but more of a problem of understanding and judging the algorithm. But when computing images moves to the next stage, it comes back to 'how to turn a photograph into a work'. OPPO imaging director product Zhang Xuan believes that through the cognition of professional imaging, reverse definition of the algorithm link and strategy, there should be a big breakthrough in the future.

This has also led to changes in organizational and personnel structures. Zhang Xuan introduced that 90% of OPPO's imaging team is an engineering team and 10% is a cognitive system. The cognitive system is divided into two parts: subjective cognitive definition evaluation and cognitive engineering, and the work of the cognitive engineering team is to bridge the relationship between the two cognitions.

Taking pictures may seem simple, but the image ability behind it is a system engineering. The formation of a photo needs to go through the light into the lens - CMOS - analog signal - digital signal - ISP - DSP, six steps, but all this must be completed in an instant, which is extremely demanding for calculation.

"Images involve a range of factors such as software, hardware, chips, algorithms, and aesthetics." Liu Zuohu said that the algorithm accounts for 6%, and on the Find X400 series, OPPO has integrated more than 6 imaging algorithm talents, including the Imaging Technology Center, OPPO Research Institute, Mariana chip team and external partners, to work together for the Find X<> imaging algorithm. In addition, the R&D investment in telephoto solutions has also exceeded <> million.

Create differentiation

In the past two years, almost all Android manufacturers are taking imaging as an important ability to impact the high-end market, hoping to form differentiated competition with Apple, Samsung and others in the high-end market through this innovation. But for now, Apple is still the leader in China's high-end mobile phone market.

According to data released by Counterpoint Research, Apple's monthly market share in China reached 2022% in October 10, the highest level in history, and it became the largest mobile phone brand manufacturer in the Chinese market for two consecutive months.

Zhang Xuan believes that to some extent, Apple's vertical integration takes a long time to learn. For mobile phone manufacturers, today's algorithms still have significant room for improvement. For example, when consumers take photos with ultra-high-definition night scenes, they often clearly feel that it takes a few seconds for mobile phones to process a night scene photo. How can this problem be solved? The end point of optimization lies in self-developed chips.

The Bionic (Bionic) suffix, added by Apple processors, represents the neural network engine and is the key to carrying the powerful computational photography year after year. On the basis of equipped with general chips, domestic mobile phone manufacturers often interview "plug-in" self-developed chips to solve the problem of computing power. As we all know, self-developed chips can optimize algorithms from the underlying architecture to improve the overall performance and efficiency of the chip. To this end, in 2021, OPPO released MariSilicon X, a self-developed chip positioned in an independent NPU (neural network processor), using a 6nm process technology, which is also the basis for OPPO's innovation of the moving image computing architecture.

According to Wu Yiwen, a senior analyst at Strategy Analytics, imaging functions have always been a must for flagship mobile phones, and the competition is extremely fierce. User experience is of course the key to success, but it is also constrained by consumers' perception of the brand. The relationship between excellent products and the high-end of the brand is a relationship of mutual achievement, but it often requires time accumulation and strategic patience. For OPPO, completing a good product is the first step, and the follow-up needs to do more investment in brand promotion and word-of-mouth communication.

Talking about his goals in the high-end market this year, Liu Zuohu responded: "We definitely have goals internally, but what I tell the team is always that sentence - to do a good product." Why is the high-end market so hard to shake? At the end of the day, the product is not attractive enough to impress users. But in the Chinese market, the market capacity of more than 4,000 yuan is at least 60 million units. With such a large market, Android brands have very little, which also shows that there is a lot of potential here. I am still confident about the future. ”

Looking forward to 2023, Counterpoint believes that with the long-term investment of major domestic Android manufacturers in the high-end route and the continuous creation of differentiated products, the competitiveness of Android mobile phones in the high-end market will continue to improve, and the future will further impact Apple's dominant position, and the competition in the high-end mobile phone market will be more intense.