"Already proved their effectiveness": Russian scientist on the use of neural networks in agriculture

- Pavel Nikolaevich, according to the press service of the Ministry of Education and Science, you have developed a special automated system for monitoring the condition of vineyards together with scientists from the All-Russian National Research Institute of Viticulture and Winemaking "Magarach" of the Russian Academy of Sciences. Please tell us more about the invention. How is the system arranged?

— We are talking about automated monitoring of the state of vineyards, which we developed together with colleagues - the head of the laboratory "Digital Technologies in Winemaking and Viticulture" Dmitry Voronin and graduate student Dmitry Kotelnikov. The technology is based on neural network analysis of images of grape leaves, which are filmed by unmanned aerial vehicles (UAVs).

The sequence of actions is as follows: a flight task is loaded into the flight controller of the UAV, according to which the drone flies around the rows of the vineyard. We found out that for optimal results, the copter must fly around each row at least three times.

Then the footage is uploaded to a stationary high-performance computing device for subsequent neural network classification. That is, each frame is analyzed to find an image of dry, diseased or pest-affected leaves and their counting is made. The data obtained is visualized by the system in the form of a heat map, which indicates in what coordinates and how much affected foliage was found.

Initially, we considered the idea of data processing directly during video shooting, but this would require installing an on-board computer on the UAV. This would increase the weight of the copter and reduce the flight time until the next recharge and replacement of batteries. Therefore, we have abandoned this approach.

• RIA Novosti
• © Denis Abramov

In the process of experimental research, it was found that for the correct functioning of the system, it is not enough to use the standard procedure for neural network classification of objects, the so-called Object Detection. This is due to the fact that the same leaves can fall on several frames, and to avoid their re-counting, we used object tracking technology based on another neural network - Object Tracking. Thus, thanks to the simultaneous use of two technologies, we have obtained high-quality results sufficient for a rapid assessment of the state of the vineyard.

As for the equipment, almost any UAV is suitable, the main thing is that they are equipped with a good video camera with a stabilizer. However, if you use devices with closed software, then you will need to equip them with a specialized GPS tracker.

What was the most challenging part of the work on the technology?

"The most difficult thing was to create a dataset for training a neural network, containing a large number of images with marked healthy and affected grape leaves, as well as programming the UAV flight controller. As part of the study, such a set consisting of more than 6 thousand images was prepared. We manually photographed leaves and grape bushes, used photo and video materials from open sources, including YouTube.

This amount of data was enough to train the neural network to independently distinguish healthy leaves from affected ones. But for an accurate classification of the causes of damage - from specific diseases or pests - such a volume is not enough.

• Gettyimages.ru

Therefore, this year we will continue to work on collecting images of grape bushes and plan to perform a procedure for marking damaged leaves indicating for the neural network the specific causes of damage. In this work, we will need the help of our colleagues from the laboratory "Plant Protection" of the All-Russian National Research Institute of Viticulture and Winemaking "Magarach" of the Russian Academy of Sciences, headed by one of the best scientists in this field, Natalia Aleinikova. We believe that it will be possible to classify some individual diseases (chlorosis, mildew, rot, etc.).

However, we will have to postpone the work on video shooting of vineyards from the air until the ban on the launch of UAVs in our region is lifted.

— What practical benefits will such technology bring? Is there already interest in it from the business side?

- Yes, there is such an interest. The technology will be widely demanded by agricultural enterprises engaged in growing grapes: it will not only increase yields, but also reduce possible financial risks. Rapid detection of foci of plant diseases in the early stages will prevent the spread of the disease and avoid major losses. Moreover, the neural network is able not only to find such foci, but also to predict the dynamics of the spread of diseases. In addition, the system can be used to control the level of humidity - drying of the soil or, conversely, excess moisture can also cause damage to grape bushes.

First of all, the developed technology is planned for implementation in the vineyard of Sevastopol State University. This site is an excellent test site for testing various digital solutions in the field of viticulture. This is due to the fact that this object was originally designed as a "Digital Vineyard" - there is an interactive map of planting bushes with GPS tags, meteorological sensors and so on.

— Is it possible to adjust the technology to other crops - for example, for cereals, fruits?

- Yes, because this system is a universal tool for detecting, classifying and counting almost any objects and can be used to solve the problems of agricultural enterprises engaged in other crops. You only need to retrain the neural network to find other objects.

Moreover, we have already adapted this technology to solve problems in another area – solar energy. The principle of operation of the technology remains the same - video recording is conducted using UAVs, just photovoltaic modules act as objects. The drone, flying over them, looks for problem areas: damaged surfaces of photovoltaic modules, pollution, dimming, etc. The map displays the points where the neural network has identified potential problems, and specialists can go there for repairs. We have already begun work on the practical implementation of technology for servicing solar power plants.

— Last year, the Ministry of Agriculture of Russia published a tender for the creation of a service using AI, which would automatically track and analyze the state of agricultural land in the country, as well as crops growing on them. What is the reason for such a high interest in the use of neural networks in agriculture?

— Now the government pays much attention to the digitalization and automation of enterprises, including agricultural ones. I believe that such technologies will significantly increase labor productivity.

The main obstacle to the introduction of such solutions was the lack and high cost of high-performance computing equipment and robotic means, including unmanned aerial vehicles. But now copters have become available, and the performance of computers has increased markedly. All this has opened up opportunities for the widespread introduction of technologies such as ours.

They have already proven their effectiveness. Neural network analysis and machine vision are successfully used in various fields to solve problems related to fuzzy logic. Fuzzy logic is a branch of logic proposed by the American mathematician Lutfi Zadeh in 1965 to analyze data that cannot be generalized as "yes" or "no," figuratively speaking. In such cases, standard software algorithms work inefficiently. But the use of fuzzy logic in programming has led to the emergence of neural networks - special programs capable of self-learning.