The United Nations predicts that the world's population is expected to reach 9.1 billion in 2050, and global demand for food will increase by 70%. However, due to the uneven development of agriculture in the world, the grain production in many regions cannot be accurately counted, so it is impossible to make reasonable plans for agricultural development in these regions. The existing grain output statistical methods are difficult to promote or require a high level of science and technology. To this end, researchers from Kyoto University used convolutional neural networks (CNN) to analyze farmland photos and efficiently and accurately count local grain production, providing a new method to promote global agricultural development.
Author | Xuecai
Editor | Sanyang
This article was first published on the HyperAI Super Neural WeChat public platform~
Affected by population growth, rising incomes and the widespread use of biofuels, global demand for food will increase by 70% in 2050.
However, due to global climate warming and decline in biodiversity, food production around the world is extremely vulnerable to environmental changes, and development is uneven among regions.
It can be seen that China, the United States, India and Brazil are the main grain-producing areas, while the grain output in the Southern Hemisphere is relatively low. Moreover, due to the low agricultural productivity of the Southern Hemisphere, its food production is difficult to accurately measure. Therefore, it is difficult for us to effectively assess local agricultural productivity, let alone provide effective means to increase production.
There are currently three commonly used grain yield statistical methods, including self-reporting, actual cutting measurements and remote sensing statistics. The first two methods are difficult to promote on a large scale, and the use of remote sensing technology will be restricted by the local scientific and technological level.
To this end, researchers from Kyoto University used convolutional neural networks (CNN) to analyze farmland photos taken on the spot, and then made statistics on local grain production. The results show that the CNN model can quickly and accurately count rice yields during the harvest and mature stages under different lighting conditions. This result has been published in "Plant Phenomics".
Paper link:
https://spj.science.org/doi/10.34133/plantphenomics.0073
experiment procedure
1. Establish a database: rice canopy photos + grain yield
The researchers collected rice photos and grain yields from 20 farmlands in seven countries. After the rice matures, use a digital camera to shoot vertically downward at a height of 0.8 to 0.9 meters above the rice canopy to obtain an RGB photo of the rice with an area of 1 m2 .
Note: The rice canopy is the dense top layer of rice branches and leaves and is the main part of the plant for photosynthesis.
Subsequently, they changed the shooting angle, time and period, and removed the rice inflorescences one by one in some experiments to explore the mechanism of the CNN model to predict yield. Finally, they obtained 22,067 RGB photos of 462 rice species from 4,820 shooting locations.
The grain yield in the experiment is the coarse grain yield, including the total weight of solid and empty rice grains. The statistically obtained grain yield is between 0.1 t/ha (tons per hectare) and 16.1 t/ha, showing a normal distribution, with an average yield of about 5.8 t/ha.
A: Coarse grain yield distribution in 7 countries;
B: Pie chart of average coarse grain yield in different countries;
C: Image of rice with the highest coarse grain yield;
D: Image of rice with the lowest coarse grain yield.
2. Yield prediction: canopy photo + CNN → Grain Yield
The CNN model, loss function, and optimizer are deployed using the Python language and PyTorch framework. Subsequently, the researchers calculated the validation loss and relative root mean square error (rRMSE) when the model training was completed by combining different Batch Sizes and Learning Rates, and obtained the optimal Batch Size (32) and Learning Rate (0.0001) of the model.
The CNN model has 5 convolutional layers in the Main Stream (MS) and 4 convolutional layers in the Branching Stream (BS) . The pooling layer of the model includes average pooling layer (AveragePooling) and maximum pooling layer (MaxPooling). The activation function is mainly rectified linear unit (ReLU), and in some parts exponential linear unit (ELU) is used. Finally, MS and BS merge to output the estimated grain yield through the ReLU layer.
The CNN model has strong resolving power for images. When the ground sampling interval (GSD, the actual distance corresponding to each pixel in the photo, as opposed to the resolution) is 0.2 cm/pixel, the correlation coefficient R 2 between the CNN model prediction results and the actual results is above 0.65 . Even if the GSD increases to 3.2 cm/pixel, the R 2 of the model can remain above 0.55.
A: The relationship between the R2 of the CNN model and the GSD of the verification set and test set photos;
B: Scatter plot of CNN model predicted output and actual output;
C & D: Schematic photos of GSD of 0.2 cm/pixel and 3.2 cm/pixel.
Further, the researchers tested the CNN model using the prediction set data. The CNN model can distinguish the difference in yield between Takanari rice and Koshihikari rice in Tokyo, and the predicted data is close to the actual data.
The team then occluded the images to explore the mechanism by which the CNN model analyzes images and predicts grain yield. They blocked specific areas of the photo with gray patches and calculated the difference in the predicted yield of the CNN model before and after blocking.
A: Photo before blocking;
B: Photo after blocking;
C: The weight of different areas of the photo on the predicted yield.
The results show that grain yield is positively correlated with the number of rice inflorescences, and negatively correlated with the proportion of stems, leaves, ground and other elements in the picture.
Therefore, the researchers verified the role of inflorescences in yield prediction through inflorescence removal experiments. They picked two inflorescences from each rice plant, photographed and counted the coarse grain yield until all the inflorescences were removed.
A: Schematic diagram of inflorescence removal experiment;
B: Photo after the inflorescence is removed;
C: Line chart of expected output and actual output;
D: Relationship between expected and actual yield during inflorescence removal.
As the number of inflorescences decreases, the yield prediction results of the CNN model continue to decrease, finally falling to 1.6 t/ha. This experiment shows that the CNN model mainly judges grain yield based on the number of inflorescences in the photo.
3. Robustness: photo angle, time and period
After verifying the CNN model's ability to predict grain yield, the researchers changed the shooting angle, time, and period to explore the robustness of the CNN model under different conditions.
The shooting angle of the photos is between 20°-90°, and the test interval is 10°. The results show that the prediction accuracy of the CNN model increases as the camera angle increases. When the shooting error is 20°, the prediction results of the CNN model are -3.7-2.4 t/ha. When the shooting angle is 60°, the prediction error is between -0.45-2.44 t/ha, which is close to the prediction result when it is 90°.
A: Schematic diagram of shooting angle experiment;
B: Photos taken from different shooting angles;
C: The difference between the predicted output and the actual output of photos from different shooting angles.
Subsequently, the camera was placed at a fixed position and took a photo of the farmland every 30 minutes to explore the impact of shooting time on the CNN model. The results show that although the lighting environment changes, the prediction results of the CNN model for all-day photos are basically stable.
A: Schematic diagram of shooting time experiment;
B: Photos taken at different shooting times;
C: Predicted yield of CNN model for photos at different shooting times.
Finally, the researchers explored the impact of shooting period on the prediction results of the CNN model. After 50% of the rice heads, they go to the farmland every week to collect photos and analyze them with a CNN model. At the early stages of rice maturity, the predicted yield of the CNN model is lower than the actual yield at harvest time because the inflorescences are not fully mature at this time.
As time goes by, the prediction results of the CNN model gradually approach the actual production. Four weeks after 50% heading, the prediction results of the CNN model are basically stable and close to the actual yield.
A: Photos taken at different shooting periods, DAH represents the number of days after heading, and DBH represents the number of days before harvest;
B: Prediction results of CNN model for photos taken at different times.
The above results jointly demonstrate that the CNN model can accurately analyze farmland photos obtained at different shooting angles, times and periods, and obtain stable yield prediction results. CNN models are robust.
Smart Agriculture: AI helps agricultural planning
According to the United Nations, the global population will reach approximately 9.1 billion in 2050. As the global population grows and incomes increase, people's demand for food is also increasing.
At the same time, the intensification, digitization and intelligence of agricultural production have continuously increased grain output per acre. From 2000 to 2019, the global agricultural land area decreased by 3%, while the output of major crops increased by 52%, and the output of fruits and vegetables also increased by about 20%.
Professional equipment such as large harvesters and drones are put into use, allowing farmers to plan their farmland accurately and conveniently. Technologies such as big data and the Internet of Things help farmers perceive farmland conditions in real time, and can also automatically adjust the environment in the greenhouse. Deep learning and large models can predict weather in advance to prevent extreme weather before it happens and alleviate the problem of traditional agriculture relying on the weather.
However, as of 2021, the number of people affected by hunger globally increased by approximately 46 million compared to the previous year, reaching 828 million. The problems of unbalanced agricultural production and imperfect systems still exist and are even more prominent.
With the help of AI, we can make better plans for local agricultural development, promote the balanced development of world agricultural production, and provide a satisfactory answer to the problem of global hunger.
Reference links:
[1] https://www.fao.org/documents/card/en/c/cc2211en
[2] https://www.deccanherald.com/opinion/smart-farming-tech-new-age-700994.html
This article was first published on the HyperAI Super Neural WeChat public platform~