AI shows how field crops develop: Software can simulate future growth based on a single initial image

Researchers at the University of Bonn have developed software that can simulate the growth of field crops. To do this, they fed thousands of photos from field experiments into a learning algorithm. This enabled the algorithm to learn how to visualize the future development of cultivated plants based on a single initial image. Using the images created during this process, parameters such as leaf area or yield can be estimated accurately.

Which plants should I combine in what ratio to achieve the greatest possible yield? And how will my crop develop if I use manure instead of artificial fertilizers? In the future, farmers should increasingly be able to count on computer support when answering such questions.

The researchers have now taken a crucial step forward on the path towards this goal. "We have developed software that uses drone photos to visualize the future development of the plants shown," explains Lukas Drees from the Institute of Geodesy and Geoinformation at the University of Bonn. The early career researcher is an employee in the PhenoRob Cluster of Excellence.

The large-scale project based at the University of Bonn intends to drive forward the intelligent digitalization of agriculture to help farming become more environmentally friendly, without causing harvest yields to suffer. The findings are published in the journal Plant Methods.

A virtual glimpse into the future to aid decision-making

The computer program now presented by Drees and his colleagues is an important building block. It should eventually make it possible to simulate certain decisions virtually—for instance, to assess how the use of pesticides or fertilizers will affect crop yield.

For this to work, the program must be fed with drone photos from field experiments. "We took thousands of images over one growth period," explains the doctoral researcher. "In this way, for example, we documented the development of cauliflower crops under certain conditions."

The researchers then trained a learning algorithm using these images. Afterwards, based on a single aerial image of an early stage of growth, this algorithm was able to generate images showing the future development of the crop in a new, artificially created image.

The whole process is very accurate as long as the crop conditions are similar to those present when the training photos were taken. Consequently, the software does not take into account the effect of a sudden cold snap or steady rain lasting several days. However, it should learn in the future how growth is affected by influences such as these—as well as an increased use of fertilizers, for example. This should enable it to predict the outcome of certain interventions by the farmer.

"In addition, we used a second AI software that can estimate various parameters from plant photos, such as crop yield," says Drees. "This also works with the generated images. It is thus possible to estimate quite precisely the subsequent size of the cauliflower heads at a very early stage in the growth period."

Focus on polycultures

One area the researchers are focusing on is the use of polycultures. This refers to the sowing of different species in one field—such as beans and wheat. As plants have different requirements, they compete less with each other in a polyculture of this kind compared to a monoculture, where just one species is grown. This boosts yield. In addition, some species—beans are a good example of this—can bind nitrogen from the air and use it as a natural fertilizer. The other species, in this case wheat, also benefits from this.

"Polycultures are also less susceptible to pests and other environmental influences," explains Drees. "However, how well the whole thing works very much depends on the combined species and their mixing ratio."

When results from many different mixing experiments are fed into learning algorithms, it is possible to derive recommendations as to which plants are particularly compatible and in what ratio.

Plant growth simulations on the basis of learning algorithms are a relatively new development. Process-based models have mostly been used for this purpose up to now. These—metaphorically speaking—have a fundamental understanding of what nutrients and environmental conditions certain plants need during their growth in order to thrive.

"Our software, however, makes its statements solely based on the experience they have collected using the training images," stresses Drees.

Both approaches complement each other. If they were to be combined in an appropriate manner, it could significantly improve the quality of the forecasts. "This is also a point that we are investigating in our study," says the doctoral researcher. "How can we use process- and image-based methods so they benefit from each other in the best possible way?"