Development of machine vision system capable of locating king flowers on apple trees

Apple blossoms grow in groups of four to six blooms attached to branches, and the center blossom is known as the king flower. This flower opens first in the cluster and usually grows the largest fruit. So, it is the key target of a robotic pollination system, according to researcher Long He, assistant professor of agricultural and .

Insect pollination has traditionally been relied upon for apple productivity. However, evidence suggests that pollination services, both from domesticated honeybees and wild pollinators, is not matching increasing demands, He noted. Due to , honeybees around the world have been dying at alarming rates. As a result, producers need alternative methods of pollination.

This study is the latest conducted by He's research group in the College of Agricultural Sciences, which is devoted to developing robotic systems to accomplish labor-intensive agricultural tasks such as mushroom picking, apple tree pruning and green-fruit thinning. The primary goal of this project, He explained, was to develop a deep learning-based vision system that could precisely identify and locate king flowers in tree canopies.

"We think this result will provide baseline information for a robotic pollination system, which would lead to efficient and reproducible pollination of apples to maximize the yield of high-quality fruits," He said. "In Pennsylvania, we still can rely on bees to pollinate apple crops, but in other regions where bee die-offs have been more severe, growers may need this technology sooner than later."

Training the machine vision system to locate king flowers was challenging because they are the same size, color and shape as the lateral blossoms in clusters, and the king flowers are typically obscured by surrounding flowers because of their central position. Raw images were labeled in two pre-defined classes: individual flowers and occluded flowers. Credit: Penn State. Creative Commons

The image augmentation process to enlarge the dataset, aimed at increasing the machine vision's precision, included rotating, cropping, scaling and flipping photos like those above. The vision system automatically located the flower clusters separately based on a two-dimensional flower density mapping approach. Credit: Penn State. Creative Commons

The image-acquisition system with a camera was mounted on a utility vehicle maneuvered between tree rows. Credit: Penn State. Creative Commons

In the machine vision system, the masks of each detected apple flower are separated from the background. Credit: Penn State. Creative Commons