Research

Agricultural engineers design early step for robotic, green-fruit thinning

Prototype ‘end-effector' capable of deftly removing unwanted apples from trees

Lead researcher Magni Hussain, a doctoral degree student in the Department of Agricultural and Biological Engineering, linked the end-effector with a robotic manipulator, demonstrating the potential of a robotic green-fruit system to remove fruit at different locations and orientations.  Credit: Penn StateCreative Commons

UNIVERSITY PARK, Pa. — Penn State agricultural engineers have developed, for the first time, a prototype “end-effector” capable of deftly removing unwanted apples from trees — the first step toward robotic, green-fruit thinning.

The development is important, according to Long He, assistant professor of agricultural and biological engineering, because manual thinning is a labor-intensive task, and the shrinking labor force in apple production makes manual thinning economically infeasible. His research group in the College of Agricultural Sciences conducted a new study that led to the end-effector.

Prior to designing the end-effector, researchers performed a series of fruit-removal-dynamics tests on three different apple cultivars to determine the forces required for robotic thinning. The device for determining needed pulling force is shown here. Credit: Penn StateCreative Commons

The apple crop is a high-value agricultural commodity in the U.S., with an annual total production of nearly 10 billion pounds and valued at nearly $3 billion, according to He, who is a leader in agricultural robotics research, previously developing automated components for mushroom picking and apple tree pruning. Green-fruit thinning — the process of discarding excess fruitlets in early summer, mainly to increase the remaining fruit size and quality — is one of the most important aspects of apple production.

“Eventually — in the next decade, we hope — this end effector will be combined with a machine vision component and a locomotion system, creating a mechanism that eventually will be able to accomplish robotic green-fruit thinning in apple orchards,” He said. “I realize this sounds wildly futuristic, but I think this is where we are headed, sooner than later, and this end-effector was the needed first step.”

The set-up for measurement of green fruit removal forces for stem cutting included a stem-holding mount (1), stem- cutting blade (2), digital force gauge (3) and force-transfer rod (4).  Credit: Penn StateCreative Commons

Prior to designing the end-effector, lead researcher Magni Hussain, a doctoral degree student in the Department of Agricultural and Biological Engineering, performed a series of fruit-removal-dynamics tests to determine the forces required for robotic thinning, using pulling or stem-cutting methods on three different apple cultivars. The work was done at Penn State’s Fruit Research and Extension Center in Biglerville, in south-central Pennsylvania.

After finding that fruit removal using the pulling method resulted in a high occurrence of spur-end stem detachment — which can leave a wound that affects neighboring fruits in a cluster — Hussain built a stem-cutting end-effector prototype and conducted extensive field tests. He tested two end-effector prototype configurations: one placing the end-effector on a handheld bar; the other integrating the end-effector with a robotic manipulator.

The success rates of green fruit removal for all end-effector-prototype experiments were over 90%, the researchers recently reported.  Credit: Penn StateCreative Commons

The success rates of green-fruit removal for all end-effector-prototype experiments were over 90%, the researchers recently reported in the Journal of the American Society of Agricultural and Biological Engineers. Linking the end-effector with the robotic manipulator demonstrated the potential of a robotic green-fruit system to remove fruit at different locations and orientations. Robotic green-fruit thinning can potentially be as selective as manual thinning, the researchers suggested.

Hussain and He are already looking to the next stage of the research and the work is complex. Hussain is now working on the machine-vision component of the overall robotic system, “trying to design it from the ground up, so to speak," he said. The challenge is significant, he conceded, because for such a vision system to work, there are several components to consider.

Researcher Magni Hussain (left) poses with the mechanism he built in the lab of Long He, assistant professor of agricultural and biological engineering (right), at Penn State’s Fruit Research and Extension Center in Biglerville, in south-central Pennsylvania. Credit: Penn StateCreative Commons

“First and foremost, it has to detect green fruit, and I've implemented a deep-learning algorithm into the system to enable it to not only detect the green fruit in a tree environment, but also generate their pixel masks,” Hussain said. “And then, on top of detecting the fruit, I'm also inserting an algorithm that will allow it to detect the orientation of the fruit. Because when you're trying to remove green fruit, knowing their orientations is important for properly orienting an end-effector for fruit removal.”

James Schupp, professor of pomology, and Paul Heinemann, professor of agricultural and biological engineering, also contributed to this research.

This research was supported in part by U.S. Department of Agriculture’s National Institute of Food and Agriculture.

 

Last Updated July 19, 2022

Contact