Robots in the field

In June 2015, the three founders of Dogtooth Technologies decided to start working on an innovation that would have an impact in the real world. With its large revenues and small margins, the agricultural industry appealed to them and they believed that there was real scope to increase productivity and decrease waste. In 2018, almost 60% of agricultural businesses experienced a shortage of workers, leaving crops unpicked. Robot workers could be the solution. After exploring dozens of ways of using computer vision in the sector, they decided that a robotic system to pick strawberries could add the most value per metres moved or unit of time.

The founders all have backgrounds in software engineering and computer vision, and the project has involved engineers from most major disciplines: mechanical, electronic, software, and firmware engineers have all contributed. Dogtooth designs and manufactures its own robot arms and cameras, and the system’s machine-learning and computer vision software is bespoke. The company developed a few prototypes, four of which have been working in the field for 18 months in the UK and Australia. Now the robots are ready to get to work on a wider scale. They are designed to work on the same farms as human pickers using existing infrastructure.

A 3D camera system mounted to the robot arms surveys a row of strawberries to detect a single strawberry and locate it. The robot arm moves in to inspect the strawberry and decide whether it is ready for picking. The device uses machine learning to detect ripeness based on the colour of the fruit, before considering other factors, such as whether it can be picked without damaging nearby fruit and whether the arm needs reconfiguring to pick it. Once the robot has decided to pick the fruit, the robot arm snips the strawberry’s stalk just above the leaves, never touching the strawberry itself.

After the strawberry has been picked, it goes through a quality control step where the robot reviews the fruit under controlled lighting and inspects it for 17 types of defect, such as bruising or mould. It also measures its mass and checks for ridges and asymmetry, before sorting the strawberry into different punnets based on quality. The picker also ensures that punnets have the correct weight of fruit in them, whereas when strawberries are picked by people the punnets are normally sorted later to ensure that they weigh the right amount. The robots work at approximately half the speed of a person, but can work for twice as many hours and save time on sorting and packing.

One of the key challenges was the control software for the arm. Dogtooth needed to optimise route-finding, which is very different to traditional robotics – for example, robots that build cars follow set routes repeatedly, whereas Dogtooth’s robots are constantly observing and making decisions about what to do next. The camera can ask the arm to move anywhere within the robot’s reach, so it needed to be programmed to ensure that it did not crash into itself or plants and to move efficiently. The robot is always learning and collects data (about yield for example), labels data and trains algorithms. The strawberries can be traced from flower to the packing house, and the company collects data on strawberries every two days for nine months of the year.

This summer, 24 strawberry-picking robots will be picking fruit in the UK before spending winter in California and Australia. Then, Dogtooth will begin work on robots that pick other soft fruits, such as raspberries.

For more information, visit their website. 

***
This article has been adapted from "Robots in the field", which originally appeared in the print edition of Ingenia 78 (March 2019).