Can (Agricultural) Robots Get A Grip?

Podcasts are awesome. Robots too. This episode from Steve Levitt’s podcast People I (Mostly) Admire entitled “Can Robots Get A Grip?” was the catalyst for this post: Can Robots Get a Grip? – Freakonomics.

The podcast is mainly about the work of Prof. Ken Goldberg at U.C. Berkeley in training robots to grasp things. This was his PhD, and what he still does. As the discussion describes, the central issue that he is grappling with is that there is noise in the system. For example, friction doesn’t always act in the way the model suggests, or there is a grain of sand in the way that adds an unpredictable force to the system, or the sensor used to see things is just not that good.

A valuable point made by Levitt is that the success we’ve had in mechanisation, especially mechanisation in agriculture, comes largely from the fact that these systems doesn’t mimic what the human did. We often redefine the task or even the system so that it fits the mechanical option. For example, we used to pull up weeds, then we had hand hoes that we used to target the weeds, then we used multiple row mechanical hoes towed behind tractors in crops sown in very neat rows. Many crops have been breed specifically to tolerate a mechanical system. No robots here, but an important point in the economic history of /adoption of technology in / innovation in agriculture.

Back to the robots. Goldberg tested tactil sense to control a grasp: using sensors to understand if the robot hand has made contact with something. Didn’t work – too many false positives (stopped grasping in mid-air) and negatives (squashed the object it was trying to grab). He’s probably tested a heap of other things too, but he gets quickly to the point in his career where robots are being trained on ML / AI type models. The big constraint here is training data – it just doesn’t exist like it does for the Large Language Models. So to create some data they used some virtual models, focusing on the likelihood of a successful grip on an object when testing of every possible pair of grip points. The genius was that they added randomness to the data: the training set was too perfect and didn’t replicate the real world and adding noise made it more realistic. This more noisy data was then use to train statistical models.

Goldberg and his team then successfully used these statistical model to create a real-life robot that was pretty OK and put into commercial use. This then allowed them to collected a heap of data off these robots to train the next generation of “robot brain” models on actual data.

Two last important points: First, all this needed to be coupled with some high quality old-fashioned engineering: making sure that the right data is being collected and the right quality machines being constructed. Secondly, there is a big difference between training models on fine motor skills compared to predictable gross motor skills. Robots that walk are much easier to make than robots that tie a shoelace. A robot that drives in a straight line is even easier.

So now, agricultural robots

Well, first, it’s questionable if robots will ever need to get a grip. There are fruit picking robots, but they usually use suction or catch a fruit they have cut loose. Not an easy problem that has been solved – finding fruit in 3D space that is of the right ripeness – but the robot are not delicately grasping and twisting or anything like that. Again, the method or system is changed to suit the mechanical solution.

There are some autonmous systems in use in agriculture that have amazing accuracy. Verdant Robotic’s Sharp Shooter or Carbon Robotic’s Laser Weeder spring to mind. But even with these awesome machines, the task is still firmly in the class of gross motor skills category (point and shoot a stream of liquid/ light energy called a “laser”).

Randomness. This was the really interesting part of this discussion. Agriculture is not a well controlled environment. Taking from the discussion in the podcast, building models on exact values seems pointless. This suggests that we should try more in dealing with the randomness, not trying to control it. Admittedly, it is rare that data collected from an agricultural system is highly precise, but that not the point. Or is it?

This rant was originally posted 2025-10-14. Updates likely.