Special objects that are difficult to grab robots

To make robotic capture more reliable, researchers are developing objects that are as complex as possible to manipulate.

Inspired by the concept of competitive images, researchers from the University of California at Berkeley study physical competitive objects carefully designed to be difficult to capture with standard robotic grips



Recently, a huge amount of research has appeared on the topic of "competitive images", which are images of objects that are modified so that it is especially difficult for computer vision algorithms to recognize them. The idea is to use these images to develop more robust computer vision algorithms, since their “adversarial” nature is a specially implemented “worst” option — and if your algorithm can deal with adversarial images, then it will probably handle most other tasks.



Researchers from the Laboratory for Automation and Engineering ( AUTOLAB ) at the University of California at Berkeley, led by Professor Ken Goldberg, have extended this concept to robotic grips. They specifically designed physical competition objects so that they were difficult to lift with conventional robotic arms. You just need to slightly correct the usual three-dimensional figures, and the standard two-finger grip will encounter all sorts of difficulties, trying to properly grasp the object.





Starting from a cube, a competitive object evolves, becoming more and more difficult to capture with a two-finger clamp - when it tries to squeeze an angular surface, the object rotates and slips out.



The main feature of these competitive objects is that they look like they are easy to grab, but with at least a two-finger clamp, it turns out to be difficult. The difference between how the objects look and their real geometry is insignificant: in one example, this is a cube with low pyramids on three of the six sides - the smallest of them has an edge angle of 10 degrees. And opposite each pyramid is an ordinary flat face, so the cube does not have opposing flat faces. Because of this, the two-finger grips working on the clamp of objects encounter problems, because if you try to clamp the surface at an angle, the compressive force will cause the object to turn, and it will often fall out of the clamp.





Parallel capture with point contacts successfully captures a regular cube (left) and cannot cope with a competitive cube developed by researchers (right)



Capture planners often look for smooth opposing surfaces that can be clamped, and since the difference between a competitive and a regular cube is small enough for a 3D sensor to miss, most robotic systems will think: “Oh look, this is a cube, it's very easy ! ”And then they will surely fail the task when the cube gets out and slips out of their grip.





Adversarial forms work in humans too - wearing thimbles that emulate the cold, insensitive steel of the robot, and using two fingers to clamp the object, the researchers confirmed that it is difficult to lift these objects



With the increasing complexity of the form, it becomes increasingly difficult to develop a competitive version. Taking the cuboctahedron (a polyhedron with eight triangular and six square faces), the researchers randomly changed the vertices of the figure (in the simulation) until they got a shape that does not have directly opposite faces. And for more complex forms, such as intersecting cylinders, competitive examples were obtained using the deep learning algorithm.





For more complex forms, such as intersecting cylinders, competitive examples were obtained using deep learning algorithms



In preliminary tests in the real world, parallel point capture attempted to lift some of these controversial objects. In each case, the estimated probability of capturing an object was 100% (based on the Dex-Net policy tested in the simulation), but in reality it turned out that successful captures of competitive cubes and cuboctahedra were successful only in 13% of cases.



Researchers say they plan to test these objects with various types of grips, as well as with vacuum grips, to see if they can create the most competitive objects of all. And remember that the goal of all this is not just to fool the miserable capture algorithms, but to develop tools that will help make the robotic capture work sufficiently (hopefully) reliable for the real world.