When you grip a heavy object, like a pipe wrench, your hand naturally uses the entire surface of your fingers, not just the fingertips. This action triggers sensory receptors in your skin to send information to your brain about the object you’re holding.
In the realm of robotics, tactile sensors that rely on cameras to gather data about grasped objects tend to be flat and small, usually positioned at the fingertips of robotic hands. These robots typically grasp objects using a pinching motion, limiting their versatility in various manipulation tasks.
Researchers at MIT have introduced a novel camera-based touch sensor known as the GelSight Svelte. This sensor is unique in its design, resembling a human finger with a long, curved shape. What sets it apart is its ability to provide high-resolution tactile sensing over a large area. The GelSight Svelte employs two mirrors to manipulate light in such a way that a single camera, positioned at the base of the sensor, can observe the entire length of the artificial finger. This breakthrough has been detailed in a pre-print publication on arXiv.
Additionally, the researchers crafted the finger-shaped sensor with a flexible backbone. By monitoring how this backbone flexes when the artificial finger touches an object, they can estimate the force applied to the sensor.
The GelSight Svelte sensors were used to create a robotic hand capable of gripping heavy objects in a manner akin to a human hand, utilizing the entire sensing area of all three artificial fingers. Furthermore, this robotic hand retained the ability to perform traditional pinch grasps seen in conventional robotic grippers.
Alan (Jialiang) Zhao, a graduate student in mechanical engineering and the lead author of the GelSight Svelte research paper, expressed the significance of this innovation: “Because our new sensor is human finger-shaped, we can use it to do different types of grasps for different tasks, instead of using pinch grasps for everything. There’s only so much you can do with a parallel jaw gripper. Our sensor really opens up some new possibilities on different manipulation tasks we could do with robots.”
To overcome the limitations of cameras used in tactile sensors, which are typically small and flat due to size and shape constraints of robotic grippers, Zhao and his team devised a solution involving two mirrors. These mirrors manipulate light in a manner that allows the camera, situated at the sensor’s base, to capture the entire length of the artificial finger.
The GelSight Svelte incorporates two types of mirrors: one flat, angled mirror opposite the camera, and one long, curved mirror along the sensor’s back. These mirrors alter the path of light rays from the camera, enabling it to observe the entire length of the artificial finger. To optimize the mirrors’ shape, angle, and curvature, the researchers developed software to simulate light reflection and refraction.
The sensor, including the mirrors, camera, and LED arrays for illumination, is housed within a flexible skin made from silicone gel and attached to a plastic backbone. The camera peers at the inner side of the gel skin, allowing it to detect contact points and measure the object’s contact surface geometry based on skin deformation. Moreover, red and green LED arrays indicate the degree of gel compression when an object is gripped, aiding in the reconstruction of a 3D depth image of the grasped object.
The plastic backbone also serves to capture proprioceptive information, such as twisting torques applied to the artificial finger when gripping objects. Machine learning algorithms are employed to estimate the applied force based on these backbone deformations.
However, bringing all these components together into a functional sensor proved to be a challenging endeavor. Zhao mentioned the difficulty of achieving the correct curvature for the mirrors to match the simulation, as well as the discovery that certain types of superglue hindered the curing of silicone. Consequently, extensive experimentation was required to create a functional sensor.
Versatile grasping
After fine-tuning the design, the GelSight Svelte underwent rigorous testing. Researchers pressed various objects, including screws, onto different parts of the sensor to assess image clarity and its ability to discern object shapes accurately.
They didn’t stop there; they harnessed three GelSight Svelte sensors to construct a robotic hand capable of executing a range of grasps. This included the familiar pinch grasp, a lateral pinch grasp, and a powerful grip that engaged all three fingers’ sensing areas. In contrast, most robotic hands, typically resembling parallel jaw grippers, are limited to pinch grasps.
The three-finger power grasp, in particular, empowers a robotic hand to securely hold heavier objects. Nevertheless, pinch grasps remain valuable for handling very small objects. The ability to switch between these grasp types with a single robotic hand enhances a robot’s versatility.
Looking ahead, the researchers have aspirations to further enhance the GelSight Svelte. They intend to make the sensor articulated, allowing it to bend at joints more akin to a human finger’s flexibility.
Monroe Kennedy III, an assistant professor of mechanical engineering at Stanford University who was not involved in this research, remarked on the significance of these optical-tactile finger sensors. He noted that they enable robots to utilize cost-effective cameras to capture high-resolution images of surface contacts. By monitoring the deformation of a flexible surface, the robot can estimate contact shapes and applied forces. Kennedy stressed that this work represents an advancement in GelSight finger design, offering improved full-finger coverage and the capacity to estimate bending deflection torques through image analysis and machine learning.
Enhancing a robot’s sense of touch, bringing it closer to human capabilities, is a crucial step in developing robots capable of handling intricate, dexterous tasks.