With the advancement of robotic technology from "pre-programmed execution" to "embodied intelligent interaction," the physical interaction capability between robots and their environment has become a critical bottleneck limiting their autonomy and adaptability. Tactile perception, as a core sensing method for robots to understand object properties, achieve fine manipulation, and ensure human-machine safety, is increasingly highlighted. However, current robotic tactile systems still lag far behind humans in perception dimensions, resolution, and signal comprehension, making it difficult to support complex, dynamic real-world tasks. Recently, a team led by Ding Wenbo from the Shenzhen International Graduate School of Tsinghua University, in collaboration with multiple research institutions, drew inspiration from pigeons' exceptional multispectral vision and non-imaging perception mechanisms to propose a biomimetic multimodal tactile sensor (SuperTac). This sensor integrates multispectral imaging, triboelectric sensing, and inertial measurement, and enables tactile signal comprehension and reasoning through the construction of a tactile language model (DOVE), aiming to propel robotic tactile perception toward "human-level" capabilities. It provides a new-generation tactile solution for fields such as intelligent manufacturing, medical assistance, and service robotics. The related findings were published in the first issue of *Nature Sensors*, marking the first article by a domestic institution as the first author unit in this journal.

Research Background

In the field of tactile sensing technology, the existing mainstream solutions primarily include electronic skin and vision-tactile sensors, but both exhibit significant shortcomings:

Although electronic skin sensors can achieve multimodal perception through multifunctional materials, improving spatial resolution relies on dense electrode arrays, which often leads to signal crosstalk, system complexity, and reduced stability, making it difficult to balance high resolution with multimodal fusion;

Visual-tactile sensors achieve sub-millimeter resolution through optical imaging and are easily integrated with computer vision models. However, their perception spectrum is typically limited to visible light, lacking the capability to fuse non-imaging modalities such as temperature, material properties, and proximity, which restricts their comprehensive sensing in multi-physical field environments;

Currently, tactile systems generally suffer from weak tactile signal interpretation capabilities, lacking intelligent models capable of integrating multimodal tactile information and performing semantic reasoning, resulting in robots being "aware but ignorant," making it difficult to achieve human-like tactile cognition and interactive decision-making.

Design and Testing

Biological Inspiration Sources

Inspired by the exceptional visual system of pigeons, this work draws on the division-of-labor mechanism among multiple types of cone cells in their retinas, particularly the ultraviolet-sensitive cells, to extend the spectral sensing range of sensors. Additionally, it simulates the specialized molecular mechanism in the pigeon retina responsible for magnetic field perception, transferring non-imaging sensing capabilities to the tactile sensing domain, thereby overcoming the limitations of traditional visual sensing.

Inspired by the principles of parallel processing and synergistic integration of multisensory information in biological neural systems, a multimodal sensing physical system was constructed: the ultraviolet band enables marker tracking and sliding recognition; the visible light band facilitates color and environmental visual perception; the near-infrared band specializes in texture and contact force distribution detection; and the mid-infrared band is responsible for temperature field measurement.

The frictional triboelectric sensing module, designed with a transparent conductive PEDOT:PSS layer, identifies materials and achieves proximity sensing through contact electrostatic charge differences. It integrates an MPU6050 inertial measurement unit to collect real-time three-dimensional posture and acceleration data, enabling collision detection and vibration analysis. Finally, an air-pressure-adjustable silicone inflation support structure forms an adaptive sensing skin, dynamically adjusting the force perception range from 0 to 7N to achieve high-fidelity contour reconstruction of complex curved surfaces.

Perceptive Skin Design

The sensing skin adopts a four-layer film stacking structure with a total thickness of only 1mm: the conductive layer is composed of a PEDOT:PSS/TPU composite film, designed with vortex line electrodes to achieve uniform signal distribution, combining high transparency and excellent conductivity; the fluorescent layer uses ultraviolet fluorescent ink, which becomes visible under UV light and transparent in the near-infrared range, enabling mode switching and marker tracking; the reflective layer consists of a silver powder/silicone composite material, featuring a one-way mirror effect that allows transparent/opaque switching across different wavelength bands via light intensity control; the support layer employs a pneumatically adjustable silicone inflatable film, not only providing mechanical support but also offering superior mid-infrared transparency compared to traditional acrylic.

Based on the light-controlled transparency of the reflective layer, the system achieves intelligent working mode switching: in tactile mode, the internal light source activates, rendering the film opaque for precise detection of surface texture and force; in visual mode, the internal light source turns off, transitioning the film to a transparent state that permits direct observation of external ambient light, forming a unique optical field modulation mechanism.

Source: Sensor Expert Network. If there is any infringement, please contact us for removal