PanoTac: A Dense Three-Axis Tactile Sensor for Robust Touch-Centric Manipulation

Binglin (Andre) Wang^1,†Xiaole (Alex) Jiang^2,†Zicheng Teng¹Xinqi Liu¹Yunzhu Li³Lerrel Pinto¹Zichen Jeff Cui^1,‡Binghao Huang^3,‡

¹ New York University² City University of New York³ Columbia University

† Equal contribution.‡ Equal advising.

External Camera (not policy input)Tactile visualization

Contact-rich manipulation benefits from both normal and shear feedback.Test Tube Preparation Demo (Autonomous, 8×)

Explore the demos

Abstract

Humans rely on vision and touch to manipulate everyday objects, using tactile feedback to determine how an interaction should proceed. When scooping grains, feeling the handle’s shape helps us infer spoon orientation, while resistance transmitted through the grasp guides our motion. These decisions depend on two essential aspects of touch: detailed contact geometry and interaction shear forces. To bring these sensing capabilities to robots, we present PanoTac, a low-cost, compliant tactile fingertip designed for dense three-axis tactile sensing. Each finger provides a 12 × 32 taxel grid, with each taxel representing three components: normal (z) and shear (x, y). Sensor characterization quantifies response repeatability and consistency across sensor instances and contact locations. We integrate PanoTac into a robotic system with a visuo-tactile learning pipeline. Across test tube preparation, grain scooping, and salt pouring, the resulting policies outperform vision-only baselines by an average of 46.7 percentage points. Ablations show that removing either spatial contact information or load feedback reduces overall performance across the task suite.

Summary Video

01 / Our Tactile Hardware

Dense three-axis tactile sensing in a compliant fingertip.

12 × 32 taxelsUp to 100 Hz$9.4 per sensor

Design & assembly

Sensor Design

Sensor Assembly

Tactile Signal Construction

Piezoresistive and magnetic measurements combined into a dense three-axis tactile field. — **Sensor Signal**

Sensor Response

Interactive visualization

02 / Data Collection System

VR Teleoperation with Tactile-Driven Audio Feedback (Play with sound to hear the feedback.)

Through VR teleoperation, we record robot actions alongside wrist-camera views and tactile observations. Changes in tactile signals drive the pitch and volume of audio channels, helping the operator hear contact changes during manipulation.

03 / Capability Experiments

Test tube preparation

Tactile feedback guides tube reorientation and insertion, then detects stopper release during removal.

90%

Grain scooping

Contact patterns reveal the occluded spoon’s orientation to guide scooping, transfer, and return.

85%

Salt pouring

Tactile weight cues distinguish filled from empty opaque bottles to decide whether to pour.

100%

04 / Failure modes

Representative failures of the baseline policies on each task.

The robot fails to insert the test tube into the rack opening.

The robot lifts the tube before the stopper has been released.

The tube slips from the grasp as the robot lifts it too early.

The robot continues wiggling even after the stopper has been released.

The stopper falls off during wiggling rather than being removed in a controlled motion.

The robot attempts to scoop with the spoon facing the wrong direction.

The robot moves the spoon through the grains but fails to scoop them up.

The robot skips pouring into the jar and returns the filled spoon to the tray.

The spoon catches on the jar during the return motion, dragging the jar over.

The robot discards a bottle that still contains salt.

The robot fails to discard the bottle after it is empty.

The robot stops pouring before all of the salt has left the bottle.

The robot attempts to pour from an empty bottle instead of discarding it.

05 / Experiment Results

Removing either spatial contact information or load feedback reduces overall success across the task suite.

Overall full-task success (%)

Vision only45.0%

Vision + dense normal array73.3%

Vision + sparse three-axis78.3%

Vision + PanoTac Ours91.7%

Indentation experiment setup and normal and shear response curves comparing three PanoTac sensors. — Sensor characterization: experimental setup and consistent normal and shear responses across three sensors.

06 / Bonus Task

We pour a slightly more dangerous material: water. Humans can often tell whether a bottle is empty just by lifting it. With PanoTac, our robot gains the same capability. The policy also generalizes to unseen water colors and to couscous, despite its different density and pouring behavior.

Acknowledgements

We thank Venkatesh Pattabiraman, Zifan Zhao, Bowen Tan, Hengkai Pan, Kanad Patel, Yibo Yan, Nikhil Bhattasali, and Kevin Wu for helpful discussions.

BibTeX

@misc{wang_panotac,
  title  = {{PanoTac}: A Dense Three-Axis Tactile Sensor for Robust Touch-Centric Manipulation},
  author = {Wang, Binglin (Andre) and Jiang, Xiaole (Alex) and Teng, Zicheng and Liu, Xinqi and Li, Yunzhu and Pinto, Lerrel and Cui, Zichen Jeff and Huang, Binghao},
  url    = {https://panotac.github.io/}
}