Spatiotemporal Tactile In-Hand Rolling: Enabling Object Identification & Pose Estimation for Low-Resolution Tactile Sensors

As robots increasingly transition into uncontrolled industrial and domestic environments, their ability to manipulate objects robustly remains a key bottleneck. While high-end anthropomorphic hands like the £100k Shadow Dexterous Hand have demonstrated feats such as one-handed Rubik's Cube solving, their cost and complexity render them impractical for deployable robots, which instead use simple grippers with limited in-hand manipulation (IHM). Compounding this is the perception problem: object pose tracking during manipulation is challenging due to occlusions and lighting constraints in vision-based systems, while tactile sensing, despite its complementary strengths, remains underutilized.

This work addresses these gaps by advancing cost-effective robotic hands and tactile-based perception for IHM. Drawing inspiration from human exploratory procedures, it focuses on tactile in-hand rolling-rolling objects between fingers to improve perception.

Key contributions include:
- Robotic hands: open-source, 3D-printed hands optimised for tactile in-hand rolling.
- Spatiotemporal tactile methods: techniques leveraging low-resolution, high-speed tactile arrays to extract contact features over time, enabling object identification, super-resolution contact-point estimation, and real-time capable pose tracking.
- Tactile sensor designs: Tactile sensors based on commercially available barometric pressure sensors, with a practical focus on ease of fabrication, repairability, and customizability.
Together, this project demonstrates that low-cost hardware and spatiotemporal tactile perception can overcome longstanding barriers in robotic IHM, paving the way for robust manipulation in real-world settings.