Vision models had billions of images to learn from. Touch models start with much less, and the data they do have comes from sensors that record contact in incompatible ways. Meta’s Sparsh, released October 31, 2024, is one attempt to deal with both problems at once.

The problem

The Sparsh paper lays out the obstacle. Labeling real tactile data with ground truth “like contact forces and slip” is hard to do at scale. That difficulty is “further compounded by sensors of various form factor differing in aspects like lighting and gel markings.” As a result, the authors write, current systems “often rely on task and sensor specific handcrafted perception models.”

What Sparsh does

Sparsh is a family of models for vision-based tactile sensors, the kind that use a camera to image a soft gel surface as it makes contact. Rather than relying on labels, it learns through self-supervision: it was pretrained on more than 460,000 tactile images using masking and self-distillation, an approach borrowed from computer vision.

To measure it, the team built TacBench, a set of six tasks ranging from identifying tactile properties to physical perception and manipulation planning. The authors report that self-supervised pretraining beat task- and sensor-specific end-to-end training by 95.1 percent on average across TacBench. Of the variants, those built on the DINO and I-JEPA methods did best.

What the numbers cover

TacBench is the authors’ own benchmark, and 95.1 percent is an average improvement over their baselines, not a success rate on a robot task. Sparsh is also scoped to vision-based tactile sensors. Meta released it alongside new hardware, including the Digit 360 fingertip and the Digit Plexus hand platform, as described in its announcement.

Why it matters for humanoids

Humanoid makers have begun feeding touch into learned control policies. Figure says its Helix 02 model takes fingertip tactile signals as input alongside cameras and joint data, and that its sensors detect forces as small as three grams. Any company training such a policy faces the question Sparsh addresses: how to get useful touch representations without labeling every contact by hand. Figure’s post does not describe how its tactile data is collected or used in training.

Whether touch models trained on one sensor work on another, which is the question TacBench was built to test, and whether humanoid makers publish what tactile data they collect and how much of it there is.