The past two years of countless, contactless Zoom weddings, funerals, holidays, and hangouts have given a new urgency to an old dream: being able to make meaningful tangible contact through screens.
In a recent survey conducted by the National Research Council, 78 percent of respondents indicated that they miss the ability to physically touch when interacting with people virtually; 63 percent saw virtual reality as a means of compensating for the loss of physical interaction caused by the pandemic. Concerns about “skin hunger” have been pushed to the surface of our collective consciousness, but our existing digital interfaces don’t permit the simplest (though often most meaningful) forms of interpersonal social touch. For the past 50 or so years, the dream of the haptics field has been that a newly virtualized touch could both complement and challenge the visuality of computer displays, restoring a missing dimension to our interactions with and through computers.
While the “goggles and gloves” model of virtual reality has lingered in our cultural imagination since the 1980s, recent commercial VR hardware releases — like Meta’s Quest, Sony’s Playstation VR, and HTC’s Vive — have been more goggles than gloves: far more focused on graphical displays than on haptic interfaces. But according to proponents, some mix of devices that allow us to feel touch sensations in VR — haptic vests, gloves, armbands, helmets, shorts, or even shoes — will give rise to a powerful haptic epistemology capable of restoring balance to our mediated sensorium. Or as Oculus/Facebook/Meta chief scientist Michael Abrash put it in 2015, just as his design team was apparently beginning work on its glove, “the first haptic VR interface that really works will be world-changing magic on a par with the first mouse-based windowing system.”
The promotional videos pushed out by companies like Meta attempt to normalize the desire for haptics
If the promise of haptics is so self-evidently appealing, if our digital disconnection makes us ever more skin-hungry, why have haptics devices consistently failed to take hold? Is it simply, as many in the field suggest, a question of not-good-enough tech that will inevitably get better?
That’s what Meta and the half-dozen other companies bringing haptic gloves to market are betting on. In November, the world was given a glimpse of Meta’s glove in this promotional release, timed to the company’s announcement of its abrupt rebranding and big metaverse push: Call it a pivot to the hands. In the demo video, we see a woman wearing the glove handling objects in a virtual environment as if they have the customary properties perceptible to the fingers and hands: She grasps, lifts, and squeezes a ball, then tosses it into the air and catches it. She gingerly places a “rubber” duck atop a block, delighting in her success as she releases the object.
Bringing a second user into the demo, Meta gets to the heart of the dream, teasing the promise of being able to touch, feel, and be felt by another person inhabiting the virtual world. The two users wave, shake hands, and then play some irreducibly touch-dependent games such as thumb war and virtual Jenga, enjoying the pleasures of mutual haptic connection, despite their physical distance. The message is clear: In the physical world, they are far away, but in the virtual environment, thanks to the gloves, they can make tactile contact.
Throughout the demo, a split-screen view shows us what the user sees in virtual reality on one side and on the other side, the glove itself: a formidably dense tangle of wires, adorned with plastic and metal actuators — the company describes them as “tiny, soft motors” — that spasm and twist in response to collisions between the virtual hand and virtual objects.
The glove looks intricate and impressive; its apparent sophistication seems to suggest how thoroughly it can simulate the complex modalities of touch. But it also resembles the apparatuses that the earliest haptics researchers — 19th century “Brass Age” experimental psychologists — devised for their studies: a small point of metal that could be pressed against the skin and quickly heated or cooled; an exoskeletal cage for studying the maximum number of points on the body that could be perceived simultaneously; crude compasses to map thresholds of spatial discrimination. There’s a visible kinship between the apparatuses for simultaneous touches (circa 1892), which used large inflatable air pockets as an experimental stimulus, and Meta’s glove, which employs tiny computer-controlled air pockets pressed against the fingertips to create the illusion of haptic contact with virtual objects.
While those in the field (and the popular press) often link haptics back to the ancient Greek haptikos, this term was initially adopted by experimental psychologists in the 1890s to describe new scientific research on touch. Haptics as a label thus has the effect of simultaneously making the field sound newer and older than it is. This can obscure the fact that the structures of touch identified by this research have been gradually revealed over the past century, through particular modes of investigation that divide touch up into discrete submodalities, map the anatomical structures responsible for producing our tactile impressions of the world, and quantify our capacity to perceive touch sensations.
Haptics today, then, can be understood as both a normative model of touch produced by a specific epistemic orientation toward tactility that arose in the late 19th century and a set of technologies, built with this model, that allow computers to artificially stimulate touch.
Touch, as revealed by this science of haptics, is a highly nuanced mode of perception that amalgamates sensations and submodalities including pressure, weight, temperature, movement, vibration, texture, and pain. A square centimeter of fingertip skin contains several layers of densely clustered tactile receptors — more than 500 Merkel discs, 100 Meissner corpuscles, 10 Ruffini corpuscles, and 10 Pacinian corpuscles — each of which is uniquely attuned to detect particular types of tactile stimulation.
The key design challenge for VR gloves involves striking the right balance of haptic fidelity (the extent to which it renders the range of touch sensations such as weight, texture, contact, and even temperature), manufacturing cost, and wearer comfort. One early exoskeleton haptic glove, the CyberGrasp, achieved a high degree of haptic fidelity, but was cumbersome to use and costly to manufacture. (When it first went on the market more than 20 years ago, it sold for $39,000.) The video game wearables company bHaptics announced at the Consumer Electronics Show in January that it will sell haptic gloves for $300 per pair, but these rely only on a single vibration motor in each finger to feed tactile sensations to the wearer, with no capacity for force display. Offerings like the HaptX Gloves DK2 (which Meta’s gloves bears a striking resemblance to), SenseGlove’s Nova ($5,000) and ContactCI’s Maestro gloves ($5,000 per pair) walk a middle ground, mainly intended for industrial applications like VR training, virtual prototyping, and telerobotics. That golden ratio for haptics has proved enduringly elusive.
The bigger challenge, however, may be cultural, beyond the power of engineers to solve on their own. Potential users of haptic gloves need to be convinced en masse that the risk of turning their hands over to these bulky and cumbersome devices will be well worth the reward promised by marketers and true believers. The process of enveloping the body in a flood of digital signals is frequently represented as a clean process: Hollywood films show interfaces seamlessly sending touch to the body (The Matrix and Ready Player One provide our key cultural touchstones); corporate press releases promise the haptic realism of touching a blade of grass or rubbing an orange. But in practice, haptics is especially messy and meaty. The skin is a fraught and vulnerable surface: It flakes and tears with rough contact, it leaks sweat, it pulses uncomfortably in response to pressure, and its nerves are often benumbed from repeated and sustained stimulation. The sensations produced by current haptic gloves frequently retain an artificial and mechanistic feel. They don’t restore a naturalistic sensation of touch; rather they call attention to the simulation’s artifice, a sort of haptic uncanny valley. Moreover, the takeover of the hands by VR gloves leaves them unavailable for other routine and intimate tasks of bodily maintenance: adjusting our glasses (or a VR headset), scratching an itch, reaching for a drink or a snack, fixing our hair, or even simply idly fidgeting will our fingers each becomes more challenging — if not impossible — while wearing them gloves. The benefits of sending hands into simulated haptic space may not be worth the costs of surrendering them to the interface.
The glove looks intricate and impressive, which suggests how thoroughly it can simulate the complex modalities of touch
Perhaps more crucially, when enveloped in Meta’s gloves, the hands will ooze data. Every motion will be captured, every gesture will be mapped, and every haptic stimulus we respond to will be recorded. Expanding biometrics to include touch could thus enable a new mode of “haptic coercion,” as Dave Birnbaum, the former head of design strategy and outreach for Immersion Corporation terms it, where digital touch can be used to prod or nudge us into making purchasing decisions preferred by a brand or advertiser.
Immersion, which entered the haptics space in 1993 and has licensed its IP to a range of companies including Sony, Microsoft, BMW, Apple, and Samsung, has already experimented with using phone vibration cues in online ads to influence purchasing decisions. Research on touch and consumption in physical space suggests that handling an object makes us more likely to buy it, even influencing the price consumers are willing to pay. Touching a branded virtual good — wrapping your hand around that shiny sword in Fortnite or rubbing the fabric of a couch in the Walmartverse — could provide that push you need to buy it.
For a company like Meta, whose business model remains grounded in attention monetization, it’s key that this digitized relationship between contact, sensation, and consumption can be captured, aggregated, and analyzed, with touch data folded into surveillance capitalism’s expanding feedback loops. The company will have an idea of what and how you like to touch and how you respond to different kinds of contact. Did the virtual texture of that shirt you’re shopping for — rendered by the glove’s tiny actuators — entice you to continue touching it and eventually purchase the physical version? If not, would slightly changing that virtual texture encourage more active touch, elicit a pleasurable biometric response, and ultimately prompt you to buy the item? A recommendation algorithm could make suggestions based on tactile patterns you’ve responded positively to in the past, or customize the feeling of a virtual object to fit your unconsciously expressed touch preferences. Understanding the microphysical relationships between touch and consumption has proved exceptionally challenging, limiting the ability of marketers to drive purchasing through touch. Using VR gloves to capture that interface between hands and brands as data has the potential to enable an unprecedented form of haptic coercion.
On top of these concerns is the looming and persistent threat of harassment in virtual worlds, not a new issue at all, but one that’s already made its way into Meta’s virtual space. The company’s fraught track record with social media suggests it is ill-equipped to navigate this challenge as it pivots to VR. The allure of inserting our hands into gloves that give us the power to feel in VR may be blunted by the threat of feeling unwanted and unpleasant touches — nonconsensual, aggressive, and even violent modes of contact enabled by these devices. While hapticians and other touch evangelists present the desire for touch as universal, the raced, classed, and gendered power differentials often encoded and maintained through touch mean that for some, interfaces that screen touch out provide a welcome respite from the lingering threat of physical harassment. Regimes of touch are maintained, performed, and policed through everyday acts of social touching, as considerations of who is allowed to touch whom, who can refuse touch without being formally or informally sanctioned, and which types of touching are acceptable and unacceptable structure our experiences of bodily autonomy. The answer to the question of “will you be punished for refusing an unwanted handshake or hug from a client?” often depends on one’s position in an intersection of social hierarchies. Meta’s recent announcement that it would introduce a four-foot “personal boundary” in its Horizon Worlds platform (notably absent any sort of tactile cue that would let you know when another avatar collided with your boundary) demonstrates the way we often turn to technological patches for social problems, while also illustrating the company’s power to shape regimes of touch in this new social space.
Watching haptics develop — and frequently stagnate — over the past three decades, I’ve continually wondered about the technology’s capacity to live up to the sweeping, transformational promises made around it. Is the cultural desire for robust and complex haptics — beyond the mundane haptics already in our phones, wearables, and game controllers — widespread enough for these devices for them to achieve the scale necessary for Abrash’s “world-changing magic”? And if such widespread desire manifests — if someone finds that golden ratio of ergonomics, haptic fidelity, and cost — will haptics be liberatory, restorative, and humanizing, or will it ultimately prove confining, exploitative, and dehumanizing?
The benefits of sending hands into simulated haptic space may not be worth the costs of surrendering them to the interface
I suspect there’s more cause for pessimism than optimism on this front. Any unifying and standardizing haptic device that emerges from the coming chaos of product releases will have to either be funded by device sales or underwritten by advertising revenue. In either case, the design process is entangled with intended use cases, applications, and business models. It can’t not be. Consequently, whatever version of touch gets extended into virtual worlds will be shot through with these sorts of instrumental considerations, bringing normative ramifications for the types of bodies and the types of touch privileged by the design process. Decisions about the gloves’ sizing and fit, the sensitivity of their motion capture algorithms to registering involuntary hand movements like tremors, and the intensity of stimuli they provide (what counts as painful or uncomfortable touch?) each embody standardizing assumptions about the hands of target users.
Given touch’s impossible complexity, any attempt to digitally remake it will be necessarily incomplete and fragmentary. Will temperature be left out? What about weight, texture, or even pain? The form of that incompleteness matters, because those choices configure the sensations we will and won’t feel in VR. As one of our foundational epistemic building blocks, touch shapes our sensory map of the external world. If haptic simulation technologies succeed at supplanting the haptic real — similar to the way manipulated and simulated images and sounds have gradually supplanted the visual and auditory real — this new haptic real will be missing crucial components of the original it’s derived from.
Despite the field’s failures to make good on its longstanding promises of social transformation, haptics has succeeded in getting us to attend more closely to questions of touch. It’s worth reflecting on the types of touch tech we’d be comfortable incorporating into our everyday practices of mediated communication. The promotional videos pushed out by companies like Meta, and the accompanying puff pieces that circulate in the popular technology press, attempt to normalize the desire for haptics. But given the relentless demands digital devices already make on our attention, we may ultimately opt to resist these efforts at taking over our hands.