Among the buzziest consumer technologies right now are “mixed reality” or “spatial computing” headsets that convincingly blend views of the real world with digital content.
A key enabling technology behind these gizmos is passthrough video, which involves blocking out all light so users must rely on cameras on the headsets to see the external world around them via real-time video playing on tiny screens. The arrangement allows users to physically interact with their environments and go about daily activities but with added digital content displayed, ranging from familiar device apps to innovative gaming scenarios. If tech companies' visions come true, users would wear these headsets for extended periods, even all day long at work and at home, ushering in new modes of human-computer and social interaction.
To put passthrough video through its paces, a diverse team of Stanford researchers recently conducted field tests alongside longitudinal analyses of their personal journeys and interpersonal interactions. As described in a new study in Technology, Mind, and Behavior, overall user experiences proved—fittingly enough—mixed, with moments of both awe and unsettlement. The researchers accordingly recommend caution regarding prolonged headset use and call for a longer-term assessment.
“Given how far headsets with passthrough video have come, it's time to dedicate serious academic thought to the psychological and behavioral effects of this technology,” said Jeremy Bailenson, the Thomas More Storke Professor in the Stanford School of Humanities and Sciences and founding director of the Virtual Human Interaction Lab (VHIL). “We want to understand the implications of living in a life in which we rely on passthrough for hours every day to see the world around us.”
Pros of passthrough
For the study, 10 research scholars in the VHIL and Bailenson himself spent at least 140 minutes over two or three sessions wearing Meta Quest 3 passthrough video headsets, which became widely available in October 2023.
The researchers engaged in a wide range of activities such as having conversations, walking outdoors, playing games, and eating and cooking food. For safety reasons, given concerns about potentially tripping over objects or encountering moving people or vehicles, a chaperone not wearing a headset remained present at all times.
The study participants attempted to examine the experience from both a hands-on, subjective perspective as well as a removed, clinical view. “We took an observational approach, more akin to naturalists, and really dove into the medium in an exploratory way,” said study co-author James Brown, a master's student in the Symbolic Systems Program.
In general, the researchers found they enjoyed many aspects of having reality filtered through passthrough. “For a lot of us, wearing a headset in public was exciting,” said study co-author Monique Tania Santoso, a doctoral student in the Department of Communication.
“It was a very novel experience being in these headsets while walking around campus, interacting with strangers, and even buying coffee,” said co-author Portia Wang, a second-year master's student in the Management Science and Engineering Department studying computational social science.
As for Bailenson, who has long followed the development of passthrough video and recalls first donning a rudimentary device back in the late 1990s, the experience was “mind-blowing” in comparison.
Source: Stanford University