SAPIENS in XR: operationalizing interaction-attention in extended reality

We examine peripheral interactions in XR environments, for which we propose a conceptual space with two specialized dimensions, interaction-attention and reality-virtuality. We also formalize the notion of an XR display to expand the application range of ambient displays from physical environments to XR. To operationalize these conceptual contributions for researchers and practitioners, we capitalize on Sapiens, an open-source event-based software architecture for peripheral interactions in smart environments, to propose Sapiens-in-XR, an extended architecture that also covers XR displays. In a simulation study based on a Poisson probabilistic model of notification delivery, we demonstrate the efficiency of the event processing pipeline of Sapiens-in-XR with an average processing time of just 18ms from event creation to delivery. We present simulations of peripheral interaction scenarios enabled by our conceptual space and Sapiens-in-XR, and report empirical results from a controlled experiment implementing one scenario, where users were asked to maintain their focus of attention in the central field of view while notifications were displayed at the attention periphery. Our results show similar user perception and the same level of user performance for understanding and recalling content of notifications in either the virtual and physical environments. Our conceptual space, software architecture, and simulator constitute tools meant to assist researchers and practitioners to explore, design, and implement peripheral interactions in XR.

Automated summary

We propose a conceptual space with two specialized dimensions, interaction-attention and reality-virtuality, to examine peripheral interactions in XR environments. We extend the application range of ambient displays to XR by formalizing the notion of an XR display. For researchers and practitioners, we introduce Sapiens-in-XR, an extended architecture that covers XR displays and capitalizes on Sapiens, an open-source event-based software architecture.