3D Interaction Group

EyeST: Eye-Enhanced Immersive Space to Think with Gaze-Driven Recommendations

The advancements in AI have raised intriguing possibilities for personalized experiences. Imagine if AI could anticipate your interests by analyzing your gaze patterns from the last hour of reading. What if it could recommend insights not just based on the pages you visited but on the specific parts of those pages that held your attention? The concept extends into immersive spaces. In XR, even though you get more space to organize massive amounts of information, you’d still have to exhaustively...

Planet Purifiers: A Collaborative Immersive Experience Proposing New Modifications to HOMER and Fishing Reel Interaction Techniques

This project presents our solution to the 2025 3DUI Contest challenge! We developed a collaborative, immersive experience that raises awareness about trash pollution in natural landscapes. In our experience, one user collects harmful pollutants while the other user provides medication to impacted wildlife using enhancements to traditional interaction techniques: HOMER and Fishing Reel. We enhanced HOMER to use a cone volume to reduce the precise aiming required by a selection raycast for more efficient pollutant collection, coined as FLOW-MATCH. To...

Guided Tours for Asynchronous Collaboration in Virtual Reality

While collaborative virtual environments have been investigated for working in different spaces at the same time, few have investigated the best practices for collaborating in those spaces at different times. We designed a system that allows experts to create engaging tours inside a virtual inspection space, preserving knowledge and insights while requiring observer interaction to follow the tour’s playback. A user study evaluating the influence of these interactions on an observing user’s information recall and user experience indicated that additional...

Towards Privacy-Aware AR: A Look into Mitigating Privacy Concerns in a Pervasive AR Future

Fig 1: An overview of our approach to creating a interactive privacy-aware AI assistant for everyday AR In this work, we present a practical design framework for a privacy-aware virtual assistant for everyday AR, focusing on educating users with a lack of knowledge regarding technical and/or privacy literacy. Our approach features human-in-the-loop to learn privacy context, provides transparency into the system state of privacy detectors, and affords the user control and the ability to provide feedback to the system. The...

Context-Aware Inference and Adaptation: A Framework for Designing Intelligent AR Interfaces

We have developed a holistic framework for the design of Intelligent AR interfaces. Using our proposed taxonomy of everyday AR contexts we proposed a framework for the design of an intelligent AR interface to infer users’ wants and needs and predict the desired adaptations to the AR interface design dimensions, virtual content, and interaction techniques. Depending on the context, the intelligent interface may make general adaptations to the whole system, or to individual apps. This work is in preparation for...

Revisiting Performance Models of Distal Pointing Tasks in Virtual Reality

Performance models of interaction, such as Fitts’ Law, are important tools for predicting and explaining human motor performance and for designing high-performance user interfaces. Extensive prior work has proposed such models for the 3D interaction task of distal pointing, in which the user points their hand or a device at a distant target in order to select it. However, there is no consensus on how to compute the index of difficulty for distal pointing tasks. We present a preliminary study...

Exploring the Benefits and Challenges of Working with AR Virtual Displays In-The-Wild

Mobile workers have limited access to display infrastructure while working on remote settings. While permanent office settings can support one or more monitors with large screen space, workers in remote settings are often reliant on the portable devices available to them at the time, such as laptops, tablets, and smartphones. Existing research has shown that virtual displays are feasible with current technology although they do not perform as well as a physical multi-monitor setup would. They have been shown to...

Welcome to the 3D Interaction Group

The 3D Interaction (3DI) Group performs research on 3D user interfaces (3D UIs) and 3D interaction techniques for a wide range of tasks and applications. Interaction in three dimensions is crucial to highly interactive virtual reality (VR) and augmented reality (AR) applications to education, training, gaming, visualization, and design. We also conduct empirical studies to understand the effects of immersion in VR and AR, the impact of natural and magic 3D interaction techniques, and usability and user experience in 3D UIs.