3D Interaction Group

The Alchemist: A Gesture-Based 3D User Interface for Engaging Arithmetic Calculations

This project presents our solution to the IEEE VR 2024 3DUI contest. We present The Alchemist, a VR experience tailored to aid children in practicing and mastering the four fundamental mathematical operators. In The Alchemist, players embark on a fantastical journey where they must prepare three potions to break a malevolent curse imprisoning the Gobbler kingdom. Our contributions include the development of a novel number input interface, Pinwheel, an extension of PizzaText, as well as four novel gestures, each corresponding...

Towards Privacy-Aware AR: A Look into Mitigating Privacy Concerns in a Pervasive AR Future

Fig 1: An overview of our approach to creating a interactive privacy-aware AI assistant for everyday AR In this work, we present a practical design framework for a privacy-aware virtual assistant for everyday AR, focusing on educating users with a lack of knowledge regarding technical and/or privacy literacy. Our approach features human-in-the-loop to learn privacy context, provides transparency into the system state of privacy detectors, and affords the user control and the ability to provide feedback to the system. The...

Context-Aware Inference and Adaptation: A Framework for Designing Intelligent AR Interfaces

We have developed a holistic framework for the design of Intelligent AR interfaces. Using our proposed taxonomy of everyday AR contexts we proposed a framework for the design of an intelligent AR interface to infer users’ wants and needs and predict the desired adaptations to the AR interface design dimensions, virtual content, and interaction techniques. Depending on the context, the intelligent interface may make general adaptations to the whole system, or to individual apps. This work is in preparation for...

Exploring the Benefits and Challenges of Working with AR Virtual Displays In-The-Wild

Mobile workers have limited access to display infrastructure while working on remote settings. While permanent office settings can support one or more monitors with large screen space, workers in remote settings are often reliant on the portable devices available to them at the time, such as laptops, tablets, and smartphones. Existing research has shown that virtual displays are feasible with current technology although they do not perform as well as a physical multi-monitor setup would. They have been shown to...

EyeST: Predicting Information Relevance from Eye Gaze Data in Immersive Space to Think

Eye gaze patterns vary based on reading purpose and complexity, and can provide insights into a reader's perception of the content. We hypothesize that during a complex sensemaking task with many text-based documents, we will be able to use eye-tracking data to predict the importance of documents and words, which could be the basis for intelligent suggestions made by the system to an analyst. We introduce a novel eye-gaze metric called `GazeScore' that predicts an analyst's perception of the relevance...

Goldilocks Zoning: Evaluating a Gaze-Aware Approach to Task-Agnostic VR Notification Placement

While virtual reality (VR) offers immersive experiences, users need to remain aware of notifications from outside VR. However, inserting notifications into a VR experience can result in distraction or breaks in presence, since existing notification systems in VR use static placement and lack task and environment awareness. We address this challenge by introducing a novel notification placement technique, Goldilocks Zoning, which leverages a 360-degree heatmap generated using eye-tracking data in order to place notifications near salient areas of the environment...

Welcome to the 3D Interaction Group

The 3D Interaction (3DI) Group performs research on 3D user interfaces (3D UIs) and 3D interaction techniques for a wide range of tasks and applications. Interaction in three dimensions is crucial to highly interactive virtual reality (VR) and augmented reality (AR) applications to education, training, gaming, visualization, and design. We also conduct empirical studies to understand the effects of immersion in VR and AR, the impact of natural and magic 3D interaction techniques, and usability and user experience in 3D UIs.