Evaluating the Use of Conversational Agents for Immersive Collaboration

Collaborative Virtual Environments (CVEs) empower geographically dispersed teams to work across time and physical space within a shared virtual workplace. Within CVEs, team members can communicate in real-time and preserve information about their work through text annotations. However, collaborators may not always be available to assist with tasks or answer questions regarding prior contributions in the CVE. Conversational agents powered by large language models (LLMs) have the potential to address these gaps in availability and knowledge by leveraging annotations and recorded actions within the CVE to answer users’ questions. However, the effectiveness of agent-mediated assistance relative to a live human collaborator remains underexplored. To address this gap, we conducted an extended reality (XR) study examining differences in task performance and user experience during immersive information-seeking collaboration. Our findings indicate that collaborating with a conversational agent yields comparable task performance and satisfaction, but with trade-offs in prompting behavior, trust, and enjoyment.

Participant view of the video-see-through collaborative virtual environment showing the virtual inspection object and a portion of the paper audit form. A: Initial configuration with the object floating above the desk. B: The participant repositions the object using the controller to locate an annotation indicator (green sphere at the center of a cell). C: The participant reviews the annotation for the highlighted cell while recording information in the paper audit form.

, , , , , , ,