{"id":3574,"date":"2026-02-11T09:44:10","date_gmt":"2026-02-11T14:44:10","guid":{"rendered":"https:\/\/wordpress.cs.vt.edu\/3digroup\/?p=3574"},"modified":"2026-02-12T16:35:45","modified_gmt":"2026-02-12T21:35:45","slug":"exploring-real-time-affective-state-detection-and-intelligent-adaptive-interventions-in-virtual-reality","status":"publish","type":"post","link":"https:\/\/wordpress.cs.vt.edu\/3digroup\/2026\/02\/11\/exploring-real-time-affective-state-detection-and-intelligent-adaptive-interventions-in-virtual-reality\/","title":{"rendered":"Exploring Real-Time Affective State Detection and Intelligent Adaptive Interventions in Virtual Reality"},"content":{"rendered":"\n

There is growing consensus in human-computer interaction (HCI) and affective computing that affect<\/em> \u2014 users\u2019 emotional and psychological states \u2014 is a critical contextual dimension for truly adaptive XR systems. While traditional context-aware systems primarily adapt to users\u2019 tasks and environmental conditions, affect-aware systems should extend this capability by continuously inferring users\u2019 affective states, reasoning about their causes within a given context, and delivering adaptive responses that help users mitigate unwanted emotions and successfully progress through their tasks. One example of where this might be particularly relevant is education, where a student’s emotions strongly influence key learning processes such as attention, memory, reasoning, and decision-making. In this context, intelligent XR (iXR) systems should aim to recognize<\/em> and adapt<\/em> to users\u2019 affective states in real time, enabling experiences that respond dynamically to how users feel and behave.<\/p>\n\n\n\n

Prior research demonstrates that affective states can be inferred by AI models analyzing physiological, behavioral, and cognitive signals, many of which are now accessible through modern head-worn displays (HWDs), such as gaze data. However, integrating real-time, AI-driven emotion recognition into XR systems remains both technically and conceptually challenging. In the realm of intelligent adaptations, existing adaptive systems typically employ rule-based mechanisms, such as modifying game difficulty based on detected stress or anxiety levels. While these methods can be effective for targeted use cases, they under-exploit the broader potential of AI to enable nuanced, context-aware adaptations.<\/p>\n\n\n\n

Our project aims to pioneer the integration of continuous, real-time affect inference within interactive XR environments, enabling meaningful adaptations tailored to users\u2019 emotional states as they occur. As a proof-of-concept, an initial study will gather both self-reported and gaze data and preliminary insights into how adaptive XR environments can detect and respond specifically to boredom. By immersing participants in a simulated educational environment, this study aims to provide valuable insights into how intelligent XR technology could be applied in relevant real-world settings, where emotions play a critical role in performance and learning outcomes.<\/p>\n\n\n\n

\"\"
Fig. 1:<\/strong> Boredom is detected as the user passively watches a presentation. Once detected, the system triggers adaptations designed to mitigate the unwanted affective state, reengaging the user with the task.<\/figcaption><\/figure>\n\n\n\n
<\/div>\n","protected":false},"excerpt":{"rendered":"

There is growing consensus in human-computer interaction (HCI) and affective computing that affect \u2014 users\u2019 emotional and psychological states \u2014 is a critical contextual dimension for truly adaptive XR systems. While traditional context-aware systems primarily adapt to users\u2019 tasks and environmental conditions, affect-aware systems should extend this capability by continuously inferring users\u2019 affective states, reasoning …<\/a><\/p>\n","protected":false},"author":494,"featured_media":3577,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[9,11],"tags":[],"ppma_author":[501,559,391],"class_list":["post-3574","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-featured","category-projects"],"jetpack_featured_media_url":"https:\/\/wordpress.cs.vt.edu\/3digroup\/wp-content\/uploads\/sites\/141\/2026\/02\/affectxr.png","authors":[{"term_id":501,"user_id":0,"is_guest":1,"slug":"francielly","display_name":"Francielly (Elly) Rodrigues","avatar_url":{"url":"https:\/\/wordpress.cs.vt.edu\/3digroup\/wp-content\/uploads\/sites\/141\/2025\/10\/D9E7D095-A562-459F-A5D2-77F36EE71D1C-scaled.jpeg","url2x":"https:\/\/wordpress.cs.vt.edu\/3digroup\/wp-content\/uploads\/sites\/141\/2025\/10\/D9E7D095-A562-459F-A5D2-77F36EE71D1C-scaled.jpeg"},"author_category":"1","user_url":"https:\/\/www.linkedin.com\/in\/franciellyrodrigues","last_name":"Rodrigues","first_name":"Francielly (Elly)","job_title":"Presidential Postdoctoral Fellow","description":"Francielly (Elly) Rodrigues is a Presidential Postdoctoral Fellow in the Department of Computer Science at Virginia Tech\u2019s College of Engineering. She holds a Ph.D. in Computational Modeling from the National Laboratory of Scientific Computing in Brazil. Her research lies at the intersection of human-computer interaction and artificial intelligence. Her interests center on context-aware and intelligent Extended Reality (iXR) systems, particularly how immersive interfaces can sense contextual information about the user, task, and environment, reason over it, and adapt accordingly. This adaptability is essential for XR to be practical and effective in everyday life. The broader impact of her work is to create XR solutions that enhance productivity, communication, well-being, and accessibility across a wide range of applications, from education and medicine to everyday tasks.<\/span>"},{"term_id":559,"user_id":494,"is_guest":0,"slug":"ryutong","display_name":"Yutong Ren","avatar_url":{"url":"https:\/\/wordpress.cs.vt.edu\/3digroup\/wp-content\/uploads\/sites\/141\/2025\/09\/Portrait-e1757962967296.png","url2x":"https:\/\/wordpress.cs.vt.edu\/3digroup\/wp-content\/uploads\/sites\/141\/2025\/09\/Portrait-e1757962967296.png"},"author_category":"1","user_url":"https:\/\/yutong195.github.io\/","last_name":"Ren","first_name":"Yutong","job_title":"Graduate Student","description":"Yutong is a first-year Ph.D. student at Virginia Tech. His research focuses on AI-driven XR systems that understand and adapt to human behavior. He studies how immersive systems can use signals such as eye tracking, interaction data, audio features, and environmental context to infer users\u2019 cognitive and affective states in real time. His work is grounded in three questions:
(1) How can XR systems model user states such as boredom, attention, distraction, and engagement from multimodal data?
(2) How can AI reason about these states in relation to task and environmental context?
(3) How can XR systems respond with adaptive interventions that improve human performance, engagement, and safety while preserving user agency?"},{"term_id":391,"user_id":331,"is_guest":0,"slug":"dbowman","display_name":"Doug Bowman","avatar_url":{"url":"https:\/\/wordpress.cs.vt.edu\/3digroup\/wp-content\/uploads\/sites\/141\/2021\/01\/professional_photo2_2019-cropped-square-smaller-scaled.jpg","url2x":"https:\/\/wordpress.cs.vt.edu\/3digroup\/wp-content\/uploads\/sites\/141\/2021\/01\/professional_photo2_2019-cropped-square-smaller-scaled.jpg"},"author_category":"1","user_url":"","last_name":"Bowman","first_name":"Doug","job_title":"","description":"Doug A. Bowman is the Frank J. Maher Professor of Computer Science<\/a> at Virginia Tech<\/a>. He is the principal investigator of the 3D Interaction Group<\/a>, focusing on the topics of three-dimensional user interfaces, VR\/AR user experience, and the benefits of immersion in virtual environments.\r\n\r\nDr. Bowman is one of the co-authors of 3D User Interfaces: Theory and Practice<\/a>. He has served in many roles for the IEEE Virtual Reality Conference<\/a>, including program chair, general chair, and steering committee chair. He also co-founded the IEEE Symposium on 3D User Interfaces (now part of IEEE VR). He received a CAREER award from the National Science Foundation for his work on 3D Interaction, and has been named an ACM Distinguished Scientist. He received the Technical Achievement award from the IEEE Visualization and Graphics Technical Committee in 2014 and the Career Impact Award from IEEE ISMAR in 2021.\r\n\r\nHis undergraduate degree in mathematics and computer science is from Emory University, and he received his M.S. and Ph.D. in computer science from the Georgia Institute of Technology.\r\n\r\nCurriculum vitae (PDF)<\/a>"}],"_links":{"self":[{"href":"https:\/\/wordpress.cs.vt.edu\/3digroup\/wp-json\/wp\/v2\/posts\/3574","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/wordpress.cs.vt.edu\/3digroup\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/wordpress.cs.vt.edu\/3digroup\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/wordpress.cs.vt.edu\/3digroup\/wp-json\/wp\/v2\/users\/494"}],"replies":[{"embeddable":true,"href":"https:\/\/wordpress.cs.vt.edu\/3digroup\/wp-json\/wp\/v2\/comments?post=3574"}],"version-history":[{"count":7,"href":"https:\/\/wordpress.cs.vt.edu\/3digroup\/wp-json\/wp\/v2\/posts\/3574\/revisions"}],"predecessor-version":[{"id":3582,"href":"https:\/\/wordpress.cs.vt.edu\/3digroup\/wp-json\/wp\/v2\/posts\/3574\/revisions\/3582"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/wordpress.cs.vt.edu\/3digroup\/wp-json\/wp\/v2\/media\/3577"}],"wp:attachment":[{"href":"https:\/\/wordpress.cs.vt.edu\/3digroup\/wp-json\/wp\/v2\/media?parent=3574"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/wordpress.cs.vt.edu\/3digroup\/wp-json\/wp\/v2\/categories?post=3574"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/wordpress.cs.vt.edu\/3digroup\/wp-json\/wp\/v2\/tags?post=3574"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/wordpress.cs.vt.edu\/3digroup\/wp-json\/wp\/v2\/ppma_author?post=3574"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}