{"id":3551,"date":"2026-02-11T09:43:56","date_gmt":"2026-02-11T14:43:56","guid":{"rendered":"https:\/\/wordpress.cs.vt.edu\/3digroup\/?p=3551"},"modified":"2026-02-12T16:36:13","modified_gmt":"2026-02-12T21:36:13","slug":"understanding-3d-mid-air-spatial-manipulation-in-immersive-environments-a-taxonomy-and-visual-representation","status":"publish","type":"post","link":"https:\/\/wordpress.cs.vt.edu\/3digroup\/2026\/02\/11\/understanding-3d-mid-air-spatial-manipulation-in-immersive-environments-a-taxonomy-and-visual-representation\/","title":{"rendered":"Understanding 3D Mid-Air Spatial Manipulation in Immersive Environments: A Taxonomy and Visual Representation"},"content":{"rendered":"\n

In this project, we propose a novel taxonomy for mid-air spatial manipulation techniques in 3D virtual environments (VEs). Mid-air spatial manipulation refers to the application of mathematical transformations that alter an object\u2019s position or orientation in virtual space, performed through gestures or actions directly in 3D space and closely resembling real-world interactions with physical objects.<\/p>\n\n\n\n

Over the years, a wide range of mid-air manipulation techniques have been proposed, and the field has shifted from one-size-fits-all solutions to a diverse set of strategies tailored to specific use cases. Designers now face a complex design space with numerous considerations, including input devices and mechanisms, interaction constraints, and strategies to map physical movement to virtual motion. Critically, these decisions must account for contextual factors such as task demands, user capabilities, system constraints, and environmental conditions.<\/p>\n\n\n\n

To support reasoning within this design space, several taxonomies of 3D interaction have been proposed, but none provides a complete view of the spatial manipulation design space or its complexity. Our novel taxonomy characterizes 3D spatial manipulation techniques across five key orthogonal components, facilitating the description, comparison, and design of mid-air spatial manipulation techniques.<\/p>\n\n\n\n

\"\"
Fig. 1: Novel Taxonomy for Classifying 3D Mid-Air Spatial Manipulation Techniques: Five components uniquely describe each operating mode of a technique. The blue boxes indicate the options within a component or subcomponent, from which a single option should be chosen. The separation method subcomponent only applies when a level of separation exists (partial or total). * [T][R] in the Transfer Function Mappings component indicates that an operating mode can include multiple spatial transformations, each with one or more associated transfer functions. Thus, this component should be repeated for each.
<\/figcaption><\/figure>\n\n\n\n
<\/div>\n\n\n\n

We also propose a visual representation that enables techniques and their operating modes to be visualized in a compact, easy-to-read format, facilitating comparison of operating modes and techniques. <\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

Finally, to demonstrate the expressiveness and practical utility of our framework, we present a curated representative set of existing manipulation techniques and discuss how they are classified and illustrated. <\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

We aim for our taxonomy and visual representation to facilitate the entire research and design process of 3D manipulation techniques, starting from identifying limitations in the existing methods, pinpointing potential areas that could be enhanced to address a specific limitation, designing innovative approaches based on these identified shortcomings, and finally, comparing those different approaches to assess their effectiveness.<\/p>\n\n\n\n

<\/p>\n","protected":false},"excerpt":{"rendered":"

In this project, we propose a novel taxonomy for mid-air spatial manipulation techniques in 3D virtual environments (VEs). Mid-air spatial manipulation refers to the application of mathematical transformations that alter an object\u2019s position or orientation in virtual space, performed through gestures or actions directly in 3D space and closely resembling real-world interactions with physical objects. …<\/a><\/p>\n","protected":false},"author":331,"featured_media":3554,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[11],"tags":[14,27,78],"ppma_author":[501,417,391],"class_list":["post-3551","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-projects","tag-3d-interaction-techniques","tag-augmented-reality","tag-virtual-reality"],"jetpack_featured_media_url":"https:\/\/wordpress.cs.vt.edu\/3digroup\/wp-content\/uploads\/sites\/141\/2026\/02\/rodrigues_fig1-scaled.png","authors":[{"term_id":501,"user_id":0,"is_guest":1,"slug":"francielly","display_name":"Francielly (Elly) Rodrigues","avatar_url":{"url":"https:\/\/wordpress.cs.vt.edu\/3digroup\/wp-content\/uploads\/sites\/141\/2025\/10\/D9E7D095-A562-459F-A5D2-77F36EE71D1C-scaled.jpeg","url2x":"https:\/\/wordpress.cs.vt.edu\/3digroup\/wp-content\/uploads\/sites\/141\/2025\/10\/D9E7D095-A562-459F-A5D2-77F36EE71D1C-scaled.jpeg"},"author_category":"1","user_url":"https:\/\/www.linkedin.com\/in\/franciellyrodrigues","last_name":"Rodrigues","first_name":"Francielly (Elly)","job_title":"Presidential Postdoctoral Fellow","description":"Francielly (Elly) Rodrigues is a Presidential Postdoctoral Fellow in the Department of Computer Science at Virginia Tech\u2019s College of Engineering. She holds a Ph.D. in Computational Modeling from the National Laboratory of Scientific Computing in Brazil. Her research lies at the intersection of human-computer interaction and artificial intelligence. Her interests center on context-aware and intelligent Extended Reality (iXR) systems, particularly how immersive interfaces can sense contextual information about the user, task, and environment, reason over it, and adapt accordingly. This adaptability is essential for XR to be practical and effective in everyday life. The broader impact of her work is to create XR solutions that enhance productivity, communication, well-being, and accessibility across a wide range of applications, from education and medicine to everyday tasks.<\/span>"},{"term_id":417,"user_id":0,"is_guest":1,"slug":"run-yu","display_name":"Run Yu","avatar_url":"https:\/\/secure.gravatar.com\/avatar\/?s=96&d=mm&r=g","author_category":"","user_url":"","last_name":"","first_name":"","job_title":"","description":""},{"term_id":391,"user_id":331,"is_guest":0,"slug":"dbowman","display_name":"Doug Bowman","avatar_url":{"url":"https:\/\/wordpress.cs.vt.edu\/3digroup\/wp-content\/uploads\/sites\/141\/2021\/01\/professional_photo2_2019-cropped-square-smaller-scaled.jpg","url2x":"https:\/\/wordpress.cs.vt.edu\/3digroup\/wp-content\/uploads\/sites\/141\/2021\/01\/professional_photo2_2019-cropped-square-smaller-scaled.jpg"},"author_category":"1","user_url":"","last_name":"Bowman","first_name":"Doug","job_title":"","description":"Doug A. Bowman is the Frank J. Maher Professor of Computer Science<\/a> at Virginia Tech<\/a>. He is the principal investigator of the 3D Interaction Group<\/a>, focusing on the topics of three-dimensional user interfaces, VR\/AR user experience, and the benefits of immersion in virtual environments.\r\n\r\nDr. Bowman is one of the co-authors of 3D User Interfaces: Theory and Practice<\/a>. He has served in many roles for the IEEE Virtual Reality Conference<\/a>, including program chair, general chair, and steering committee chair. He also co-founded the IEEE Symposium on 3D User Interfaces (now part of IEEE VR). He received a CAREER award from the National Science Foundation for his work on 3D Interaction, and has been named an ACM Distinguished Scientist. He received the Technical Achievement award from the IEEE Visualization and Graphics Technical Committee in 2014 and the Career Impact Award from IEEE ISMAR in 2021.\r\n\r\nHis undergraduate degree in mathematics and computer science is from Emory University, and he received his M.S. and Ph.D. in computer science from the Georgia Institute of Technology.\r\n\r\nCurriculum vitae (PDF)<\/a>"}],"_links":{"self":[{"href":"https:\/\/wordpress.cs.vt.edu\/3digroup\/wp-json\/wp\/v2\/posts\/3551","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/wordpress.cs.vt.edu\/3digroup\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/wordpress.cs.vt.edu\/3digroup\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/wordpress.cs.vt.edu\/3digroup\/wp-json\/wp\/v2\/users\/331"}],"replies":[{"embeddable":true,"href":"https:\/\/wordpress.cs.vt.edu\/3digroup\/wp-json\/wp\/v2\/comments?post=3551"}],"version-history":[{"count":13,"href":"https:\/\/wordpress.cs.vt.edu\/3digroup\/wp-json\/wp\/v2\/posts\/3551\/revisions"}],"predecessor-version":[{"id":3573,"href":"https:\/\/wordpress.cs.vt.edu\/3digroup\/wp-json\/wp\/v2\/posts\/3551\/revisions\/3573"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/wordpress.cs.vt.edu\/3digroup\/wp-json\/wp\/v2\/media\/3554"}],"wp:attachment":[{"href":"https:\/\/wordpress.cs.vt.edu\/3digroup\/wp-json\/wp\/v2\/media?parent=3551"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/wordpress.cs.vt.edu\/3digroup\/wp-json\/wp\/v2\/categories?post=3551"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/wordpress.cs.vt.edu\/3digroup\/wp-json\/wp\/v2\/tags?post=3551"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/wordpress.cs.vt.edu\/3digroup\/wp-json\/wp\/v2\/ppma_author?post=3551"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}