Design Goals
In large tracked areas, natural interaction with data is yet a challenge. Previous approaches require users to either hold a device as a “Magic Wand” to enable them to select and manipulate objects, or wear trackable equipment on their hands, which is cumbersome and unnatural. In the real world the process for selecting and manipulating objects includes looking at the object and reaching it with hands. A natural data manipulation interface should utilize these actions. Such interfaces can enable user to select and manipulate data object based on gaze direction and hand gestures. Our goal is to design a Natural User Interface (NUI), which allows seamless and effective data selection and manipulation in a large tracked space.
Application Context
A NUI design application or 3D design tool can benefit from such a natural interaction techniques. The idea can be used to design a virtual 3D toolbox. User will be able to observe different virtual objects around him and select and manipulate objects. User should be able to move, rotate or scale the selected object. User will be able to navigate through this VE by walking in the tracked space. For example, if we consider this as a virtual store, user will be able to walk around the store and look at different products in the shelves. User can grab products, move, rotate and examine them and put them in his shopping cart. Selection can be done using either gaze or hand gestures and manipulation will done through hand gestures. Although gaze selection might be more accurate but hand selection is more natural which is the purpose of this design.
Technology
We intend to use Leap Motion Controller for bare-hand gesture detection. Leap will be placed either on the HMD or will be hanging from user’s neck and be placed in front of user’s chest. It depends on which one works better when prototyped. To visualize the VE, we will be using an Oculus Rift HMD. One problem with this technology setup is that Leap needs a wired connection and most probable user is required to carry a tablet or laptop. We believe a Microsoft Surface tablet may be the easiest solution to this problem, as it provides USB port, windows programmability and it’s light weighted. We will use the CUBE in MAC building, which contains a 40×48-fts tracked area.