Mixed Reality & the HoloLens
Many people in the technology industry know about several up-and-coming simulation environments. Some have been popularized, such as virtual reality (VR) with the Oculus (completely submersive visual interactive environment), and augmented reality (AR) with games like Pokemon Go (layers visual simulations on top of the real world, but they don’t interact). One “reality” that isn’t as well known is mixed reality (MR), or hybrid reality. Mixed reality is an emerging technology that is sort of in the middle of virtual reality and augmented reality; mixed reality is working to combine simulations and the real world so that they can interact in real time. One company really trying to push forward in this growing technology is Microsoft, particularly with one of their products, the HoloLens.
/cdn.vox-cdn.com/uploads/chorus_image/image/60046903/20160329-8A0A9044.0.0.jpg)
Image 1a. The HoloLens device. Shows the adjustable circular headpiece to better secure the device to the user’s head.
Image 1b. The HoloLens device. This has a better view of how the user would wear the device, as well as showing the cameras and lenses that display the digital projections.
Overview of the HoloLens
The Microsoft HoloLens is a wearable headset that can feature several different forms of reality (VR, AR, MR), but its specialty and marketing really emphasizes the use of the equipment for mixed reality. The device is self described as the “first fully self-contained, holographic computer.” (1). Others describe the product as “smart glasses,” even though the HoloLens seems to be much more than that. The HoloLens also provides speakers near the ears, which allows users to experience virtual sounds, alongside those in their environment. It allows for spatial sound, so users may be able to follow sound in their virtual environment. The device has several cameras (several of which can be seen in Image 1b above), which allows the technology to recognize different surfaces and have spatial awareness. The cameras also allow the user to show others what they see; the device can live stream what the user is seeing (real environment, virtual surroundings, point of view recording). The HoloLens uses motion tracking and gesture input as controls. These controls are relatively intuitive, so users may be able to quickly pick up how to operate the device.
Image 2a. Static demonstrations of HoloLens abilities. Shows multitasking abilities, such as video calling someone and showing/working/interacting with holographic prototypes of products. In this scenario, the HoloLens user may be streaming what they see to the person in the video call.
Image 2b. Static demonstrations of HoloLens abilities. Shows an interactive, 3D holographic model of an animation.
Microsoft HoloLens Info Session
Microsoft held an informational recruiting session on campus at Virginia Tech at the end of August that also shared a lot of information about the HoloLens. One of the recruiters, Ishita, is a Program Manager for HoloLens, so she shared quite a bit about her experience working with the device, hopes for the software, and gave a live demonstration of using a HoloLens (videos 1a, 1b are below of the live demonstrations). (2).
Video 1a. As a Microsoft recruiter and a full-time HoloLens Program Manager, Ishita demonstrates the HoloLens and its abilities. Demonstrates how the device can live stream the user’s mixed reality experience and the interactive 3D holograms. (https://youtu.be/AF-DVudRTPo)
Video 1b. As a Microsoft recruiter and a full-time HoloLens Program Manager, Ishita demonstrates the HoloLens and its abilities. Shows how useful this technology can be in planning a space, especially before buying or trying to move in physical items. (https://youtu.be/TcnCDfASMgo)
Current Uses
Ishita, a Program Manager for HoloLens, shared her expertise on the technology and how clients have used it in the past. She shared how some companies have used the HoloLens to save money, waste, and time. One example of this was for a client who used the HoloLens to design a set of stairs for their store. This client happened to redesign their stairs frequently, so they previously had a costly system in place. This system comprised of designing the stairs architecturally on paper, moving to making a prototype out of cardboard, building the prototype onsite, and receiving feedback. Typically, this client would go through several iterations of this process, spending a lot of money and creating a lot of unnecessary waste in the meantime. Once introduced to the HoloLens, this client ended up saving thousands of dollars, produced less waste from their design process, and saved a lot of time “building” and altering the prototype. (2). Below is a video that also shows a scenario in which the HoloLens saved money, resources, and time through building holographic prototypes.
Video 2. Microsoft HoloLens partner with thyssenkrupp (a steel company, which, for the purposes of this video, help build a stair lift for an elderly couple). The video shows how the HoloLens collects data and can build a virtual version of a physical object. Although not exactly the same as the example given above with the client building staircases, it shows how useful the HoloLens can be in creating virtual prototypes to save money, resources, and time. (https://www.youtube.com/watch?v=IgtHnCJJUMU).
Future Uses
Currently, the HoloLens is mainly marketed towards commercial clients, not necessarily for personal use, but that may change in the near future. Ishita, a Program Manager for HoloLens, gave examples of how the technology could be used in the future, and how she has personally benefitted from the device. The HoloLens could be used in an educational setting, such as a biology class. The students could learn about the human body’s anatomy, to scale, by interacting with a 3D hologram. This could enable people to learn more about the human body, or even anatomy of smaller animals, without ever having to do another dissection again (sounds great for all those squeamish, yet still curious, students!). The hologram visualizations could also be hugely advantageous to those students who are visual learners – imagine learning chemical reactions through atomic structure visualizations, or learning about historical figures or animals with holograms! There are so many educational possibilities with the HoloLens. On a personal level, Ishita shared that she benefited from working with the device – she planned her living room with it! She was able to see what furniture fit, where it looked best, what flow and energy of the room worked for her – all without moving physical furniture around a million times. (2). Below is a video that shows how useful virtually planning a space through the HoloLens can be, rather than doing it all physically.
Video 3. Microsoft HoloLens and Microsoft Layout. This video is a promotional video for Microsoft HoloLens, but also does a great job of capturing how the technology can help plan a physical space, all virtually. (https://www.youtube.com/watch?v=rK7l6Gq16WA).
HoloLens & Creative Computing Studio
The HoloLens could be incredibly useful for our class project, especially if anyone was planning a physical product that needed prototyping. Personally, I would love to do a project with the HoloLens – it would just take some creativity to decide what to do, because there are so many options. Virginia Tech actually has a few HoloLens devices on campus that designated people can sign up to use or check out. Considering that there are very few sets compared to a large Computer Science and other technology departments, it may not be very feasible to experiment with a HoloLens for the time we have this semester. Currently, it may be better to work with another similar reality-altering headset that is more affordable or more available during this semester for our capstone projects. If someone has an extra couple thousand dollars lying around, you might want to take a peek at the HoloLens Development Edition. (3). It would be a really cool opportunity to start working with this technology now, start developing for it now, and test and play other HoloLens apps and games before it is marketed towards individual use. Honestly, I hope to see more of this type of technology in the future!
Still curious?
Here is the Official Microsoft HoloLens YouTube Channel: https://www.youtube.com/channel/UCT2rZIAL-zNqeK1OmLLUa6g
Resources
- https://www.microsoft.com/en-us/p/microsoft-hololens-commercial-suite/944xgcf64z5b/2bhf?icid=MRVRcat_CP3_HoloLensCommercial_09162018&activetab=pivot%3aoverviewtab
- AWC x Microsoft: Women’s Welcome Back Panel. HoloLens Program Manager, Ishita Ganotra.
- https://www.microsoft.com/en-us/p/microsoft-hololens-development-edition/8xf18pqz17ts/md7v?activetab=pivot:overviewtab