Microsoft HoloLens Review

Listen up — Microsoft has done it. The Microsoft HoloLens is Augmented Reality’s “Oculus Moment.” This is big. In the AR community, this is what we’ve been waiting for. For those not, the future is about to sneak up on them with a product that is going to blow minds.

Given GigantiCo’s readership is one largely familiar with AR (Augmented Reality), I’m not going to give a deep explanation beyond a cultural reference to Tony Stark’s Iron Man HUD (Head’s Up Display), and a definition I crafted a few years ago, “Special effects, rendered in real time, over real life.” Bruce Sterling tells me he disapproves of my definition of AR, but in the context of HoloLens, I believe it is fitting.

The first thing I noticed at Microsoft’s private New York HoloLens event was the pronunciation. For those already aware of HoloLens (mostly industry insiders), I have only ever heard “Holo” pronounced like Han Solo. But no, every Microsoft employee at the event pronounced “Holo” like Hologram, and without any pause between syllables. I think they’re going to have trouble pronunciation-policing that.

My second surprise was that the demo experience was entirely gaming. I had anticipated an enterprise demo, as most of the pre-release promotional videos I have encountered had indicated an enterprise focus. I showed up and it was all gameplay. My friend Nate, a recruitment subcontractor for game developers, told me the day before I attended the demo that he is recruiting for the HoloLens team, so I guess that should have been no small clue (if you’re a game developer, give Nate a shout).

Regardless of end user application, a gaming scenario makes sense for the demo as it best demonstrates the HoloLens’ wide range of technologies: 3D sound, gaze-tracking, 3D scanning, real-to-virtual content registration, stereoscopic projection, frame rate latency — convincing gameplay is technologically demanding. If it can succeed as a gaming device, everything else is child’s play.

The demo takes place in a small room, kitted with minimal decor to hint at a family den. Some preparation is needed to use the device. The ergonomics of the fitting are well designed. The head-mounting is made of two nested rings, the inner of which wraps onto the user’s head, and a rotor knob tightens to fit. The external ring then pivots the stereoscopic lenses up and down to position best on the user’s face. NOTE: It works over glasses! I wore mine, and the device was still comfortable enough that I forgot I was wearing them. It was also lighter than I had expected.

Before gameplay can begin, the user must map the room. The device has a robust array of visual sensors, with multiple cameras/scanners on both sides, and in the center. The user must turn-around slowly in a circle as a SLAM (Simultaneous Localization and Mapping) process is used, similar to my Occipital Structure Sensor, where a low polygon wireframe model can be seen building-out its understanding of the environment — this alone is a cool visual to see. Once the user has turned in a full circle, the device indicates when the scan is complete and gameplay can begin.

The game starts by utilizing 3D audio. A banging, crushing sound reveals where alien invaders will enter the room. Turning to face the direction of the noise, cracks begin to appear in the wall. A column bursts into the room, opening a portal — a spaceship launch bay through which the invaders will stage their attack.

AR being an emerging technology, UI (User Interface) precedent is still being establishing. Here Microsoft uses gaze-tracking — the device senses when you have a baddy in your sights. Gesture tracking of an index-finger-tap in the air works as a trigger for shooting. Over time a super-power builds up that can freeze all aliens using a voice command.

Like most games, this one has escalating levels, increasing in difficulty. The first alien invaders are small flying biomechanical insects shooting fireballs. In the next level, robotic scorpions that crawl the walls join in the attack. The final level climaxes with a mothership and full armada of defenses. The gameplay is very physical. I’m not a gamer, but I am an avid dancer, casually speaking, and the gameplay was athletically demanding. Jumping about the room, ducking and dodging fireballs, I did break a sweat. Small boast: I was the only user in our group to complete the full demo without being killed.

In my review, I’m less concerned about this particular game than I am with how the demo demonstrates the device’s capabilities. From gesture tracking, gaze-tracking, 3D mapping, 3D registration, head tracking, voice recognition, frame rate latency … in every conceivable metric, the HoloLens met or exceeded expectations. Additionally, given all the jumping around the room dodging fireballs, the device stayed comfortable, and never came loose or needed adjustment, and I never experienced any visible lag with the head tracking. The registration was completely convincing.

My one concern going in was the rumor about the narrow field of view (FOV)¹. I can report back that the field of view is not at all the concern one might assume. In Virtual Reality (VR), FOV is more apparent because the device is typically face-fitting, and anything out of FOV is masked in black. Hence if the FOV is narrow in VR, the experience can be claustrophobic, and the user feels like they’re looking into a scene through a hole in a fence. This does not apply in AR. That is because the user’s peripheral view is of the room they are in. In VR, any negative space around the FOV is blacked out. In AR, anything outside the user’s FOV is simply their existing view of the real world. So the FOV feels as wide as real life. And any clipping effect at the edges of the FOV is also minimal because blank space is alpha-masked². It is only when an object exits frame that the edge of the FOV is momentarily revealed.

To be clear, the content does not seamlessly blend with the real life room. The resolution is low-ish but acceptable. Good alpha-channel rendering avoids jagged pixel edges where virtual content meets reality. The projected content is “bright” with sufficient opacity (i.e.: It doesn’t have a “ghosty” appearance, but is opaque). The bright quality also makes the virtual content pop, standing out against the background.

One thing I was unable to test was occlusion. The room had no obstructions in it for virtual content to be obscured behind. Late coming competitor MagicLeap (funded by Google) has teased with a few seconds of YouTube video showing a comic robot under a table that is momentarily occluded behind the table leg. I cannot speak to whether HoloLens is capable of virtual content occlusion effects, but given their 3D scanning process, my suspicion is that it does. It is possibly the one thing the demo lacked. I doubt that a demonstration of occlusion was left out of the demo for lack of capability, but more likely for sake of liability. With users leaping about to avoid fireball shots, a column or other obstruction in the room could have proven dangerous.

I did ask the Microsoft representatives in our group’s post-demo exit interview how much of the HoloLens technology came from Microsoft’s ODG (Osterhout Design Group) patent portfolio acquisition last year. They were unable to answer. Subsequently a relationship of mine in the post-production special effects industry informed me of conversations with Microsoft about AR HMDs that predates the ODG deal. So while the IP acquisition appears to have accelerated their timeline, HoloLens is more than a mere a productization of ODG’s technology.

One closing observation that cannot be overstated: HoloLens is wireless. It is not just that it doesn’t have a cable connecting it to a console, it has no console. The entire HoloLens computer system is housed within the headset device — all processing is onboard. This final note cannot be overstated. No console, no wireless network latency or connectivity issues, and NO CABLES!

In conclusion, HoloLens is the real deal.

For those of us in the AR space who have been waiting for an off-the-shelf AR HMD, it is now here. This is going to be big. Kudos to Microsoft and everyone on the HoloLens team. You’ve done it.

The Microsoft HoloLens is currently on a limited demo tour, by appointment only, to select major cities in the US. Check their schedule to get an appointment.

The HoloLens Developer Kit (DVK) is said to be shipping in early Q1 2016, at a price of $3000.00. HoloLens could hit the consumer market as early as Q4 2016.

Addendum

This demo was performed at a Microsoft event earlier this year. I have edited a clip below to the relevant portion.: