The HoloLens 2 augmented reality headset may now be used during driving | Immersive Technology

Microsoft has entered into a joint venture with the German automaker Volkswagon to create an augmented reality (AR) GPS navigation technology fuelled by the tech giant’s HoloLens 2 MR headset. Microsoft’s ‘Moving Platform’ functionality has been created to support drivers of Volkswagen vehicles. It does so by offering instant navigation and security-related insights when a person is on a road trip.

Dr. Andro Kleen, head of the data science team, Volkswagen Group Innovation, said in the company’s blog post that its team believes that MR insight, is by far the most straightforward information that can be catered to customers in an attempt to optimise their journey. He emphasised the visuals being offered by the mixed reality technology are congruent with what drivers have to process. Kleen said that these visuals are not vastly different from what a person would see whilst driving.

Volkswagon has reportedly been testing with augmented reality navigation for quite some time. The company started working on a research study called ‘Race Trainer’ back in 2015. Augmented reality technology has been amalgamated with self-driving cars to instruct drivers about how they can race. As they drove, the technology utilised the HoloLens 2 headgear to project virtual directions such as steering and brake tidbits on top of their actual surroundings. Sadly, the project was halted owing to tracking concerns. These were identified by observing that as soon as the car moved, the HoloLens 2 holographic vision also did the same.

Marc Pollefeys, Director, Microsoft’s Mixed Reality and AI Lab in Zurich, Switzerland, said that there were a lot of discussions regarding the actual efficacy of the technology in real-world use cases. Several scenarios were outlined and tallied against the expectations that were existent for the tech within those. There was a lot of enthusiasm related to the collaboration and Microsoft wanted to employ the HoloLens to solve certain problems.

Microsoft and Volkswagen joined up once in 2018 to address the long-standing dilemma. The team led by Polyfeys created an algorithm for the HoloLens 2 to analyse and factor in vehicle movement to facilitate a safe AR-guided experience that does not compromise monitoring. Pollefey and his colleagues had to operate remotely from their residences during the development phase, due to the COVID-19 pandemic limitations that existed then in Switzerland.

Following some preliminary testing, the team could perform some in-person testing by using Volkswagon vehicles, within proximity of the Redmond campus of Microsoft. At the early stage, the experimentation was done on boats, whilst later cars were used. The researchers tracked the location of the car and compared it to the VR headset by implementing a three-dimensional positioning system. This allowed them to fixate 3D information atop the actual surroundings of a user. A real-time 3D map, for instance, maybe displayed above the vehicle’s dashboard, enabling the user to check their actual position without turning away their eyes from the road.

As per Microsoft, the Moving Platform concept has so far caught the imagination of certain nautical groups.