Apple looks to its first augmented reality headset for next product breakthrough – NZ Herald

Self-declared “mad scientist” Bertrand Nepveu wanted to take a sip of beer but couldn’t see the bottle while wearing his start-up’s virtual reality headset.

So the dreadlocked engineer looked for a solution to not being
able to see real objects while being plunged into a virtual world. His company Vrvana repurposed the cameras in its “Totem” prototype, which had been designed to sense a user’s position, to allow users to also view their surroundings.

The breakthrough led Apple to acquire Vrvana for $30mn in 2017. That acquisition is just one of a dozen purchases in the fields of virtual and augmented reality that the iPhone maker has made over the past six years.

Apple’s own headset, the long-awaited culmination of all those deals and a huge investment in research and development, is shaping up to be its most significant new product since the iPhone.

Apple’s Worldwide Developers Conference in San Francisco on June 6 could be the $2tn company’s last big gathering before the headset is unveiled as early as this autumn, giving it a final opportunity to rally fans and chart a course to a new generation of computing hardware.

Little is known about Apple’s new product, but Vrvana’s “pass through” video system is expected to be a key feature. The idea is to turn the enclosed world of VR headsets inside-out, enabling the device to allow users to also see the physical world around them overlaid with digital images.

“We re-spun everything to merge AR and VR, for the first time, in a single device,” said Nepveu, who declined to discuss his work while at Apple before leaving last year.

Such breakthroughs have led to confidence that Apple may once again launch a device that could transform the tech, media and entertainment industries as it has previously with the iPod, iPhone and iPad.

“I do believe there is a path that you [won’t] need your phone and [augmented reality] glasses will evolve to be the next computing platform,” said Cristiano Amon, chief executive of Qualcomm, chip supplier to the world’s largest device makers, including Apple, Samsung and Facebook’s parent Meta.

While few analysts expect AR to go mainstream overnight, researchers at Citi expect that 1bn people — roughly the number of iPhone users today — will be wearing headsets by 2030, supporting a market worth up to $2tn in revenue.

To achieve that lofty target, Apple and its Silicon Valley rivals are racing to solve technical challenges. For many, the way VR cuts off users from the world is awkward and anti-social. Some users have complained they feel vulnerable or silly wearing bulky headsets. Enough gamers injure themselves for “best VR accidents” to proliferate on YouTube.

Those issues have led existing headsets made by groups such as Meta to remain a niche product with few regular users other than gamers.

Timoni West, vice-president of emerging tools at Unity, a video games software company that works closely with Apple and iPhone app developers, calls this “the biggest problem” in VR.

Most people have little interest in wearing something that West compares to a scuba mask. “It can still be kind of scary,” they said. “And for a class of people who want to be able to see everyone else in the space with them, then it’s kind of a non-starter.”

The VR market remains tiny in comparison to PCs and smartphones, but it is currently dominated by Meta. Sales of its Quest 2 headset reached an estimated 10mn units last year, giving Meta 78 per cent of the VR market, according to research house IDC. That gives Meta a significant head start over Apple in understanding what consumers want and luring app developers on to its platform.

Meta now has 17,000 employees working on “metaverse” tech, with a budget of about $3bn per quarter. Mark Zuckerberg has said its higher-end “Cambria” headset — another hybrid VR/AR device with pass-through video similar to those rumoured for Apple’s first device — will launch later this year.

That investment makes it unlikely that Apple’s first headset will do to the Quest what the iPhone did to the BlackBerry anytime soon.

“We’re nowhere near to [Apple chief executive] Tim Cook stepping up on stage and saying, ‘this is the device that replaces your smartphone’,” says Sam Cole, CEO of FitXR, a VR fitness app.

Multiple innovations are needed in fields as varied as battery power, user interface and physical comfort for VR and AR headsets to move from niche gaming applications to become part of a daily routine for tens of millions of people.

When Apple created the iPhone, its biggest breakthrough was to replace a physical keyboard with a multi-touch screen.

Matt Tullis, a senior executive at Tencent-backed start-up UltraLeap, predicts that using natural hand gestures, rather than games controllers or joysticks, will be essential to taking AR mainstream. “Hand-tracking will be to [AR] what multi-touch was to the iPhone,” he said.

Tipatat Chennavasin, a VR investor at the Venture Reality Fund, said he expects Apple’s first headset to be an expensive device aimed at professional designers and developers, the same customers who first adopted the Macintosh decades ago. “This is not the iPhone moment, this is the Mac moment,” he said.

Apple’s advantages over Meta are seen in its expertise in blending services, software and hardware, including the design of its own ultra-efficient chips. It also has an army of millions of iPhone app developers ready to rewrite their software to run on what is rumoured to be called Apple’s “RealityOS” platform.

Sahar Fikouhi, founder of ARki, an iPhone AR app for viewing architectural models in 3D, experimented with the Microsoft Hololens but said only Apple’s platform could reach a wider audience.

“We are trusting that Apple is going to be the company that produces a headset with mass-market appeal,” she said.

Written by: Patrick McGee, Tim Bradshaw and Cristina Criddle
© Financial Times