Google's new Tech Turns VR Headsets Invisible Using 3D Modelling of Face

By Vishal Goel, | February 23, 2017

Mixed reality headset removal app experiment from Daydream Labs and Google Research. (Youtube)

Mixed reality headset removal app experiment from Daydream Labs and Google Research. (Youtube)

Google's Research arm collaborated with Daydream Labs to develop a new technology that lets spectators see full faces of the Virtual Reality headset users. The aim of the technology is to allow less awkward demonstrations of VR technology and experiences on video.

The technology first constructs a 3D model of the user's face using a colour and depth camera, a process which takes around one minute to complete. Next, the user is handed with a modified HTC Vive headset that is capable of tracking eye-gaze. Also, for a fully immersive video demo, the user is shot in front of a green screen while they are engaged in a VR experience.

Like Us on Facebook

The specialized camera setup is calibrated and time-synced with the headset in such a way that it becomes easy to replace the headset with the 3D face model and line it up with the visible portion of the face of the user.

Superimposing the digital top half of a person's head onto a real face can look unnatural (the phenomenon is called the "uncanny valley"). Therefore, the team decided to depict a "scuba mask effect" which shows the 3D face model composited with a translucent headset, reducing the chance of viewers getting distracted by imperfections and artefacts of the 3D model.

In the blog post, Google noted that this solution will go beyond improving VR and mixed reality experiences: "Headset removal is poised to enhance communication and social interaction in VR itself with diverse applications like VR video conference meetings, multiplayer VR gaming, and exploration with friends and family.

Going from an utterly blank headset to being able to see, with photographic realism, the faces of fellow VR users promises to be a significant transition in the VR world, and we are excited to be a part of it."

The GIFs in the blogpost show that the company's implementation is not entirely perfect yet, as one can still tell that there is some digital trickery involved in making the top half of the user's face visible, reports The Next Web. Still, it is certainly an improvement over watching someone in a VR experience with their headset on.

Google says that it will make the tech available in select YouTube Spaces, its studios that are open to popular creators on its video platform.


©2024 Telegiz All rights reserved. Do not reproduce without permission
Real Time Analytics