Facial Capture for XR

Using a camera on a mobile phone, an AI app can capture the user's facial model with different expressions/motions, and build a database. While the user is wearing a VR/AR headset, the face can be captured by head-mounted cameras (HMCs) and matched with a facial model in the database in real-time, and this model is rendered back into the virtual space. Thus users can have a face-to-face meeting while wearing an AR/VR headset.

Related publications