Labor für Virtuelles Studio / Virtuelle Realität

Abstract: Virtual avatars in the virtual studio using markerless tracking for body and face [DE]

Motion Capture offers limitless possibilities for the creation and control of realistic virtual avatars and the development for that technology is progressing constantly. There are numerous implementation options, both with or without markers, with a tracking suit or with normal everyday clothing. In the practical part of this work, a virtual avatar is controlled in real time with the help of marker-less tracking, which is applied to both the face and the body independently. The tracking data is combined in the render engine Unity. After that the avatar will be put in a virtual environment. The body will be tracked by using the Captury Live system of the virtual studio at the Düsseldorf University of Applied Sciences. Various systems are available for the face, starting from a simple webcam up to using a depth camera (e.g. iPhone X). In this case an iPhone XR is used. An evaluation and demonstration round off the topic.

Keywords:

Motion Capture, Face Capture, Face Tracking, Markerless Tracking, Virtual Avatar, Face Animation, Body Animation, Realtime Tracking

Supervisor:

Prof. Jens Herder, Dr. Eng./Univ. of Tsukuba
Philipp Ladwig, M.Sc.

Location:

The research took place at the Virtual Sets and Virtual Environments Laboratory.