Labor für Virtuelles Studio / Virtuelle Realität

Abstract: Gesture recognition using a depth sensor and machine learning techniques

Advances in depth sensing provide great opportunities for the development of new methods for hu- man computer interactivity. With the launch of the Microsoft Kinect sensor, high-resolution depth and visual sensing has become available for widespread use. As it is suitable for measuring distances to objects at high frame rate, such kind of sensors are increasingly used for 3D acquisitions, and more generally for applications in robotics or computer vision. The aim of this survey is to implement a gesture recognition system using the Kinect version 2 of Microsoft in order to interact with a virtual TV weather studio. The Kinect sensor was used to build up a dataset, which contains motion recordings of 8 different gestures and was build up by two different gesture training machine learning algorithms. Then, the system is evaluated in a user study, which allows a direct comparison and reveals benefits and limits of using such technique. Finally, it is given an overview of the challenges in this field and future work trends.

Keywords:

Gesture recognition, Interaction, Kinect, Machine learning, Virtual studio

Supervisor:

Prof. Jens Herder, Dr. Eng./Univ. of Tsukuba
Prof. Dr.-Ing. Thomas Bonse

Ort:

The bachelor thesis was conducted at the Virtual Sets and Virtual Environments Laboratory.