Abstract: Automatic Camera Control and Video Switching in Live Studio Production using Deep Learning Algorithms [DE]
Manuel Garni, Automatic Camera Control and Video Switching in Live Studio Production using Deep Learning Algorithms, Hochschule Düsseldorf, Master Thesis, 18.02.2021.
Robotic, track-mounted and remote-controlled camera equipment is widely used in virtual
reality and news studios. Technologies and research are currently limited to predefined
positions and automated person tracking. Mostly, such systems are not taking into account
the image composition demands and additional layers of virtual cinematography, which only
fully integrated solutions can offer. Real time control of cameras is enabled by markerless
actor and camera tracking – positions of persons and parameters of equipment in the studio
are continuously collected and transmitted. Based on this data, a semi-automatic prototype
has been implemented and used in production. The prototype uses image composition parameters
in a 3D rendering environment. Along the prototype, requirements and tasks for deep-learning
supported camera and switching control are outlined, foremost the data collection.
This thesis introduces an approach of robotic camera control using industry-standard
communication protocols. Relying on simple interaction, methods are proposed, providing
automatic control, but requiring no human intervention by default. Interaction methods
and data collection approaches are provided for future implementations, that rely on
the infrastructure of a virtual studio.
virtual cinematographer, camera control, switcher control, camera placement, camera framing,
virtual environment, virtual studio, image composition, robotic camera system, AI director, deep learning, neural networks
Prof. Jens Herder, Dr. Eng./Univ. of Tsukuba
Prof. Dr.-Ing. Thomas Bonse
The research took place at the Virtual Sets and Virtual Environments Laboratory.