HS Düsseldorf >> 
Fachbereiche >> 
Institute >> 
Organe und Gremien >> 
Verwaltung >> 
Hochschulbibliothek >> 
Campus IT >> 
Suche & Hilfe >> 

FB 1 Architektur >> 
FB 2 Design >> 
FB 3 Elektro- & Informationstechnik >> 
FB 4 Maschinenbau >> 
FB 5 Medien >> 
FB 6 Sozial/Kulturwiss. >> 
FB 7 Wirtschaft >> 

Aktuelles >> 
Studium >> 
Personen >> 
Showroom >> 
Studienbüro >> 
Dekanat >> 
Internationales >> 
Forschung & Entwicklung  >> 

Projekte >> 
Labore, Pools & Studios  >> 

HSD

Hochschule Düsseldorf
University of Applied Sciences


FB5

Fachbereich Medien
Faculty of Media

Labor für Virtuelles Studio / Virtuelle Realität



Publikationen >>
Presentationen >>
Forschung >>
Technologie >>
Kooperationen >>
Galerie >>
Abschlussarbeiten >>
Events >>
Lehre >>
Anfahrt >>
Labor Suche >>
Aktuelles >>

Abstract: Server-based Virtual Studio [DE]

Oliver Mertens, Server-based Virtual Studio, Hochschule Düsseldorf, Master Thesis, 21.10.2021.

Virtual studios are typically based on a local interconnection of specialised devices that perform all necessary steps to mix camera and sound recordings with the elements of a virtual environment in real time. This thesis addresses the questions of how far this process can be performed entirely in an IP-based network using a central server, what latencies occur and how the synchronisation of multiple data streams can be established. A scenario is considered in which video and camera tracking data from mobile devices is transmitted using protocols and technologies such as RTP and WebRTC and synchronised using clock synchronisation and timestamps during further processing. A server application was implemented to test the coordination of multiple clients that can be connected via WebRTC. These clients were developed as a web app that acts as a camera source and a Unity application to render the received video streams using GStreamer together with 3D content. Furthermore, an AR app for Android was implemented that uses RTP to send a video stream and corresponding tracking data directly to the Unity application. By setting up a production pipeline in a local network, both the quality of synchronisation and occuring latencies are determined. It is shown that the server is able to coordinate the clients and enables a dynamic establishment and termination of connections. The tracking data could be reliably assigned to the associated video images by using their timestamps after a transmission and the synchronisation of two video streams was shown to be feasible in principle, but could not be robustly established with the present implementation and required manual readjustment. The final output had a latency between 0.46 s and 0.54 s when using the AR app, so it is suggested to render a feedback image directly in the AR app, which was displayed after 0.2 s at the latest.

Keywords:

Virtual studio, video streaming, WebRTC, RTP, server-client architecture, clock synchronization, timestamps, GStreamer, ARCore, Unity

Supervisor:

Prof. Jens Herder, Dr. Eng./Univ. of Tsukuba
Prof. Dr.-Ing. Thomas Bonse

Location:

The research took place at the Virtual Sets and Virtual Environments Laboratory.

VirtuellesStudio

HSD FB 5 VSVR

30.10.2023

Seitenanfang ^^ 

Kontakt >> 
Suche >> 
Impressum >>