HS Düsseldorf >> 
Fachbereiche >> 
Institute >> 
Organe und Gremien >> 
Verwaltung >> 
Hochschulbibliothek >> 
Campus IT >> 
Suche & Hilfe >> 

FB 1 Architektur >> 
FB 2 Design >> 
FB 3 Elektro- & Informationstechnik >> 
FB 4 Maschinenbau >> 
FB 5 Medien >> 
FB 6 Sozial/Kulturwiss. >> 
FB 7 Wirtschaft >> 

Aktuelles >> 
Studium >> 
Personen >> 
Showroom >> 
Studienbüro >> 
Dekanat >> 
Internationales >> 
Forschung & Entwicklung  >> 

Projekte >> 
Labore, Pools & Studios  >> 


Hochschule Düsseldorf
University of Applied Sciences


Fachbereich Medien
Faculty of Media

Labor für Virtuelles Studio / Virtuelle Realität

Publikationen >>
Presentationen >>
Forschung >>
Technologie >>
Kooperationen >>
Galerie >>
Abschlussarbeiten >>
Events >>
Lehre >>
Anfahrt >>
Labor Suche >>
Aktuelles >>

Abstract: Realtime Relighting of a Video Stream in an Augmented Virtuality Scene Using Markerless Motion Tracking [DE]

Till Davin, Realtime Relighting of a Video Stream in an Augmented Virtuality Scene Using Markerless Motion Tracking, Hochschule Düsseldorf, Master Thesis, 10.02.2021.

The combination of real and virtual image elements is inevitable in virtual productions and mixed reality applications. One major problem in combining real and virtual elements in a single image lies in the visual discrepancy between virtual and real content, whether it is in the form of different lighting conditions, shadows or wrong object occlusion. These differences are easily noticeable to the human eye and counteract the desired immersion. This thesis seeks to address these challenges in augmented reality and virtual production, and presents an implementation of realtime-relighting of a video stream with the light setting of a virtual scene as a possible solution. The goal of this work is to develop an application that matches the recording of an actor with that of a virtual scene, and blurs lighting differences for the human eye. This matching will be done in realtime to meet the needs of today’s mixed reality applications and virtual productions. After investigating different possible approaches to solving the relighting problem, a system is realized that utilizes image processing and modern graphics calculations to relight the input video stream. First the two-dimensional contour is extracted from the video image through image processing. The contour is then used to generate a displacement texture. With this displacement texture the two-dimensional video plane is given three-dimensional depth. The video stream then can be relit with the virtual light in the scene and the acquired normal vectors and depth information from the generated geometry. Further, an integrated markerless motion-tracking system creates the possibility to correctly position the geometry in the scene. Image results, advantages, and disadvantages of the proposed system are discussed and evaluated.


Real-Time Rendering, Markerless Motion Tracking, Virtual Studio, Augmented Virtuality, Relighting, Light Estimation


Prof. Jens Herder, Dr. Eng./Univ. of Tsukuba
Prof. Dr. rer. nat. Christian Geiger


The research took place at the Virtual Sets and Virtual Environments Laboratory.




Seitenanfang ^^ 

Kontakt >> 
Suche >> 
Impressum >>