 |
Presentation: 3DGS, AR, and VR for Virtual Productions - Vizrt Days 2026
Jens Herder, 3DGS, AR, and VR for Virtual Productions - Vizrt Days 2026, Munich, 20. May 2026.
Virtual productions can benefit from Augmented Reality (AR) and Virtual Reality (VR)
to facilitate remote control and easy on-location use. We demonstrate how AR can
assist in setting up studio lights and configuring broadcast studios.
We also present the development of AR applications for controlling broadcast studio
equipment using video-see-through head-mounted displays (HMDs).
These applications integrate various classic studio devices and support features
like multiviewer/video switcher, lighting control, prompting and camera robotics.
User interfaces utilise hand and eye tracking to keep hands free and speed up
workflows. Additional information aids operator decision-making during live
production. Recent advances in Gaussian Splats for rendering and capturing
environments enhance set design. AI can even generate environments from a single
image, further speeding up the process. As VR and AR become more widespread,
a virtual studio will showcase VR and AR user experiences. Marker-less motion
tracking (MoCap) not only facilitates interaction with virtual objects or avatars
in front of the camera but also resolves occlusion. Furthermore, it enables
robotic camera control for sophisticated framing, taking graphical
objects into account.
Keywords:
Mixed Reality, Augmented Reality, MoCap, 3DGS, Co-located AR, Remote AR/VR
Additional Material
| Description | Time | Access |
| Video recording of the presentation by Vizrt |
31:11 Min | Vimeo 2K >> |
| Presentation slides | 34 Slides |
78,8 MB >> |
Demonstrations
The main hall with the stage for presentations and panels was scanned on the day
before and become a virtual set. The set got video textures for live feeds included.
Interactive physics were demonstrated with chairs which could be kicked away by
people on the virtual stage. Chairs could even fall down from the stage. This was
possible because the scan for 3DGS including also bounding boxes. The live
demonstration was getting live video from the AI keyer, including camera
and person tracking data. A virtual flight through the hall was used as an
opener.
Another demonstration showed virtual presence could be used to control remotely
the light in the virtual studio in Düsseldorf using VR and 360 video feed. The 3DGS
could also be experienced in VR on Vison Pro HMD.
Links
- 3DGS - Gaussian Splats für Virtuelle Produktionen, VR und AR
- Vizrt - Virtual Studio Software for Research and Education at the HS Düsseldorf, University of Applied Sciences
- Captury Live - Markerless actor tracking at the Hochschule Düsseldorf, University of Applied Sciences
Productions
- WDR: Der Haushalts-Check mit Yvonne Willicks - Wandfarbe für zuhause: Preis, Qualität und Wirkung!
- Presenting VR in a Virtual Studio, Production partly at the Virtual Studio of the Hochschule Düsseldorf, Synchronisation of two tracked cameras and two HMDs, 13.5.2026
- Interachtive Virtual Studio - "Aquaboom"
- Production at the Virtual Studio of the Hochschule Düsseldorf with 3DGS generated using Marble, 2025.
- Interactive Virtual Studio - MoCap - Tech Demo "Bulletproof"
- Production at the Virtual Studio of the Hochschule Düsseldorf with Viz Artist 5 / Unreal 5 and Captury Live, 2023.
Publications
- Augmented Reality for Lighting Adjustment in a Virtual Studio
- Maja Michaelis and Jens Herder, in "Augmented Reality - Situated Spatial Synergy",
IntechOpen, DOI=10.5772/intechopen.10124242, October, 2025
- Distributed Metaverse: Creating Decentralized Blockchain-based Model for Peer-to-peer Sharing of Virtual Spaces for Mixed Reality Applications
- Bektur Ryskeldiev, Yoichi Ochiai, Michael Cohen, Jens Herder,
Proceedings of the 9th Augmented Human International Conference,
Seoul, Republic of Korea, ACM, DOI=10.1145/3174910.3174952, Februar 7-9, 2018
- Augmented Reality for
Studio Broadcast Systems - from Live Video Switching using Contextual
Information to Configuration and Control of Studio Equipment
- Nico Balthazar, Jannik Hölper, and Jens Herder,
Extended Reality: International Conference, XR Salento 2026, Otranto, Italy,
June 16–20, 2026, Proceedings. Springer Nature, 2026.
- Kosteneffektive, volumetrische Erfassung von Baugruben unter Verwendung von AR
- Mathias Langmann, Jens Herder, Monika Jarosch, 9. DEUTSCHES GEOFORUM 2021,
Bedeutung der Geoinformation für Gesellschaft und Wirtschaft, 25.11.2021.
Theses
- NeRFs and Gaussian Splatting for Sets at Virtual Studio
- Christopher Grimm, Hochschule Düsseldorf - University of Applied Sciences, Bachelor thesis, 07.11.2024.
- Immersive Design of Sign
Language Avatars for Video Streams in VR: Analyzing the Impact of Visual Customization Elements
- Joshua Lasse Einhoff, Hochschule Düsseldorf - University of Applied Sciences, Bachelor thesis, 22.09.2025.
- Development of an AR Interface for Live Video Switching using Contextual Information to Aid Decision-Making
- Nico Balthazar, Hochschule Düsseldorf - University of Applied Sciences, Bachelor thesis, 11.02.2026.
- Development of an AR application for controlling and configuring television studio components
- Jannik Hölper, Hochschule Düsseldorf - University of Applied Sciences, Master thesis, 24.03.2026.
- Development of an Automated Camera Control System for Virtual Production Using Real-Time Person Tracking
- Lars Hötte, Hochschule Düsseldorf - University of Applied Sciences, Bachelor thesis, 24.02.2026.
-
- Integration of Computer-Generated Content in Gaussian Splatting Scenes: Shadow Catching and Hybrid Light Estimation
- Tobias Preuss, Hochschule Düsseldorf - University of Applied Sciences, Master thesis, 5.05.2026
|
|
