HS Düsseldorf >> 
Fachbereiche >> 
Institute >> 
Organe und Gremien >> 
Verwaltung >> 
Hochschulbibliothek >> 
Campus IT >> 
Suche & Hilfe >> 

FB 1 Architektur >> 
FB 2 Design >> 
FB 3 Elektro- & Informationstechnik >> 
FB 4 Maschinenbau >> 
FB 5 Medien >> 
FB 6 Sozial/Kulturwiss. >> 
FB 7 Wirtschaft >> 

Aktuelles >> 
Studium >> 
Personen >> 
Showroom >> 
Studienbüro >> 
Dekanat >> 
Internationales >> 
Forschung & Entwicklung  >> 

Projekte >> 
Labore, Pools & Studios  >> 


Hochschule Düsseldorf
University of Applied Sciences


Fachbereich Medien
Faculty of Media

Labor für Virtuelles Studio / Virtuelle Realität

Publikationen >>
Presentationen >>
Forschung >>
Technologie >>
Kooperationen >>
Galerie >>
Abschlussarbeiten >>
Events >>
Lehre >>
Anfahrt >>
Labor Suche >>
Aktuelles >>

Captury Live - Markerless actor tracking at the Hochschule Düsseldorf, University of Applied Sciences [DE]


Captury: Control room Captury Live: Person overlayed with avatar Captury Live: Camera within the light Captury Live: Single camera at the truss
Captury Live: FBX Motion Capturing Captury Live: Vision Processor Captury Live: Back of pc with firewire cables and koaxial range extension Captury Live: Extrinsic Calibration
Captury Live: Avatar Skeleton Captury Live: Avatar in Unity with live control Captury: Camera viewfinder shows mixed scene Captury: Actor position controls moving head
Captury: Actor position controls (real) light spot Captury: Shadow based on avatar Captury: Occlusion and refraction based on actor position Captury: Avatar copies movement


Short Description


A Captury Live system with 10 cameras (optional with 18 cameras) from The Captury GmbH is installed at the Virtual Studio of the Hochschule Düsseldorf, University of Applied Sciences.

The system is used for realtime control of virtual environments and virtual (tv) productions as well as for motion capturing (MoCap). Without the need of markers attached to the actors, the system can track 1 to 3 people in the working volume. The system delivers 22 position and rotation data per actor. It can be used for following example applications:

  • Control of avatars
  • Full-body interactions in space
  • Resolving the depth relation within a Virtual Studio
  • Control of a robotic camera
  • Control of a moving head
  • Motion capturing (MoCap) for animations
Following software environments or formats are supported at the virtual studio:
  • Viz Artist
  • Unreal
  • Unity
  • FBX
  • OSC
The system substitutes the Openstage system from Organic Motion, which was used for many developments and productions.




The media player is loading...


Interactive Virtual Studio 2019


The set was developed as part of the class interactive virtual studio within the master program media informatics at the Faculty of Media at the Hochschule Düsseldorf, University of Applied Sciences. The set shows a table with a translucent figure, demonstrating reflections and refractions. Talent is part of the scene as video texture, which moves in depth using the marker less talent tracking system, enabling correct occlusion. The production was performed live, that is, tracking, rendering and compositing took place in real-time. The Tracking Hub software for distributing camera tracking data is provided as part of a cooperation with Vizrt Austria. During recording, a manually operated camera Jib-arm (JanJib) from Egripment was used. The camera movements create a motion parallax, so that the space unfolds and the audiences experience the depth clearly. The Captury Live system provides skeletal data of actors for control of a virtual set like animating an avatar or using a shadow of an avatar. The highlight of this production was controlling a moving head using the position data of the talent. The light follows the person as well as the virtual light at the same position, which also is used for casting shadows.


Development team


Oliver Mertens, Manuel Zohlen


Production team


Dennis Amuser, Lara Bertram, Jens Herder, Oliver Mertens, Antje Müller, Manuel Zohlen




Virtual Studio, Actor tracking, Interaction, Shadow cast, Occlusion, Reflections, Refractions, Moving head


Tools and Credits

Tracking Hub software by Vizrt
Camera systems - Jibarm (JanJib) by Egripment
Markerless actor tracking - Captury Live by The Captury GmbH
Unreal Rendering Engine
Moving Head Varyscan P6, JB-Lighting, provided by the Presentation and lighting technology laboratory



Description SizeFiletype
Interactive Virtual Studio 2019 - Time 5:05233 MBmp4 >> 
MoCap recording of a walk (data)618 KBFBX >> 
MoCap recording of a walk (video)1,4 MBmp4 >> 



Semi-Automatic Camera and Switcher Control for Live Broadcast
Jeff Daemen, Jens Herder, Cornelius Koch, Philipp Ladwig, Roman Wiche, and Kai Wilgen, International Conference on Interactive Experiences for Television and Online Video (TVX'2016), Chicago, IL, USA, ACM, DOI=10.1145/2932206.2933559, June 22-24, 2016.
Four Metamorphosis States in a Distributed Virtual (TV) Studio: Human, Cyborg, Avatar, and Bot - Markerless Tracking and Feedback for Realtime Animation Control
Jens Herder, Jeff Daemen, Peter Haufs-Brusberg, and Isis Abdel Aziz, Virtual Realities, pp 16-32, Springer International Publishing, DOI=10.1007/978-3-319-17043-5_2, ISBN 978-3-319-17042-8, April 1st, 2015.
SpiderFeedback - Visual feedback for Orientation in Virtual TV Studios
Jonathan Simsch and Jens Herder, 11th International Conference on Advances in Computer Entertainment, ACE 2014, Funchal, Portugal, November 11-14, 2014.
Four Metamorphosis States in a Distributed Virtual (TV) Studio: Human, Cyborg, Avatar, and Bot
Jose Burga, Jeff Daemen, Sascha Charlie Djuderija, Maren Gnehr, Lars Goossens, Sven Hartz, Peter Haufs-Brusberg, Jens Herder, Mohammed Ibrahim, Nikolas Koop, Christophe Leske, Laurid Meyer, Antje Müller, Björn Salgert, Richard Schroeder, and Simon Thiele, 10th International Conference on Visual Media Production (CVMP 2013), short paper, London, November, 2013.
Markerless Actor Tracking for Virtual (TV) Studio Applications
Jeff Daemen, Peter Haufs-Brusberg, and Jens Herder, International Joint Conference on Awareness Science and Technology & Ubi-Media Computing (iCAST 2013 & UMEDIA 2013), HC-2013 Track, Hamamatsu/Aizu-Wakamatsu/Duesseldorf, November, 2013.


Actor Tracking for rule-based Camera and Switcher Control in a Virtual (TV) Studio
Production at the Virtual Studio of the Hochschule Düsseldorf, 2014 and 2015.
Explore the Grand Canyon! - Interactive Virtual Studio
Production at the Virtual Studio of the Hochschule Düsseldorf for the Studio5 day, SS 2015.
Subtle Animations using Talent Tracking in a Virtual (TV) Studio
Production at the Virtual Studio of the FH Düsseldorf, 2014.
SpiderFeedback - Rope-based, visual feedback for orientation in Virtual Studios
Production at the Virtual Studio of FH Düsseldorf, 2014.
VRON: Virtual Reality over Networks - FOUR METAMORPHOSIS STATES IN A DISTRIBUTED VIRTUAL (TV) STUDIO: HUMAN, CYBORG, AVATAR, AND BOT - A Virtual Studio Production of the Department of Media
Production at the Virtual Studio of FH Düsseldorf, 2013.
Markerloses Darsteller Tracking für Anwendungen im Virtuellen Studio
Production at the Virtual Studio of FH Düsseldorf, 2013.


Markerloses Moderatorentracking im virtuellen Studio unter Verwendung von Silhouetten mehrerer Kamerabilder
Jeff Daemen, Bachelor thesis, 13.5.2013
Erweiterung des markerlosen Personentrackings für virtuelle Studio Live-Animationen
Isis Abdel Aziz, Bachelor thesis, 2.12.2014
Virtuelles Studio



Seitenanfang ^^ 

Kontakt >> 
Suche >> 
Impressum >>