Captury Live - Markerless actor tracking at the Hochschule Düsseldorf, University of Applied Sciences [DE]
A Captury Live system with 10 cameras (optional with 18 cameras) from The Captury GmbH is installed at the Virtual Studio of the Hochschule Düsseldorf, University of Applied Sciences.
The system is used for realtime control of virtual environments and virtual (tv) productions as well as for motion capturing (MoCap). Without the need of markers attached to the actors, the system can track 1 to 3 people in the working volume. The system delivers 22 position and rotation data per actor. It can be used for following example applications:
Following software environments or formats are supported at the virtual studio:
- Control of avatars
- Full-body interactions in space
- Resolving the depth relation within a Virtual Studio
- Control of a robotic camera
- Control of a moving head
- Motion capturing (MoCap) for animations
The system substitutes the Openstage system from Organic Motion, which was used for many developments and productions.
- Viz Artist
The media player is loading...
Interactive Virtual Studio 2019
The set was developed as part of the class interactive virtual studio within the master program media informatics at the Faculty of Media at the Hochschule Düsseldorf, University of Applied Sciences.
The set shows a table with a translucent figure, demonstrating reflections and refractions. Talent is part of the scene as video texture, which moves in depth using the marker less talent tracking system, enabling correct occlusion.
The production was performed live, that is, tracking, rendering and compositing took place in real-time. The Tracking Hub software for distributing camera tracking data is provided as part of a cooperation with Vizrt Austria.
During recording, a manually operated camera Jib-arm (JanJib) from Egripment was used. The camera movements create a motion parallax, so that the space unfolds and the audiences experience the depth clearly. The Captury Live system provides skeletal data of actors for control of a virtual set like animating an avatar or using a shadow of an avatar. The highlight of this production was controlling a moving head using the position data of the talent. The light follows the person as well as the virtual light at the same position, which also is used for casting shadows.
Virtual Studio, Actor tracking, Interaction, Shadow cast, Occlusion, Reflections, Refractions, Moving head
Tools and Credits
- Tracking Hub software by Vizrt
- Camera systems - Jibarm (JanJib) by Egripment
- Markerless actor tracking - Captury Live by The Captury GmbH
- Unreal Rendering Engine
- Moving Head Varyscan P6, JB-Lighting, provided by the Presentation
and lighting technology laboratory
|Interactive Virtual Studio 2019 - Time 5:05||233 MB||mp4 >> |
|MoCap recording of a walk (data)||618 KB||FBX >> |
|MoCap recording of a walk (video)||1,4 MB||mp4 >> |
- Semi-Automatic Camera and Switcher Control for Live Broadcast
- Jeff Daemen, Jens Herder, Cornelius Koch, Philipp Ladwig, Roman Wiche, and Kai Wilgen, International Conference on Interactive Experiences for Television and Online Video (TVX'2016), Chicago, IL, USA, ACM, DOI=10.1145/2932206.2933559, June 22-24, 2016.
- Four Metamorphosis States in a Distributed Virtual (TV) Studio: Human, Cyborg, Avatar, and Bot - Markerless Tracking and Feedback for Realtime Animation Control
- Jens Herder, Jeff Daemen, Peter Haufs-Brusberg, and Isis Abdel Aziz, Virtual Realities, pp 16-32, Springer International Publishing, DOI=10.1007/978-3-319-17043-5_2, ISBN 978-3-319-17042-8, April 1st, 2015.
- SpiderFeedback - Visual feedback for Orientation in Virtual TV Studios
- Jonathan Simsch and Jens Herder, 11th International Conference on Advances in Computer Entertainment, ACE 2014, Funchal, Portugal, November 11-14, 2014.
- Four Metamorphosis States in a Distributed Virtual (TV) Studio: Human, Cyborg, Avatar, and Bot
- Jose Burga, Jeff Daemen, Sascha Charlie Djuderija, Maren Gnehr, Lars Goossens, Sven Hartz, Peter Haufs-Brusberg, Jens Herder, Mohammed Ibrahim, Nikolas Koop, Christophe Leske, Laurid Meyer, Antje Müller, Björn Salgert, Richard Schroeder, and Simon Thiele, 10th International Conference on Visual Media Production (CVMP 2013), short paper, London, November, 2013.
- Markerless Actor Tracking for Virtual (TV) Studio Applications
- Jeff Daemen, Peter Haufs-Brusberg, and Jens Herder, International Joint Conference on Awareness Science and Technology & Ubi-Media Computing (iCAST 2013 & UMEDIA 2013), HC-2013 Track, Hamamatsu/Aizu-Wakamatsu/Duesseldorf, November, 2013.
- Actor Tracking for rule-based Camera and Switcher Control in a Virtual (TV) Studio
- Production at the Virtual Studio of the Hochschule Düsseldorf, 2014 and 2015.
- Explore the Grand Canyon! - Interactive Virtual Studio
- Production at the Virtual Studio of the Hochschule Düsseldorf for the Studio5 day, SS 2015.
- Subtle Animations using Talent Tracking in a Virtual (TV) Studio
- Production at the Virtual Studio of the FH Düsseldorf, 2014.
- SpiderFeedback - Rope-based, visual feedback for orientation in Virtual Studios
- Production at the Virtual Studio of FH Düsseldorf, 2014.
- VRON: Virtual Reality over Networks - FOUR METAMORPHOSIS STATES IN A DISTRIBUTED VIRTUAL (TV) STUDIO: HUMAN, CYBORG, AVATAR, AND BOT - A Virtual Studio Production of the Department of Media
- Production at the Virtual Studio of FH Düsseldorf, 2013.
- Markerloses Darsteller Tracking für Anwendungen im Virtuellen Studio
- Production at the Virtual Studio of FH Düsseldorf, 2013.
- Markerloses Moderatorentracking im virtuellen Studio unter Verwendung von Silhouetten mehrerer Kamerabilder
- Jeff Daemen, Bachelor thesis, 13.5.2013
- Erweiterung des markerlosen Personentrackings für virtuelle Studio Live-Animationen
- Isis Abdel Aziz, Bachelor thesis, 2.12.2014