HS Düsseldorf >>
Fachbereiche >>
Institute >>
Organe und Gremien >>
Verwaltung >>
Hochschulbibliothek >>
Campus IT >>
Suche & Hilfe >>

FB 1 Architektur >>
FB 2 Design >>
FB 3 Elektro- & Informationstechnik >>
FB 4 Maschinenbau >>
FB 5 Medien >>
FB 6 Sozial/Kulturwiss. >>
FB 7 Wirtschaft >>

Aktuelles >>
Studium >>
Personen >>
Showroom >>
Studienbüro >>
Dekanat >>
Internationales >>
Forschung & Entwicklung >>

Projekte >>
Labore, Pools & Studios >>


	Hochschule Düsseldorf University of Applied Sciences


	Fachbereich Medien Faculty of Media


	Labor für Virtuelles Studio / Virtuelle Realität

Publikationen >>

Events >>

Lehre >>

Anfahrt >>

Labor Suche >>

Aktuelles >>

Captury Live - Markerless actor tracking at the Hochschule Düsseldorf, University of Applied Sciences [DE]




		$Captury: Occlusion and refraction based on actor position$

Short Description

A Captury Live system with 8 cameras from The Captury GmbH is installed at the Virtual Studio of the Hochschule Düsseldorf, University of Applied Sciences.

The system is used for realtime control of virtual environments and virtual (tv) productions as well as for motion capturing (MoCap). Without the need of markers attached to the actors, the system can track 1 to 3 people in the working volume. The system delivers 22 position and rotation data per actor. Additionally finger movements can be recorded. It can be used for following example applications:

Control of avatars
Full-body interactions in space
Resolving the depth relation within a Virtual Studio
Control of a robotic camera
Control of a moving head
Motion capturing (MoCap) for animations

Following software environments or formats are supported at the virtual studio:

Viz Artist
Unreal
Unity
FBX
OSC

The system substitutes the Openstage system from Organic Motion, which was used for many developments and productions.

Video

The media player is loading...

Interactive Virtual Studio 2019

The set was developed as part of the class interactive virtual studio within the master program media informatics at the Faculty of Media at the Hochschule Düsseldorf, University of Applied Sciences. The set shows a table with a translucent figure, demonstrating reflections and refractions. Talent is part of the scene as video texture, which moves in depth using the marker less talent tracking system, enabling correct occlusion. The production was performed live, that is, tracking, rendering and compositing took place in real-time. The Tracking Hub software for distributing camera tracking data is provided as part of a cooperation with Vizrt Austria. During recording, a manually operated camera Jib-arm (JanJib) from Egripment was used. The camera movements create a motion parallax, so that the space unfolds and the audiences experience the depth clearly. The Captury Live system provides skeletal data of actors for control of a virtual set like animating an avatar or using a shadow of an avatar. The highlight of this production was controlling a moving head using the position data of the talent. The light follows the person as well as the virtual light at the same position, which also is used for casting shadows.

Development team

Oliver Mertens, Manuel Zohlen

Production team

Dennis Amuser, Lara Bertram, Jens Herder, Oliver Mertens, Antje Müller, Manuel Zohlen

Keywords

Virtual Studio, Actor tracking, Interaction, Shadow cast, Occlusion, Reflections, Refractions, Moving head

Tools and Credits

Tracking Hub software by Vizrt
Camera systems - Jibarm (JanJib) by Egripment
Markerless actor tracking - Captury Live by The Captury GmbH
Unreal Rendering Engine
Moving Head Varyscan P6, JB-Lighting, provided by the Presentation and lighting technology laboratory

Downloads

Description	Size	Filetype
Interactive Virtual Studio 2019 - Time 5:05	233 MB	mp4 >>
MoCap recording of a walk (data)	618 KB	FBX >>
MoCap recording of a walk (video)	1,4 MB	mp4 >>
Example for Mocap data in AR	web	glb/usdz >>

Publications

Semi-Automatic Camera and Switcher Control for Live Broadcast: Jeff Daemen, Jens Herder, Cornelius Koch, Philipp Ladwig, Roman Wiche, and Kai Wilgen, International Conference on Interactive Experiences for Television and Online Video (TVX'2016), Chicago, IL, USA, ACM, DOI=10.1145/2932206.2933559, June 22-24, 2016.
Four Metamorphosis States in a Distributed Virtual (TV) Studio: Human, Cyborg, Avatar, and Bot - Markerless Tracking and Feedback for Realtime Animation Control: Jens Herder, Jeff Daemen, Peter Haufs-Brusberg, and Isis Abdel Aziz, Virtual Realities, pp 16-32, Springer International Publishing, DOI=10.1007/978-3-319-17043-5_2, ISBN 978-3-319-17042-8, April 1st, 2015.
SpiderFeedback - Visual feedback for Orientation in Virtual TV Studios: Jonathan Simsch and Jens Herder, 11th International Conference on Advances in Computer Entertainment, ACE 2014, Funchal, Portugal, November 11-14, 2014.
Four Metamorphosis States in a Distributed Virtual (TV) Studio: Human, Cyborg, Avatar, and Bot: Jose Burga, Jeff Daemen, Sascha Charlie Djuderija, Maren Gnehr, Lars Goossens, Sven Hartz, Peter Haufs-Brusberg, Jens Herder, Mohammed Ibrahim, Nikolas Koop, Christophe Leske, Laurid Meyer, Antje Müller, Björn Salgert, Richard Schroeder, and Simon Thiele, 10th International Conference on Visual Media Production (CVMP 2013), short paper, London, November, 2013.
Markerless Actor Tracking for Virtual (TV) Studio Applications: Jeff Daemen, Peter Haufs-Brusberg, and Jens Herder, International Joint Conference on Awareness Science and Technology & Ubi-Media Computing (iCAST 2013 & UMEDIA 2013), HC-2013 Track, Hamamatsu/Aizu-Wakamatsu/Duesseldorf, November, 2013.