HS Düsseldorf >> 
Fachbereiche >> 
Institute >> 
Organe und Gremien >> 
Verwaltung >> 
Hochschulbibliothek >> 
Campus IT >> 
Suche & Hilfe >> 

FB 1 Architektur >> 
FB 2 Design >> 
FB 3 Elektro- & Informationstechnik >> 
FB 4 Maschinenbau >> 
FB 5 Medien >> 
FB 6 Sozial/Kulturwiss. >> 
FB 7 Wirtschaft >> 

Aktuelles >> 
Studium >> 
Personen >> 
Showroom >> 
Studienbüro >> 
Dekanat >> 
Internationales >> 
Forschung & Entwicklung  >> 

Projekte >> 
Labore, Pools & Studios  >> 

HSD

Hochschule Düsseldorf
University of Applied Sciences


FB5

Fachbereich Medien
Faculty of Media

Labor für Virtuelles Studio / Virtuelle Realität



Publikationen >>
Presentationen >>
Forschung >>
Technologie >>
Kooperationen >>
Galerie >>
Abschlussarbeiten >>
Events >>
Lehre >>
Anfahrt >>
Labor Suche >>
Aktuelles >>

Captury Live - Markerless actor tracking at the Hochschule Düsseldorf, University of Applied Sciences [DE]

 

Captury: Control room Captury Live: Person overlayed with avatar Captury Live: Camera within the light Captury Live: Single camera at the truss
Captury Live: FBX Motion Capturing Captury Live: Vision Processor Captury Live: Back of pc with firewire cables and koaxial range extension Captury Live: Extrinsic Calibration
Captury Live: Avatar Skeleton Captury Live: Avatar in Unity with live control Captury: Camera viewfinder shows mixed scene Captury: Actor position controls moving head
Captury: Actor position controls (real) light spot Captury: Shadow based on avatar Captury: Occlusion and refraction based on actor position Captury: Avatar copies movement

 

Short Description

 

A Captury Live system with 10 cameras (optional with 18 cameras) from The Captury GmbH is installed at the Virtual Studio of the Hochschule Düsseldorf, University of Applied Sciences.

The system is used for realtime control of virtual environments and virtual (tv) productions as well as for motion capturing (MoCap). Without the need of markers attached to the actors, the system can track 1 to 3 people in the working volume. The system delivers 22 position and rotation data per actor. It can be used for following example applications:

  • Control of avatars
  • Full-body interactions in space
  • Resolving the depth relation within a Virtual Studio
  • Control of a robotic camera
  • Control of a moving head
  • Motion capturing (MoCap) for animations
Following software environments or formats are supported at the virtual studio:
  • Viz Artist
  • Unreal
  • Unity
  • FBX
  • OSC
The system substitutes the Openstage system from Organic Motion, which was used for many developments and productions.

 

Video

 

The media player is loading...

 

Interactive Virtual Studio 2019

 

The set was developed as part of the class interactive virtual studio within the master program media informatics at the Faculty of Media at the Hochschule Düsseldorf, University of Applied Sciences. The set shows a table with a translucent figure, demonstrating reflections and refractions. Talent is part of the scene as video texture, which moves in depth using the marker less talent tracking system, enabling correct occlusion. The production was performed live, that is, tracking, rendering and compositing took place in real-time. The Tracking Hub software for distributing camera tracking data is provided as part of a cooperation with Vizrt Austria. During recording, a manually operated camera Jib-arm (JanJib) from Egripment was used. The camera movements create a motion parallax, so that the space unfolds and the audiences experience the depth clearly. The Captury Live system provides skeletal data of actors for control of a virtual set like animating an avatar or using a shadow of an avatar. The highlight of this production was controlling a moving head using the position data of the talent. The light follows the person as well as the virtual light at the same position, which also is used for casting shadows.

 

Development team

 

Oliver Mertens, Manuel Zohlen

 

Production team

 

Dennis Amuser, Lara Bertram, Jens Herder, Oliver Mertens, Antje Müller, Manuel Zohlen

 

Keywords

 

Virtual Studio, Actor tracking, Interaction, Shadow cast, Occlusion, Reflections, Refractions, Moving head

 

Tools and Credits

Tracking Hub software by Vizrt
Camera systems - Jibarm (JanJib) by Egripment
Markerless actor tracking - Captury Live by The Captury GmbH
Unreal Rendering Engine
Moving Head Varyscan P6, JB-Lighting, provided by the Presentation and lighting technology laboratory

Downloads

 

Description SizeFiletype
Interactive Virtual Studio 2019 - Time 5:05233 MBmp4 >> 
MoCap recording of a walk (data)618 KBFBX >> 
MoCap recording of a walk (video)1,4 MBmp4 >> 

 

Publications

Semi-Automatic Camera and Switcher Control for Live Broadcast
Jeff Daemen, Jens Herder, Cornelius Koch, Philipp Ladwig, Roman Wiche, and Kai Wilgen, International Conference on Interactive Experiences for Television and Online Video (TVX'2016), Chicago, IL, USA, ACM, DOI=10.1145/2932206.2933559, June 22-24, 2016.
Four Metamorphosis States in a Distributed Virtual (TV) Studio: Human, Cyborg, Avatar, and Bot - Markerless Tracking and Feedback for Realtime Animation Control
Jens Herder, Jeff Daemen, Peter Haufs-Brusberg, and Isis Abdel Aziz, Virtual Realities, pp 16-32, Springer International Publishing, DOI=10.1007/978-3-319-17043-5_2, ISBN 978-3-319-17042-8, April 1st, 2015.
SpiderFeedback - Visual feedback for Orientation in Virtual TV Studios
Jonathan Simsch and Jens Herder, 11th International Conference on Advances in Computer Entertainment, ACE 2014, Funchal, Portugal, November 11-14, 2014.
Four Metamorphosis States in a Distributed Virtual (TV) Studio: Human, Cyborg, Avatar, and Bot
Jose Burga, Jeff Daemen, Sascha Charlie Djuderija, Maren Gnehr, Lars Goossens, Sven Hartz, Peter Haufs-Brusberg, Jens Herder, Mohammed Ibrahim, Nikolas Koop, Christophe Leske, Laurid Meyer, Antje Müller, Björn Salgert, Richard Schroeder, and Simon Thiele, 10th International Conference on Visual Media Production (CVMP 2013), short paper, London, November, 2013.
Markerless Actor Tracking for Virtual (TV) Studio Applications
Jeff Daemen, Peter Haufs-Brusberg, and Jens Herder, International Joint Conference on Awareness Science and Technology & Ubi-Media Computing (iCAST 2013 & UMEDIA 2013), HC-2013 Track, Hamamatsu/Aizu-Wakamatsu/Duesseldorf, November, 2013.

Productions

Actor Tracking for rule-based Camera and Switcher Control in a Virtual (TV) Studio
Production at the Virtual Studio of the Hochschule Düsseldorf, 2014 and 2015.
Explore the Grand Canyon! - Interactive Virtual Studio
Production at the Virtual Studio of the Hochschule Düsseldorf for the Studio5 day, SS 2015.
Subtle Animations using Talent Tracking in a Virtual (TV) Studio
Production at the Virtual Studio of the FH Düsseldorf, 2014.
SpiderFeedback - Rope-based, visual feedback for orientation in Virtual Studios
Production at the Virtual Studio of FH Düsseldorf, 2014.
VRON: Virtual Reality over Networks - FOUR METAMORPHOSIS STATES IN A DISTRIBUTED VIRTUAL (TV) STUDIO: HUMAN, CYBORG, AVATAR, AND BOT - A Virtual Studio Production of the Department of Media
Production at the Virtual Studio of FH Düsseldorf, 2013.
Markerloses Darsteller Tracking für Anwendungen im Virtuellen Studio
Production at the Virtual Studio of FH Düsseldorf, 2013.

Thesis

Markerloses Moderatorentracking im virtuellen Studio unter Verwendung von Silhouetten mehrerer Kamerabilder
Jeff Daemen, Bachelor thesis, 13.5.2013
Erweiterung des markerlosen Personentrackings für virtuelle Studio Live-Animationen
Isis Abdel Aziz, Bachelor thesis, 2.12.2014
Virtuelles Studio

HSD FB 5 VSVR

30.10.2023

Seitenanfang ^^ 

Kontakt >> 
Suche >> 
Impressum >>