Labor für Virtuelles Studio / Virtuelle Realität

Abstract: Immersive Design of Sign Language Avatars for Video Streams in VR: Analyzing the Impact of Visual Customization Elements [DE]

Sign languages are the primary way of communication for millions of Deaf and Hard of Hearing worldwide, and written languages do not provide equivalent accessibility. Recent advances in sign language avatar technology and the rapid adoption of consumer VR/AR devices open up new possibilities for delivering accessible content in immersive environments. This thesis aimed to develop and evaluate a prototype application of immersive sign language avatars for video streams on the Apple Vision Pro. The prototype allowed for the free placement of avatars in both VR and AR modes. Animations were created using motion capture and integrated into the system. Usability and perception were evaluated through a mixed method user study that included questionnaires, open interviews, and eye tracking analysis with sign language experts. The results showed that usability aspects such as ease of placement and background control were positively rated. However, participants criticized the lack of facial animation and hand-shape accuracy, highlighting avatar quality as the main limitation. The immersive multi-avatar scene was regarded as promising but constrained by the use of identical avatars. The findings suggest that sign language avatars are most suitable as a supplement to human interpreters in consumer contexts such as entertainment or public information services. Improving animation quality and customization options will be key to broader acceptance of sign language avatars in immersive media, along with creating research environments in which Deaf and Hard of Hearing individuals are empowered to participate on equal terms.

Keywords:

Virtual Reality, VR, Augmented Reality, AR, Human-Computer Interaction, Human-Avatar Interaction, Sign Language, Avatars, German Sign Language, Signing Avatars

Supervisor:

Prof. Jens Herder, Dr. Eng./Univ. of Tsukuba
Prof. Dr.-Ing. Thomas Bonse

Location:

The research took place at the Virtual Sets and Virtual Environments Laboratory.