The Enhanced Virtual Reality (EVR) system aims to glorify the role of the music performer by amplifying her or his persona and live performance at a concert. The authors believe that there are many different ways to approach EVR, and some of these are being explored in a series of VR experiments. For example, it seems consideration should be given to enhancing the performer’s appearance in some ways such as using virtual costumes, makeup, alterations to the body, etc. Alternatively, the performer’s appearance could be completely transformed by mapping her or his motions onto a virtual, custom-designed avatar. This could be used to create an embodied narrative.

  ​

 

The present project aims to create a VR experience for a live concert scenario. The prototype system works as follows. A Kinect V2 motion tracker is used to capture the motions of a musician performing live. This data is fed to the Unity game engine, which causes a virtual character to move in response to the musician’s movements. A visual from the game engine is extracted to be presented in synchrony with the audio.

The example scene “Duke Skellington Plays the Xylobone” renders the musician as a skeleton sending back music of bone-like sounds from the afterlife. The experience of watching the EVR performance transcends the mere combinations of the sounds, to the point at which the music and the narrative fuse together into an Enhanced Virtual Reality experience, aimed at immersing an audience member into a whole new nexus of sound, music, image, and experience.

This document is licensed according to the Creative Commons Attribution-ShareAlike 2.0 Generic license, which can be found at https://creativecommons.org/licenses/by-sa/2.0/ The virtual band scene in Fig. 3 is from https://www.flickr.com/photos/bagogames/25962114290/