³ÉÈË¿ìÊÖ

We met Florian Schweiger, Senior Technologist for ³ÉÈË¿ìÊÖ Research & Development and Jenny Chapman, Innovation producer for ³ÉÈË¿ìÊÖ iWonder, to chat about the 3D Dancer

Tell us about your 3D Dancer

FS: This dance video features a ballerina and has five moments where the video becomes 3D data and the user can twist, turn and explore the scene. Each moment is annotated, this allows the user to learn more about our bodies and how we have used them over the years to tell stories. After each moment the scene then transforms back into video.

How were each of you involved?

FS: Well, this is a collaboration between the REACT Project, ³ÉÈË¿ìÊÖ Research & Development and ³ÉÈË¿ìÊÖ iWonder. My job was on the technical side of this project, coordinating the efforts of all the partners that made this demo happen. Besides the 3D video player you see here, I was mainly responsible for the 3D capture of the ballerina in our blue-screen studio.

JC: In my day-to-day job for ³ÉÈË¿ìÊÖ iWonder I explore new technologies and audience behaviours. I then advise and influence the development of the iWonder site. When I saw a demo of ³ÉÈË¿ìÊÖ Research and Development’s work in turning video into data I immediately wanted to use this in a Learning context.

What were you trying to achieve with this project?

JC: I know very little about dance, particularly ballet and I wanted to see if this technology could present that information in a compelling way. I was also trying to see if this technology would work with different stories or in another context such as sport.

FS: The technical goal was to show that 3D data could be extracted from ordinary video without complicated motion capture technology that would interfere with the dancer’s performance.

And you succeeded…

FS: Yes. We recorded the dance using 9 HD cameras surrounding the performance area in our blue-screen studio. The images were processed to create a 3D model of the ballerina. A video was edited together, showing the ballerina in a simple virtual space. The video automatically pauses at key moments and the display cross-fades to a view of the 3D model rendered in the web browser that exactly matches that of the camera. When the user has finished exploring the scene, the virtual viewpoint is automatically returned to match that of the camera and the video is faded up again.

Sounds complicated, what is the technology?

FS: We used a prototype system that we built with other partners in the REACT project that can create 3D models directly from video. We then turned the model into a form that can be rendered in a modern web browser using ‘WebGL’ technology.

And has it turned out how you hoped?

JC: I wasn’t sure what to expect! During the filming there was a lot of tech involved like motion sensors and our poor dancer spent a long time with things attached to her.

FS: …which, to be fair, were only used to record reference data for comparison. For the results shown here we could spare our dancer the trouble of having markers attached to her body. So yes, we’ve achieved what we wanted to.

What do you hope to learn from 3D Dancer being on Taster?

JC: I want to see if audiences like interacting with video in this way – and if they learn anything new from it.

FS: If video can become 3D then we can do some amazing things as a broadcaster in the digital world.

Exciting times! Whats next?

JC: Watch this space as we have just been filming a man making a chicken out of a 3D box…