Set to BMI songwriter David Bowie’s iconic track “Heroes,” Creative Director Melissa Painter and her key collaborators have crafted a multidimensional experience using 2-D, virtual reality, and augmented reality headsets. The stage is the historic Theater at Ace Hotel in Los Angeles where the film component was created, and the plot is to immerse the audience in a fiercely athletic and romantic dance that lets them chase the dancers around a pool and on stage, scale through a labyrinth of stunning architecture, and even reach out and hold the dancers in the palm of their hands. It is truly “A Duet in Mixed Reality” and will find its premiere at Sundance 2017, starting Friday, January 20, at Claim Jumper on Main Street in Park City.
BMI caught up with the award-winning filmmaker/author to find out how and why she and her team of fellow innovators were inspired to take on such an ambitious project. However, this is not the first deep dive Painter has made into the world of immersive content, having premiered previous projects and feature films to enthusiastic audiences at Sundance, SXSW, New York, Cannes, and Telluride film festivals. Here’s what she had to say.
Pictured: Melissa Painter
What technology/equipment is used in the performance?
You experience the story in three ways: traditional film, an Oculus Gear VR headset, and the Microsoft HoloLens AR headset. All together it is a duet in mixed reality.
How does the audience participate?
You as a participant are invited in a multiplicity of ways to interact with the dance. The technology in each experience provokes you to be in your own body and to move. The VR challenges you to look closely, and all around you, in order to catch the details hidden within the dance and the architecture. In the Augmented Reality, you move your own body through it, and change it as you do so. You are able to talk to the experience, to touch it, and your path through it alters it. The entire project brings you closer to the athleticism of dance. Each experience consecutively allows deeper and deeper levels of interactivity.
How and where can this be experienced after Sundance?
Sundance is the world premiere. Oculus will be releasing the VR portion of the experience. The full installation will travel to a Helios Dance Theater’s live performance at the Theater at Ace Hotel DTLA on March 5, and then again in LA at Unity’s Vision Summit in May. We hope it will continue to travel beyond those events and that we will have a chance to release the AR portion through the HoloLens app.
What have the reactions been so far?
For many of the people who have tried the experience to date, it is their first time inside augmented reality. I think for all of us as first users (none of us had created with or used this technology before we started this!), and for the people who have seen it so far, it is hard not to be struck with the magic, surprise and awareness that we are truly stepping into a new frontier. It lights up everyone’s imagination - to see a technology that offers such new potential. The digital world is 100% off the screen and in multi-dimensions.
What is the significance of the name “Heroes”?
We first saw the dance with this song—Laura Gorenstein Miller, the choreographer, had made the dance initially without music, and could not find the right song for it—we felt the power of the match. It struck us how Bowie was both inviting and daring us to be heroic. The song is celebratory and encouraging. He reminds us that even if we can’t always sustain those moments of strength, love, or larger than life daring, we still need to catapult ourselves into them. I remember the first time I saw him sing it live, when I was in high school. He was such a heroic figure. Music can help us feel those moments of potential in a visceral way—for me dance and sports do that too—moments of watching human potential at its fullest.
Bowie dares us to be engaged. Something super necessary right at this moment. The story of our piece is about humans being heroic and moving into new realities. The fierceness and athleticism of the dance within it remind us of human potential. The woman and the man carry each other, passing their strength and daring back and forth between each other. Sometimes the woman flat out has to pull the man along with her.
What was the inception point for this project?
We decided to make this, and to do so in the three media, the moment we first saw Laura’s duet set to “Heroes.”
I had worked in emergent media for the past three years, and throughout it had been witness to capabilities that intrigued me: photogrammetry, highly accurate motion capture, the incredibly high resolution capability of the Nokia Ozo camera. I also have both danced, and worked in dance intermittently, my whole life. I knew this was a chance to bring three loves together, the tech experimentation, dance, and architecture.
Why is music and short form a great place to start experimenting with these emergent technologies?
The early days of music videos were laboratories—rapid prototyping cycles for new forms of editing, design, and visual effects. Songs, which are a form of storytelling that is powerful, bite sized, and timeless, gives us firm emotional and humanistic ground to stand on to make the kind of mini experiments we need to make now, to learn from the medium and learn from the work the community is creating in it.
I think that we all know that when film first appeared, it altered human consciousness, created new languages, and enabled empathetic and cathartic storytelling. It took us to parts of the world we could never see, and allowed us to predict new futures.
These new tools, which are characterized by being immersive, interactive, able to blend real and imaginary, and digital and physical, and to exist in multi-dimensions, are powerful delivery mechanisms. I think many of us are experimenting in this space because we believe that these new kinds of experiences can and will change culture.
How did the song drive the creative for the piece?
The song defined the core story elements: the shifts in scale that happen in it as the dancers go from human to giant, and the sense of bravery strength and celebration we wanted the piece to have.
Why is Bowie in particular such a great match for this mixed reality experience?
Bowie was such an exceptionally daring artist in all ways: creating identities, storyworlds, and music across all media. He adopted the new, making it his own at each step. He has been an inspiration to whole generations of artists, certainly to me.
For more information on performances of Heroes at Sundance 2017, visit: http://www.sundance.org/projects/heroes.