Blast from the recent XR past: a VR DAYS 2020 Article

CENTRE STAGE

TOBY COFFEY, HEAD OF THE NATIONAL THEATRE’S IMMERSIVE STORYTELLING STUDIO, LOOKS FOR ‘A SYMBIOTIC RELATIONSHIP BETWEEN TECHNOLOGY AND STORYTELLING’. MOMCHIL ALEXIEV INTERVIEWS HIM ABOUT THE FUTURE OF THE PERFORMING ARTS.

How does the National Theatre in London transform performances into immersive storytelling to reach new audiences?

‘Some work is independent of the repertoire and some is inextricably linked or provoked by it,’ says Toby Coffey, head of the National Theatre’s Immersive Storytelling Studio. For the last five years, the studio has produced a series of immersive experiences. All Kinds of Limbo is an original music piece and performance, created in response to NT’s production, Small Island.

‘We have created a communal room-scale VR experience where the audience and performers are in the same space together.’ Through the use of VR glasses, viewers step into the show, alongside life-size ‘holograms’ of the musicians. Madame Kalamazoo does not really happen in virtual reality, but it is still interactive. ‘We are not driven by technology. If we can create an immersive story via old-fashioned email, we’ll do it.’ In the new application that was launched during the pandemic, Madame Kalamazoo is a storyteller who arrived from the Blue Moon at the start of lockdown. She writes funny daily stories for children and sends them to you by email. In the stories, the children are the main characters. ‘I think the lockdown has been a catalyst for a lot of digital conversations that have been around for a while’, says Coffey. The National Theatre had to cancel all planned tours, but the National Theatre at Home platform reached 20 million viewers over the course of four months. ‘This shows that the home market is really significant and I think that goes beyond lockdown. This is a much more tangible way to go forward than we considered it to be.’ And the studio has even bigger plans. ‘One of the things we will be doing in our next projects is look at how we can better create works that are intended to be consumed across different types of hardware.’

How can we stage a performance that some people can watch on TV, some in VR, some in MR?
Basically, the same performance, but perceived in different ways.’

Any final advice for creators?
‘People who are guardians of new work in this area should support a symbiotic relationship between technology and storytelling, rather than one coming after the other, and allow artists time to get used to the technology and create great work.”

 

NEW HORIZONS Magazine download link:
https://www.christopherlafayette.com/wp-content/uploads/2020/11/magazine_vrdays_2020.pdf

Deconstructing the Experience: Meaning–Making in Virtual Reality Between and Beyond Games and Films

In this paper me and my colleague Prof. Gianmarco Giuliana (Università degli Studi di Torino) deconstruct the so–called experience of Virtual Reality by enriching the cognitive and phenomenological explanations of VR’s renown “immersiveness” with a semiotic perspective and approach that highlight the quilting of the different dimensions of meaning–making involved in virtual reality. We do this first of all by highlighting how VR mixes key aspects from previous media and by looking at the socio–cultural aspects involved in the attribution of “realism” and “immersion”. Then, in the second part, we focus on the uniqueness of VR’s technology and highlight the consequent cognitive processing and the resulting phenomenological situation in which such experience occurs. In the third part, we underline the narrative essence of this experience as well as the birth of a new spectatorship allowing for VR to work as intended. In the fourth part, we look at the philosophical and anthropological background granting meaning to the experienced virtual utopic body and other space. Finally, we criticize the widespread conception of a non–mediated experience grounded on the embodiment and phenomenological conditions of the virtual worlds, and propose a different triadic explanatory model that helps to complexify our understanding of meaning–making in VR.

Links:
Academia
Meaning-Making in Extended Reality
I Saggi di Lexia

#
Semiotics,
Sociology,
Digital Media,
Embodiment,
Digital Games,
Cinema,
Virtual Worlds,
Culture and Cognition,
Immersion and Experience,
Virtual Reality

Published: 2020

How I approached the VR film medium.

…Left, Right, Left, Right… I am sitting alone on this remote island, closing and opening one eye after the other while trying to concentrate on the horizon. It is an ancient practice which pirates have mastered during their early travels across the Indian Ocean. The purpose of this technique their teacher had put like this: “to train the eye in a manner that enables it to see behind itself”. Funny, it reminds me of the opening lines of Johan van der Keuken’s film Face Value. Only that in his case he refers to the video camera.

…Left, Right… Pirates never really understood the precise meaning of the teacher’s words. For some of them it was about trying to see the way they used to. For me it is about learning over and over again. I see it as an invitation. A stimulus to “let go” and to forget everything I know. I claim it is, on the one hand, about regaining the ability to question everything I see, and on the other hand a way to come up with a different perspective. Strangely enough it makes me think of immersive cinema.

…Left, Right, Left… Flint used to lead these visual exercises but after he disappeared, the parrot of Long John Silver, who had memorized many of the captain’s phrases, was repeating it all the time. It was as if we had a new “ignorant master”.

…Left… I liked that. It made me think that such a master doesn’t force knowledge but rather helps you to find your path, your own way to it. What if this is applied in film?

…Right… If a film technique questions our very own cognition that would be a way to go.

…Left, Right, Left… A different use of the eye means a new condition of spectatorship and a profoundly different way to communicate and discuss knowledge. But every communication starts with an agreement and agreement is based on trust.

…Follow me and I will lead you to victory!… one of the parrot’s favorite quotes from Flint.

The old Captain used to say also one other thing: …Trust is like a building and a building is as strong as its foundations…

Thank you, Flint and thank you ignorant master. But what are the foundations? While asking this, both my eyes are open and focus on an approaching cruise ship that just appeared as a dot on the horizon. I wonder if the people on it will be interested in my interpretation of this old exercise.

The best thing I saw at IFA Berlin

IFA Berlin is one of the biggest consumer electronics exhibition in the world. I jumped on a train from Amsterdam and went there with the intention to visit everything related to VR. And there was a lot to see: HTC Vive, Sony Morpheus for developers and gamers, tons of Samsung Gear VR with various previews, Zeiss VR One for the high range VR glasses, lots of mid range VR glasses, Arte’s cross-platform for virtual reality, various Oculus demos and Roller Coasters… This was all interesting, but nothing that I didn’t already know about, until this: http://www.fraunhofer-cingo.com/

Fraunhofer institute has developed an amazing software for 3D sound experience! It analyzes any type of audio input your video has and adapts it in 3D space. All you need is your smartphone with VR glasses, and any, yes any!!! type of headphones and you have 360° video and sound.

Why is this so great?

Because until recently, and I speak from personal experience, to make a 360° video and have corresponding 360° sound was a huge challenge. For me it was amazing to see such a good and well functioning solution. I honestly thought it’ll take much longer.

So, I tried it and it worked great! (not perfect yet, I noticed some slight mismatches between spacial movements and sounds but it was nothing compared to the overall effectiveness) Oh, and one more thing, you can use it to simulate 5.1 (or any other set like 4.0 and so on) only with the two speakers of your computer, phone or smartphone!

Another indispensable tool for immersive film is here, check it out:)

http://www.fraunhofer-cingo.com/en/ueber-fraunhofer-cingo.html

Eyes Light Bodies Bright

I was looking into my way of seeing a particular night and place. It struck me already while filming back in 2009. Later in post I have been “jumping around” this material on various occasions. It took time for it to ferment and for me to understand that sound has never been part of it.

silent; mini DV ( 787 x 576 pixels ); sample: