Deconstructing the Experience: Meaning–Making in Virtual Reality Between and Beyond Games and Films

In this paper me and my colleague Prof. Gianmarco Giuliana (Università degli Studi di Torino) deconstruct the so–called experience of Virtual Reality by enriching the cognitive and phenomenological explanations of VR’s renown “immersiveness” with a semiotic perspective and approach that highlight the quilting of the different dimensions of meaning–making involved in virtual reality. We do this first of all by highlighting how VR mixes key aspects from previous media and by looking at the socio–cultural aspects involved in the attribution of “realism” and “immersion”. Then, in the second part, we focus on the uniqueness of VR’s technology and highlight the consequent cognitive processing and the resulting phenomenological situation in which such experience occurs. In the third part, we underline the narrative essence of this experience as well as the birth of a new spectatorship allowing for VR to work as intended. In the fourth part, we look at the philosophical and anthropological background granting meaning to the experienced virtual utopic body and other space. Finally, we criticize the widespread conception of a non–mediated experience grounded on the embodiment and phenomenological conditions of the virtual worlds, and propose a different triadic explanatory model that helps to complexify our understanding of meaning–making in VR.

Links:
Academia
Meaning-Making in Extended Reality
I Saggi di Lexia

#
Semiotics,
Sociology,
Digital Media,
Embodiment,
Digital Games,
Cinema,
Virtual Worlds,
Culture and Cognition,
Immersion and Experience,
Virtual Reality

Published: 2020

How I approached the VR film medium.

…Left, Right, Left, Right… I am sitting alone on this remote island, closing and opening one eye after the other while trying to concentrate on the horizon. It is an ancient practice which pirates have mastered during their early travels across the Indian Ocean. The purpose of this technique their teacher had put like this: “to train the eye in a manner that enables it to see behind itself”. Funny, it reminds me of the opening lines of Johan van der Keuken’s film Face Value. Only that in his case he refers to the video camera.

…Left, Right… Pirates never really understood the precise meaning of the teacher’s words. For some of them it was about trying to see the way they used to. For me it is about learning over and over again. I see it as an invitation. A stimulus to “let go” and to forget everything I know. I claim it is, on the one hand, about regaining the ability to question everything I see, and on the other hand a way to come up with a different perspective. Strangely enough it makes me think of immersive cinema.

…Left, Right, Left… Flint used to lead these visual exercises but after he disappeared, the parrot of Long John Silver, who had memorized many of the captain’s phrases, was repeating it all the time. It was as if we had a new “ignorant master”.

…Left… I liked that. It made me think that such a master doesn’t force knowledge but rather helps you to find your path, your own way to it. What if this is applied in film?

…Right… If a film technique questions our very own cognition that would be a way to go.

…Left, Right, Left… A different use of the eye means a new condition of spectatorship and a profoundly different way to communicate and discuss knowledge. But every communication starts with an agreement and agreement is based on trust.

…Follow me and I will lead you to victory!… one of the parrot’s favorite quotes from Flint.

The old Captain used to say also one other thing: …Trust is like a building and a building is as strong as its foundations…

Thank you, Flint and thank you ignorant master. But what are the foundations? While asking this, both my eyes are open and focus on an approaching cruise ship that just appeared as a dot on the horizon. I wonder if the people on it will be interested in my interpretation of this old exercise.

The best thing I saw at IFA Berlin

IFA Berlin is one of the biggest consumer electronics exhibition in the world. I jumped on a train from Amsterdam and went there with the intention to visit everything related to VR. And there was a lot to see: HTC Vive, Sony Morpheus for developers and gamers, tons of Samsung Gear VR with various previews, Zeiss VR One for the high range VR glasses, lots of mid range VR glasses, Arte’s cross-platform for virtual reality, various Oculus demos and Roller Coasters… This was all interesting, but nothing that I didn’t already know about, until this: http://www.fraunhofer-cingo.com/

Fraunhofer institute has developed an amazing software for 3D sound experience! It analyzes any type of audio input your video has and adapts it in 3D space. All you need is your smartphone with VR glasses, and any, yes any!!! type of headphones and you have 360° video and sound.

Why is this so great?

Because until recently, and I speak from personal experience, to make a 360° video and have corresponding 360° sound was a huge challenge. For me it was amazing to see such a good and well functioning solution. I honestly thought it’ll take much longer.

So, I tried it and it worked great! (not perfect yet, I noticed some slight mismatches between spacial movements and sounds but it was nothing compared to the overall effectiveness) Oh, and one more thing, you can use it to simulate 5.1 (or any other set like 4.0 and so on) only with the two speakers of your computer, phone or smartphone!

Another indispensable tool for immersive film is here, check it out:)

http://www.fraunhofer-cingo.com/en/ueber-fraunhofer-cingo.html

Eyes Light Bodies Bright

I was looking into my way of seeing a particular night and place. It struck me already while filming back in 2009. Later in post I have been “jumping around” this material on various occasions. It took time for it to ferment and for me to understand that sound has never been part of it.

silent; mini DV ( 787 x 576 pixels ); sample: