Deconstructing the Experience: Meaning–Making in Virtual Reality Between and Beyond Games and Films

Posted on by
Reply

In this paper me and my colleague Prof. Gianmarco Giuliana (Università degli Studi di Torino) deconstruct the so–called experience of Virtual Reality by enriching the cognitive and phenomenological explanations of VR’s renown “immersiveness” with a semiotic perspective and approach that highlight the quilting of the different dimensions of meaning–making involved in virtual reality. We do this first of all by highlighting how VR mixes key aspects from previous media and by looking at the socio–cultural aspects involved in the attribution of “realism” and “immersion”. Then, in the second part, we focus on the uniqueness of VR’s technology and highlight the consequent cognitive processing and the resulting phenomenological situation in which such experience occurs. In the third part, we underline the narrative essence of this experience as well as the birth of a new spectatorship allowing for VR to work as intended. In the fourth part, we look at the philosophical and anthropological background granting meaning to the experienced virtual utopic body and other space. Finally, we criticize the widespread conception of a non–mediated experience grounded on the embodiment and phenomenological conditions of the virtual worlds, and propose a different triadic explanatory model that helps to complexify our understanding of meaning–making in VR.

Links:
Academia
Meaning-Making in Extended Reality
I Saggi di Lexia

#
Semiotics,
Sociology,
Digital Media,
Embodiment,
Digital Games,
Cinema,
Virtual Worlds,
Culture and Cognition,
Immersion and Experience,
Virtual Reality

Published: 2020

How I approached the VR film medium.

Posted on by
Reply

…Left, Right, Left, Right… I am sitting alone on this remote island, closing and opening one eye after the other while trying to concentrate on the horizon. It is an ancient practice which pirates have mastered during their early travels across the Indian Ocean. The purpose of this technique their teacher had put like this: “to train the eye in a manner that enables it to see behind itself”. Funny, it reminds me of the opening lines of Johan van der Keuken’s film Face Value. Only that in his case he refers to the video camera.

…Left, Right… Pirates never really understood the precise meaning of the teacher’s words. For some of them it was about trying to see the way they used to. For me it is about learning over and over again. I see it as an invitation. A stimulus to “let go” and to forget everything I know. I claim it is, on the one hand, about regaining the ability to question everything I see, and on the other hand a way to come up with a different perspective. Strangely enough it makes me think of immersive cinema.

…Left, Right, Left… Flint used to lead these visual exercises but after he disappeared, the parrot of Long John Silver, who had memorized many of the captain’s phrases, was repeating it all the time. It was as if we had a new “ignorant master”.

…Left… I liked that. It made me think that such a master doesn’t force knowledge but rather helps you to find your path, your own way to it. What if this is applied in film?

…Right… If a film technique questions our very own cognition that would be a way to go.

…Left, Right, Left… A different use of the eye means a new condition of spectatorship and a profoundly different way to communicate and discuss knowledge. But every communication starts with an agreement and agreement is based on trust.

…Follow me and I will lead you to victory!… one of the parrot’s favorite quotes from Flint.

The old Captain used to say also one other thing: …Trust is like a building and a building is as strong as its foundations…

Thank you, Flint and thank you ignorant master. But what are the foundations? While asking this, both my eyes are open and focus on an approaching cruise ship that just appeared as a dot on the horizon. I wonder if the people on it will be interested in my interpretation of this old exercise.

The best thing I saw at IFA Berlin

Posted on by
Reply

IFA Berlin is one of the biggest consumer electronics exhibition in the world. I jumped on a train from Amsterdam and went there with the intention to visit everything related to VR. And there was a lot to see: HTC Vive, Sony Morpheus for developers and gamers, tons of Samsung Gear VR with various previews, Zeiss VR One for the high range VR glasses, lots of mid range VR glasses, Arte’s cross-platform for virtual reality, various Oculus demos and Roller Coasters… This was all interesting, but nothing that I didn’t already know about, until this: http://www.fraunhofer-cingo.com/

Fraunhofer institute has developed an amazing software for 3D sound experience! It analyzes any type of audio input your video has and adapts it in 3D space. All you need is your smartphone with VR glasses, and any, yes any!!! type of headphones and you have 360° video and sound.

Why is this so great?

Because until recently, and I speak from personal experience, to make a 360° video and have corresponding 360° sound was a huge challenge. For me it was amazing to see such a good and well functioning solution. I honestly thought it’ll take much longer.

So, I tried it and it worked great! (not perfect yet, I noticed some slight mismatches between spacial movements and sounds but it was nothing compared to the overall effectiveness) Oh, and one more thing, you can use it to simulate 5.1 (or any other set like 4.0 and so on) only with the two speakers of your computer, phone or smartphone!

Another indispensable tool for immersive film is here, check it out:)

http://www.fraunhofer-cingo.com/en/ueber-fraunhofer-cingo.html

Filmmaking and virtual reality. How different and how similar?

Posted on by
2

Below are a few short reasonings in regards to my next project which is a 360 degree film tale.

Film can use the instruments of VR to tell stories. But where does one start and the other end? Is this process leading to something that will relate to film the same way smartphones relate to landlines?

We can start with the fact the both propose an immersive experience through the use VR glasses. If we leave special effects and interaction aside, what remains as unique characteristic is the feeling of being somewhere else. Even the complete darkness before a game or during a film is a digital “dark room”.

mind map 1

So we have our common ground, namely immersive space.

Another common element is interaction.

It is there by default, because the sensor of the Oculus Rift is detecting the orientation of the head and respectively the body of the viewer. This makes it possible to look around and choose what to “exclude” from the entire picture. The spectator frames the story the way his/her will or instinct desire.

This is a significantly different from traditional storytelling in cinema. We are transported to a different place, immersed in the fabric of the film. The director is still leading, but in a way that leaves us a full horizon for exploration. Space acquires distinct importance as a narrative element.

I would say, we can safely conclude that there is interaction already when simply watching a movie with VR glasses.

There is though, a big turning. How much of an interaction does one want to use? From keyboards to smells and haptic devices, everything is possible. Since we are talking about film and storytelling there are two things to mention in that regard:

1. It is extremely easy to loose the sense of being immersed in a film story, if some kind of controller is used. Because of the nature of present time VR glasses and sets, this interaction immediately brings us out of the film and transforms the experience into some sort of a video game.

2. When interaction means that the viewer is altering the storyline, the experience is different. One thing is if I am free to look around, another, if I can actually change the film destiny. Even if we only use eye tracking to make choices in the film, it makes us conscious of ourselves and subsequently may draw us away from the story.

To summarise: Immersive feeling and basic interaction are the largest areas of overlapping between Film and VR.

Anything that steps out of that border seems to form new categories and aggregates.

 

Eyes Light Bodies Bright

Posted on by
Reply

I was looking into my way of seeing a particular night and place. It struck me already while filming back in 2009. Later in post I have been “jumping around” this material on various occasions. It took time for it to ferment and for me to understand that sound has never been part of it.

silent; mini DV ( 787 x 576 pixels ); sample:

 

VR and Ethics

Posted on by
Reply

This is an article on The Ethics of Virtual Reality from 1992. It covers the main problematics of the concept in philosophy and ethics. It elaborates on the cultural and technical dimension of VR and introduces various points of view. A short quote:

“Virtual reality therefore has the effect of reality upon us, though we recognise that it is not properly real. VR is best described as a simulation, as opposed to a representation (Baudrillard, 1983). If I make a model of something, say a chair, then my model is a representation if I never lose my belief that it is the original chair that is the real object. On the other hand, if I make a model of something, say the surface of the moon, and in navigating the imaginary terrain I come to believe that the model is real, then I am in a simulation. The term “virtual reality” is a precise expression of this latter concept for we are, at the same time, admitting the fact that we have created a model (which is unreal) and admitting that we are treating it as reality.

The significant point about this definition is that VR is essentially subjective. VR is an experience and not a piece of technology. I can curl up with a good novel and claim to be in virtual reality, whilst I can don the most expensive headset and data glove yet remain perfectly aware of the fakeness of what I observe. This is important because it follows from this definition that many symbolic structures in society can be viewed as “virtual reality” (for example, cases where a computer model of an organisation does not reflect the underlying reality). We must therefore understand virtual reality as being the expression of a deep philosophical problem caused by our commitment to symbolic structures whose existence now obscures the reality of the underlying object.”

http://www.cs.waikato.ac.nz/oldcontent/cbeardon/papers/9201.html