Immersive journalism_Franklin _NewMediaLit_Bratislava, Slovakia

Immersive journalism

Iliana Franklinwww.mediaframestudios.com

Leonardo da Vinci Project 2012-2014:New Media Literacy for Media Professionals

Partners: SKAMBA (SK), Media21Foundation (BG), FOPSIM (MT) and Videovest (RO).

This presentation was prepared by SKAMBA and is free to use under condition of acknowledging autorship

Immersive journalism (IJ): The Real-time interaction with information

“Well-crafted journalism always aims to elicit a connection between the audience and the news story. Creating that connection via different kinds of ‘immersion’ has long been considered ideal.” – Peña (2010)




Definition

IJ puts an audience member directly into the event using 3D gaming and immersive technologies that create a sense of "BEING THERE" and offer the opportunity to personally engage with a story.

What is Immersive journalism?




The IJ construct

• Spatial Participation - A PLACE ILLUSION;

• Contextualization the story;

• Audio and video captured at a real event or place

• The story should reflect good journalistic/documentary practices

In general, IJ is likely to include the following:




The IVR construct

“Gone Gitmo,” - (Peña, 2010) “Hunger in Los Angeles”- (Peña, 2011)

•Immersive Virtual Reality and the Body•The concept of presence•Plausibility




Combined Techniques for Simulation

Combined Technologies are used to construct a simulated world for fully immersive experience of the news report in which audiences can suit up, walk around, and interact with environment:

•Game development tool unity 3D•A head mounted display with motion tracking•Live audio collected from the real scene•Motion capture to animate the characters

“Hunger in Los Angeles” did premiere at Sundance Film Festival 2012

MOCAP FACILITY




Put On A Helmet, And You're In The Story

WHY VIRTUAL REALITY JOURNALISM IS THE FUTURE MODEL OF JOURNALISM?

Holds potential for journalism, as well as for fictional storytelling

Rises issues of objectivity and subjectivity

Virtual reality walkthroughs will be the future of journalism” (Peña, 2013)




Historic background“...new journalism gained momentum in the 1960s because the ‘who,

what, when, where, why and how’ model did not capture the emotional essence of social movements occurring at the time” - Mills (1974:xvii);

“...frequently involves the first person voice”

- (Conley, 1997: 225);

Debate: “More honest and entertaining?”

or

“Allows form to rule content, and that it is dishonest?”




The Ultimate Interface

Ultimate Display for human computer interaction - “where the computer is directly attuned to ... brain activity;

(Schneiderman, 1983)

Such an interface can be made possible using “a brain–computer interface (BCI).” - (Pfurtscheller & Neuper, 2001)




BCI system

A brain-computer interface (BCI) is a communication system enabling users to send commands to computers by means of brain activity via a headset fitted with electrodes.

Possible Paradigms




Translation into sensible representation

Applications




Application of BCI is used to teach children with attention deficit hyperactivity disorder (ADHD)” - Forbes (2012)

Human–Computer Interface Issues in Controlling Virtual Reality With Brain–Computer Interface

Mapping Thoughts




“We are interested in the mapping between the thoughts used to activate the BCI and the resulting functionality in the VE, that is, if a person thinks about kicking a ball, the mapping would be natural if they would see their virtual foot kick a virtual ball, that would actually be kicked away. ” – (Friedman, Leeb, Pfurtscheller, Slater, (2010)

PRESENCE-TELEOPERATORS AND VIRTUAL ENVIRONMENTS

customizable, anthropomorphic

virtual bodies, or avatars

What is self-presence?




“The concept of self-presence provides a framework for understandinghow people connect to virtual self-representations on three distinct levels (body, emotions, and identity) .”- Ratan and Hasler, (2010)

Immersion and sensorimotor contingencies

IVR SYSTEM

Head and Body Tracking

Video And Audio

Haptic Sensations

Literary Techniques

Virtual Location




In a HMD, the displays are mounted close to the eyes and head tracking ensures that the left and right images are updated according to the head movements of the participant with respect to the underlying virtual environment.

HEAD MOUNTED DISPLAY







“Participant will see their avatar as if it were actually their own body and they will be offered a first-person point of view.

Depending on the extent of body tracking, the movements of the virtual body will match those of the movements on the person’s real body.” (Peña,2010)

HEAD MOUNTED DISPLAY SYSTEM

Stereo Vision




Without stereo vision humans cannot perceive the 3D nature of environment around them; Same idea is used in IVR system to calculate the positions of a specific point in 3D

Interaction

Participants can effect changes in the environment




“The Room”, Lionhead Studio, MGS - Moleneaux, P., (2010), YouTube

Online virtual worlds - Unity

Unity




Oculus SDK




The next generation Virtual Reality http://www.oculusvr.com/rift/

Oculus Rift

Immersive technologies




First Person View

• IPSRESS: The experience begins with a first person view of the avatar before changing to first-person perspective

• The muffled audio is designed to sound like it is coming from another room and only fragments can be heard clearly

Point Of View




360-Degree View From New York’s Tallest Building, Time Magazine

360 degree view

Photography and time-lapse




360-degree VR broadcasts for live events




HMD FOR 360 DEGREE BROADCAST

3d stereoscopic video recording

Jaunt camera-360 degrees

Hollywood's-virtual-reality-blockbustersNew-360-degree-camera-capture-VR-movies




Embodied experience

Share body with someone else

The idea is NOT simply experience what embodiment of the robot feels like, but to also complete a regular task in work as a journalist: interviewing people for a story.“ - Peña, (2010)




Augmented Reality

Quick Response Code (QR)




Online tools for journalists

Many editors today look for strong level of digital literacy and evidence of online engagement skills from journalists

Shorthand




Shorthand




Thinglink




Zentrix




WireWAX




3D PLATFORMS, VIRTUAL REALITY AND VISUAL LANGUAGE AS A TOOL FOR STORYTELLING

Alex Roman Silestone- 'Above Everything Else' , film (2010)

http://vimeo.com/15630517

Photorealistic 3D rendering




Visual content creation tools

Adobe Creative cloud and Autodesk




Serious games as news

Serious games as news fall into broad categories:

•editorial•educational•expository•advocacy/activist




Editorial: Newsgames

The project’s main idea was to use the language of videogames to describe current events while conveying a timeless maxim: violence begets more violence.

September 12th (Frasca, 2001)




Newsgame

Newsgames can persuade, inform, and titillate; make information interactive; recreate a historical event; put news content into a puzzle; teach journalism; and build a community. (Chris DeLeon, 2013)




Educational Newsgame

Te player assumes the role of a freelance journalist who has just arrived in Jerusalem facing challenges. The goal is to create and get an article published for a newspaper by collecting quotes from the dialog in the game.

Global Conflicts: Palestine (Hanson, 2007)




Educational Newsgame

Interactive News Museum in Washington, DC

Exhibitions at theNewseum




Expository Newsgame

While the game play is adapted from an existing military third-personshooter game engine, the scenes and missions are re-enactments of battles from the war in Iraq and Afghanistan based on news accounts, interviews, and DOD reports.

KUMAWAR, (KUMA Reality Games, 2004)




Activist Newsgame

The player is a refugee from a Darfuri family and must evade the Janjaweed militia patrols on trips for water and firewood.

The game is meant to increase empathy for victims of genocide by positioning the player within a game environment where the hazards refer to actual tragedy.

Darfur is Dying (Ruiz, 2006)




http://secondlife.com/whatis/

Second Life Real world build by users




Virtual Gitmo: Human Rights in Second Life

A virtual but accessible version of the prison, in contrast to the real but inaccessible prison on web platforms - IJ rebuilt it in Unity

Guantanamo Bay




Project Syria

An IJ Experience




Walljumpers-Transmedia

There are 23 border walls on the face of the earth, divisions that mark cultural differences Walljumpers leap over those barriers in search of humanity’s shared qualities using documentary

film, Second Life, mashinima, mobile and Google Earth. - Weil and Pena (2009)




The most bizarre designs

Zurich University of the Arts – motor simulator of flapping wings




Embodied experience in an unchangeable narrative

How these three concepts—place illusion, plausibility, and virtual body ownership—can be harnessed to generate immersive journalism ?




Open-ended questions • What are key ethical and legal aspects/issues of immersive journalism?

• If this is based on virtual game - principles, can it have potentially, in the long - term, the same negative consequences - at least in some cases? (e,g, psychiatric obsession - dependency)

• Is the virtual world, really that different from our real world, and how does this possible difference impinge on the fields of journalism?

• If, by definition, interactive art is predetermined indeterminacy, what is at stake for the author or the spectator in presuming that there is a choice? What are the limits of interactive subjectivity?

• What if the explicitly limited choice was used tactically to thwart simple dualisms such as masculine/feminine, subject/object, inside/outside, public/private, moral/amoral, real/virtual, passive/interactive?

• What universes we will be invited to explore and how rich and complex these will be — and why we should want to visit them in the first place?

•




Open-ended questions

• What is real and rich behind the highly sophisticated technical objects, the technologies of virtual environments?

• How to update the wooden horse of Troy type of subterfuge, allowing societies to invent the physical and mental means of measuring their capacities for evolution and adapting themselves to the artificial transformations they themselves bring about?

• Have researchers taken a direction which will lead them, with different forms of objectivity, into a metaphorical trap?

• What are the dimensions of human identity today?

• How the world will react to these constantly progressing new technologies?

• When we know that all the possible variables are already recorded in the interactive programme, what does this freedom of choice really amount to?

• What will our society turn into if everyone is walking around in goggles and gloves pretending they are somewhere that doesn't even exist?




ReferencesNonny de la Peña, Peggy Weil, Joan Llobera, Elias Giannopoulos, Ausiàs Pomés, Bernhard Spanlang, Doron Friedman, Maria V. Sanchez-Vives, and Mel

Slater. “Immersive Journalism: Immersive Virtual Reality for the First Person Experience of News,” Presence: Teleoperators and Virtual Environments 19, no. 4 (2010): 291 – 301

Maria V Sanchez-Vives, Mel Slater (2005) From presence to consciousness through virtual reality. Nat Rev Neurosci 6: 4. 332-339

Mel Slater (2009) Place illusion and plausibility can lead to realistic behaviour in immersive virtual environments. Philos Trans R Soc Lond B Biol Sci 364: 1535. 3549-3557

Peggy Weil and Nonny de la Peña, “Avatar Mediated Cinema” ACM International Conference Proceeding Series; 352 (2008): 209-212

gonegitmo.blogspot.com

Tracy Fullerton, “Documentary Games: Putting the Player in the Path of History,” in Playing the Past: Nostalgia in Video Games and Electronic Literature, ed. Zach Whalen and Laurie Taylor (Vanderbilt University Press, September 2008).

Keith Halper, "Kuma Games Press Release," http://ww.kumagames.com/press.

"Towards Immersive Journalism: The IPSRESS Experience". YouTube. 2009-07-15. Retrieved 2013-08-02.

Çizmeci , M (2009) 3d head tracking and VRML editor for the Responsive workbench system

Leverenz, K (2008) Immersion Journalism , Labsome.

Kramer, Mark. ‘Breakable Rules for Literary Journalists.’ Sims, Norman & Kramer, Mark.

“Introduction to Literary Journalism.” New York, USA: Ballantine Books, 1995.

Biocca, Frank. "Communicating Within Virtual Reality: Creating a Space for Research." Journal for Communication 42.4 (1992): 5-22.

Carr, Clay. "Is Virtual Reality Virtually Here?" Trainingand Development 46.10 (1992): 36-41.

Newquist, Harvey P. "Virtual Reality's Commercial Reality." Computerworld 30 March 1992: 93-95.

Peterson, Ivars. "Looking Glass Worlds." Science News 4 January 1992: 8-10.

Ratan, R., Hasler, B. Exploring Self-Presence in Collaborative Virtual Teams, Psychology Journal, 2010, Volume 8, Number 1, 11 -31




Immersive journalism_Franklin _NewMediaLit_Bratislava, Slovakia

Presentations & Public Speaking

Transcript of Immersive journalism_Franklin _NewMediaLit_Bratislava, Slovakia