Crafting Stories for Virtual Reality

Melissa Bosworth, Lakshmi Sarah

• Publisher: Routledge
• Year: 2018
• ISBN:
  9781351584371
• Edition: 1
• Topic: Languages & Linguistics
• Subtopic: Journalism
• Index: Languages & Linguistics

We are witnessing a revolution in storytelling. Publications all over the world are increasingly using immersive storytelling—virtual reality, augmented reality and mixed reality—to tell compelling stories. The aim of this book is to distill the lessons learned thus far into a useful guide for reporters, filmmakers and writers interested in telling stories in this emerging medium. Examining ground-breaking work across industries, this text explains, in practical terms, how storytellers can create their own powerful immersive experiences as new media and platforms emerge.

Section II
Immersive Media and Storytelling Styles

3 Immersive Narratives and News

Stories are compasses and architecture, we navigate by them, we build our sanctuaries and our prisons out of them, and to be without a story is to be lost in the vastness of a world that spreads in all directions like arctic tundra or sea ice.

(Rebecca Solnit, The Faraway Nearby)

Case Studies

Clouds Over Sidra, United Nations VR

Limbo, Guardian VR

Climbing Giants, Black Dot Films for National Geographic

The Occupation of Alcatraz that Sparked an American Revolution, Seeker VR

We Who Remain, Emblematic Group, AJ+ and NYT-VR

Video, Euronews (various projects)

VRtually There, YouTube and USA Today

Take an immersive journey to witness the lives of the few who remain in the war-torn Nuba Mountains of Sudan, or of the water protectors standing up against the oil industry at their camp in Standing Rock, North Dakota. Gaze over miles of forest canopy from the crown of a giant sequoia tree, or glide through a ghost-like city to catch a glimpse of life as a refugee. All of these experiences are possible in immersive narrative form.

We use the term immersive narratives to describe non-interactive 360-degree stories within which the viewer has the ability to look around in a scene, but not move within that scene or interact with elements in it. Some pieces in this category might also be referred to as “Cinematic Virtual Reality” and “Immersive Cinema.” The form can also include live breaking news, and short, experiential 360-degree videos.

All these pieces can be described as having three degrees of freedom (DOF), because the viewer can turn their head on three axes—horizontal, vertical, and tilting left and right. That is, they have full rotational freedom to look around without translational movement, i.e. actually walking around in the scene. Standard monoscopic and stereoscopic 360-degree videos fall into this category, as do graphically rendered experiences in which the viewer has a locked position in the scene.

360-degree video has been the most readily adopted form used for journalistic storytelling and by many creators new to immersive media. In part, this has been because 360-degree videos are easily shareable on social media platforms such as Facebook and YouTube. They can be viewed on most mobile devices and monitors and do not require the use of a head-mounted display (HMD) or devices to track the viewer’s movement around a space.

This format is also arguably the easiest way to tell a linear story using immersive media, and it is less likely to run up against challenging issues that arise with other forms—such as the ethics of re-creating a scene for a walk-around piece, or the labor-intensive nature of creating an interactive game-style experience or a branching narrative.

Media outlets have been trying everything from live coverage and extreme sports to behind-the-scenes perspectives at factories and concerts. A common principle is to take the viewer somewhere they do not have access to, but another way of thinking about the story is to provide an intimate experience by taking on another person’s perspective.

The case studies in this chapter look at pieces that incorporate 3D graphics, video, animation and photos. Based on interviews with the projects’ creators, we’ll survey the technology used in production, including camera rigs, video stitching and editing, 3D modeling and volumetric capture. We’ll look at storytelling techniques including pacing, sound, structure, location of characters in a scene, and tools for guiding the viewer.

Clouds Over Sidra

United Nations VR (UNVR)

January 2015

Clouds Over Sidra, the story of a young girl and her family in a refugee camp in Jordan, was the United Nations’ first VR project. Produced by Within (called Vrse at the time)¹ and the United Nations VR team, the eight-minute piece first screened in January 2015 at the World Economic Forum in Davos, Switzerland. The story broke new ground as a format for documentary and journalistic work. This was about 10 months before the New York Times sent Google Cardboard headsets out to subscribers, and before Facebook and YouTube introduced their 360 video players.

Barry Pousman, who was working for DiscoveryVR and consulting part-time for the UN, received a call in 2014 from creative director to the United Nations, Gabo Arora, who was looking for a director of photography. Arora was helping Within founder Chris Milk (who some call the “grandfather of VR”) produce a series of three films for UNVR: Clouds Over Sidra, Waves of Grace and My Mother’s Wing.

Pousman has extensive experience in traditional film, but he believes VR filmmaking has an additional capacity to evoke emotional connections with subjects. “We have an opportunity, through media, to take our audiences to places we never have before,” he says.

Traveling to a Refugee Camp

Clouds Over Sidra was filmed in the Za’atari refugee camp in Jordan and follows 12-year-old Sidra, originally from Syria, around her temporary home.

Based on several interviews with Sidra, the narration of a voice actor brings the story of Sidra and her family to life, offering a glimpse into daily life in a refugee camp. She describes her school, and briefly talks about the men who go to the gym, the boys who wrestle and the computer room—even though girls are not allowed to play games. She describes the bakery and says, “the smell of bread on our walk to class drives us mad sometimes.”

Some of the most powerful scenes are those that break the fourth wall. In one instance, kids look at the camera on the way to school—making the viewer feel as though they are making eye contact. In another scene the creators pair a powerful visual with a compelling audio narration: Sidra says, “There are more kids in Za’atari than others right now. Sometimes I think we are the ones in charge,” as children giggle and gather around the camera.

Chris Milk’s 2015 TED talk² on empathy came out just a few months after Clouds Over Sidra published. In it he discusses the aim of the project, and how it allows people to see the lives of others and feel as though they have taken a trip to a refugee camp.

Early Days of 360 and VR

Prior to filming Clouds Over Sidra, Pousman had never seen a 360-degree video, so the first thing he did when he received a call from Milk and Arora was to fly to the Within office in Los Angeles. There, the Within team showed him a few camera prototypes for 360 video capture. (Pousman describes one as “a circle of little cameras.”) They discussed microphones and directional sound. Milk showed an example of the rig they filmed with on-stage during his March, 2015 TED Talk. Within is protective of its camera details and avoids sharing specifics of their custom rig.

Pousman watched his first piece of 360-degree video in headset in November, 2014. The piece was a single shot of Andy Lesniak, the chief creative technologist for Within and the camera’s inventor. Pousman recalls that he saw a dog and then a person who picked up the dog and held it out toward the camera. “Immediately you feel it is real, in your personal bubble …” Pousman says, “I got goosebumps, I felt transported to that living room.”

Pousman and Arora spent four days filming in the Za’atari refugee camp in Jordan. The Within team initially thought the output of this type of camera would be one continuously filmed scene. Pousman had seen demos in Los Angeles, and didn’t imagine the ability to edit the content using standard video editing software such as Adobe Premiere.

The footage, which featured Sidra in different scenes over the course of the four-day shoot, worked to create a narrative that feels as though it occurs over the course of a single day. “I don’t think they imagined the footage we came back with would be so compelling and so intimate,” Pousman says.

“There is some creative freedom and poetic license,” Pousman said, describing how some scenes were filmed over multiple days and how they wanted to incorporate different aspects of the camp within the narrative—including places where Sidra didn’t normally go.

Within used proprietary stitching software to match the images from different cameras so the final 360 video looked seamless.

They decided to use a voice-over actor, narrating in English because the original interviews were conducted in Arabic. Because they were initially aiming for an English-speaking international audience, many users would have had to read the subtitles in addition to taking in the 360 experience. Subtitles can be difficult to follow in VR, and they can also, as Pousman says, pull the viewer out of the piece—“rather than seeing the film as an experience they are in.” In other words, having to read subtitles can break presence.

Lessons on Transporting the Viewer

Despite the new format and experimental nature of the piece, Pousman says there’s not much he would change about Clouds Over Sidra beyond the height of one particular shot; “In a very nitpicky way, I would just lower the camera four inches,” he says of the third shot in the sequence (at the :51 minute mark) in which Sidra’s family is introduced to the viewer. He says he always puts himself in the shoes of his audience to see how they would experience a piece, trying not to get too distracted by thinking about traditional film editing techniques, such as quick cuts: “directors get caught up thinking that it is a film. Actually it is much more like a play,” he says.

For Pousman, the project is all about taking the viewer to a place they would not have access to. “It turns out that we were able to put them in the shoes of a girl,” he says. VR added an element that made people say: “Oh my god, I feel like I went there, I have had an experience,” he says. Clouds Over Sidra took a risk by making something longer than what was being done at the time.³ But as Pousman says, “As long as the story is compelling enough, you will just stay in there.”

Emotional Impact and Reach

The very first edition of the project was released in time for the World Economic Forum in 2015. When asked about impact, Pousman said (anecdotally), “Half the people who do it cry.” Through the Within app, the UN screening at Davos, the UN General Assembly, and the World Humanitarian Summit, Pousman says at least hundreds of thousands of people have watched. Since 2015, the piece has been translated into 15 languages.

After Clouds Over Sidra, UNVR published Waves of Grace, My Mother’s Wing, and a piece on Nepal earthquake recovery. Waves of Grace is the story of a young woman during the Ebola epidemic in Liberia. My Mother’s Wing follows a mother in the Gaza strip coping with losing two of her children in a bombing and the piece on the Nepal looks at the state of Kathmandu, just three days after the 2015 earthquake.

Takeaways

Scripted voice-over:

• Subtitles can break presence, limiting the immersive effect by reminding viewers they are watching a screen. Use of an actor for voice-over is an alternative.

Against the rules of film:

• Breaking the fourth wall through brief eye contact with the camera can place the viewer in the scene.
• Long cuts take inspiration from theater, rather than traditional film.

Transporting the viewer:

• Take the viewer somewhere they wouldn’t otherwise have access to.
• Sensory cues, such as the mention of the smell of baking bread, can give an additional suggestion of “being there.”

About This Project

URL: https://with.in/watch/clouds-over-sidra/

Interview Date: June, 2017

Interview Subject: Barry Pousman, Director ...

Table of contents

• Cover
• Half Title
• Title Page
• Copyright Page
• Dedication Page
• Table of Contents
• List of Figures
• Acknowledgments
• Introduction: Evolution of Media: Where Are We Now?
• Section I Context
• Section II Immersive Media and Storytelling Styles
• Section III Bringing It All Together
• Glossary
• Index