Students show how wearable technology could alter the idea of narrative

Jun 11, 2014 by Mike Fricano
From this broadcast studio control room, Cole Baker could not only view what each participant was seeing via the Google Glass camera but also send instructions to each person via the wearable computing device. Credit: Jeff Burke/UCLA

The IMAX camera has pushed filmmakers to think in bold vistas that capture dazzling cityscapes or majestic mountains. High-speed Internet has spurred the rise of serialized dramas that can be marathon-watched.

As technology evolves, so too does storytelling as writers try to conceive of ideas that take advantage of a new device's signature capabilities.

So how will a set of glasses that puts the Internet, GPS and a high-def camera just above your right eye shape the narratives of the future?

Students in the UCLA School of Theater, Film and Television (TFT) have spent this year trying to figure that out by creating stories for and with Google Glass. Last July, Google announced that TFT was one of just five film schools in the country selected to participate in the Glass Creative Collective.

"If you're a student at UCLA, you should have these options to really engage in things that people have never done before," said Jeff Burke, assistant dean for technology and innovation at TFT and the instructor who designed the class.

The freedom to experiment, and perhaps establish ideas for how best to use Glass as a tool for creative expression, attracted students from TFT, computer science and even business. "Everybody's actually learning, as opposed to being told how to do something out of the book," said film major Cole Baker, who referred to Glass as "a freaking computer on your face."

To figure out how storytelling could work uniquely on Glass, TFT created a two-quarter class called "Location-based and Audience-aware Storytelling" that would culminate with two finished projects. First, the class started examining how storytelling has evolved by reading the likes of Umberto Eco, who introduced stories that were crafted through dialogue with an audience.

Using Glass, students realized, would require them to develop a whole new form of participatory storytelling. "Trying to imagine ways to use Glass beyond its use as a point-of-view camera was one of the bigger obstacles we had," film major Zoe Sandoval said.

Writing scripts for a device that can simultaneously record and display media while you wear it meant writing for performers who would be free to use their hands and maintain eye contact while being fed dialogue or instructed to do something in real time through Glass and, if desired, an audience that can be wearing Glass as well.

The class wasn't just an opportunity for TFT students to work with cutting-edge technology. Working with Alex Horn, staff software developer at UCLA REMAP—a partnership between TFT and the Henry Samueli School of Engineering and Applied Science that explores the intersections of technology with culture—students wrote software so that the writers could utilize Glass the way they wanted to.

"As a tech enthusiast, I was stoked," said Joon-Sub Chung, a computer science grad student. "I wanted to take the course because my current major focus is on mobile technologies. I wanted it to be fun, challenging and open a new gate to technological challenges. It delivered."

By early December, two projects were selected for the class to complete. In "Bodies for a Global Brain," Eben Portnoy's interactive web series, two characters, Ada and Raymond, who meet online before dating, believe the Internet is becoming a conscious entity. They decide to essentially give themselves over to this emerging technological consciousness (the eponymous Brain) and, via Glass, they are sent its "thoughts."

Students show how wearable technology could alter the idea of narrative
Wearing Google Glass allowed the actors in "Bodies for a Global Brain" to get their dialogue from Twitter.

n reality, Portnoy, who is a master's student in film, wrote a script that contained his instructions for the actors and some original dialogue, but also some placeholder dialogue for the tweets that would be mined in real-time from the Twitterverse during the actual filming.

For this fictional story about characters embodying "the global brain," student programmers Chung and Karan Chugh had to create a "machine-learning" algorithm that would automatically select real tweets that would become dialogue for the actors. REMAP's Horn created a program to send these selected tweets to Glass for the actors to say.

To teach the program about the language of emotions beforehand, students manually categorized a set of 1,500 representative tweets into one of 12 "intentions," like "bond," "inform," "manipulate" and "evade." Then during filming, an off-screen operator triggered the program to select real-time tweets from a library of more than 1 million that matched the emotions that Portnoy wanted the actors to convey at that moment.

"Show me you want me," the actor who played Raymond said to Ada while they were arguing.

Sometimes, though, the tweets didn't create appropriate dialogue.

"Can my yoga pants get out of the laundry now so I can wear them?" one actor said, reading the tweet that appeared on Glass.

Some of the resulting lines may sound like nonsense—not at all surprising given the newness of the technology—but that is also the point, Portnoy said.

"I'm fascinated by the paradox of social media," said Portnoy, who described Glass as "training wheels for cyborgs." "I think there is an amazing potential for collectivity, but a threat of isolation and removal from the physical world through distraction. I wanted to create a narrative experience infused with that paradox."

"They created a piece that experimented with the very notion of how dialogue is created," said Burke, who was both surprised by and impressed with the project. "The Global Brain project is a semi-dadaist experiment; mixing elements of the absurd with structured story."

Portnoy's desired endgame for "Bodies for a Global Brain" is to incorporate live segments during which an online audience can influence what Ada and Raymond say in real time.

This video is not supported by your browser at this time.

The second project, "Grace Plains," uses Glass to combine live-action role-playing with participatory theater in which participants solve a murder mystery. To accommodate the scope of the project—six roles for Glass-wearing participants plus two for other actors, three film sets, and video and audio control rooms—the performance was held at Google's YouTube Space L.A. building.

Upon arrival, participants were greeted by someone from "Grace Plains Research Laboratory" who welcomed them to a meeting with scientist Erinne Kirschner, who was going to talk about her latest advances in artificial intelligence. The participants were given a quick orientation about how to use Glass and instructions about the characters they'd be playing—a general, an investor and a filmmaker, among others.

Participants in Grace Plains wearing Google Glass as they sit at a chess board. Participants in Grace Plains wearing Google Glass as they sit at a chess board. Credit: Jeff Burke/UCLA

Once the program began, the invited guests discovered that Kirschner had been electrocuted by what seems to be the artificial intelligence. They then had to figure out what happened and ultimately whether to free, destroy or contain the , played by the digitally feminized voice of Grace Plains co-creator and film grad student Pierre Finn.

To guide the experience, the TFT student administrators fed clues and prompts, like "Is that Senator Moser? You could team up with her. Power is only a campaign donation away," to participants via Glass. In television studio control rooms, students monitored each participant's point-of-view camera

The project drew heavily on video-game writing, which incorporates a non-linear story structure that relies on participants' decisions to open some narrative paths and close off others. Writers Finn and Baker had to create a narrative with a few pre-determined points that drove toward set endings, but also allowed for participants to decide, for example, how best to provoke or cooperate with other players.

Participants in the Grace Plains project get individualized instructions sent to them via Google Glass, which they can then choose to act on however they want. Credit: Jeff Burke/UCLA

"We wanted people to be engaged in an experience with each other and with an environment in a way that didn't require them to be constantly checking a device," Burke said. The biggest "complaint" participants had was that they wanted the experience to last longer than an hour to give them more time adjust to wearing Glass.

No pre-existing technology would have allowed the students the capability of creating projects with this level of wearable interactivity on this scale, Burke said.

Essential to the Glass project was Burke and the TFT's commitment to mashing up storytelling and technology through programs like REMAP and "the Glass class."

"This type of cross-disciplinary work is something UCLA does well and should be known for," Burke said. "Projects like this give us a chance to engage with innovations that are having a significant impact in the world."

Explore further: Google challenges nonprofits on ideas to use Glass

add to favorites email to friend print save as pdf

Related Stories

Students to explore filmmaking with Google Glass

Jul 30, 2013

Beauty is in the eye of the Google Glass wearer. At least that's what the Internet search giant hopes a handful of young filmmakers will discover. Google is enlisting film students from five colleges to help it explore how ...

Using Google Glass in the operating room

Mar 04, 2014

At this point, only a few people are using Google Glass, Google's wearable eyeglasses-like computer. The device isn't yet commercially available.

UA medical students work with Google Glass

Nov 05, 2013

Two fourth-year students at the University of Arizona College of Medicine – Phoenix received one of an estimated 6,000 Google Glass units that were distributed earlier this year.

Recommended for you

Microsoft unveils Xbox in China as it faces probe

12 hours ago

Microsoft on Wednesday unveiled its Xbox game console in China, the first to enter the market after an official ban 14 years ago, even as it faces a Chinese government probe over business practices.

A smart wristband for nocturnal cyclists

Jul 29, 2014

Five EPFL PhD students have developed a wristband that flashes when the rider reaches out to indicate a turn. Their invention was recognized at a European competition.

Lenovo's smart glasses prototype has battery at neck

Jul 28, 2014

China's PC giant Lenovo last week offered a peek at its Google Glass-competing smart glass prototype, further details of which are to be announced in October. Lenovo's glasses prototype is not an extreme ...

Sapphire talk enlivens guesswork over iPhone 6

Jul 27, 2014

Sapphire screens for the next iPhone? Sapphire is second only to diamond in hardness scratch-proof properties, used in making LEDs, missiles sensors, and on screens for luxury-tier phones. Last year, the ...

User comments : 1

Adjust slider to filter visible comments by rank

Display comments: newest first

daykatrina
not rated yet Jun 12, 2014
The creativity conjured up by the independent human cortex will never be matched. The poor souls will end up in the world of synthetic illusion and adulterated reality.