Scholar explores changing gestures of digital age

March 28, 2016 by Tom Winterbottom
Musician Onyx Ashanti creates wearable appendages that make sounds through a computer interface according to his movements. Stanford scholar Vanessa Chang is examining how such gesture-based digital technology is redefining artistic expression. Credit: Vanessa Chang

When we sit down to write, for the most part we now do it on a computer. Of course, that was not always the case.

"A pen and paper used to be the main medium of expression through writing, involving certain gestures and technological interactions. But now, many school districts are controversially dropping cursive from their curriculum in favor of digital technologies," said Vanessa Chang, a Geballe Dissertation Prize Fellow at the Stanford Humanities Center this year.

Chang sees this change as symbolic of a broader trend: "That switch from pen to keyboard and from paper to screen plays into a larger development in human interactions with technological innovations."

Chang's research, which centers on artistic production and performance, reveals how gestures that are central to those interactions have taken on new forms in the .

"I am interested in what a gesture contains and what it involves, as well as how those movements mediate the interaction between people and things, between subjects and machines," she said.

Chang cites musical performance as one example where this changing landscape is particularly visible.

"With inventive wearable instruments, artists like Imogen Heap and Onyx Ashanti are using an evolving repertoire of gestures and sounds to craft avant-garde on-stage performances," she said.

Where once there were instruments on stage, these musicians are creating innovative interfaces between movement, music and digital technology.

Heap, a British musician, has worked with engineers to develop a pair of musical gloves loaded with a series of sensors. Using what seems like nothing but her voice, and body movements, Heap is able to construct and manipulate a digital soundscape in her performances.

For the artists embracing these new technologies, their performances represent a novel and authentic freedom of expression in the digital age that is facilitated, somewhat ironically, by technological innovation.

"The more that technology is present, the more that it impacts our bodies and our gestures. As it develops, it is becoming increasingly integrated to the point where it is almost disappearing into the body," Chang said.

Gestures, technology, art

Chang sees this change as part of a broader shift in contemporary life.

Her dissertation, "Tracing Electronic Gesture: A Poetics of Mediated Movement," studies writing, drawing, dance and music performance to understand movement at the intersection of art and technology.

"We come to know the world through our movements. They embody our creativity, agency and subjectivity. As a site of expression and contestation, artistic practice is a productive place to explore how movement is changing alongside developments in technology," Chang said.

In an age of digital devices, our interactions with smartphones, tablets or laptops are mediated by a new set of gestures that are becoming fundamental to contemporary existence, whether swiping on our phone or playing a digital drum machine.

With the changing landscape of digital and electronic technologies, movements and gestures have also evolved, a change that Chang sees most clearly in the artistic and aesthetic realm.

"That idea led me to look at the space between gestures and technology more closely," Chang said.

"We often think about this in terms of authenticity. Listening to vinyl, sitting at a desk with pen and paper or standing at an easel is sometimes understood as being somehow more authentic than the fake, superficial digital world."

According to Chang, it is not that straightforward.

"Along with those traditional forms of artistic expression, a new poetics of gestures and movement creates a sense of reinvented authenticity in the digital age, in which technology plays a defining role."

Making music

Chang coordinated the Graphic Narrative Project at Stanford, a collaborative space for the study of comics and graphic novels.

"These forms of mixed media popular culture are vital in telling the story of human civilization but have only recently become a part of academic discussions," she said.

From that interest in comics and literature, Chang developed an eye for the physical dimension of artistic expression in popular culture and expanded her research to encompass animation, dance and literature.

It is in music, however, that Chang most clearly sees a gesture-based digital technology redefining artistic expression.

"In recent years, there has been a decoupling of human bodies and traditional instruments in many emerging music performances. Electronic instruments, laptops and nascent technologies have quickly become very prominent in the live setting," she said.

That transition has brought about a change in the gestures involved.

"When a drummer has a drum kit on stage, they require a certain physical gesture – the hand hitting the stick on the drum skin – to make a sound in the analog world," Chang said.

"With a digitized instrument, that same gesture is no longer necessary, and yet gestures of a different sort continue to play a fundamental role in performance and in the unique interaction between artist and audience."

Chang highlights the example of Onyx Ashanti, an avant-garde musician who uses a 3-D printer to create innovative wearable appendages that make sounds through a computer interface according to his movements.

"In his gestures, he embodies an innovative relationship between sound and space," she said.

This sort of performance gesture makes supposedly "flat" and "inauthentic" digitally sampled sounds and media more present, more embodied and more alive. They are art practices that affirm the presence of our bodies in a way that is new.

The classically trained pianist Jeremy Ellis is a well-known early adopter, using a profoundly technologized digital drum machine in his performances.

"He performs with small 'black boxes' that have a series of pads and buttons that he programs in advance. Using similar gestures to piano playing, his fingers move quickly over the various pads and settings to create a layered, live performance that is defined by a new and unique series of ," Chang said.

In both cases, there is a persistence of older gestural forms, such as piano techniques in Ellis' work and the breath and fingering patterns from the saxophone in Ashanti's work.

Explore further: Improving the experience of the audience with digital instruments

Related Stories

Seeing isn't required to gesture like a native speaker

March 21, 2016

People the world over gesture when they talk, and they tend to gesture in certain ways depending on the language they speak. Findings from a new study including blind and sighted participants suggest that these gestural variations ...

Recommended for you

Ancient parrot fossil found in Siberia

October 26, 2016

(—A Russian paleontologist has discovered a parrot fossil uncovered in Siberia several years ago—the first evidence of parrots living in Asia. In his paper published in Biology Letters, Nikita Zelenkov describes ...

Ancient burials suggestive of blood feuds

October 24, 2016

There is significant variation in how different cultures over time have dealt with the dead. Yet, at a very basic level, funerals in the Sonoran Desert thousands of years ago were similar to what they are today. Bodies of ...


Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.