Audial information—assigning a sound to data—is a part of the modern world. You might recognize the concept as the Geiger counter, the device that emits an increasing rate of clicks correlated to growing levels of radiation. Other examples include sonar, cockpit displays and a ticking clock.

As basic as some sounds can be to relay information, there is a growing contingent of creative types who bring a sense of soul to their sound data. To illustrate how audio samples might paint pictures of information for the ears, the Center for the Performing Arts at Penn State will host events by multimedia artist-explorer Paul D. Miller, aka DJ Spooky, and artist-academic Mark Ballora.

‘Arctic Rhythms’

Starting in 2007, Miller, a multimedia artist, author and influential turntable DJ, traveled to the north and south polar regions to record data related to the effects of climate change. He returned to New York City with inspiration for a book and a collection of music, “Of Water and Ice.” As an emphatic believer in the concept of climate change, he said he felt moved to put the data to use and created “Arctic Rhythms.”

Miller will present his multimedia travelogue at 7:30 p.m. March 23 in Eisenhower Auditorium. The presentation will feature compositions infusing hip-hop beats and live classical instrumentation by a Penn State graduate string quartet to create the musical equivalent of what climate change looks like.

He said he hopes that his audio program will help people see the issue of climate change as a recognizable pattern and how it might affect other aspects of humanity—including political unrest and global health issues.

“The role of the arts isn’t necessarily about policy,” he said, “but the arts play a very powerful role in getting people to reframe the way we think about our present moment. ... The intense social change that happened with the civil rights movement wouldn’t have been possible without music. So, that’s where, I would say, you need more and less of composition, music and art about climate change, because I think that’s where you are going to see a lot more, for lack of a better word, immediacy.”

He said he wanted people to really think about climate change and saw that opportunity through his musical experiments and soundscapes.

“A symphony or a string quartet imitating the sound of ice is just as powerful as policy and/or primary science,” Miller said. “Imagine if science had better soundtracks. I think it would have more impact.”

Miller said he sees the value of musically sonifying sets of information and composing musical works based on the link between historic mathematics and the algorithms of today, but he stresses the importance of a certain amount of flair in its presentation. He said he realizes that sets of data-driven information, featuring hard math and science, aren’t always palatable. As a digital DJ, he bases his creations on the model of collage and layering.

“I’m an artist, I mean, (raw information) doesn’t work. It doesn’t look cool, it doesn’t sound cool,” he said. “They need to have additional pull, gravity to pull them into the mix. That’s what this is about. It’s layers, and it’s bits and pieces that can easily be remixed, too.”

“It’s gotta be accessible.”

‘Seeing with Your Ears’

Mark Ballora wants to take Miller’s idea of making information accessible to a more literal place.

The Penn State associate professor of music technology made headlines in the recent past with his sonification contributions to musical works by Grateful Dead percussionist Mickey Hart, a film by Nobel laureate and astrophysicist George Smoot and last year’s release of his squirrel data not-so-symphonies. He teeters on the edge of pure science and musical creativity, but he said he leans more toward the role of the artist than of the scientist.

In turning data into sound, “You’re mapping them to pitch, or volume or other characteristics like that, which is usually what I do,” the self-described “artist with science envy” said.

He will host the lecture “Seeing with Your Ears: Visualizing Data with Sound” at 5:30 p.m. March 23 in 100 Life Sciences Building. The event will give attendees a chance to see how their own sounds can be assigned to official scientific information.

Ballora is fond of his 2014 Antarctic ice project, in which he and then-Penn State graduate student Matthew Kenney merged tracks representing solar radiation, ground temperatures and ice. The result is a new-age sonic representation of a day in the life of Antarctica. As informative as that audio portrait is supposed to be, Ballora said he sees a more real-world application for his sonification projects. He’s applying for grants to study the effect of audio segments in the learning process.

“If you’re presenting kids with some information about the climate and storms, how does it affect their learning experience if there’s a soundtrack to that?” he said. He expects that audio-enhanced information would be helpful for learners of all abilities.

“We’d be looking at different kinds of programming for different kinds of audiences and seeing how it helps them,” he said. “We’d be interested in how it helps kids either on the autism spectrum or who have visual impairments.”

Ballora said he was inspired after hearing an NPR report about a computer program called Exonify, created for a nonsighted astronomer so she could “hear” data.

“Then they mentioned, ‘Oh some of her sighted colleagues use it, too, because there are patterns that they could hear that they couldn’t always see,’” he said. “That just emphasizes what I’m saying about why that’s useful. It’s really interesting that this was made for someone who couldn’t see, so it was made for reasons of accessibility, but then it brought out these characteristics of the data.

“We think this is promising in education because it plays upon this really low level of perceptual capacity that we have as humans, that we’re just sensitive to things that we hear. We don’t all like the same kind of music, but everybody’s passionate about some kind of music.”

Ballora equates the enhanced, accessible, musical delivery of information with images of far-off galaxies.

“They never just show you a photograph of space. They color it, and they process it in a lot of ways to make it look appealing,” he said. “So just doing something for the ears, it’s not that different, you’re just working with a different sense and what its capabilities are. The eyes do certain things and the ears do certain things, but it’s trying to make art of information in a way that’s both appealing and informative.”