Surround Sound

Eyebots in Fallout 3 play music from the '40s

The current HASTAC forum on Sound and Technology has me thinking about the way sound is used in video games. In particular, I have been thinking about the difference between the way sound is used to create atmosphere and the way it is used to create a sense of space. Below is the basic distinction I made on the forum with a bit more consideration.

By “atmosphere” I mean how sound plays into the overall aesthetic and feel of the world being presented. It is the way Bioshock uses sound effects, NPC dialogue, and music to flesh out the richness of Rapture. Another example is Fallout 3, the game I am currently playing (finally), which punctuates a near future post-apocalyptic landscape with ‘40s Big Band music. Together they give the game a bizarre tone that feels both historical and futuristic, but at the same time it is a perfect extension of the game’s central alt-history conceit which basically wonders what would happen if WWII led into a nuclear holocaust.

This is something different from the way sound contributes to an experience of 3D space. In Fallout 3, again, because it is an open-world game and enemies can come at your character from any angle, one has to rely on hearing the enemies when they are not in one’s line of vision. There is no peripheral vision in games, the limit of onscreen display is a hard limit, so sound becomes really important for locating one’s character – and potential threats – in virtual space.

From the point of view of immersion, sound should ideally blend into the background. Atmpospheric sounds work seemlessly with spatial sounds to compose the virtual world without becoming singularly noticable in doing so. I have found Fallout 3‘s background soundtrack, for example, to be pretty successful at this, as most of the time I am not aware there is any music is playing at all.

And, yet, there are often those times in games in which the relationship between atmospheric and spatial sounds is more complicated. Take for example, driving in Grand Theft Auto 4. The sound of the engine, tires, other cars contribute to the sense of moving through space. At the same time, the radio station one selects changes the tenor of that movement. There is something really cold about mowing down pedestrians while listening to talk radio, while listening to Barry White as you ramp your motorbike into the Hudson is pretty funny.

So, video game sound can mix atmosphere and space in lots of different ways. For example, the moments when the instrumental background soundtrack and sound effects give way to full vocal tracks in Red Dead Redemption produced some of the most substantial emotional experiences I’ve ever had with a game. Sound in games, then, is not just about creating a world, but also the affect that can emerge from an unexpected confluence of atmosphere and space.

Interactivity and the Law

gam_madworld_490 (1)This week, the Supreme Court heard arguments in Schwarzenegger v. Entertainment Merchants Association, which the state of California is basically asking for an exception to the First Amendment with regard to violent video games. It is an extremely important case, which could very well decide the future of the medium. So far the conversation has been a little weird, as Justice Scalia seems to think this is all patently ridiculous. Thankfully, even though not particularly knowledgeable about contemporary gaming, having spent much of the session on Mortal Kombat (1992), the Court appears justifiably skeptical of California’s measure, which is too broad and ill-defined. They seem to rightly recognize that this is a gateway provision, pitched as protecting children but could be referred to later to censor other “objectionable” media.

Central to the Court’s complaint is a concern that such a proposition might escalate to include other forms of currently protected free speech. This manifested as a broad comparative discussion of media forms. Violence in games was compared to violence in rap lyrics, in cartoons, and in Grimm’s Fairy Tales. In short, as Justice Ruth Bader Ginsburg summerizes, “Why are video games special?”

Justice Roberts pushed California for a response:

Chief Justice John Roberts: I think that misses Justice Ginsburg’s question, which is: Why just video games? Why not movies, for example, as well?

Morazzini: Sure, Your Honor. The California legislature was presented with substantial evidence that demonstrates that the interactive nature of violent — of violent video games where the minor or the young adult is the aggressor, is the — is the individual acting out this — this obscene level of violence, if you will, is especially harmful to minors.

California’s argument rests on two basic assumptions.

  1. that the violence in gaming is ultra violent.
  2. That gaming is interactive, which makes it worse.

The first tenet became the focus of conversation, as Justice Kagan and Sotomayor wanted to know if eventually the provision would extend to things like movies and Bugs Bunny cartoons. The second tenet, however, went pretty much unchallenged. Judge Roberts, in fact, echoed the sentiment later, stating  “In these video games, the child is not sitting there passively watching something; the child is doing the killing. The child is doing the maiming.”  In Morazzini’s statement, there is, at least, some distance between game experience and life experience — acting aggressively in-game is potentially harmful – even if there is no explanation as to why. Justice Roberts, on the other hand, conflates virtual and real violence: “the child is doing the killing.”

Here, in the continued confusion about interactivity, is a real cause for concern and why there is perhaps no better time to have a serious discussion of immersion and interactivity.  What bothers me about the logic attached to interactivity being used here, is that it fails to grasp interactive media as an phenomena in and of itself. I understand the need to compare gaming with other media in order to establish a legal precedent on which to ground this case; that is expected and actually helps gaming’s cause from what I can tell. The issue, though, is when people say like-movies-but-interactive, they almost always mean like-movies-but-more-so. In this case, games are violent like movies, but interactive, so the effect is greater. Most of the time this argument is left at that, as if it is tacitly understood that interaction intensifies everything. But, when drawn out, the logic frequently goes where Justice Robert takes the argument, collapsing represented action with physical action