More than Words: Astonishment and Special Effect in Multimedia

In this paper, I contend that current approaches to multimedia privilege
language and narrative over visual and auditory effect, and that such
privileging maintains Enlightenment hierarchical arrangements of word above
image, of mind above body, and of intellect above emotion. Important areas
in humanities computing have operated within these binaries to their
academic benefit. Yet, while a focus on language and narrative has helped
some lines of humanities computing gain academic legitimacy, it has also had
the effect of sustaining linguistics-based and narrative-based critical
approaches to multimedia that ultimately prove inadequate to understanding
works based substantially upon visual and auditory elements. If we are to
understand the complexities of how we experience multimedia, then we need to
look more closely at how visual and aural elements function within
interactive environments. This is a key critical task for humanities
computing scholars who are now working more and more with new media elements
such as image, animation, video, and audio.

I focus on the multimedia genre of computer games to demonstrate that the
attraction to computer games and, by implication, other genres of rich
multimedia, is not based so much upon linguistic and narrative meaning, as
some have argued.[1] Rather, a significant component of the appeal of
computer games is our astonishment at special effects and, importantly, at
our participation in their technological performance. In this respect,
analysis of multimedia may benefit from special effects film theory, in
which technological astonishment is key to understanding our fascination
with spectacular, special-effects driven film. In short, some film
theorists argue that when we watch special-effects film we oscillate between
a willing suspension of disbelief and our astonishment at the technological
cinematic display.[2] Our enjoyment of playing computer games is driven by
a similar oscillation between willing immersion and technological awe with
the added ingredient of our participation in the technological performance.

The major parts of this paper are:

1. Linguistic bias in the humanities and humanities computing
1.1. Humanities
1.2. Humanities Computing
1.3. Hypertext Theory
2. Narrative bias in major studies of multimedia art and entertainment
3. Special effects, astonishment, and film theory
4. Performing astonishment in multimedia
5. Conclusion: Humanities computing and computer games

1. Linguistic bias of humanities and humanities computing
Until recently, the primary object of humanities computing research has been
linguistic forms of text. The word has taken precedence over image and
sound, and humanities computing research has chiefly focused on the problems
of linguistic text. There are several reasons for this logocentrism:

1.1 The systemic privileging of the written word in the humanities:
Even with the substantial growth of academic areas such as film studies,
cultural studies, pop culture, and visual culture, written text remains the
privileged form of intellectual expression, both in the works we study and
in the works we produce.

1.2 The disputable assumption that word-based computational research is
easier to perform than image- or audio-based computational research:
Words may seem easier to analyze computationally because alpha-numeric
characters are discrete and definable units, whereas image and sound may
seem more difficult to reduce to the finite level of individual units for
computational analysis.

1.3 The academic success of hypertext theory:
As one of the most widely-accepted areas of humanities computing, hypertext
theory has enjoyed success for its intersections with the most influential
critical theories in humanities scholarship--post-structuralism and its
derivatives--which are built substantially upon linguistics-based
conceptions of meaning.

A linguistic focus may be suitable for studying written works, but
multimedia works normally include a mixture of word, image, and sound, or
they may even rely upon image and sound alone. As such, multimedia should
be understood within a framework where the qualities of image and sound that
are different from written text can be recognized and understood as
different. This seems especially important for multimedia theorists to
remember, for one of the defining features of multimedia is the combination
of different media types. Oddly, the "multi" in multimedia is too often
neglected.

2. Narrative bias in major studies of computer games
In this section, I will quickly summarize dominant approaches to studying
computer games. Although few significant humanities-based studies of
computer games have been published, those that exist, for all their
strengths, tend to prioritize narrative over other element. Drawing from
Espen Aarseth’s Cybertext and Janet Murray’s Hamlet on the Holodeck, I
demonstrate that even when these studies try to build new frameworks for
studying computer games as interactive narratives, they emphasize narrative
over special effects, which are cast as empty and secondary to the potential
for computer games to move beyond the superficial.

3. Special effects, astonishment, and film theory
As a step to thinking about special effects in multimedia, I introduce
neo-formalist film approaches to special effects. Using Tom Gunning’s
"Cinema of Attractions" as a springboard, some film theorists argue that
special effects film should be analyzed not within the context of narrative,
an approach drawn from literary studies and applied to written and oral
works substantially different from film. Instead, special effects film
should be analyzed from the perspective of the viewer’s astonishment at the
special effects, and especially at the technology that produces them.

4. Performing astonishment in multimedia
In the final section of this paper, I look directly at astonishment in
computer games. In general, as players progress through today’s computer
games from level to level, the special effects tend to get more dazzling, as
if the prize for solving puzzles or beating opponents is the chance to test
their game play in a more difficult, but more visually and aurally stunning
environment. In first-person shooters like Doom, Quake, and Half-Life,
monsters and opponents get bigger and more fantastical; in strategy games
like Age of Empires II, buildings, communities, soldiers and war machinery
get larger and more visually detailed; similarly, in simulation games like
the SimCity series, buildings and transportation systems grow in size and
become more visually prominent and futuristic looking; and in
puzzle-adventure games like the remarkably rich Eve from Peter Gabriel's
Real World Multimedia, mudflats become Edenic gardens and music studios
become populated with music samples and animated image sequences for the
user to assemble into interactive music videos. In all cases, players are
given more astonishing special effects as they progress through the game,
suggesting that our experience of computer games is structured in
significant part by the increasing intensity of special effects.

Narrative is not absent from these computer games, but it is not the primary
reason for their appeal, nor would the games necessarily be any better if
narrative were strengthened. It is the technological spectacle and the user
’s participation in performing the effects that makes computer games fun.
If there is a narrative at work in computer games greater than a frame upon
which to display technological magic, then it is the narrative of the
special effects technology itself. Indeed, magazine and user reviews of
computer games frequently contextualize a game within the history of special
effects software and hardware. In this way, the astonishment of computer
games not only includes our awe at performing the special effects
technology, but also our awe at having access to the technology at all.

This final point is important to emphasize, for it suggests that part of the
enjoyment of multimedia, whether computer game, home encyclopedia, scholarly
resource, or e-commerce web site, is our amazement at our access.

5. Conclusion: Humanities computing and computer games
In the conclusion to this paper, I connect the place of astonishment in
computer games with the academic goals of humanities computing by posing and
providing provisional answers to a series of related questions: What is a
special effect and can the treatment of a special effect make it more
serious and less sensational? How do we account for the role of
astonishment over computer technology’s ever-increasing power to handle and
manipulate word, image, and sound? Should we avoid the temptation to
incorporate special effects in our academic computer productions, and
thereby avoid tainting the seriousness of our work? Or do we treat
astonishment seriously and re-evaluate the role of emotional response in
building intellectual understanding? As humanities computing scholars deal
more and more with non-linguistic elements in their work, these questions
may be fruitfully considered within the context of computer games and other
forms of multimedia art and entertainment.

Notes:

[1] See Aarseth and Murray.

[2] See Gunning, Landon, and Ndalianis.

Works Cited:

Aarseth, Espen. Cybertext: Perspectives on Ergodic Literature. Baltimore:
Johns Hopkins UP, 1997.

Gunning, Tom. "The Cinema of Attractions: Early Film, Its Spectator and the
Avant-Garde." Early Cinema: Space, Frame, Narrative. Eds. Thomas Elsaesser
with Adam Barker. London: BFI, 1990, pp. 56 - 62.

Landon, Brooks. The Aesthetics of Ambivalence: Rethinking Science Fiction
Film in the Age of Electronic (Re)production. Westport, Conn.: Greenwood
P., 1992.

Murray, Janet. Hamlet on the Holodeck: The Future of Narrative in
Cyberspace. Cambridge, Mass.: MIT Press, 1997.

Ndalianis, Angela. "Special Effects, Morphing Magic, and the 1990s Cinema
of Attractions." Meta-Morphing: Visual Transformation and the Culture of
Quick Change." Ed. Vivian Sobchack. Minneapolis, MN: U Minnesota P., 2000,
pp. 251 - 71.