<audio>Digital Humanities</audio>: The Intersections of Sound and Method

A wide range of interdisciplinary scholarship on sound has sparked investigations into the cultural histories of aurality and sound reproduction, the politics of the voice and noise, urban soundscapes, ethnographic modernities, acoustemologies, and the sonic construction of gender, race, and ethnicity.[i] These important qualitative studies, moreover, have in recent years been supplemented by large-scale quantitative analyses of speech and music datasets, several of which have been underwritten by the International Digging into Data Challenge, including the “Structural Analysis of Large Amounts of Music” (SALAMI) and the “Mining a Year of Speech” projects. Yet a lingering textual bias within digital humanities – largely a product of the field’s emergence from textual and literary studies – has obscured the significance of this work for the field, often preventing meaningful overlap. Copyright restrictions, the difficulties of archiving audio formats, and the general lack of tools for researching and writing in audio have contributed to the difficulty of working with sound in digital projects. Aside from the occasional use of CD appendixes or supplementary websites, for example, many studies have not taken full advantage of the affordances of digital media to produce scholarship that integrates audio content into scholarly argumentation. It is against this backdrop that leading sound theorist Jonathan Sterne has argued that “existing digital humanities work has largely reproduced visualist biases in the humanities” (2011).
By identifying and highlighting four research initiatives clustered around audio artifacts, this panel aims to bring sound scholarship and digital humanities into a more meaningful conversation with each other. As these projects demonstrate, sound is materially constituted, containing invisible environmental fingerprints or leaving physical traces in artifacts; and, further, is performative and temporally mediated. Thus to access and analyze sound requires not only a new approach to “tool making” within digital humanities, but a deeper engagement with media studies, archival science, and creative forms of scholarship more generally. As Trettien and Lingold’s Soundbox initiative shows, the methodological vibrancy of the field is also predicated on innovation and reform of our critical infrastructures, including the development of publication environments that can take advantage of the cross-medial character of much sound research. Elliott’s kits for cultural history, for example, allow users to experience the past through multiple sensory channels, including sight, sound, and touch; and Clement and Kraus’s work incorporates extensive spectrographic analysis. Thus a larger aim of the panel is to draw attention to the richly synaesthetic nature of digital sound studies.
Access and Analysis, Tanya Clement (15 minutes)

There are few analysis tools available for humanists interested in accessing and analyzing audio archives that comprise significant artifacts of bygone oral traditions represented in storytelling, speeches, oral histories, and poetry performances. In response to this lack, the iSchool at UT-Austin and the Illinois Informatics Institute (I3) at the University of Illinois at Urbana-Champaign (UIUC) hosted a year-long NEH-funded Institute for Advanced Topics in the Humanities called High Performance Sound Technologies for Analysis and Scholarship (HiPSTAS). HiPSTAS included twenty humanities junior and senior faculty and advanced graduate students as well as librarians and archivists interested in analyzing large audio collections. As this speaker will address, HiPSTAS has yielded significant results for audio big data analysis in the humanities including an implementation of the ARLO (Adaptive Recognition with Layered Optimization) software, a machine learning application for analyzing sound on Stampede, an NSF petascale HPC system at the Texas Advanced Computing Center. Originally developed to classify and analyze bird calls by extracting audio features and displaying the audio data as a spectral graph (Downie et al. 2008, Punyasena et al. 2012), ARLO has also been used by humanists as part of HiPSTAS to extract basic prosodic features such as pitch, rhythm and timbre for matching, discovery (clustering) and automated classification (prediction or supervised learning) (Figure 1). This talk will discuss how significant sonic patterns of interest to humanists are discoverable using ARLO with the PennSound poetry archive and the University of Texas Folklore Center Archives, among other collections.

Fig. 1: This spectrogram, created in ARLO, shows Gertrude Stein reading “Some such thing” from her novel The Making of Americans; each row of pixels is a frequency band presented across an X-access of time.
Media Archaeology, Devon Elliott (15 minutes)

Recent research in media archaeology (Fuller 2005, Gitelman 2006, Kirschenbaum 2008) underscores why the material particulars of technology matter where questions of culture are concerned. This research is frequently anchored in archival documents—including lab notebooks, patents, and engineering journals—that correspond with technological experiments. Building on this research, this talk shares initial findings from the “Kits for Cultural History” project, a collaboration between the Maker Lab in the Humanities (UVic), the Lab for Humanistic Fabrication (Western), and several memory institutions across Canada. The project involves making physical kits that encourage scholars to reconstruct historical experiments through the use of schematics, facsimiles, and rich media. Audio is central to a number of these kits, especially kits that focus on sound reproduction. Not only does it add another modality to research that is usually text-based or visual in character. It also emphasizes how any media history is a history of the senses: a history of how embodied behaviors like listening relate recursively with technological developments. With audio in mind, the talk argues for the relevance of experimental reconstruction to digital humanities, highlighting the importance of: 1) old technologies to contemporary computing practices, 2) multimodal learning and applied methods to media history, 3) integrating museum collections into these methods, and 4) understanding sound as necessarily material, subject to techniques commonly found in, say, textual studies. These four points draw together domains all too often parsed: visual and sonic paradigms, critical thinking and critical making, media archaeology and digital humanities.
Signal and Noise, Kari Kraus (15 minutes)

Twentieth-century recorded sound, like the first electric power system, originated in Thomas Edison’s Menlo Park laboratory shortly before the turn of the century (Hughes, Morton). In the decades that followed, sound technology and power transmission would continue to develop in tandem. In this presentation we introduce an unexpectedly useful consequence of the historic entanglement of sound and electricity: the ability to code our past for time and place. A new collaboration at the University of Maryland aims to recover the date and time on which an historic recording was made based on analysis of incidentally captured traces of small variations in the electric power supply at the time of recording (Oard, et al; Su, et al). Although the field of audio forensics has used such Electric Network Frequency signatures to authenticate contemporary recordings for over a decade, our project seeks to extend the period for which baselines are available a further half century into the past. We do this by assembling recordings that were made at known times and comparing their ENF signatures with the signatures in recordings for which we lack such provenance information.
After summarizing the results of our initial experiments, we focus on implications for archival practice, including retention of the original ENF signal across media formats (Figure 1), and conclude on a theoretical note: because ENF is traditionally dismissed as electronic noise by audio engineers and regarded as non-semantic in character, it poses an interesting challenge to the well-established archival concept of “significant properties.”

Fig. 2: Analog recordings that have undergone digital conversion and reformatting will often contain two or more ENF signatures: an original and a recaptured signature. The spectrogram in the image on the left shows the ENF trace from a 1962 magnetic tape recording of an oval office meeting during the Kennedy administration and a separate ENF trace embedded at the time of digitization. In the figure on the right, two signatures overlap. We have developed preliminary techniques for distinguishing these multiple traces.
Publication and performance, Whitney Trettien and Mary Caton Lingold (15 minutes)

Soundbox is a collaborative exercise in producing and publishing sonic scholarship. Its main research output is an edited digital collection bringing together a vanguard of emerging scholars and critical artists engaging in sonic scholarship, from exhibits and installations to digital essays, soundscapes, and speculative digital tools.
While the original goal of the project was to show, through example, the wide range of possibilities for an amplified digital humanities, the impossibility of publishing this work through standard scholarly venues – that is, those that facilitate the forms of peer review required for advancement in the profession – has become clear as the project proceeds. Because of concerns over long-term maintenance of digital scholarship, database-driven platforms like Omeka, Scalar, and Wordpress are quickly becoming the standard publishing format for digital work. Designed around arguments written in text and image, though, these platforms are largely inadequate to scholarship that integrates sound beyond the occasional linked audio clip. Thus the potential for amplified scholarly production opened up by, for instance, creative, small-scale, targeted uses of the HTML5 audio tag remains largely unrealizable within an increasingly calcified digital publishing infrastructure – a fact with ongoing consequences for what “counts” as digital humanities scholarship.
Using Soundbox’s experience as a case study, the speakers address the structural biases that continue to silence digital humanities. We argue for balancing the need for long-term maintenance and accessibility with a pluralistic approach that does not foreclose the possibilities of new forms and formats.
References

Attali, Jacques (1985). Noise: The Political Economy of Music. Minneapolis: University of Minnesota Press.
Cavarero, Adriana (2005).For More Than One Voice: Toward a Philosophy of Vocal Expression. Stanford, Calif: Stanford University Press. Print.
Dolar, Mladen (2006). A Voice and Nothing More. Cambridge, Mass: MIT Press.
Downie, J. S. , Tcheng, David K., and Xiang, Xin (2008). “Novel interface services for bioacoustic digital libraries” in Proc. 8th ACM/IEEE-CS Joint Conf. on Digital Libraries. New York: ACM, 2008: 423-423.
Fuller, Matthew. (2005). Media Ecologies: Materialist Energies in Art and Technoculture. Cambridge, Mass: MIT Press.
Gitelman, Lisa. (2005) Always Already New: Media, History, and the Data of Culture. Cambridge, Mass: MIT Press.
Hirschkind, Charles. (2006) The Ethical Soundscape: Cassette Sermons and Islamic Counterpublics. New York: Columbia University Press.
High Performance Sound Technologies for Access and Scholarship project blogs.ischool.utexas.edu/hipstas/
Hughes, Thomas Parke. (1983) Networks of Power: Electrification in Western Society, 1880-1930. Baltimore: Johns Hopkins University Press. Print.
Josephson, Matthew (1992). Edison: A Biography. New York: J. Wiley. Print.
Kirschenbaum, Matthew (2008). Mechanisms: New Media and the Forensic Imagination. Cambridge, Mass: MIT Press.
LaBelle, Brandon (2010). Acoustic Territories: Sound Culture and Everyday Life. New York: Continuum.
Meintjes, Louise (2003. Sound of Africa!: Making Music Zulu in a South African Studio. Durham: Duke University Press.
Morton, David. (2004). Sound Recording: The Life Story of a Technology. Westport, CT: Greenwood Press. Print.
Moten, Fred (2003). In the Break: The Aesthetics of the Black Radical Tradition. Minneapolis: University of Minnesota Press.
Oard, D., M. Wu, K. Kraus, A. Hajj-Ahmad, H. Su, R. Garg (2014). “It’s about Time: Projecting Temporal Metadata for Historically Significant Recordings.” Forthcoming, Proceedings of the 2014 iConference. Berlin, Germany. 4-7 March 2014. ACM Digital Library. Web.
Ochoa, Ana Maíra (2006). “Sonic Transculturation, Epistemologies of Purification and the Aural Public Sphere in Latin America.” Social Identities 12.6: 803-25.
Punyasena, Surangi W., Tcheng, David K., Wesseln, Cassandra, Mueller, Pietra G (2012). “Classifying black and white spruce pollen using layered machine learning.”New Phytologist 196.3: 937-944.
Rodgers, Tara (2010).Pink Noises: Women on Electronic Music and Sound. Durham, NC: Duke University Press.
Smith, Mark M (2006). How Race Is Made: Slavery, Segregation, and the Senses. Chapel Hill: University of North Carolina Press. Print
Smith, Mark M (2001). Listening to Nineteenth-Century America. Chapel Hill: University of North Carolina Press. Print.
Sterne, Jonathan. The Audible Past: Cultural Origins of Sound Reproduction. Durham, NC: Duke University Press, 2003.
Sterne, Jonathan. (2011) “Audio in Digital Humanities Authorship: A Roadmap.” (essay in progress) Super bon! Online:superbon.net/?p=1915. Accessed June 7, 2013.
Sterne, Jonathan (2012). MP3: The Meaning of a Format. Durham: Duke University Press.
Su, H., Garg, R., Hajj-Ahmad, A., Min Wu (2013). “ENF Analysis on Recaptured Audio Recordings.” Proceedings of the 2013 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP). Vancouver, BC, Canada. 26-31 May 2013. 3018-3022. Web.
Thompson, Emily Ann (2002). The Soundscape of Modernity: Architectural Acoustics and the Culture of Listening in America, 1900-1933. Cambridge, Mass. : MIT Press.
Toop, David (2010). Sinister Resonance: The Mediumship of the Listener. New York: Continuum.
Weheliye, Alexander G. (2005). Phonographies: Grooves in Sonic Afro-Modernity. Durham, NC: Duke University Press.
For cultural histories of aurality, see entries for Smith 2001, 2006 in the bibliography; for sound reproduction, Sterne 2003, 2012; for the politics of the voice, Cavarero 2005, Dolar 2006; and noise, Attali 1984; for urban soundscapes, see entries for Toop 2010, Thompson 2002, Labelle 2010; ethnographic modernities and acoustemologies are covered in Hershkind 2006 and Ochoa 2006. The sonic construction of gender receives treatment in Rodgers 2010 and Martin 1991; for race and ethnicity, see Weheliye 2002, Moten 2003, Smith 2006, and Meintjes 2003.