Integrating Score and Sound: "Augmented Notes" and the Advent of Interdisciplinary Publishing Frameworks

paper, specified "long paper"
  1. 1. Joanna Elizabeth Swafford

    University of Virginia

Work text
This plain text was ingested for the purpose of full-text search, not to preserve original formatting or readability. For the most complete copy, refer to the original conference program.

1. Introduction
While sound studies is experiencing a resurgence in literary criticism, academic arguments involving sound are nearly impossible to make in traditional print media. An article could include an excerpt of a score, but only scholars who can read music would be able to understand it. Likewise, articles or books could include audio files externally, as did Nicholas Temperley’s special edition of Victorian Studies, which included a cassette tape with the songs discussed in the articles. 1 However, these solutions do not address the central problem: readers will have difficulty finding the exact musical phrases mentioned in articles, and those with less musical expertise will be left out of the conversation entirely. Newer options for incorporating music in academic articles include SoundCite (, a tool that lets users embed sound clips in websites, Scalar (, a publishing framework that lets users annotate media, and the strategy of assigning a QR code to each audio excerpt and inserting these into a print article, as Jennifer Wood suggests. 2 None of these options integrates the audio with the score: SoundCite will only let users hear the audio, Scalar only supports textual annotations of media files, and QR codes require readers to have smart phones, which vastly limits the audience for the article. To address these problems, I have built two tools: "Songs of the Victorians" (, an archive and analysis of musical settings of Victorian poems with an interactive framework that highlights each measure of a score in time with its music, and "Augmented Notes" (, a public humanities tool that allows users who do not know how to program to build their own sites like "Songs of the Victorians."

2. Overview of "Songs of the Victorians"
"Songs of the Victorians" melds the archive and the scholarly article. It examines both high- and low-brow Victorian settings of contemporaneous poetry by integrating scores, audio files, and scholarly analytical commentary in an interactive environment to help users understand both the literary and musical elements of the argument. As an archive, it provides audio files of each song and archival-quality scans of first-edition printings of each score. For every song, the user can listen to the audio while each measure of the score is highlighted in time with the music, as the archive page for William Balfe’s "Come into the Garden Maud" demonstrates ( The project also functions as a collection of scholarly articles in which each song includes an analysis of the song’s interpretation of the text. When the commentary discusses a particular measure, the users can click on an icon of a speaker, which will play the relevant excerpt of the audio file and highlight the score so they can hear for themselves the effect the commentary describes, as in the analysis page for Caroline Norton's "Juanita" (

Fig. 1: Musical excerpt from the analysis page of Caroline Norton’s “Juanita”

"Songs of the Victorians" includes Caroline Norton’s "Juanita," Sir Arthur Sullivan’s setting of "The Lost Chord," and two settings of Tennyson's Maud: a parlor song by Michael William Balfe and an art song by Sir Arthur Somervell. The site furthers scholarship for bibliographers, musicologists, Victorianists, and cultural studies scholars alike. More generally, this new framework, which enables critics to describe musical arguments to non-musicians, facilitates this interdisciplinary approach of bringing music and literature together. It also preserves the musical and cultural afterlives of well-known poems, as many of these scores have either disintegrated and been lost to time or are only available in select libraries. "Songs of the Victorians" empowers users regardless of musical training: those who cannot read music can overcome their feelings of intimidation at a musical score and can better understand the ideas described in the analysis, whereas those who can read music will still benefit, since few people can hear in their mind the music on the page.

3. Overview of "Augmented Notes"
After the success of "Songs of the Victorians," I used its framework to produce "Augmented Notes" (, a generalized, public humanities tool to allow anyone to develop similar websites. "Augmented Notes" eliminates the need for users to understand programming by creating archive pages, like those from "Songs of the Victorians," which users can tweak and redesign. It is simple to use, as the site only requires audio files and images of the score to produce an archive page. After the audio and image files are uploaded, users are taken to a page where they click and drag to draw boxes around each measure (they can also edit the sizes and order of these boxes), indicating what portion of the score should be highlighted when that measure plays.

Fig. 2: Box-drawing page of “Augmented Notes”

Users can also optionally upload an MEI file--the TEI-based scholarly standard for music--for the score if they already have measure positions recorded in MEI. Users then set the times at which the highlighting box changes position through a "time editing" page.

Fig. 3: Time editing page of “Augmented Notes”

The site brings together the measure and time information, saving them in a JSON file, which enables each measure of the song to be highlighted in time with the music. Users then click "Download Zip" to download a zip file with the HTML, CSS, and JavaScript files necessary for a complete archive page, which they can then restyle themselves.

"Augmented Notes" also has a sandbox ( through which users who would like to experiment with the technology but do not themselves have the requisite files can try it out. "Augmented Notes" is already being used by scholars, both for archival purposes (such as the "Performing Romantic Lyrics" project from the University of South Carolina) and for pedagogical purposes such as generating interactive scores for use in music classrooms. Since this tool produces websites with integrated audio and scores, it empowers users to preserve cultural archives, whether their materials include classical music, unpublished manuscripts, popular music, or folk music and traditional tunes from around the globe.

4. Implications
This presentation will discuss the projects in greater detail, complete with live demonstrations and an explanation of the underlying technology to show their digital as well as scholarly innovations. I will explain the rationale for my choice of poems, settings, sound files, and editions for "Songs of the Victorians," as well as my plans for future collaboration and expansion for both projects. The presentation will illustrate the sorts of arguments that this framework can enable: for example, my examination of Sir Arthur Sullivan’s setting of Adelaide Procter’s "A Lost Chord" challenges the received interpretation that the poem merely describes a domestic, uncomplicated religious moment of transcendence. Likewise, Caroline Norton’s "Juanita" has been considered a conventional song that preserves the traditional rules of courtship and parlor propriety, but my analysis of the music helps us see that it critiques the Victorian institution of marriage as imprisoning. I will conclude by exploring the ways in which "Songs of the Victorians" is itself a Victorian endeavor, as it uses new technology to collect, analyze, and bring together Victorian music and poetry, thereby giving voice to the silent page.

5. Funding
This project was made possible by fellowships from NINES and the Scholars’ Lab.

1. Temperley, Nicholas (1986). Music in Victorian Society and Culture: A Special Issue of Victorian Studies. 30.1.

2. Wood, J. (2013). Noisy Texts: How to Embed Soundbytes in Your Writing. Burnable Books. Ed. Bruce Holsinger. (Accessed 30 October 2013).

If this content appears in violation of your intellectual property rights, or you see errors or omissions, please reach out to Scott B. Weingart to discuss removing or amending the materials.

Conference Info


ADHO - 2014
"Digital Cultural Empowerment"

Hosted at École Polytechnique Fédérale de Lausanne (EPFL), Université de Lausanne

Lausanne, Switzerland

July 7, 2014 - July 12, 2014

377 works by 898 authors indexed

XML available from (needs to replace plaintext)

Conference website:

Attendance: 750 delegates according to Nyhan 2016

Series: ADHO (9)

Organizers: ADHO