Two new tools for multimodal editions

1. Introduction

From the earliest e-Text centers at Oxford1 and the University of Virginia 2, through the development of the TEI in the late 1980s3, to the publication of image-based editions in projects like The William Blake Archive 4, digital editing projects have long been a core activity of the Digital Humanities. Until very recently, though, the limitations of reliably available technology and complicated intellectual property laws have kept all but the most adventurous editors from venturing beyond the media of text and image to multimodal editions that incorporated video and audio recordings. Over the past year, however, the New York Public Library for the Performing Arts has released two new digital editing projects: an online tool for producing editions of dance video and a mobile app for publishing multimodal, variorum editions of musical theater libretti. This paper will examine both projects and what they imply for the future of critical edition building in the 21st century.
2. Antecedents

Although most digital editions produced by digital humanities scholars have presented only text and image, over the past two decades, a few pioneers attempted to produce multimodal texts of various kinds. In the 1993s the Voyager Company produced a series of Hypercard and CD-ROM based multimodal editions (such as a Companion to Beethoven’s 9th Symphony) 5. A 2002 project led by Janet Murray produced a critical digital edition of the film, Casablanca 6. More recently, the Maryland Institute for Technology in the Humanities, Music Theatre Online, produced with an NEH Digital Humanities Startup grant by the Maryland Institute for Technology in the Humanities, linked several libretti of the 2008 musical Glory Days to mp3 files from live performances7. Although each was an interesting experiment that advanced the field of digital editing, the Voyager CD-ROMs were never financially successful 8 , copyright restrictions prevented the widespread release of Murray's Casablanca edition, and Music Theater Online never found a large audience.
Over the last five years, though, several new technologies have emerged which have made the production and dissemination of multimodal editions much easier. The now widely adopted, video-and-audio-friendly HTML 5 specification and code libraries like Popcorn.js have made it much easier to embed and control audio and moving image recordings on web pages without relying on external plugins like Adobe's Flash or Microsoft's Shockwave. Cloud-based streaming services with public APIs such as Rdio9 and YouTube10 allow digital editors to make use of content hosted by third parties (making rights clearance less of a concern). Further, the ever accelerating migration of users from desktops and laptops to mobile devices and their app-based ecosystem makes it possible to publish these editions on hardware that has become a comfortable technology for long-form reading (thereby overcoming some of the ergonomic obstacles that prevented wide-spread adoption of many earlier digital editions).
3. Our projects

Fig. 1: Two video synchronized and shown together

Fig. 2: The editing environment
In October of 2013, the New York Public Library for the Performing Arts released a collection of over 1000 hours of video of dance performance via a new web-based collections portal. In addition, we released a web-based video editor, based on the JavaScript library Popcorn.js, which allows users to synchronize different videos of the same event (or work) together to be watched side-by-side (thereby creating a kind of multimodal variorum edition) [see figure A]. The tool also allows segments of multiple videos from various sources (e.g. NYPL Archives and YouTube) to be edited together into a playlist and annotated (either with text or additional videos) to create a video critical edition [see figure B]. The individual elements of any "edition" created with this tool will only be played in locations where rights agreements allow (for instance, in one of the 88 branches of New York Public Library), but any publicly viewable content and textual annotations can be seen anywhere in the world.

Fig. 3: Libretto: The Android App
In February of 2014, the Library will also release an NEH-funded eBook app (provisionally titled Libretto) for Android operating systems capable of presenting variorum editions of texts linked to audio recordings. Readers will be able to switch among variant versions of the same text, and (through an implementation of the new ePub 3.0 ebook standard) click portions of the text to hear associated audio (e.g. music associated with lyrics in an opera libretto) [see figure C]. In most cases, the music should be bundled with the text, but, in cases where obtaining a license to redistribute the music proves impossible, the editor may link the reader to an online store (such as Amazon.com) to purchase recordings which will automatically be synched to the text based on timestamps included in the editorial metadata.
4. Analysis

It is our hypothesis that scholarly users of the multimodal editions created for these environments will find the multichannel approach we have taken useful. However, relatively little research has been done in exactly this context. The Voyager company's CD-ROMS and the Casablanca edition were similar, but designed only for desktops and optical media, and so represented a different experience than we hope to provide with both our mobile eBook reader and the web-based video editor. However, by July, the video editing tool will have be public for nine months and Android app will have been public for about five. During this period we will track usage of both, and in this short paper will report on the response of both editors and “readers” of these new tools for multimodal editions and what it suggests for the scholarly editors of the future.
References

1. IT Services, 13 Banbury Road. The University of Oxford Text Archive. Text. ota.ahds.ac.uk/Web. 28 Oct. 2013.
2. About The Etext Center | University of Virginia Library Digital Curation Services.www.digitalcurationservices.org/digital-stewardship-services/etext Web. 28 Oct. 2013.
3. TEI guidelines. www.tei-c.org/Guidelines/P5/.
4. The William Blake Archive Homepage.www.blakearchive.org/blake/ Web. 28 Oct. 2013.
5. Brown, Geoffrey. Developing Virtual CD-ROM Collections: The Voyager Company Publications. International Journal of Digital Curation 7.2 (2012): 3–20. www.ijdc.net. Web. 28 Oct. 2013.
6. Here’s Looking at Casablanca. National Endowment for the Humanities. Accessed October 28, 2013. www.neh.gov/humanities/2005/septemberoctober/feature/here%E2%80%99s-looking-casablanca.
7. Music Theatre Online.mith.umd.edu/mto Web. 28 Oct. 2013.
8. Reid, Calvin. Voyager Shakeup: In Wake of Stein’s Exit, It’s up for Grabs. Publishers Weekly, November 18, 1996. General OneFile.
9. Rdio - Welcome to the Rdio API. Accessed October 28, 2013. developer.rdio.com.
10. YouTube JavaScript Player API Reference - YouTube — Google Developers. Accessed October 28, 2013. developers.google.com/youtube/js_api_reference.