Supercomputing in the Humanities: A Robust Model for Interacting with Copyright-Sensitive Multimedia Content

Introduction
Music Information Retrieval (MIR) and Music Digital Library (MDL) are inter-related multidisciplinary research endeavors that strive to develop innovative content-based searching schemes, novel interfaces, and evolving networked delivery mechanisms in an effort to make the world’s vast store of music accessible to all. Some teams are developing “Query-by-Singing” systems (e.g., [1,2]), some “Query-by-Note” systems (e.g., [3,4]), some “Query-by-Example” systems (e.g., [5,6]), some comprehensive music recommendation and distribution systems (e.g., [7,8]), some musical analysis systems (e.g., [9,10]), and so on. Good overviews of MIR/MDL’s interdisciplinary research areas can be found in [6,11,12].
This poster outlines a tera-scale supercomputing resource model that promises to have significant implications for the future of humanities computing. It specifically details the framework for the "International Music Information Retrieval System Evaluation Laboratory" (IMIRSEL) being constructed at the University of Illinois at Urbana-Champaign. IMIRSEL is an integral sub-component of the author's "Music Information Retrieval / Music Digital Library Evaluation Project." The "Evaluation Project" has an intial four-year time-span (1 October 2003 to 30 September 2007) and is being supported via significant funding from both the Andrew W. Mellon Foundation and the National Science Foundation. The project is a co-operative undertaking involving researchers from the Graduate School of Library and Information Science, the National Center for Supercomputing Applications (NCSA), the Faculty of Music, the Music Library, Electrical Engineering and the Computer Science department of UIUC. One central goal the of the project is the creation and refinement of secure access mechanisms that will allow the manipulation of a unique, large-scale standard corpus of sensitive (i.e., copyright issues) multimodal (e.g., text, symbolic, audio and metadata) music materials for the research and evaluation use of the international MIR/MDL and humanities computing research communities.
Current Problem
The MIR/MDL communities have long recognized the need for a scientific evaluation paradigm. A formal resolution expressing this need was passed, 16 October 2001 by the attendees of the Second International Symposium on Music Information Retrieval (ISMIR 2001). (See http://music-ir.org/mirbib2/resolution for the list of signatories.) The resolution highlights the fact that MIR/MDL research requires access to music materials consisting of inter-connected collections of symbolic (e.g., scores, MIDI, MusicXML, etc.), textual (e.g., lyrics, libretti, reviews, analyses, etc.), audio (e.g., MP3, WAV, etc.) and metadata (e.g., bibligraphic records, etc.) information. However, 1) music is expensive; 2) music rights-holders are notoriously litigious; and 3) recent changes to copyright law in the United States have put into question the very existence of “public domain” sources of audio recordings (see [6]). These three facts, when taken together, have effectively stopped the development of any large-scale, community-accessible, test collections comprising the necessary audio, symbolic and metadata representations. Notwithstanding that some private research institutions have acquired substantial collections for their in-house use only, such collection holders do not make them accessible to others in the community for fear of becoming the objects of expensive civil and criminal litigation.
Thus, the key to developing a useful and robust evaluation testbed comprising the necessary depth and breadth of content hinges on addressing the sensitive nature of music materials (i.e., the copyright issue). Furthermore, if one can overcome the very real hesitations of rights-holders (i.e., the music companies) by demonstrating to them that their property is secure and being used only for research purposes (which ultimately might also benefit them), it follows that this project and other humanities computing projects like it would be in a better position to extend the depth and breadth our test collections to include other copyright sensitive materials (i.e., videos, films, rare manuscipts, multiple versions of publications, etc.). The implications are profound as we would then be able to fully explore the important interconnections between all forms of cultural expression amenable to computational analyses.
The IMIRSEL Solution
We have begun construction of the world’s first-and-only, internationally-accessible, large-scale MIR/MDL testing and development database. This multiformat collection is housed at the UIUC's NCSA. Formal transfer and use agreements are being finalized with HNH Hong Kong International, Ltd. (http://www.naxos.com), the owner of the Naxos and Marco Polo recording labels. This will afford the MIR community research access to HNH’s entire catalogue of Classical, Jazz, and Asian digital recordings. This generous gesture on the part of HNH represents approximately 30,000 audio tracks or about three terabytes of digital audio music information. All Media Guide (http://www.allmusic.com) has also agreed to follow HNH’s lead, enabling UIUC/NCSA to incorporate its vast database of music metadata within the same test collection. All Media’s dataset includes descriptive catalogue records, discographies, and recording classifications. We are also exploring a variety of options concerning the acquistion/creation of related symbolic music data but, as of the time of writing, no formal decisions have be reached.
Overiew of the IMIRSEL Frameworl
Figure 1. Overiew of the IMIRSEL system framework.
Given the unique opportunity that these rights-holders have afforded the MIR/MDL communities, it is important that our testing and evaluation database be constructed with three central features in mind:
security for the property of the rights-holders, which is especially important if we are to convince other rights-holders to participate in the future;
accessibility for both internal, domestic, and international researchers; and,
sufficient computing and storage infrastructure to support the computationally- and data-intensive techniques being investigated by the various research teams.
To these ends, we are exploiting the expertise and resources of NCSA and its Automated Learning Group (ALG), headed by Prof. Michael Welge. NCSA's systems have been designed to be secure. Certificate-based authentication for all users as well as means for encrypting data and data transfers are fundamental to NCSA’s security protocols. The ALG has developed a data-to-knowledge system, D2K, which supports all phases of the data mining process. D2K was originally designed to provide data mining professionals with a flexible “sandbox” for developing and evaluating the performance of a range of supercomputing techniques on a variety of data sets. Using the D2K technology as a starting point, we are creating a secure “Virtual Research Lab” (VRL) for each participating research team. These VRLs will provide secure access to the test collection and the resources necessary to conduct large-scale MIR evaluation experiments. Simply put, we enhance the security of the valuable music data by bringing the research teams to the collection, rather than distributing the collection willy-nilly around the globe.
To make interaction with the underlying supercomputing resources simpler (i.e., ultimately invisible) for the international, domestic and internal research teams, we are incorporating another ALG application, D2K-SL. D2K-SL builds upon current D2K modules to provide a set of pre-defined applications that guide users through the supercomputing process. These tools will be instrumental in supporting the multidisciplinary nature of MIR/MDL research. In addition, we hope that these D2K-SL applications can be used to address other related research thrusts, such as new MIR/MDL techniques, new interface designs and the development of protocols to make the proposed MIR/MDL GRID a reality [13].
Summary and Future Work
If humanities computing is going to realize its full potential in the 21st century, it must overcome the copyright issues that are impeding it from expanding it analyses and praxis across and between all forms of cultural expression. MIR/MDL and humanities computing research require largescale respositories of multimodal materials that can be held in common among all interested parties. These materials must be accessible to all legitimate research teams in order to facilitate a scientifically valid basis for comparing and contrasting the efficacy of the myriad techniques and approaches being explored. The IMIRSEL system being developed at UIUC can be seen as a prototype for future humanities test collections. By incorporating a amalgem of state-of-the-art supercomputing computational, storage and security techniques, the IMIRSEL test database is designed to create a flexible and accessible--yet secure--research enivornment that we hope will inspire other researchers and content-holders to participate in robust, multimodal and largescale humanities supercomputing research projects that incorporate both public domain and copyright-sensitive cultural content.
Acknowledgements
Drs. Don Waters and Suzanne Lodato, both of the Andrew W. Mellon Foundation, are thanked for their moral and financial support. Karen Medina, Joe Futrelle, William Birmingham and Mike Welge are also thanked for their valuable contributions and suggestions throughout the project. This project is also support by the National Science Foundation (NSF) under Grant Nos. NSF IIS-0340597 and NSF IIS-0327371.
References
[1] Haus, G. and Pollastri, E., "An Audio Front End for Query-by-Humming Systems." Second International Symposium on Music Information Retrieval, Bloomington, IN, USA, pp. 65-72, 2001.
[2] Birmingham, W., Dannenberg, R. B., Wakefield, G. H., Bartsch, M., Bykowski, D., Mazzoni, D., Meek, C., Mellody, M., and Rand, W., "MUSART: Music Retrieval Via Aural Queries." Second International Symposium on Music Information Retrieval, Bloomington, IN, USA, pp. 73-81, 2001.
[3] Doraisamy, S. and Rüger, S. M., "A Comparative and Fault-tolerance Study of the Use of N-grams with Polyphonic Music." Third International Conference on Music Information Retrieval, Paris, France, pp. 101-106, 2002.
[4] Pickens, J., "A Comparison of Language Modeling and Probabilistic Text Information Retrieval Approaches to Monophonic Music Retrieval." International Symposium on Music Information Retrieval, Plymouth, MA, USA, 2000.
[5] Haitsma, J. and Kalker, T., "A Highly Robust Audio Fingerprinting System." Third International Conference on Music Information Retrieval, Paris, France, pp. 107-115, 2002.
[6] Downie, J. S., "Music Information Retrieval." Annual Review of Information Science and Technology, vol. 37, pp. 295-340, 2003.
[7] Pauws, S. and Eggen, B., "PATS: Realization and User Evaluation of an Automatic Playlist Generator." Third International Conference on Music Information Retrieval, Paris, France, pp. 222-230, 2002.
[8] Logan, B., "Content-Based Playlist Generation: Exploratory Experiments." Third International Conference on Music Information Retrieval, Paris, France, pp. 295-296, 2002.
[9] Kornstädt, A., "The JRing System for Computer-Assisted Musicological Analysis." Second International Symposium on Music Information Retrieval, Bloomington, IN, USA, pp. 93-98, 2001.
[10] Barthélemy, J. and Bonardi, A., "Figured Bass and Tonality Recognition." Second International Symposium on Music Information Retrieval, Bloomington, IN, USA, pp. 129-136, 2001.
[11] Byrd, D. and Crawford, T. C., "Problems of Music Information Retrieval in the Real World." Information Processing and Management, vol. 38, pp. 249-272, 2002.
[12] Futrelle, J. and Downie, J. S., "Interdisciplinary Communities and Research Issues in Music Information Retrieval." Third International Conference on Music Information Retrieval, Paris, France, pp. 215-221, 2002.
[13] Dovey, M. J., "Music GRID: A Collaborative Virtual Organization for Music Information Retrieval Collaboration and Evaluation." The MIR/MDL Evaluation Project White Paper Collection, Champaign, IL: GSLIS, pp. 50-52, 2002.