CloudPad - A Cloud-based Documentation and Archiving Tool for Mixed Reality Artworks

CloudPad – A Cloud-based Documentation and Archiving Tool for Mixed Reality Artworks
Giannachi, Gabriella, Centre for Intermedia, University of Exeter, g.giannachi@exeter.ac.uk
Lowood, Henry, Stanford Libraries, Stanford University, lowood@stanford.edu
Rowland, Duncan, Mixed Reality Lab, University of Nottingham, dar@cs.nott.ac.uk
Benford, Steve, Mixed Reality Lab, University of Nottingham, sdb@cs.nott.ac.uk
Price, Dominic, Mixed Reality Lab, University of Nottingham, djp@cs.nott.ac.uk
This paper reflects on the process of designing and building a documentation and archiving tool named CloudPad on the basis of its first evaluation at Stanford Libraries and the San Francisco Art Institute in September 2010. The paper explores the value of CloudPad and its ability to document individual users’ replay of an artwork within the context of performance documentation and new media archiving, speculating on its possible use within a number of curatorial, educational and creative contexts that are relevant to digital humanities.

The CloudPad was developed in 2010 by a team in Horizon RCUK-funded digital economy research and involved staff in performance studies and computer science from the Universities of Exeter and Nottingham, with partners from Stanford Libraries, the Ludwig Boltzman Institute Media.Art.Research, The San Francisco Art Institute, British Library, Blast Theory, and the University of Sheffield. The work developed out of the team’s intention to research novel theoretical and practical approaches for the documentation and archiving of mixed reality performances and artworks that span both digital and physical entities (Benford and Giannachi 2011), allowing users to engage with the materials creatively over time and from different locations. The project benefitted from previous research conducted by members of the team through the AHRC-funded Presence project (2004-9), which used second life and a wiki to document practices spanning from performance art, to video art and new media, including work in virtual reality CAVE, and the EPSRC-funded Creator project (2008-9) which used an e-science tool, the digital replay system, to generate synchronised annotations about a mixed reality performance (DRS). The project also benefitted from the findings of the e-dance project (2007- 9), which was jointly funded by AHRC, JISC and EPSRC, and conducted by colleagues from the Universities of Bedfordshire, Leeds, Manchester and Open University. This adopted access grid technologies for developing new approaches to choreographic composition, involving the use of the Memetic toolkit for recording, replaying and annotating sessions in access grid. Finally, the project was developed in dialogue with artworks such as Lynn Hershman Leeson’s RAW/WAR feminist film archives (2010), sosolimited’s interactive archival performances, and current thinking in new media documentation (e.g. Costello 2005, Depocas et al 2003, Jones and Muller 2008 and Dekker 2011, among others).

Technically the CloudPad was designed as a customisable web-based platform aiming to facilitate the synchronised playback and mash-up of cloud-based media entities such as video or audio files, as well as webpages and photographic materials, together with layers of user annotations. It took a novel approach to the archiving and replay of pervasive media experiences by making use of Web 2.0 technologies (DiNucci 1999) rather than grid technologies. CloudPad users were empowered to view the repository as a living document in which they could leave their own impression of an experience (both of the original event recordings as well as any thematic connections or annotations provided by other visitors and subject experts). Previous interactive systems designed for the replay of events for analysis lack this level of emergent reflection (see Brundell 2008), treating the corpus of recorded material as essentially immutable. To enable this, the CloudPad made use of internet-based storage, which means that media from a wide variety of different sources could be included in a presentation (for example YouTube videos can be included and synchronised with images from Flickr). This was accomplished by the use of HTML5 (see w3.org), an emerging web standard that enables collaborative interactive applications to be developed which run inside a web browser (Murray 2005).

As an initial form of content to assess the operation of the CloudPad we utilised a ‘bespoke’ documentation of Blast Theory’s Rider Spoke that was recorded by our team when the work occurred at the ars electronica festival in Linz in 2009. Rider Spoke is a location-based game for cyclists developed by Blast Theory in collaboration with Mixed Reality Laboratory at the University of Nottingham as part of the European research project IPerG. The work encouraged participants to cycle around a city in order to record personal memories and make statements about their past, present and future that were associated with particular locations (see figure 1).

Blast Theory, Rider Spoke. Participant listening to recordings. Copyright Blast Theory.

Full Size Image

To collect a documentation that addressed the complexity of this work, we developed a hybrid approach. This included the collection of documentations pertaining to the artists and technologists’ descriptions of the works (in terms of original aims, interim analyses and final evaluations), as well as documentations of the user experience (see Jones and Muller 2008 and Depocas et al 2003), the latter recorded from a variety of points of view (e.g. first person, third person) and through a number of technologies (e.g., video, GPS, Wi-Fi) and perspectives (see figures 2, 3 and 4).
Linz documentation. Participant captured via first person point of view.

Full Size Image

Linz documentation. Participant captured via third person point of view.

Full Size Image

Linz documentation. Participants journey through the city captured on googlemaps.

Full Size Image

The overall analytical approach was interdisciplinary, thus including different and potentially even contrasting accounts of the event (see Chamberlain et al 2010). These accounts were presented through a number of historic, canonic and participant ‘trajectories’ (see figure 5).
Matt Adams’ annotation about a participant linking first and third person perspectives in a canonic trajectory.

Full Size Image

By historic trajectories we defined a historic event, i.e. a participant’s experience as documented in a video; by canonic trajectories we defined an expert user’s set of annotations through these materials; and by participant trajectories we defined the CloudPad user’s own annotations (Giannachi et al 2010). This architecture does not privilege a single viewpoint and encourages creative use of both the historic materials and their canonic annotations. Arguably, every replay, producing participant trajectories, re-constitutes the work.
The CloudPad evaluation showed that users did not only envisage adopting the CloudPad for purposes of documentation and archiving, but also wanted to use it curatorially, to present work to others and engage users in annotating materials, for example in an online exhibition, academically, to write ‘visual essays’, and creatively, to make artwork. We have seen that the CloudPad offers scholars, artists and students the possibility to document, archive, curate and create synchronised variable media mash-ups from existing digital resources. These mash-ups, which show how users have engaged with the original documentation stored on CloudPad, build an invaluable resource for those who may be interested in how a core documentation or archive is navigated and interpreted over time. In other words, the CloudPad is not only a documentation and archiving tool, it also documents and archives itself, generating contextual footprints or traces and possibly even re-enactments of every replay of the original materials. This paper reflects on the advances generated by this particular functionality in terms of performance documentation, preservation, and re-enactment.

Acknowledgements
We gracefully acknowledge the RCUK funded Horizon digital economy research and the AHRC funded Riders Have Spoken project. We would like to thank Blast Theory and staff at the Ludwig Boltzman Institute Media.Art.Research, Katja Kwastek, Dieter Daniels and Ingrid Spoerl in particular, who facilitated the documentation of Rider Spoke in Linz, and our participants and volunteers who gave their time to make this documentation possible. We would also like to thank the staff and students at the San Francisco Art Institute, Stanford Libraries and St Jose State University for providing crucial feedback that informed the writing of this paper.

References:
Benford, S. and Giannachi, G. 2011 Performing Mixed Reality, the MIT Press Cambridge, Mass.

Brundell, P., Tennent, P., Greenhalgh, C., Knight, D., Crabtree, A., O’Malley, C., Ainsworth, S., Clarke, D., Carter, R. and Adolphs, S. 2008 “Digital Replay System (DRS) - a tool for interaction analysis, ” Proceedings of the 2008 International Conference on Learning Sciences, Utrecht: ICSL, June 23-24, 2008

Chamberlain, A., Rowland, D., Foster, J., Giannachi, G. 2010 “Riders Have Spoken: Replaying and Archiving Pervasive Performances, ” Leonardo Transactions, 43.1 90-1

Costello, B., Muller, L., Amitani, S., and Edmonds, E. 2005 “Understanding the Experience of Interactive Art: Iamascope in beta_space, ” ACM 2005, vol. 123 49-56

Dekker, Cosetta Saba, Julia Noordegraaf, Barbara Le Maître and Vinzenz Hediger (eds.) forthcoming 2011 Preserving and Exhibiting Media Art: Challenges and Perspectives, University Press Amsterdam

Depocas, A., Ippolito, J., Jones, C. 2003 Permanence through Change: The Variable Media Approach, Guggenheim Museum Publications New York

DiNucci, D. 1999 History Fragmented Future Recovered, October 2010 (link)

Giannachi, G., Rowland, D., Benford, S., Price, D. 2010 “The Documentation and Archiving of Mixed Media Experiences: the Case of Rider Spoke, ” Digital Futures, Nottingham, 11-12 October 2010

Horizon, October 2010 (link)

IPerG, October 2010 (link)

Jones, J. and Muller, L. 2008 “Between Real and Ideal: Documenting Media Art, ” Leonardo, 41.4 418-41

Murray, G. 2005 Asynchronous JavaScript Technology and XML (Ajax) With the Java Platform, October 2010 (link)