Virtual Touch. Towards an Interdisciplinary Research Agenda for the Arts and Humanities

Virtual Touch. Towards an Interdisciplinary Research Agenda for the Arts and Humanities
Bentkowska-Kafel, Anna, Centre for Computing in the Humanities, King’s College, London, UK, anna.bentkowska@kcl.ac.uk
Giachritsis, Christos, SyMoN lab, University of Birmingham, UK, c.giachritsis@bham.ac.uk
Prytherch, David, User-lab, Birmingham Institute of Art and Design, Birmingham City University, UK, david.prytherch@bcu.ac.uk
The term 'haptics' encompasses two areas of study: human and machine haptics. The first relates to the study of the perception of the world through the sense of touch. It includes proprioception (one’s awareness of one’s own body position in space) as well as cutaneous information (one’s awareness of skin deformations). Machine haptics relates to the design and development of devices that simulate the haptic properties of physical objects. In principle, they are incorporated in virtual environments and allow users to experience tactile properties of virtual objects such as size, shape, weight, compliance and texture.

Virtual Artefact. A Different Approach
The virtual artefact has firmly established itself as a research tool within several disciplines of the Arts and Humanities. Many art and material culture historians and professionals rely on digital records and visualisations of artefacts in their research, teaching and practice. We have witnessed, from the 1990s onwards, how the virtual artefact has increasingly become photo-realistic and interactive, and how it continues to evolve. The virtual artefact can now be part of a complex, collaborative research environment. With the enhanced technical specifications comes the interest in exploring the research potential of virtual artefacts further. We are here concerned with enhanced simulation of the real experience of physical objects through the application of haptic interfaces, or virtual touch technologies. We believe that the addition of virtual touch would also contribute to greater usability of the existing, often neglected electronic resources and libraries of 3D artefacts.

Research into the use of haptic interfaces–that is devices engaging the sense of touch in virtual environments–in the Arts and Humanities is in its infancy. Although virtual simulation of physical touch has resulted in important advances in other disciplines–such as medicine, neuro-science, telemanipulation control systems and product design–the potential of such applications to humanities scholarship has not yet been explored. Very few researchers in the Arts and Humanities have had an opportunity to experience haptic devices first hand and to develop a critical understanding of such systems and the perceptual processes involved. There have been some important and promising developments in the area of heritage science, such as the Haptic Museum in the US, being the work of Margaret McLaughlin et al. (2000) at the University of Southern California, Annenberg and the Los Angeles County Museum of Natural History; the Museum of Pure Form (Bergamasco et al., 2005) and ‘Touching the Untouchable: Increasing Access to Archaeological Artefacts by Virtual Handling’ in the UK, supported by the Arts and Humanities Research Council and Engineering and Physical Sciences Research Council.

The authors are investigating certain fundamental questions:

Is virtual touch likely to enhance the ways in which we carry out and communicate research in those areas of the Arts and Humanities that employ 3D visualisations of material culture?
How much do we learn about artefacts by touching them?
Would simulating this experience through a haptic interface enhance virtual fieldwork?
Who can possibly benefit from this experience and how?
If ‘seeing with vision that feels, feeling with fingers that see’ (Goethe, 1788) is possible, to what extent can this experience be mediated by a haptic system?
To Touch or not to Touch?
A notice ‘DO NOT TOUCH’ is familiar in many museums and heritage sites and the reasons why this measure is necessary are commonly understood. Most museum managers and curators embrace the notion that touching object collections are important. However, many of the most tactually alluring artefacts present conservation problems that are difficult to overcome. Yet, the existence of ‘DO NOT TOUCH’ simply demonstrates the visitor’s compelling ‘NEED TO TOUCH’ the artefact. This need originates from the fact that the sense of touch can provide us with haptic-specific information about an object such as absolute size, material compliance, texture, temperature and weight, which cannot be reliably obtained through the audio-visual senses. Therefore, a solution which would allow visitors, curators and researchers in Arts & Humanities to experience these tactile properties without compromising the integrity of the artefact is likely to be very welcome and highly valuable.

The latest haptic technologies can provide a way forward which satisfies both the visitors’ and researcher’s desire to handle and the museums’ necessity to conserve collections for future generations. An early 'proof of concept' pilot study was conducted with randomly arriving museum visitors, which looked at the potential for the substitution of real object handling with touching virtual 3D replicas via a low resolution, low cost, commercially available haptic interface (PhanTom Omni) coupled with a stereoscopic (3D) visual display (Fig. 1).

Fig. 1 'Feeling' a virtual replica of an object from the Potteries Museum, Stoke-on-Trent, UK, using the portable haptic system based on a PhanTom Omni, SenseGraphics 3D Mobile Immersive Workbench. Photo: D. Prytherch

Full Size Image

The visitor feedback offered interesting insights into the potential of the haptic display of virtual objects to enhance the experience of museum visitors by allowing them to interact with virtual artefacts through touch. An informal trial with a blind member of museum staff demonstrated that he was able to identify the shape of the object and gain some insight into the surface carving and texture of the object, despite the real artefact being untouchable. The same portable haptic system was used in the classroom when teaching a masters module in Digital Arts and Culture and received valuable feedback from the students (Fig. 2).
Fig. 2 Haptics in the classroom. A postgraduate course in Digital Arts and Culture, Centre for Computing in the Humanities, King's College London, UK, 24 November 2010. Photo: A. Bentkowska-Kafel. Insert: Laser scanning of the Bride (Potteries Museum, Stoke-on-Trent, UK) to create a virtual 3D replica. Photo: D. Prytherch.

Full Size Image

Archaeologists, art historians, restorers, palaeographers and other specialists who examine artefacts through touch to assess and authenticate their material, execution and other tactile qualities, might also be interested in haptic access to material ex situ when direct access is not possible. The capacity of human haptic perception–the ability to perceive the world through the sense of touch–to differentiate qualities of material, is incredibly rich and we learn much about the form and composition of an artefact from our sensitivity to thermal conduction and fine surface textural qualities through touch. Current commercial haptic devices do not make effective use of tactile (or cutaneous) cues, that is the physical properties of objects which are perceived through skin mechanoreceptors. Technologies that do operate at this level are either experimental, lab-based systems or, if commercially available, are extremely expensive, as well as being complex to setup and maintain. Certain limitations inherent in the whole concept of virtual representations of objects, notably weight and balance, which are dependant not on the virtual object itself, but on the properties of the specific device with which we feel it, need to be resolved.

The virtual artefact has opened up unprecedented possibilities for new research into the material culture of the past. Digital technology supporting visualisation of heritage is becoming ever more sophisticated and many projects of this kind seek to better the accompanying scholarly apparatus. The publication of the London Charter for the Computer-based Visualisation of Cultural Heritage (version 2.1, February 2009), is a step in this direction. Some scholars working in this area claim that their visualisations of ancient sites re-create the real human experience of ‘being there’. Some believe that digital visualisation itself may ‘include sight, hearing, and potentially in the future, smell, taste, and touch’ (Mudge, 2011). The lack of tactile experience is one of the significant issues inherent to digital visualisation of heritage. In addition, our limited knowledge of the real-life processes that we are trying to simulate through the use of digital media is always a challenge. We need to better understand the perceived discrepancies between the real object and its virtual record.

'Feeling with a Seeing Hand' (Goethe, 1788)
In the process of gaining knowledge about the real world, vision allows us to explore the environment instantly and provides us with information about object properties such as 3D shape, colour, texture, condition, relative size, relative distance as well as events such as motion or changes of all these visual properties. Even when looking close-up is possible our knowledge is still incomplete. This is because vision alone does not allow us to perceive important physical properties of materials such as weight, centre of mass, temperature, surface roughness, hardness and actual size. In order to perceive these properties we need to engage touch. Touch allows us to perceive the properties which are essential for effective interaction with the physical world. By exploring an object through touch, we can form a better understanding of its purpose and use, and develop insights to inform us about its past users. For example, it is not possible to appreciate the physical endurance of an ancient warrior just by viewing a shield he might have used. For this, it is necessary to engage touch: lift and carry it.

Although still relatively primitive (comparing to advances in visual displays), current haptic interfaces (HI) allow a range of haptic and tactile interaction with virtual objects, from the exploration with one or multiple fingers with one or two hands (Giachritsis et al., 2009; Giachritsis et al., 2010; Monroy et al., 2008) (Fig. 3) to more immersive experience involving movements from arm joints.

Fig. 3 Bimanual manipulation of a virtual object using precision grip with the haptic interface device Master Finger 2.

Full Size Image

Haptic applications vary from simple touch of the virtual object to real-life probes used as medical or artistic tools for working on object surface. The extensive industrial and academic research in advancing HI shows the impact of these technologies on the way people learn, train, work and entertain themselves in virtual environments. Despite their limitations, HI technologies already offer significant advantages to users of virtual environments by allowing them a more realistic interaction with virtual objects.
Recent advances in recording art objects, made possible by such technologies as photogrammetry, touch probing and 3D structured light or colour laser scanning, have augmented the already considerable body of virtual artefacts, created primarily through Virtual Reality (VRML) modelling, with records of unprecedented accuracy. The E-Curator project carried out by the University College London Museum Collections and partners, used 3D colour scans of various artefacts captured with an Arius scanner, to develop a traceable, grid-based dissemination and visualisation system. This system enables museum curators and conservators to identify and assess objects remotely in a collaborative, networked environment. The addition of haptic interfaces engaging the sense of touch seems an obvious future development. The question is which particular aspects of human touch these devices should simulate so that they could be useful to heritage professionals?

Towards a Research Agenda for the Arts and Humanities
If the promise of more intuitive and multisensory computing–than the one with which we are familiar today–is real, we should investigate how these future developments in information technologies may affect our own disciplines. We argue for the need to explore the potential benefit of haptic interaction as an addition to audio-visual experiences of virtual artefacts. How much do we learn about artefacts by touching them? Would simulating this experience through a haptic interface enhance virtual fieldwork? Who can possibly benefit from this experience and how? If Goethe's (1788) ‘seeing with vision that feels, feeling with fingers that see’ is possible, to what extent can this experience be mediated by a haptic system? We will not be certain of the answers, nor have influence over the future of multisensory computing, unless we explore whether current developments and wide-ranging research meets our varied expectations.

The authors propose a research agenda in this area is made from the four positions: 1) digital scholarship of material culture; 2) the contribution of human vision and touch in the perception and appreciation of real and virtual environments; 3) haptic interface design and applications, and 4) haptic access to 3D records of art objects within networked collaborative environments.

The authors seek to investigate the level of interest of those engaged in historical and cultural studies in researching the potential benefits of adding virtual touch to virtual artefacts for the advancement of cultural heritage scholarship and education. Expressions of such interest and comments are welcome.

References:
Brewster, S.A. 2005 “The Impact of Haptic ‘Touching’ Technology on Cultural Applications, ” Digital Applications for Cultural Heritage Institutions, Hemsley, J., Cappellini, V., and Stanke, G. Ashgate Aldershot 273-284

E-Curator Project, University College London, 14 March 2011 (link)

Frisoli, A., Jansson, G., Bergamasco, M. and Loscos, C. 2005 “Evaluation of the Museum of Pure Form Displays Used for Exploration of Works of Art at Museums, ” Proc. World Haptics Conference 2005, Pisa, 18-20 March 2005 (link)

Giachritsis, C., Barrio, J., Ferre, M., Wing, A., and Ortego, J. 2009 “Evaluation of Weight Perception During Unimanual and Bimanual Manipulation of Virtual Objects, ” Third Joint Eurohaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, Salt Lake City, UT, 18-20 March 2009

Giachritsis, C.D., Garcia-Robledo, P. Jr., Barrio, J., Wing, A.M. and Ferre, M. 2010 “Unimanual, Bimanual and Bilateral Weight Perception of Virtual Objects in the Master Finger 2 Environment, ” 19th IEEE International Symposium on Robot and Human Interactive Communication, Principe di Piemonte - Viareggio, Italy, 12-15 September 2010

Goethe, J.W. von 1988 Roman Elegies, VII [1788], Routledge London

London Charter for the Computer-based Visualisation of Cultural Heritage, 14 March 2011 (link)

McLAUGHLIN, M., L., Sukhatme, G., Shahabi, C., Medioni, G. and Jaskowiak, J. 2000 “The Haptic Museum, ” Proc. EVA Conference on Electronic Imaging and the Visual Arts, Florence, 2000

Mudge, M. forthcoming “Transparency for Empirical Data, ” Paradata and Transparency in Heritage Visualisation, A. Bentkowska-Kafel et al. eds Ashgate Aldershot (link)

Monroy, M., Oyarzabal, M., Ferre, M., Campos, A. and Barrio, J. 2008 “MasterFinger: Multi-finger Haptic Interface for Collaborative Environments, ” Proceedings of Eurohaptics ’08, Madrid, 10-13 June 2008 411-419

Prytherch, D. and Jefsioutine, M. 2007 “Touching Ghosts: Haptic technologies in museums, ” The Power of Touch: Handling Objects in Museum and Heritage Contexts, Pye, E. Left Coast Press Walnut Creek, CA 223-40

Touching the Untouchable: Increasing Access to Archaeological Artefacts by Virtual Handling, 14 March 2011 (link)