Sublime Complexities and Extensive Possibilities: Strategies for Building an Academic Virtual Reality System

Dynamic encounters with three-dimensional assets make virtual reality (VR) an attractive medium for innovations in teaching and research. One of VR’s most compelling advantages is an immersive experience: researchers and the public are no longer separated from digital artifacts; instead, they share a space with them. Such experiences engage the whole body. They provide a 360° event, eliminating the restrictions of computer screens. For the classroom and individual researchers, cost has previously hindered widespread adoption of VR in classrooms and by individual researchers. Fortunately, prices have fallen as technical solutions have evolved. Another obstacle, perhaps more complex and profound, is building useful academic VR systems. VR platforms are not equally capable; different systems tend to facilitate radically different actions and engagements. Tools, features, and types of engagements need identified, imagined, and built. Our presentation explores approaches to these system-design challenges while building the Oklahoma Virtual Academic Laboratory (OVAL), a scholar-oriented, human-centered VR system.

OVAL is the first generation of general-purpose, multiuser academic VR systems, and is free to download and use. Our first speaker, Matt Cook, explains the importance of collaborators across campus and institutions when designing and testing VR systems like OVAL. External to the University of Oklahoma, OVAL has hosted a range of multi-campus virtual tours, including early archaic caves in Arizona, Syrian ruins at Palmyra, and 3D-scanned (live) sea turtles from around the world. Internally, OVAL has been used across a range of academic disciplines to test the pedagogical impact of immersive visualization. Each of these unique implementations represents an iterative hardware and software design process, ultimately providing today’s students the means to quickly engage with complex 3D data in a way that preserves embodied interfacing and naturalistic visual depth-cues. Matt discusses the motivations, hardware & software considerations, usability testing, and documented pedagogical impact of OVAL across a range of associated academic disciplines.

Importantly, specific disciplinary needs preclude a definitive version of the OVAL software and make designing a VR system an intricate puzzle. Different disciplines require different types of engagements and functionality, and some applications demand stringent preservation practices (e.g. Law School). For architecture, as part of the creative process, student needs to move
through
the structures they design, the experience revealing previously overlooked design flaws associated with accessibility, layout, and scale. For biology, students studying various protein molecules need to analyze VR content from the outside, turning and magnifying it, examining its features, and recording their analysis as a distributable video output. Beyond presenting the history of this open access VR system, then, Matt discusses how sorting through these divergent disciplinary considerations results in a more robust VR system with numerous academic applications. He also discusses human subjects testing and its need to refine and validate design features, as well as metrics and instrumentation useful to understand and document the value of VR in the classroom.

Our second speaker, Bill Endres, from the English department, specializes in medieval manuscripts. With funding from the OU Humanities Forum, Bill built a 2-person travelling VR workstation. He has loaded 3D models of the eighth-century St Chad Gospels into it, including multispectral and post-processed renderings for recovered content. For comparing artistic techniques, he includes metal work, Pictish stone carvings, and related pages from other illuminated manuscripts. Bill discusses how the VR workstation allows him to engage experts and the public to test features and imagine new possibilities for VR. During the presentation, Bill will quickly demonstrate the traveling VR workstation and make it available throughout the conference. Before DH 2019, Bill will have presented the workstation at the Medieval Academy’s Annual Meeting, International Congress on Medieval Studies, and at the University of Glasgow.

Medieval studies has numerous artifacts that benefits from encounters in VR, including manuscripts,
mappae mundi
, swords, tapestries, cathedrals, and stone crosses. Bill discusses the simple benefit of having VR’s 360° field of visual. For example,
mappae mundi
are quite large. On a computer screen, it is impossible to explore details without losing track of the whole. VR’s 360° visual field eliminates this problem.

Because such possibilities regularly hinge on the human element rather than the technical, Bill discusses the complexities of human sensory experience. Human sensory experience is anything but simple. Rather than the popular notion of the Aristotelian five senses, neuroscientists have determined that humans have twenty-two to thirty-three senses, depending on how specific each sense is defined. For example, the sense of proprioception allows someone to sense where the parts of their body are, necessary input for performing simple tasks. It is the sense evaluated in the common sobriety test: with eyes closed, touch the tip of your nose. Proprioception provides significant information when encountering physical objects, such as a sense of size, as it relates to the human body. Size offers clues about artifacts like a manuscript, signifying whether it was meant for display, study, or travel. When examining high-resolution photographs on a computer, proprioception provides information about the body in relations to mouse and screen, not to the manuscript.

But human sensory experience has a further complexity: it is constructed. For example, when sitting on an airplane, if you glimpse the walkway being withdrawn, you might experience the airplane backing away. In reality, the plane is still. The constuctedness of sensory experience allows for two possibilities. One, to translate sensory experience into VR, ratios and ratios of different sensory data can be used. For example, cold makes an object feel more rigid. This reflects learned experiences: cold generally makes an object stiffen. Therefore, to reproduce the experience of stiffness when turning a page of parchment, rather than manipulating pressure, manipulating temperature might prove more advantageous. Two, sensory experience can be generated in alternative ways. Researchers at the University of Bristol have invented a device that uses ultrasound to generate geometric shapes mid-air, which can be felt. Ultrasound shows promise for providing haptic experiences of parchment, such as feeling its contours and layered pigments. Bill discusses work with Julie Williamson at the University of Glasgow on this possibility.

Our final speaker, John Grime, provides further disciplinary range to the team. John has a PhD in physics and chemistry. His background includes developing algorithms to explore biomedical phenomenon. John develops the platform architecture for OVAL, and he discusses some of the technical underpinnings for building different multidisciplinary tools from the same base code of the program. But thinking across disciplines is likewise highly productive. For example, in OVAL we are designing a feature that allows someone to experience the sky and its effects on lighting at Neolithic sites; such control over lighting is also important for an advanced imaging technique called reflectance transformation imaging (RTI). RTI software generates a single file from a series of photographs taken with different directional lighting. The file allows control over lighting to reveal surface details. Whether for a Neolithic site or RTI, code to control lighting for one contributes to code for the other.

Furthermore, digitally grounded sensory experiences are not limited by the rules of sensory input in the physical world. John discusses one of the latest features in OVAL: “Janus,” which dynamically warps 360° of visual content to fit within the field-of-view of VR headsets. This allows users to analyze their environment or collection of artifacts as if their vision extends behind their heads.

From its earliest days, the digital humanities (humanities computing) have been collaborative. Nowhere is this more evident or important than designing VR. Sorting out how humans construct experience and how to reconstruct it in dynamic ways in VR is complex and challenging. However, as with all things complex and challenging, there are extensive rewards. VR provides unprecedented ways to engage in research and teaching by presenting content in a 360° environment that enables body-centered interactions and expanded representational characteristics for objects. It teaches us about ourselves, our artifacts, and our world. As with other research in the digital humanities, designing and building VR invites us to understand human experience more fully.