Misremembering Machines: A Creative Collaboration on Memory in AI-driven Storytelling

Context and collaboration
The Applying AI to Storytelling project was a high-risk research and development project at the intersection of the digital humanities, computer science, and the creative industries, funded by Innovate UK. The project embedded a university-based Research Software Engineer (RSE) from a Digital Humanities (DH) lab in the core development team of a start-up company pioneering interactive storytelling techniques, with a particular focus on character-based chatbot technologies powered by artificial intelligence. The project therefore sits at a crossroads between academic DH, literary-critical analysis, the creative industries, computer science, and software engineering. In doing so it represents the emergence of a new strand in DH practice that seeks to take lessons learned over decades of incubation within universities and cultural heritage organisations into the wider world. The work is also representative of a wider socio-cultural convergence of advanced technologies with creativity and academic research.
1

Research Software Engineering (RSE)  is growing rapidly in the United Kingdom, United States, and Australasia and is increasingly being used as an umbrella category that DH practitioners can contribute to
2. That is the case at our university, where DH teams work within a wider RSE community comprising specialists in bio-informatics, imaging science, and data analysis.
3 While RSE teams in the science and technology sector have long-established relationships with industry partners--creating a significant traffic in ideas, methods, and personnel across HE and industry boundaries-- DH RSE teams have until now focused mostly on developing relationships with the cultural heritage sector rather than industry.  Embedding a DH RSE within a creative industry SME as a connection point between industry and academia aims to facilitate a similar traffic in ideas and methods between the Creative Industries, RSEs and academics while at the same time demonstrating the value of DH to higher education and government sectors that are increasingly focused on the creation of tangible economic value.

DH methods are well suited to such an undertaking, providing an important element in both the creative and technical processes, and assisting with the translation of research in digital literary studies,
4 media history,
5 post-humanism,
6 and narrative design
7 to a commercial product that has captured the interest of companies  in the United States and Europe. The partner company involved in this project is developing a system that accelerates the creation of AI-driven storytelling,  recommending possible dialogue to the author using natural language processing and visually managing the complex branching structures these narratives require.  This system is a product that is currently in the beta stage and already licensed to external companies for the creation of interactive stories, as well as film and television productions.

Our collaboration extends established techniques in immersive storytelling and ‘extended reality’ (XR) into the commercial world of product design and development.
8 This kind of work is being encouraged by the UK government through a variety of funding mechanisms, as it aims to maximise the country’s natural advantages in cultural and creative production.
9 The partnership used the company’s core product to produce a ground-breaking translation of an immersive theatre piece into an AI-driven game written in the Unity engine.  The show relies on audience participation, assigning them tasks and roles and changing the direction of the narrative based on their choices. The adaptation places audience members with a cast of AI characters, using machine learning to inform their personalities, intent, and emotional interactions with the player.

The problem of memory
The key focus of the collaboration with the DH team is the ‘problem’ of memory that results from characters who ‘remember’ what the player has said and done.  This question is what brought the tech company CEO to our DH team in the first instance: they had reached a point where their technology had moved beyond their writers’ understanding of narrative. Their goal in partnering with a DH RSE was to better understand the literary-critical issues associated with digital characters who can ‘remember’ information given to them by readers, and improve the writing product they aim to sell into the creative industries. Perhaps surprisingly, given the long history of ‘network fiction’
10 that stretches back to the electronic literature movement of the early 1990s and beyond to ‘Multi-user Dungeons’ of the 1970s, they were struggling to find writers (even when collaborating with writers with experience designing video games and interactive fiction) comfortable with the radical level of emotional interaction support for memory creates. The persistence of memory in narrative has been noted as an affordance of even simple hypertext fiction,
11 but holds additional micro-level implications for interactions with players, and profound macro-level implications for the overall direction of a narrative, when ‘super-charged’ with AI.  AI characters in the game remember, misremember and outright lie, challenging the player’s decisions, and even their own memory of what has transpired in the game.

Maximising the potential for this to support high quality narrative is non-trivial, as is developing an elegant writing tool capable of supporting such new modes of imagination and reader interaction. In building this prototype the project faced a number of overlapping, multidisciplinary challenges.  The narrative authoring system had to be comprehensible to non-technical users -- especially writers -- to be used effectively, while at the same time retaining technical scalability and sustainability. The optimisation of the machine learning algorithms to create convincing character interactions required a critical examination of their output in both literary and social science terms, identifying how conscious and unconscious bias informs the author’s conception of the character and the player’s responses to it.
This convergence of technical, creative, academic and commercial factors make the project a natural fit for Research Software Engineers working within Digital Humanities.  Difficulties in structuring the dialogue progression within the UI, or in how the algorithm follows particular narrative paths and not others, are often technical, conceptual and cultural questions all at once.  These interrelated problems require not only the range of skills provided by a partnership between academia and the creative industries, but the high-risk, experimental nature of the work demands the strong connective tissue that Digital Humanities provides in mediating across scholarly, technical and business languages.  

Versatile and eclectic RSE
The unique nature of this collaboration, and evidence of the significant value DH skills and mindsets offer the commercial product development process, is demonstrated by the fact that the lead RSE is also a writer who has worked in immersive theatre.  This relatively rare set of skills enabled him to work with the show’s author on the adaptation; developing characters, designing narrative, and writing and editing dialogue. At the same time he provided granular, detailed feedback to the development team on the usability of the UI, and the challenges that both he and the writer encounter in structuring a shared immersive experience as a single player interactive game.  
Rather than being only a conduit for the academic and creative sides of the collaboration, therefore, the lead RSE was involved in both the creative and technical processes that created the final product.   As the project evolved, changing priorities and deadlines led the RSE to take on a more creative role, becoming the primary writer of a showcase demo for the immersive theatre project and developing the narrative structure for the adaptation of a novel into the system.  As a result of this change in responsibilities, the RSE identified issues as a writer that he then helped solve as a developer. As an example, in order to better test the narratives he was creating, he developed a tool to automatically run a set of player responses, effectively creating a unit test for the story.  He then used this tool as a writer to edit and refine the narrative paths inside the AI-driven system.
This model of collaboration created an intermediate space where DH methods allow seamless movement between the roles of developer and writer, often on the same question. This ability to not just translate between the creative spaces of game development and writing, but to inhabit both simultaneously, is essential to the success of creative collaboration, and more accurately reflects the current state of digital creativity in the independent sector.

A participatory model of creative co-production
An important secondary aspect to the research is the development of a model capable of defining optimal modes of working between DH RSE teams and the creative industries. The model was created through an ethnographic approach pioneered in the Science and Technology Studies (STS) community by researchers such as Latour and Woolgar, Knorr-Cetina,
13 and Fujimura
14 in conjunction with industrial theories such as lean production (Ohno)
15 and disruptive innovation (Christensen).
16 The project’s Principal Investigator (PI)  developed the model by embedding himself in the creative process, observing interactions between writers, technologists, business people, and the DH RSE over the course of 12 months. The insights gained will be used to generate a best practice model to facilitate a mutually beneficial partnership between academia and industry, one that can be used by other DH RSE teams, increasing capability across the global community and demonstrating the special value DH has for companies and governments increasingly struggling to recruit and retain staff with convergent skill sets spanning creative and technical fields.

Effective collaboration between universities and industry is essential for the higher education sector to maintain its leading status in research and innovation, and to ensure industry can resolve the tension between convergent creative and technological pressures. Alongside this industry innovation, of course, is the equally important parallel development of a critical infrastructure within the humanities to effectively engage, modify and critique these emerging forms of narrative.  This project provides an example of how Digital Humanities RSE teams can act as key participants in this process, using their blended culture of development and research to avoid an excluded middle, where academic and creative teams lack the vocabulary, practical experience, and cultural experience to collaborate effectively together.

Bibliography

Smithies, James, ‘AI, DH, and the Automation of Labour’, in
The Digital Humanities and the Digital Modern
. Basingstoke: Palgrave Macmillan, 2017, pp.79 - 111.

Brett, Alys, Michael Croucher, Robert Haines, Simon Hettrick, James Hetherington, Mark Stillwell, and Claire Wyatt. “Research Software Engineers: State of the Nation Report 2017.” Zenodo, April 6, 2017.

Anderson, Sheila, and Tobias Blanke. “Taking the Long View: From e-Science Humanities to Humanities Digital Ecosystems.”
Historical Social Research
37, no. 3 (2012): 147–64.

For a classic example see Siemens, Raymond George, and Susan Schreibman, eds.
A Companion to Digital Literary Studies
. Blackwell Companions to Literature and Culture 50. Oxford: Blackwell Publishing, 2008.

For a classic example see Kittler, Friedrich A.
Gramophone, Film, Typewriter
. Writing Science. Stanford, Calif: Stanford University Press, 1999.

For a classic example see Hayles, N. Katherine.
How We Became Posthuman: Virtual Bodies in Cybernetics, Literature, and Informatics
. Chicago: University of Chicago Press, 1999.

For a classic example see Landow, George P.,
Hyper/Text/Theory
. Baltimore ; London: Johns Hopkins University Press, 1994 and subsequent editions in 1997 and 2006.

Atkinson, Sarah, and Helen W. Kennedy. “Extended Reality Ecosystems: Innovations in Creativity and Collaboration in the Theatrical Arts.”
Refractory: A Journal of Entertainment Media
30, no. 2018 (July 16, 2018): 10.

HM Government. “Culture Is Digital.” London: Department of Media, Culture, and Sport, March 2018; Innovate UK. “The Immersive Economy in the UK: The Growth of Virtual, Augmented and Mixed Reality Technologies.” London: HM Government, 2018.

Ciccoricco, David.
Reading Network Fiction
. Tuscaloosa: The University of Alabama Press, 2007.

Barnet, Belinda.
Memory Machines: The Evolution of Hypertext
. London: Anthem Press, 2013.

Latour, Bruno, and Steve Woolgar.
Laboratory Life: The Construction of Scientific Facts
. Princeton University Press, 1979.

Knorr-Cetina, Karin.
The Manufacture of Knowledge: An Essay on the Constructivist and Contextual Nature of Science
. Pergamon Press, 1981.

Fujimura, Joan H. “Constructing ‘Do-Able’ Problems in Cancer Research: Articulating Alignment.”
Social Studies of Science
17, no. 2 (1987): 257–93.

Ohno, Taiichi.
Toyota Production System: Beyond Large-Scale Production
. Cambridge, Mass: Productivity Press, 1988.

Christensen, Clayton M.
The Innovator’s Dilemma: When New Technologies Cause Great Firms to Fail
. The Management of Innovation and Change Series. Boston, MA: Harvard Business School Press, 1997.