Complex Space-Time Modelling, Visualising and Performing Literary and Historical Chronotopes

Overview
In seeking to understand human activity using computational methods, researchers in the Digital Humanities must inevitably wrestle with the complexities of space and time. Considered separately, these situating vectors of human experience pose challenges of perspective, direction, and, of course, uncertainty: both in how they are perceived and in the form of their documentation. When combined, and addressed as a cognitive matrix within which knowledge is constructed and narrative accounts are traced, that complexity is compounded. The challenges entailed are conceptual, theoretic and practical, and the panel aims to bring these diverse approaches into a productive dialogue.
The panelists will offer prismatic perspectives on primarily historical human experience and knowledge construction in relation to varying notions space-time, in six short papers. These will afford a keyhole view into the research undertaken from each of the projects, as it is described in the abstracts, but specified for its particular resonance with the work of one of more of the other participants. Some participants will report on research in progress, others on work produced and evaluated, while one surveys the work of past pioneers with an argument for its reconsideration. Collectively, they will inform a lively dialogue bridging knowledge representation, data modelling, semantic annotation, machine learning, urbanism, social and cultural history, literary and courtroom chronotopes, and critical artistic and design practice.
The studies are concerned with how space-time defines and is defined by human experiences: of pleasure and trauma, creativity and crime, security and insecurity; and how it is represented in textual, graphical, audiovisual and extended reality forms, empirically and imaginatively. They address questions of mobility, place-making, displacement and co-presence, as sites of complex space-time relations, in every century from the seventeenth to the present. All though, with one exception, confront common challenges around how best to identify, extract, model and compare the spatio-temporal data that inheres in the historical or contemporary record. And that one exception serves as a keystone, suggesting possible solutions to some of these questions by treating space-time as an interface metaphor that may render the multi-dimensionality of humanistic data legible in terms that are disciplinarily congenial.
It is hoped from the resulting dialogue between panel members will produce some pointers to how organising concepts and frameworks from one approach - chronotopes, dimensionality reduction, thick-mapping, fuzziness and spatial syntax - may prove unexpectedly applicable across conventional boundaries, and how these may be usefully mediated by new or existing digital tools and techniques.

Deep mapping cultural venues as a historical space-time interface (A)
Claartje Rasterhoff and Weixuan Li
Why are some places and periods considered to be more innovative than others? In addressing this question, humanities scholars and social scientists have pointed to a complex interplay of historical circumstances, urban conditions, talented individuals, as well as local supporting institutions (Hall 1998; O’Hagan and Hellmanzik 2008; O’Hagan and Borowiecki 2010; Serafinelli and Tabellini 2018). Individual studies tend to favor the one factor over the other, but collectively they increasingly employ spatial concepts and methods, visible in the mapping of spatial clustering, social networks, and migration patterns of actors or objects (Currid and Williams 2010; Brosens et al. 2017; Williams 2018). In these approaches, it is often assumed that urban space facilitated the creation and the transmission of innovation over time and place, but it is still unclear how this would have worked in practice. Where and how were (new) ideas negotiated and challenged, adopted and adapted? Would this have differed across time and place? If so, how can we present the changes over time and space?
Inspired by time-space geography (Pred 1984; Hägerstrand 1985), we propose a methodological framework for embedding complex spatial thinking and visualizations into existing research on the historical relationships between place and innovation, and create multi-layered deep maps with Geographical Information System (GIS) to capture the dynamics and the uncertainties in all its complexity (Bodenhamer et al. 2015). And we test it by dynamic digital mapping of a particularly lively and well-researched time and place: seventeenth-century Amsterdam. At the heart of our approach is a visual representation of cultural production and consumption as dynamic and performed activities embedded in (semi-)public life. We argue that we can observe and represent these activities in specific private and public urban spaces (so-called cultural venues), and that, more broadly, a focus on events and venues contributes to the integration of temporal and spatial perspectives in the digital humanities (Nijboer and Rasterhoff 2018). After all, it is in specific venues in specific urban environments that time and space cross paths, because historical activities relating to the cultural life of urban dwellers can only take place in a physical location on a particular time (Crow 1985).
The early modern period is, moreover, particularly interesting and challenging, because there were few formal cultural venues, as in the case of Amsterdam. Deep mapping culture venues in early modern Amsterdam forces us to explore available sources, and, more importantly, to rethink how we can analyze and visualize time and space while allowing for uncertainties and ambiguities. Historians have observed that many activities that we might now consider as “cultural” such as theatre, music, but also art auctions and dice games were taking place in taverns or inns (
herbergen), which in turn were often run by cultural entrepreneurs or their family members (Hell 2017). We investigate the role of these venues in the city’s various creative industries, and as spaces where entertainment and cultural consumption may have converged to give way to popular innovations.

Extracting the historical experience of urban and domestic space-time from the Old Bailey corpus by a novel application of machine learning (B)
Alex Butterworth and Simon Wibberley
The courtroom records of the Old Bailey teem with the everyday life of eighteenth and nineteenth century London, around those moments when mundane existence is more or less abruptly intersected by acts of criminal transgress. Dense sequences of human experience are described, often in multiple accounts, in an attempt to forge a consensus version of events. These are transcriptions of the legal process as it remediates the configuration of space-time that defined the original criminal event, through the formalising chronotopes of the courtroom. The temporal and spatial dimensions of the original events, already subject to uncertainty and contestation, become further layered and involuted through narrative testimony.
These concatenated constructions of space-time can be disentangled by close reading. However, it is only when the corpus is viewed at scale that it reveals historically significant patterns of human behaviour within the urban and domestic environment: the lived experience of the historical actors, and the micro-incidents that comprise it. To access these insights, we are developing a novel methodology for the semantic annotation of the historical corpus of the Old Bailey that employs supervised machine/deep learning technologies: a process in which the humanist scholar is continuously involved in assessing and categorising outputs, to iteratively generate additional training data, in a virtuous cycle.
The approach deploys a collection of advanced natural language processing methods to automatically annotate textual elements in the initial phase. The subsequent examination of these entities by their distribution, individually by type and in combination, within the texts and in relation to their discourse structure, make it possible to identify and code fragments of a semantic graph as representing events. These events, in turn, can be mapped or otherwise visualised in their physical context - of crime scene or courtroom reconstruction - and further analysed and validated using measures of coherence and plausibility that leverage the analyst’s intuitive knowledge of space-time relations.
This method will generate a rich set of semantic annotations, enabling new kinds of research questions to be posed of the corpus, at three levels of granularity:
macro: eg. what patterns can be discerned in the spatial distributions of event over time (of day, year or century), or in the frequency of reference in specific types of courtroom discourse?
meso: eg. what types of events, considered as assemblages of activity, place, time and person, characterise particular crimes?
micro: eg. what events and locations characterise the itineraries of thieves and their victims, and what causes deviation from these norms?

Visualising the uncertain: How abstract representations of spatio-temporal data can help us unlock large-scale heritage collections (C)
Eirini Goudarouli, Johannes Liem, Steven Hischorn, Charles Perin
Since 2014, a major archive has undertaken the digitisation at scale of analogue, hand-written War Diaries from the First World War (WWI), documenting the story of the British Army and its units on the Western Front, using an interactive crowd-sourcing platform (https://www.operationwardiary.org/). Crowd-workers have digitised more than a million diary pages. During this process, essential information about military units, including labels for casualties, unit strength, weather, everyday army life, soldiers names and grades, military activities at the front and non-military activities behind the line, have been tagged and annotated. However, the transcription process introduced uncertainty on many levels due to missing records or pages, misspellings, illegible passages or lost diaries.
Some uncertainties derive from the particular type of data that has been gathered, the circumstances of its creation, and its post-processing. These may be the result of human errors: the British soldiers who wrote the diaries made misspellings, faithfully transcribed, while the crowd-workers also introduced new typos, leading to further ambiguity regarding names of people and locations. Or they may be intrinsic to the historical record, as where spatio-temporal uncertainty arises when several places are mentioned on one single day in the diary. Close reading is then required to determine in which order the places were visited by the troops or whether a place was mentioned as a troop location or other reasons.
These types of uncertainty may lead to ambiguous geographic coordinates during the geo-referencing process (assigning a numeric geographic location to a place name). As part of the collaboration between Archives and its university partner, an approach was developed that we term GeoBlobs: an abstract representation of spatio-temporal data, which visualises uncertain spatio-temporal data derived from the handwritten military diaries (Liem et al., 2018).
GeoBlobs offer one possible solution to these problems. They are an abstract representation of moving entities on a map with uncertain positions. Instead of showing a unit at a given point in time, GeoBlobs convey an unordered estimation of the possible locations over a temporal window using enclosed shapes, or blobs. To this end, we apply heuristics to weigh each location within the temporal window. At the representation stage, it is possible to specify dynamically which (weighted) sets of location are considered to form the GeoBlobs. For other visualization approaches of spatio-temporal uncertainty in a cultural heritage context please refer to Windhager et al. (2018).
Different form and style parameters can influence the visual appearance of GeoBlobs and their semantics. Uncertainty can be expressed through sketchy or blurry styles, while the use of multiple GeoBlobs allows us to compare units and show different probabilities. Animation or overlays communicate additional context like military (e.g. fighting, raiding, fixing trenches) or non-military activities (e.g. playing football, boxing, resting, washing).
The GeoBlob project aims to reveal stories of the soldiers’ day-to-day life behind the lines, which will lead towards a narrative visualization (Riche et al., 2018) for communicating “the life behind the trenches” that cannot be found in our history books (Grayson, 2016).

*As part of this paper, a short video of a prototype displaying the GeoBlobs in action will be presented.

Spatial allusion, temporal recurrence and cognitive uncertainty: visualising chronotopic structure in a literary text (D)
Duncan Hay and Alex Butterworth
The mapping of literary texts has become a commonplace of Digital Humanities research in recent years. Whilst many projects have interpreted this mapping as the accurate georesolution of textual name references, others have used computational methods to explore the place-making mediated by text. Often, these approaches have aimed to locate the text, using GIS methods.
This study abstracts itself further from an indexical relationship to the physical world, focusing instead on the narratological role of chronotopic relationships within literary texts: how varying modes of time and space are deployed and inter-operate, curdling together to create and recall the textual experience of situated event. It explores this construction of narrative through the process of designing an interactive data visualisation interface, using semantic annotations of a literary text, manually produced during an initial process of close reading. The annotations encode semantic entities as types of place, time, person and event, along with textual cues and referential clues, more or less explicit and probabilistically interpreted by the reader.
Conceived as an experimental exploration of the problems and potential analytical value of the computational modelling and visualisation of literary chronotopes, this phase of the project addresses texts that are manageable in length but pose significant challenges: allusion and anaphora, uncertainty and ambiguity, recurrence and duplication, cognitive lacunae. It embraces the notion of literary text as cognitive scaffolding, whose effect lies as much in the absence of description and specificity as its presence, while grounding its analysis through the identification of chronotopic anchor points within more fluid constructions of contextualising time and space.
The case study is Coleridge’s
The Rime of the Ancient Mariner. The poem adheres to the simple past tense in its progressive dynamic, yet operates through a temporal scheme of analepsis and recurrence between a cursed present and the framing device of a confessional future, with spatial allusion playing a complex role. The paper will offer critical reflections on designing exploratory visualisations of fragments of semantic graphs in fluid and dynamic configurations; on the most useful and appropriate schema for such semantic annotation; on the practicability of scaling such annotation using supervised deep learning methods; and on the insights gained by such analysis both for literary study and the authoring of novel digital literary forms.

Only Expansion: Composing Temporal Structures For Augmented Audio Experiences In The Anthropocene (E)
Duncan Speakman
This paper will present the practice based research outcomes of an augmented audio urban artwork, offering an account of how merging realtime processing of the listener’s immediate sonic environment with remote field recordings can offer new critical approaches to temporal perception and contemporary ecological thinking.
The artwork under examination uses custom mobile technology to create an urban audio walk that both remixes the immediate sound environment of the audience and combines it with field recordings from remote locations. In the experience participants wear headphones that also contain binaural microphones, the signal from these microphones is fed through DSP software in bespoke handheld devices before being fed back to the headphones. In this way the voices of passing pedestrians might become a resonant choir, or a bus engine may form a rhythmic counterpoint. The field recordings are sourced from a series of international locations all undergoing major environmental shifts, so the sound of the wind in the city where the audience experiences the piece may become merged with wind recordings from the Tunisian Sahara. Through the combination of field recordings with processed and raw microphone signals, an interface is created between the listeners presence, the immediate space and remote locations.
Within the arc of the composition the audience is invited to seek out types of location through textual prompts often as simple as single words such as “drift” or “border”. In this way the work offers a site responsive rather than site specific experience, and the absence of cardinal guidance forces the audience to navigate the urban space through direct physical and sensory engagement. Drawing on over a decade of international practice in the creation of locative audio walks by the author, the paper considers new compositional structures for works using augmented audio technologies, focusing on the layering of different temporalities within urban environments.
The effect that is produced when the audience’s lived experience of walking through the work are layered with the timescales represented within the field recordings speaks to Timothy Morton’s idea that we are currently living with the uncanny sense of existing on two timescales simultaneously. Our everyday human actions feeding into processes that extend far beyond our lifetimes. This experience is considered within the context of Anja Kanngieser’s proposal that “sound can help to differentiate the sweeping universality - and hence the seeming unchangeability - that the Anthropocene poses,” and that “sound renders apparent that the world is not for humans. The world is rather with humans.” (Kanngeiser, 2015).

By situating the audience within the layered temporality of the work, not just physically, but as an active contributor to the soundscape, this inquiry offers new approaches to augmented audio as a way of inhabiting, communicating and knowing an entangled world. It begins not with distant stories being collected and delivered, but at the site of the audience experience, and expands outwards from there through the transversality of sound.

Exploring how historical visual classifications can inform data modelling using semantic web technologies for the digital humanities (F
)

Charles van den Heuvel
Digital humanists who seek to subject complex realities to computational methods of analysis must first build and manage complex models, conceptualised in terms that embrace but exceed the familiar dimensions of space and time. This challenge has a long history that may be traced back at least to the early 20
th century and this paper will survey that tradition for how it may inform contemporary approaches in the digital humanities.

At the turn of the twentieth century, scholars from diverse disciplines were concerned with knowledge organization at a global level, aiming to reconcile new insights from evolutionary theory, the emerging disciplines of psychology and physics (with its novel notions of spacetime) and traditional philosophical and humanist epistemology. Three-dimenstional objects and imagistic thinking played a crucial role in the classificatory systems and knowledge modelling developed by Paul Otlet, Patrick Geddes, Wilhelm Ostwald, H.G.Wells and others. Their approach was conceived to allow both automatic information retrieval and analysis, and a synthesising of knowledge that retained its dynamic complexity. Novel techniques of dimension reduction nevertheless challenged the multidimensional habits of humanistic thought - a tension that the mathematician Shiyali Ranganathan, who developed the multidimensional Colon classification, described in 1951:
“Thought is multi-dimensional. But we are one-dimensional beings – that is we still prefer all things to be handled to be arranged in one-dimension […] This means that classification is essentially a transformation of a many-dimensional universe into a uni-dimensional, uni-directional one. Machine tools are expected to perform this transformation.”
As the computer proved its capacity to analyse large quantities of scientific information and to make complex calculations, there was criticism of the “efficient” approaches developed by the information sciences. Gerard Cordonnier (1944) had already tried to define “intellectual space” and advocated classification as a collection ordered by points of view. Joseph Licklider (1965) would reassert the value of spatial analogies in the face of linear methods, observing the relative absence from the contemporary debate of reflections on concepts such as “information space,” “semantic space” or “the space of knowledge.” Parallel to these (visual) conceptualizations of space in library and information sciences, historians formulated theories of time and periodization (Pot 1951, 1999), distinguishing types of duration (Braudel, see Tomich, 2012). Their aim was to equip their field of conceptual modelling with rigorous defined spatial metaphors of multidimensionality: to imagine what Kubler described as the “shape of time.”
It was, in some senses, the conjunction of these aspirations that Ted Nelson had sought to reify in Project Xanadu (1960s to 80s), espousing the use of viewable units of meaning: “vunits” whose contents are linkable and transclusible (showing their contents). By 2007, Nelson would become frustrated that the World Wide Wide had come to imitate “paper under glass” rather than a vision of true hypertexutality. This paper argues that a decade on, as the humanities increasingly embrace semantic web technologies, reflection on past attempts to explore multi-dimensional conceptualization of time and space can significantly inform current aspirations for the inclusive modelling of complexity in humanistic enquiry.

Bibliography

A.

Andersson, Å.E., et al. (2014). Location and spatial clustering of artists.
Regional Science and Urban Economics 47, 128–137.

https://doi.org/10.1016/j.regsciurbeco.2013.09.008

Bodenhamer, et al. eds. (2015).
Deep maps and spatial narratives. Indiana University Press.

Brosens, K., et al. (2017). Visualizing and Analyzing Complex and Dynamic Networks of Flemish Tapestry Entrepreneurs (1640–1720).
Leonardo, Journal of the International Society for the Arts, Sciences and Technology 50(5), 503-503.

Crow, T.E. (1985).
Painters and public life in eighteenth-century Paris. Yale University Press

Currid, E., Williams, S., 2010. The geography of buzz: art, culture and the social milieu in Los Angeles and New York.
Journal of Economic Geography 10, 423–451. https://doi.org/10.1093/jeg/lbp032

Hall, P. (1998).
Cities in Civilization: Culture, Technology, and Urban Order. London: Weidenfeld & Nicolson; New York: Pantheon Books

Hell, M.,
De Amsterdamse herberg (1450-1800): Geestrijk centrum van het openbare leven. Nijmegen, Van Tilt.

Hägerstrand, T. (1985). Time-geography: focus on the corporeality of man, society, and environment.
The science and praxis of complexity, pp.193-216.

Nijboer, H., Rasterhoff, C. (2018). Linked Cultural Events: Digitizing Past Events and Analysing the Creative City, in S. Munster et al.,
Digital Research and Education in

Architectural Heritage, 5th Conference DECH 2017 and 1st Workshop UHDL,

Communications in Computer and Information Science series Dresden, Germany
March 30-31 2017. Springer Verlag, 22-33

O’Hagan, J., Borowiecki, K.J. (2010). Birth Location, Migration, and Clustering of Important Composers: Historical Patterns.
Historical Methods: A Journal of Quantitative and Interdisciplinary History 43, 81–90.

https://doi.org/10.1080/01615441003729945

O’Hagan, J., Hellmanzik, C. (2008). Clustering and Migration of Important Visual Artists: Broad Historical Evidence.
Historical Methods: A Journal of Quantitative and Interdisciplinary History 41, 121–136.

https://doi.org/10.3200/HMTS.41.3.121-136

Pred, A. (1984). Place as historically contingent process: Structuration and the time
‐geography of becoming places.
Annals of the association of american geographers, 74(2), pp.279-297.

Serafinelli, M., Tabellini, G. (2018). Creativity over Time and Space,
CEPR Discussion Paper 12365.

Williams, H. (2018). Artists and the city: mapping the art worlds of eighteenth-century Paris.
Urban History, 1-26.

C.

Grayson, R. (2016). A Life in the Trenches? The Use of Operation War Diary and Crowdsourcing Methods to Provide an Understanding of the British Army’s Day-to-Day Life on the Western Front.
British Journal for Military History, 2(2): 160-185.

Liem, J., Goudarouli, E., Hirschorn, S., Wood, J. and Perin, C. (2018). Conveying Uncertainty in Archived War Diaries with GeoBlobs.
IEEE VIS 2018 Electronic Conference Proceedings.

Riche, N.H., Hurter, C., Diakopoulos, N. and Carpendale, S. (eds.) (2018). Data-Driven Storytelling. Milton: Chapman and Hall/CRC.

Windhager, F., Filipov, V. A., Salisu, S. and Mayr, E. (2018). Visualizing Uncertainty in Cultural Heritage Collections.
EuroVis Workshop on Reproducibility, Verification, and Validation in Visualization (EuroRV3).

E.

Kanngieser, A. (2015) Geopolitics and the Anthropocene: Five Propositions for Sound. GeoHumanities [online]. 1 (1), pp. 80-85.

Morton, T. (2013) Hyperobjects : Philosophy and Ecology After the End of the World [online]. Minneapolis: Univ Of Minnesota Press.

F.

Cordonnier, G. (1944) Classification et Classement.
Extrait du Bulletin d’Information Scientifique et Technique.

Heuvel, C. van den (2012) Multidimensional Classifications: Past and Future Conceptualizations and Visualizations,
Knowledge Organization 39(6), pp. 446-460.

Heuvel, C. van den (2014) Historical Roots of Information Sciences and the Making of E-Humanities. In: Rens Bod, Jaap Maat & Thijs Westeijn (Eds.),
The Making of the Humanities Volume III The Modern Humanities. Amsterdam University Press, pp. 465-478

Kubler, G. (1962),
The Shape of Time. Remarks of the History of Things, New Haven and London: Yale University Press

Licklider, J.C.R. (1965)
Libraries of the future. Cambridge: Mass.: The M.I.T. Press.

Nelson, T. H. 2007/07/11
BACK TO THE FUTURE: Hypertext the Way It Used To Be

http://xanadu.com/XanaduSpace/btf.htm

.

Pot, J.H.R.van den (1951)
De periodisering der geschiedenis: Een overzicht van theorieën. ‘s-Gravenhage: W.P. van Stockum.

Pot, J.H.R. van den (1999)
Sinndeutung und Periodisering der Geschichte: eine systematische Übersicht der Theorien und Auffassungen, Leiden: Brill.

Ranganathan, S.R. (1951) Colon Classification and its Approach to Documentation. In: J.H. Shera, and M.E. Egan (eds.),
Bibliographic Organization. Papers presented before the Fifteenth Annual Conference of the Graduate Library School July 24-29, 1950. Chicago: The University of Chicago Press.

Tomich, D.W.
(2012) “The Order of Historical Time: The Longue Durée and Microhistory”. In The Longue Durée and World-System Analysis. Edited and with an introduction by Richard E. Lee, Albany: State University of New York, 9-34.

Overview

Shaw, R. (2018). A Deep Gazetteer Of Time Periods,

DH

2018

:

283-284

Sekino, T .(2018). Representation and Comparison of Uncertain TemporalData Based on Duration, 2018 Pacific Neighbourhood Consortium Annual Conference and Joint Meetings (PNC)
https://ieeexplore.ieee.org/document/8579465