Diagramming the Complexities of Historical Processes: From Ontology-based Modeling to Diagrammatic Reasoning

Introduction
This paper presents ongoing foundational theoretical and practical work on the application of ontology-based modeling to represent and visualize the complexity of knowledge disseminated in historical narratives. In short, the new approach combines modeling informed by philosophical ontology and philosophy of history with semiotically founded visualization of historical processes in order to support historical understanding.
The following quote from Munslow (2007) lends itself as an appealing summary of the character of historical narratives: “In writing a history for the past we create a semiotic representation that encompasses reference
to it, an explanation
of it and a meaning
for it.” What role could information visualization tools play in this context? As Champagne (2016) remarks, “[h]istorians occasionally use timelines, but many seem to regard such signs merely as ways of visually summarizing results that are presumably better expressed in prose.” He challenges this view and argues that timelines could support the historian in gaining novel historical insights. The main cognitive funtion of timelines is according to Champagne (2016: 40) the “logical conjunction by visual juxtaposition”. Furthermore there is also the potential of abductive reasoning: “Timelines, however, are more likely to surprise us, by showing us past events that we would have never otherwise considered chunking. Hence, in addition to historical scholarship expressed in regular prose, consulting diagrammatic signs can foster the discovery of patterns essential to a fuller understanding of the past” (Champagne, 2016: 40). This is especially more likely if there are synchronoptic timelines showing historical events of different categories—i.e. not only political events, but also economic or cultural events etc. (That is the approach conducted by Peters and Peters (1952) in their
Synchronoptische Weltgeschichte.) For example, such parallel timelines could possibly be used as a tool to support periodization (see possible use case reported by Luyt (2015)).

Problem statement and objectives
A “visual historiography” (Roegiers and Truyen, 2008) can quite easily be done via the temporal, spatial, and thematic context of information about historical events, but without explicitly stated relations between events it is questionable how useful that could be in supporting historical research. The big advantage of that approach is of course, “that one is able to represent the complexity of a historical subject, without having to fill out the gaps, or having to choose between different interpretations, but using an [information integration] architecture that places the subject in its context(s)” (Roegiers and Truyen (2008: 70) as cited in Sabharwal (2015: 57)). The problem with such a ‘visual historiography’ is that it cannot support visual contextualization done by the historian during conceptualizing complex interrelations of historical events—including not only temporal, spatial, and thematic relations, but also causal relations (incl. the motivation and roles of historical actors involved in the events), mereological, and constitutive relations of complex (e.g. composite) events.
Anyhow, explicit modeling of event structure and relations is necessary because without a more fine-grained representation of the structure and interrelations of events visualization tools are indeed limited to bare juxtaposition. Digital history demands information visualizations beyond simple timelines (see for example Drucker and Nowviskie, 2003). Diagrammatic approaches for multiperspectival analysis and synopsis of historical sources are needed. There are rarely technical implementations and just a few theoretical approaches to the development of such tools for multiperspectival exploration of historical sources (cf. Shaw, 2008: 90).
See for example Drucker (2011) in “Humanities Approaches to Graphical Display”: “At best, we need to take on the challenge of developing graphical expressions rooted in and appropriate to interpretative activity.” Jänicke (2016) argues that the reason for this slow progress is the still deficient collaboration between researchers in information visualization and digital humanities. This brings us back to the announced modeling approach, because conceptualizing can be considered as modeling: “the more schematic the conceptualization in a discipline, the more its practitoners are likely to engage with models rather than concepts” (McCarty, 2005: 25). Historians construct concepts in order to understand historical events. The method for this kind of explanation is known as colligation—the construction of colligatory concepts—in philosophy of history. Walsh (1951 S. 59) defines colligation as “the procedure of explaining an event by tracing its
intrinsic relations to other events and locating it in its historical context” (Walsh (1951 S. 59) as cited in Shaw (2010 S. 11)). An adequate modeling of colligatory concepts and the relations between the concepts—the colligatory relations (Shaw, 2008)—is the precondition for “semantic tools” (Shaw, 2013) based on such an explicit representation of the past.

Related work
The digital edition (Behrendt et al., 2010) of Peter’s (already mentioned)
Synchronoptische Weltgeschichte (Peters and Peters, 1952) visualizes parallel timelines showing historical events from different categories (political events, economic events, cultural events, etc.). However, it does not show the inner structure of complex events or processes and does not provide typed relations between events (see Shneiderman, 1996). Interestingly the tool provides visual contextualization of events based on the knowledge organization in its event database (see Fig. 1): related events are retrieved based on their common index terms.

Fig. 1 Screenshot of
Der Digitale Peters (Behrendt et al., 2010) showing contextualized historical events

The older tool
SemTime (Jensen, 2003) provides a solution to these typical shortcomings of timeline visualizations by introducing Semantic Timelines to visualize complex timelines with sub-timelines and different types of relations between historical events. Newer projects concentrate not only on the granularity of events but also on the the details of biographies, i.e. the modeling and visualization of the roles of (historical) persons in events (Trame et al., 2013; Hyvönen et al., 2018).

The VICODI (Visual Contextualisation of Digital Content) project (Nagypál et al., 2005) used semantic web ontologies as basis for visual contextualization. Based on the top-level ontology DOLCE (Gangemi et al., 2002) the SHO (Spatial History Ontology) was developed by Grossner (2010) to overcome the shortcomings of exisiting ontologies in the modeling of spatial information in event ontologies. HERO (Historical Event Representation Ontology) is also founded on DOLCE and focuses on the modeling of different types of roles (thematic, social, and also perspectival) (Goy et al., 2018). DOLCE and DOLCE-DnS Ultralite (DUL) respectively contains the Descriptions and Situations (DnS) Ontology Design Pattern (ODP) (Gangemi and Mika, 2003). DnS allows the modeling of different perspectives on entities. A CRM (Doerr, 2003) based alternative to model perspectives or interpretations is the MIDM (Multiple Interpretation Data Model) (Ruymbeke et al., 2017). A much simpler modeling approach for different perspectives is SEM (Simple Event Model) (Hage et al., 2011).

Approach
A crucial requirement for the approach presented here is the representation of different perspectives on historical events. Perspectival explanation or “synoptic judgement” (Mink, 1987) is a main task of the historian (cf. Levy, 2001: 70). I have chosen DUL as top-level ontology for the modeling examples described in the following paragraphs because of its constructivist design principles and especially because its DnS ODP fits very well to our requirement of modeling colligations and the different perspectives or interpretations of historical events. Thus, a Description represents the conceptual relations which were grasped by the historian in a synoptic judgement.
In biomedical ontologies (so far the main domain of applied ontology) Descriptions are used to represent medical diagnoses. There is indeed an interesting similarity between the synoptic judgements of a historian and the medical diagnoses of a physician: “The best analogy I can suggest for the way in which synoptic judgments are reached is that of a physician’s diagnosis—a combination of broad medical knowledge, relevant evidence drawn from various tests, a knowledge of various theoretical possibilities for explanation, and skill in seeing which interpretation of the evidence works best in a particular case—the difference being, of course, that the physician deals primarily with law-bound physiological processes, the historian primarily with human conduct and purposive action” (Schroeder, 1997).

Fig. 2 shows a screenshot from an experimental tool that draws diagrams of causal narratives. The example is from Theda Skocpol’s
States and Social Revolutions: A Comparative Analysis of France, Russia, and China. Skocpol (1979) describes the historical process which led to the French revolution in narrative form. In order to visualize the historical process consisting of three sub-processes, the mereological and causal relations had to be represented in a knowledge base according to the reconstruction of Skocpol’s narrative done by Mahoney (1999).
I am planning to adapt Semantic MediaWiki (Krötzsch et al., 2007) extensions (e.g. Weller and Maleshkova, 2016) for the creation of ‘ontological hypertext’ as base for visualization using constraint-based layout algorithms (e.g. inspired by visualizations in biology as in Hoffswell et al., 2018).

Fig. 2 Diagram of the causal narrative based on the reconstruction and visualization by Mahoney (1999: 1166) (The nodes represent events extracted from the narrative and the edges represent causal links between them. The causally linked events 33 and 34 for example are summarized as follows in Mahoney’s narrative analysis: “Pressures for creation of the Estates-General” and “King summons the Estates-General”. 37 finally marks the state collapse and the outbreak of the revolution.)

Skocpol (1979) combines macrosocial and idiographic historical research (cf. Mahoney, 1999: 1189). Fewer idiographic detail is represented in the example from the CEWS (Conflict Early Warning Systems) project (Schmalberger and Alker, 2001a) (see Fig. 3). CEWS focuses on the phase sequences (escalation and de-escalation) in conflict processes. The CEWS Explorer was developed as a tool to visualize and compare conflict phase sequences and different perspectives on them as described in causal narratives about conflicts. In my talk I argue that a remake of this approach can benefit from an ontology-based representation of conflict phases and different perspectives on phases and causes for change of phases as seen from different conflict parties or other actors involved in the conflict.

Fig. 3 Diagram of the first two episodes of the Transnistria conflict (according to the narrative by Vorkunova, 2001) (Nodes represent conflict phases and edges represent transitory events. Note that divergent perspectives on conflict episodes can be added via synchronoptic views.)

According to Peirce’s diagrammatic reasoning approach a diagram should be constructed under the rules of a “system of representation” (CP 4.418). The ontology-based knowledge representation of historical events provides as well a framework for the construction of such representation systems. Similar to the so-called two-level theory of social revolutions in the first example (see diagrammatic representation of the theory in Goertz and Mahoney (2005: 509)) the so-called visual grammar of possible conflict phase sequences in Fig.  4 is a representation system that pretends all possible sequences of different types of conflict phases within conflict episodes. The system is used as “system of diagrammatization” (NEM IV:318) in order to construct diagrams for specific conflict trajectories.

Fig. 4 Visual grammar for possible sequences of conflict phases (according to Schmalberger and Alker, 2001a)

As there is less granularity, i.e. no complex composite events, a simpler ODP can be used to represent conflict phase transitions. Fig. 5 shows an exemplary phase transition modeled with the ODP Transition

. Based on the ‘system of diagrammatization’ and some empirical data about conflicts diagrams of possible conflict sequences similar to the famous diagram of the Cuban missile crisis could be created (see Fig. 6).
The CEWS Explorer was also built for counterfactual analysis of conflicts (see Schmalberger and Alker, 2001b).

Fig. 5 Modeling the change of a conflict’s state with the Transition ODP and classification of the conflict phase types and the transitory event type with knowledge organization systems

Fig. 6 Diagram for counterfactual exploration of possible decisions of the conflict parties and resulting outcomes of the Cuban crisis according to the game tree based counterfactual analysis by Joxe (1963) (from Bertin, 1967)

Conclusion
There are two feasible use cases for the presented modeling and visualization approach—followed by a more demanding one:

preparation and communication of research results,
knowledge representation and knowledge visualization for public history, and
generation of new insights with information visualization tools for digital history.

The added value for the diagrammatic communication of research findings originates in the explicit representation of historical knowledge: A complex historical process can be visualized for better communication of research results on the base of the explicitly represented entities and their relations. As Lange (2013: 46) notes in his textbook on comparative-historical methods, it is recommended to use “diagrams to represent clearly the argument of causal narration, to make the causal claim more explicit”.
Supported by a ‘system of diagrammatization’, a (public) historian is enabled to represent and visualize the essential event relations of a “basic story” (cf. Perfetti et al., 1995: 2)—excluding more granular expert knowledge about the historical process in focus. In knowledge visualization projects for public history additional diagram types could be used in order to provide the “collateral knowledge” (Hoffmann, 2005) necessary for the public audience (non-historians) to better understand the historical events—e.g. via concept maps or knowledge mapping in general (Davies, 2011).
It has been shown in public history’s neighboring discipline history didactics, that concept maps “can help learners to analyse and synthesise knowledge” (Tzeng, 2010). This is especially valuable to provide insight into the different perspectives on an issue (for example in divergent conflict histories or national memory narratives). The third use case lies beyond knowledge visualization. It is aimed at interactive information visualization tools in the sense of a “knowledge generator” (Drucker, 2014): using diagrams to support historians during research in order to create new knowledge. Formalization of the complexities and subtleties of expert knowledge is the necessary precondition for building diagrammatic reasoning tools for historical understanding.
See Frank (2018) for more details about the semiotic foundations of this systematic diagrammatic reasoning procedure by means of systematically experimenting on a knowledge base (created by ontology-based knowledge modeling) in order to infer new knowledge. Hence, the requirements of “total explicitness and absolute consistency” (McCarty, 2005: 5) for formal knowledge modeling are large obstacles towards such ‘thinking tools’. Case studies in digital history have to reveal if the proposed diagrammatic reasoning approach is suitable. Besides historical conflicts the ontology-based modeling approach is planned to be used to prepare a collection of historical travelogues to be published as digital editions enhanced with interactive map-based visualizations. We will apply ODPs to model journeys and historical routes described in the travel narratives.
See also the Linked Places project’s conceptual model of historical geographic movement:

Bibliography

Behrendt, H. R., Burch, T. and Weinmann, M. (eds). (2010).
Der Digitale Peters: Arno Peters’ Synchronoptische Weltgeschichte. Frankfurt am Main: Zweitausendeins.

Bertin, J. (1967).
Sémiologie Graphique. Les Diagrammes, Les Réseaux, Les Cartes. Mouton.

Champagne, M. (2016). Diagrams of the Past: How Timelines Can Aid the Growth of Historical Knowledge.
Cognitive Semiotics,
9(1): 11–44.

Davies, M. (2011). Concept mapping, mind mapping and argument mapping: What are the differences and do they matter?
Higher Education,
62(3). Springer: 279–301.

Doerr, M. (2003). The CIDOC CRM: An Ontological Approach to Semantic Interoperability of Metadata.
AI Magazine,
24(3).

Drucker, J. (2011). Humanities Approaches to Graphical Display.
Digital Humanities Quarterly,
5(1).

Drucker, J. (2014).
Graphesis: Visual Forms of Knowledge Production. (MetaLABprojects). Harvard University Press.

Drucker, J. and Nowviskie, B. (2003).
Temporal Modelling: Conceptualization and Visualization of Temporal Relations for Humanities Scholarship. Technical Report.

Frank, I. (2018). Diagrammatische Denkwerkzeuge in den Digital Humanities – Ansatz zur zeichentheoretischen Grundlegung. (Ed.) Fricke, E., Posner, R. & Siefkes, M.
Zeitschrift für Semiotik,
39(1-2/2017). Stauffenburg Verlag: 51–81.

Gangemi, A. and Mika, P. (2003). Understanding the Semantic Web through Descriptions and Situations. In,
On The Move to Meaningful Internet Systems 2003: CoopIS, DOA, and ODBASE, vol. 2888. (Lecture Notes in Computer Science). pp. 689–706.

Gangemi, A., Guarino, N., Masolo, C., Oltramari, A. and Schneider, L. (2002). Sweetening Ontologies with DOLCE. In Gómez-Pérez, A. (ed),
Knowledge Engineering and Knowledge Management: Ontologies and the Semantic Web. Springer, pp. 166–81.

Goertz, G. and Mahoney, J. (2005). Two-Level Theories and Fuzzy-Set Analysis.
Sociological Methods & Research,
33(4): 497–538.

Goy, A., Magro, D. and Rovera, M. (2018). On the role of thematic roles in a historical event ontology.
Applied Ontology,
13(1): 19–39.

Grossner, K. (2010). Representing historical knowledge in geographic information systems. Santa Barbara: University of California. PhD thesis.

Hage, W. R., Malaisé, V., Segers, R., Hollink, L. and Schreiber, G. (2011). Design and use of the Simple Event Model (SEM).
Web Semantics: Science, Services and Agents on the World Wide Web,
9(2): 128–36.

Hoffmann, M. H. G. (2005). Problems of Understanding in Conflicts and a Semiotic Solution. In,
IACM 18th Annual Conference.

Hoffswell, J., Borning, A. and Heer, J. (2018). SetCoLa: High-level constraints for graph layout. In,
Eurographics Conference on Visualization (EuroVis), vol. 37. ( 3).

Hyvönen, E., Leskinen, P., Tamper, M., Tuominen, J. and Keravuori, K. (2018). Semantic National Biography of Finland. In,
Proceedings of the Digital Humanities in the Nordic Countries 3rd Conference (DHN 2018). Helsinki, Finland: CEUR Workshop Proceedings, Vol-2084, pp. 372–85.

Jänicke, S. (2016). Valuable Research for Visualization and Digital Humanities: A Balancing Act. In,
Workshop on Visualization for the Digital Humanities, IEEE VIS 2016.

Jensen, M. (2003).
Visualizing Complex Semantic Timelines. NewsBlip Technical Report.

Joxe, A. (1963).
La Crise Cubaine de 1962, éléments Principaux de Décision, Au Cours de La Crise Chause. Document Paris: EPHE Groupe d’études mathématiques des problèmes politiques et stratégiques.

Krötzsch, M., Vrandečić, D., Völkela, M., Haller, H. and Studer, R. (2007). Semantic Wikipedia.
Journal of Web Semantics,
5(4). Elsevier: 251–61.

Lange, M. (2013).
Comparative-Historical Methods. SAGE Publications.

Levy, J. S. (2001). Explaining Events and Developing Theories: History, Political Science, and the Analysis of International Relations. In Elman, C. and Elman, M. F. (eds),
Bridges and Boundaries: Historians, Political Scientists, and the Study of International Relations. (BCSIA Studies in International Security). MIT Press, pp. 39–83.

Luyt, B. (2015). Replacing the ideology of information by exploring domains of knowledge: A case study of the periodization of philippine history and its application to information studies.
Journal of Documentation,
71(6): 1289–99.

Mahoney, J. (1999). Nominal, ordinal, and narrative appraisal in macrocausal analysis.
American Journal of Sociology,
104(4): 1154–96.

McCarty, W. (2005).
Humanities Computing. Palgrave Macmillan.

Mink, L. O. (1987).
Historical Understanding. Cornell University Press.

Munslow, A. (2007).
Narrative and History. Palgrave-Macmillan.

Nagypál, G., Deswarte, R. and Oosthoek, J. (2005). Applying the Semantic Web: The VICODI Experience in Creating Visual Contextualization for History.
Literary and Linguistic Computing,
20(3): 327–49.

Peirce, C. S. (1976).
The New Elements of Mathematics. Vols 1 – 4. The Hague: De Gruyter Mouton.

Peirce, C. S. (1931–58AD).
Collected Papers of Charles Sanders Peirce. Vols 1 – 8. Cambridge: Harvard University Press.

Perfetti, C. A., Britt, M. A. and Georgi, M. C. (1995).
Text-Based Learning and Reasoning: Studies in History. Lawrence Erlbaum.

Peters, A. and Peters, A. (1952).
Synchronoptische Weltgeschichte. Frankfurt am Main: Universum-Verlag.

Roegiers, S. and Truyen, F. (2008). History is 3D: Presenting a framework for meaningful historical representations in digital media. In,
New Heritage: New Media and Cultural Heritage. Routledge, pp. 67–77.

Ruymbeke, M. V., Hallot, P. and Billen, R. (2017). Enhancing CIDOC-CRM and compatible models with the concept of multiple interpretation. In,
SPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, vol. IV-2/W2.

Sabharwal, A. (2015).
Digital Curation in the Digital Humanities: Preserving and Promoting Archival and Special Collections. (Chandos Information Professional Series). Chandos Publishing.

Schmalberger, T. and Alker, H. R. (2001a). A Synthetic Framework for Extensible Conflict. In Alker, H. R., Gurr, T. R. and Rupesinghe, K. (eds),
Journeys Through Conflict: Narratives and Lessons. Rowman & Littlefield Publishers, pp. 318–53.

Schmalberger, T. and Alker, H. R. (2001b). Exploring alternative conflict trajectories with the cews explorer. In Alker, H. R., Gurr, T. R. and Rupesinghe, K. (eds),
Journeys Through Conflict: Narratives and Lessons. Rowman & Littlefield Publishers, pp. 354–94.

Schroeder, P. W. (1997). History and International Relations Theory: Not Use or Abuse, but Fit or Misfit.
International Security,
22(1).

Shaw, R. (2008). Event Gazetteers for Navigating Humanities Resources. In,
Proceedings of the 2nd PhD Workshop on Information and Knowledge Management. (PIKM ’08). ACM, pp. 89–92.

Shaw, R. (2010). Events and Periods as Concepts for Organizing Historical Knowledge. Berkeley: University of California. PhD thesis.

Shaw, R. (2013). A semantic tool for historical events. In,
Proceedings of the 1st Workshop on Events: Definition, Detection, Coreference, and Representation. Association for Computational Linguistics, pp. 38–46.

Shneiderman, B. (1996). The Eyes Have It: A Task by Data Type Taxonomy for Information Visualizations. In,
Proceedings of the 1996 IEEE Symposium on Visual Languages. (Visual Languages). IEEE Computer Society, pp. 336–43.

Skocpol, T. (1979).
States and Social Revolutions: A Comparative Analysis of France, Russia, and China. Cambridge University Press.

Trame, J., Keßler, C. and Kuhn, W. (2013). Linked Data and Time – Modeling Researcher Life Lines by Events. In Tenbrink, T., Stell, J., Galton, A. and Wood, Z. (eds),
Spatial Information Theory. Cham: Springer International Publishing, pp. 205–23.

Tzeng, J.-Y. (2010). Designs of concept maps and their impacts on readers’ performance in memory and reasoning while reading.
Journal of Research in Reading,
33(2). Blackwell Publishing: 128–47.

Vorkunova, O. A. (2001). Escalatory Dynamics in the Moldova-Dniestr and Chechnya Conflicts. In Alker, H. R., Gurr, T. R. and Rupesinghe, K. (eds),
Journeys Through Conflict: Narratives and Lessons. Rowman & Littlefield Publishers, pp. 103–27.

Walsh, W. H. (1951).
An Introduction to Philosophy of History. London: Hutchinson.

Weller, T. and Maleshkova, M. (2016). Cognitive Process Designer – An Open-Source Tool to Capture Processes according to the Linked Data Principles. In,
The Semantic Web: ESWC 2016 Satellite Events. Springer.