DeXTER: A post-authentic approach to heritage visualisation

paper, specified "long paper"
  1. 1. Lorella Viola

    University of Luxembourg, Luxembourg

Work text
This plain text was ingested for the purpose of full-text search, not to preserve original formatting or readability. For the most complete copy, refer to the original conference program.

Cultural heritage institutions and academics are resorting more and more to visual representations of cultural heritage material as a way to enhance access to collections for users’ appreciation and research purposes (Windhager et al., 2019). However, scholars have pointed out (Drucker, 2011; 2013; 2014; 2020; Windhager et al., 2019) how a critical approach to visualisation is still largely missing and how on the whole, user-interface (UI) design still shows a functional and task-driven approach, oriented towards satisfying a need for information rather than towards eliciting curiosity, engagement and reflection. With this presentation, I aim to contribute to the urgent need for the establishment of a critical data and visualisation literacy in current task-driven approaches to interface design which continue to see the user as a consumer and to operate within a problem solver model. Drawing on critical posthumanities (Braidotti, 2017; 2019), I here propose a critical framework to digital cultural heritage and digital cultural heritage visualisation. With this approach —which I labelled “post-authentic framework” (Viola, 2021) —I want to initiate a discussion and critique of the fetishization of empriricism and technical objectivity in digital knowledge creation. To exemplify how the post-authentic framework works in practice, I present the design choices for developing the tool DeXTER – DeepteXTminER, an interactive visualisation app to explore enriched cultural heritage material. The discussion will revolve around the challenges facing product design, with specific reference to visualising the ambiguities and uncertainties of network analysis (NA) and sentiment analysis (SA). DeXTER is currently loaded with Chroniclitaly 3.0 (Viola and Fiscarelli, 2021a), a digital heritage collection of Italian American newspapers published in the USA by Italian immigrants between 1897 and 1936.

My argument for a post-authentic framework to digital cultural heritage builds upon recent digital heritage positions (see for instance Cameron, 2021; Goriunova, 2019; Jones et al., 2018) and extends on posthuman critical theory which understands the matter as an extremely complex assemblage of “forces, entities, and encounters” (Braidotti, 2017, 16). A post-authentic framework to digital cultural heritage understands curatorial interventions as cyclic processes that shape and are shaped by humans, entities, and processes connected to each other according to the various forms of power embedded in computational processes and beyond. As these processes are never neutral, their implementation requires constant critical supervision. Whilst exploiting the new opportunities offered by computational technologies, a post-authentic framework rejects an uncritical adoption of digital methods and it contests the main discourse that still presents such techniques and outputs as exact, final, objective and true.

NA is a method that models pairwise relations (i.e., edges) between objects (i.e., nodes) (Biggs, Lloyd, and Wilson, 1986). When applied to cultural heritage material, NA is typically used to investigate how referential entities (i.e., people, organisations, places) are connected with each other. However, the guiding assumption that modelling how entities relate to each other provides adequate explanations of social phenomena conceals the fact that the results are highly dependent on which entities are modelled and more importantly, which ones are not. NA therefore tends to present as accurate conclusions that may be based on over-represented actors or conversely, on underrepresented categories. When NA is applied to cultural heritage material, this issue is particularly significant as the resulting analysis can impact on the provided interpretation of the past.
Another critical issue of NA is related to the intrinsic nature of the technique itself. Nodes are understood as discrete objects, i.e., completely independent from each other, meaning that they are modelled to remain stable regardless of the relationship between them. This offers a rather artificial view of the analysed phenomena as in reality, actors are always affected by mutual relations (Drucker, 2020, 180). Although extremely significant, this issue is often obfuscated by attractive visualisations which present as accurate what in reality is only approximated information (ibid., 87).

DeXTER introduces a counter narrative in the main discourse —public and academic —by documenting, explaining and motivating all the data selections and interventions in the project’s GitHub repository (Viola and Fiscarelli, 2021b). In this way, the management of data is acknowledged as problematic, an act of constant interpretation and manipulation which transforms, selects, aggregates, and ultimately creates data. In the case of NA, DeXTER openly acknowledges that the displayed entities are not ALL the entities in the collection but in fact a representative, yet small selection. This choice –as explained in the app’s documentation –had the advantage to provide a less crowded network graph; on the other hand, however, it resulted in a loss of representation of the less occurring entities.

DeXTER’s interface allows users to carry out efficient information retrieval. As said earlier, NA visualisations are typically static, and nodes and edges remain unchanged no matter their relation. In DeXTER it is always possible to explore the changing relationships between entities over time or at specific intervals; this is done by swiping the time bar on the top left of the interface which reflects the collection’s timespan, i.e., 1897-1936 (Figure 1). This allows users to obtain a more realistic representation of how the modelled entities were mentioned by migrants over time or at specific points in history. Users can also select/deselect specific titles and observe how entities of interest were mentioned by migrants in titles of different political orientation and geographical location.

Figure 1: DeXTER default landing interface for NA. The red oval highlights the time bar

DeXTER exposes the data multi-dimensional complexity by allowing users to explore three types of networks: two entity-focused graphs and one issue-focused network. In the entity-focused graph (i.e., ego-network), users start from a selected entity of their choice (i.e., the ego) to explore the net of entities (i.e., the alters) most frequently mentioned in the same sentence as the selected entity. The network also displays the number of times entities were mentioned together, the titles in which they were mentioned and the prevailing emotional attitude of the sentence in which they were mentioned. If ego-network is not selected, the graph displays also the relations among the alters. Figures 2 and 3 show the different entity-focused displays of the same entity/ego when selecting/deselecting the ego-network option. Finally, it is possible to obtain additional points of view by starting from a specific issue (i.e., issue-focused network) so as to research which actors were mostly mentioned on a specific day (or days) by a specific newspaper or several (Figure 4).

Figure 2: DeXTER entity-focused NA - egocentric network

Figure 3: DeXTER entity-focused NA

Figure 3: DeXTER entity-focused NA

Figure 4: DeXTER issue-focused NA

Figure 4: DeXTER issue-focused NA

SA aims to identify the prevailing emotional attitude in a given text, e.g., positive, negative, or neutral. However, these labels are typically presented as if they were unambiguous categories universally accepted. As a way to acknowledge the ambiguities behind a ‘sentiment score’, we implemented a fluid visualisation through colour gradients which go from a darker shade of blue for the lowest score (i.e., negative) to a darker shade of red for the highest score (i.e., positive). We also chose pastel shades as opposed to solid shades as an open way to acknowledge SA as problematic and to promote a visualisation that would not be presented as precise and accurate. The description of how the sentiment categories have been identified, how the classification has been conducted, what the scores mean and how they have been rendered in the visualisation, and how the results have been aggregated is been thoroughly documented in the dedicated OA GitHub repository (Viola and Fiscarelli, 2021b).

Finally, the data behind the interface is accessible and downloadable through the tab ‘Data’ which remains always active from anywhere in the interface. Users can choose to either download the entire data-set or the data-set that reflects their specific selections operated through the available filters (e.g., title, time interval, frequency, entity), which also remain visible at all times. The intention is to emphasise the continuous making and re-making of data and to encode this process of forming, arranging and interpreting data within the interface itself. Moreover, in this way, DeXTER also advocates transparency, traceability, and accountability. Indeed, a post-authentic framework to digital cultural heritage primarily acknowledges the collective responsibility of building a source of knowledge for current and future generations and frames it as honest and accountable, unfinished and receptive to alternatives.

Biggs, Norman, E. Keith Lloyd, and Robin J. Wilson. (1986).
Graph Theory, 1736-1936. Oxford [Oxfordshire] ; New York: Clarendon Press.

Braidotti. (2017). “Posthuman Critical Theory.”
Journal of Posthuman Studies 1 (1): 9.

Braidotti, Rosi. (2019). “A Theoretical Framework for the Critical Posthumanities.” Theory,
Culture & Society 36 (6): 31–61.

Cameron, Fiona. (2021).
The Future of Digital Data, Heritage and Curation in a More-than Human World. Abingdon, Oxon; New York, NY: Routledge.

Drucker, Johanna. (2011). “Humanities Approaches to Graphical Display.”
Digital Humanities Quarterly 5 (1).

———. (2013). “Performative Materiality and Theoretical Approaches to Interface.” DHQ:
Digital Humanities Quarterly 7 (1).

———. (2014). “Visualizing Temporality: Modelling Time from the Textual Record.” Queen Mary University of London, March 25.
———. (2020).
Visualization and Interpretation: Humanistic Approaches to Display. Cambridge, Massachusetts: The MIT Press.

Goriunova, Olga. (2019). “The Digital Subject: People as Data as Persons.”
Theory, Culture & Society 36 (6): 125–45.

Jones, Siân, Stuart Jeffrey, Mhairi Maxwell, Alex Hale, and Cara Jones. (2018). “3D Heritage Visualisation and the Negotiation of Authenticity: The ACCORD Project.”
International Journal of Heritage Studies 24 (4): 333–53.

Viola, Lorella. (2021). “Keynote: The Importance of Being Digital.” In
HistoInformatics2021 Workshop - 6th International Workshop on Computational History. Co-Located with JCDL2021. University of Illinois.

Viola, Lorella, and Antonio Maria Fiscarelli. (2021a).
ChroniclItaly 3.0. A deep-learning, contextually enriched digital heritage collection of Italian immigrant newspapers published in the USA, 1898-1936. Zenodo.

Viola, Lorella, and Antonio M. Fiscarelli. (2021b).
DeXTER (DeepTextMiner): A Deep Learning, Critical Workflow to Contextually Enrich Digital Collections and Visualise Them (version v1.0.0). Zenodo.

Windhager, Florian, Paolo Federico, Gunther Schreder, Katrin Glinka, Marian Dork, Silvia Miksch, and Eva Mayr. (2019). “Visualization of Cultural Heritage Collection Data: State of the Art and Future Challenges.”
IEEE Transactions on Visualization and Computer Graphics 25 (6): 2311–30.

If this content appears in violation of your intellectual property rights, or you see errors or omissions, please reach out to Scott B. Weingart to discuss removing or amending the materials.

Conference Info

In review

ADHO - 2022
"Responding to Asian Diversity"

Tokyo, Japan

July 25, 2022 - July 29, 2022

361 works by 945 authors indexed

Held in Tokyo and remote (hybrid) on account of COVID-19

Conference website:

Contributors: Scott B. Weingart, James Cummings

Series: ADHO (16)

Organizers: ADHO