The Inevitability of a Reproducibility Crisis in the Digital Humanities

Once the purview of conventional science, sophisticated computational and algorithmic modeling methods are becoming ubiquitous approaches for digital humanists. However, unlike scientific approaches to research, digital humanities as a field lacks parallel paradigms or a research integrity framework related to reproducibility, also referred to as “replicability” in certain disciplines. Inherently, there is tension between humanistic approaches to inquiry and scientific or computational methods. However, as the methods and approaches of digital humanists increasingly mirror the methods and approaches of scientists in dealing with research data, it is becoming increasingly necessary to address the question of whether and how to apply scientific research integrity principles to digital humanities. 
Peels and Bouter (2018) claim replication in the humanities is possible and desirable because research in these fields follows an epistemic process that, while discipline-specific, can be replicated.  Because the digital humanities create knowledge through the use of algorithms and digital tools, the possibility and desirability of replication is even higher. Replication can be facilitated through the sharing of data and software that underlie research projects (Sikk, 2020). Unfortunately, the activity of sharing research materials is rare among both humanists more broadly and digital humanists more specifically.  
To complicate matters, the rise in availability of third-party licensed text mining datasets from digital publishing vendors, library databases, and content providers, is enabling access to previously unavailable corpora and collections to analyze. Working with their institutional libraries, scholars may be able to access the text mining files for materials in their collections through platforms like the TDM Studio or the Gale Scholar Lab, they may purchase the files for specific publications, or in some cases they may simply request the data free of charge as part of the library’s contract with the content provider. Unfortunately, these third-party licensed datasets are often subject to stringent legal terms and conditions. The potential inability of researchers to legally share the underlying data of their publication(s) presents a challenge regarding research transparency. 
Through a brief overview of institutional case studies, this short presentation will explore the developing discourse surrounding reproducibility of digital humanities research and the subsequent inevitability of a digital humanities replication crisis contextually instigated by neoliberal constraints.

Bibliography

Peels, R., and Bouter, L. (2018). The possibility and desirability of replication in the humanities.
Palgrave Communications
,
4
(1): 1–4.

https://doi.org/10.1057/s41599-018-0149-x

Sikk, K. (2020). Towards reproducible science in the digital humanities
. Digital History & Hermeneutics. Retrieved from

https://dhh.uni.lu/2020/05/19/towards-reproducible-science-in-the-digital-humanities-how-to-publish-your-data-and-code-alongside-your-research-with-the-help-of-zenodo/