Advanced Manuscript Analysis Portal (AMAP): An Interactive Visual Language Environment for Manuscript Studies

Introduction
Application of state-of-the-art digital methods in the fields of Digital Paleography and Manuscript Studies has long been a challenging task, even with the proliferation of techniques within the field of Document Image Analysis (DIA) (Kasturi et al., 2002). Several reasons can be attributed to this. From a methodological perspective, many of these techniques are black boxes (Hassner et al., 2015), whose results cannot be completely scrutinized and understood. From a development perspective, there is a distinct lack of accessibility to the various published and open-source services and methods. Furthermore, many require technically complicated configurations to execute them. Projects like DIVA (Würsch et al., 2016) attempt to overcome the latter by making them available as web services. However, one still requires programming experience to use the methods and create computational solutions for scholarly research questions.

Motivation
Recently, Visual Language-based applications like AppInventor (Wolber, 2011) and Blockly (Trower and Gray, 2015) have gained a lot of attention. By using an intuitive visual syntax, instead of textual syntax, they let non-programmers to create computational solutions without the overhead of learning a traditional programming language (Narayanan and Hübscher, 1998). However, such Visual Language (VL) environments do not exist for Manuscript Studies. Although there are tools like VMR CRE (Mahony and Bodard, 2016) and DigiPal (Stokes et al., 2014) (along with its latest incarnation archetype.ink) that are interesting in their own setting, as yet, they do not provide a programming-like environment to implement DIA methods. DIVA comes very close to this idea, but mainly focuses on providing a seamless backend for other programs to take advantage of. However, very recently, Würsch et al. (2019) have come up with a new workflow design tool DIVA-DIP that involves users in the design and execution of workflows.
In this context, we introduce the Advanced Manuscript Analysis Portal (AMAP). The aim of AMAP is twofold. On the one hand, it offers a largely self-usable toolbox that humanists can use to build solutions themselves. On the other hand, it facilitates communication between experts from Computer Scientists and Humanists (Rajan and Stiehl, 2018b). Allowing users to jointly develop solutions minimizes the black box problem as they better understand their final system of interacting DIA modules. AMAP also strives to support reproducibility and data provenance. It further intends to encourage exploration of various relevant tools and algorithms.

AMAP Design and Implementation
AMAP has been designed to be platform independent and, thereby, allows the utilization of mobile devices. The architecture is entirely based on web-based technologies to facilitate this. It was ideally designed to be used on large-touch based devices and encourages touch-based interaction and collaboration (Rajan and Stiehl (2018a)).

Figure 1: AMAP Workspace (Screenshot)
AMAP in its current version consists of a central workspace that serves as the main canvas for interaction. Images (Digitized Manuscripts) are the central focus of this design space. They can be imported individually or in multiples. If the latter, they are imported as a virtual pile that can be stacked and unstacked as necessary. All other kinds of operations and methods that act upon images are visualized as virtual objects that can be attached or detached to a specific image or image pile. The images can also be subsetted and the derived subsets can be further processed independently. Any subsetting will maintain a visible connection to the main image that it is derived from and, hence, preserve provenance.
Various DIA methods are available as objects called action chips, which get attached to the right-side of an image. They can either return a completely processed image or just image segmentations with specific Regions of Interest (ROI). If an action chip modifies the images itself, the source image is directly changed to reflect the new modification. This is to maintain the focus on the image itself. In case of action chips for image segmentation, the ROIs are shown as selection boxes that can be used to create image subsets. The parameters of DIA methods are visualized as knobs. Thus, abstracting the type and the range of the parameter space. Experimenting by various parameters is now simply a matter of selecting the value from a given range as provided by the knob. Filters that will not directly affect the image but only change the visual appearance temporarily are made available as plugs. These are attached to the bottom part of an image and can affect image characteristics such as transparency and brightness.
The chips can also be connected to each other to form an experiment processing chain or a workflow. Any changes in parameters of the methods in the chain are instantly propagated to the next elements. The intermediate results for the source images are kept for viewing and inspection. If two chips are compatible in terms of chainability, they intuitively click together, whereas, incompatible objects either do not click or repel each other. In this way, users with little or no DIA experience are guided towards configuring mutually compatible chips. It is also possible to create loops for large-scale experimenting and, hence, find the optimal parameter range for a given problem setting.
All DIA operations performed in the workspace are logged in terms of timestamps, I/O and the various processes (with the associated parameters) that the source image has gone through. Logging is essential to perform well-documented scientific experiments and saves effort by avoiding experiment replication and improving on previous experiments. AMAP also provides various virtual tools that can be used for paleographic purposes. For instance, tools such as scales and protractors can be used to measure individual features of characters or images.
The workspace is live and gives immediate feedback by being always reflective of the current status of overall processing in a chain. The entire workspace can be saved, allowing work continuity. It also enables sharing experiments among the community and will foster greater transparency and reproducibility.
By providing an exploratory environment, scholars will have better access to the latest DIA techniques and will be empowered to create computational solutions themselves. It also provides them a hands-on experience to communicate with external developers. AMAP currently supports image processing, image segmentation, basic keyword spotting, keypoints visualization, OCR and writer identification.

Conclusion
In our short paper, we have briefly introduced Advanced Manuscript Analysis Portal (AMAP) for programming with open source DIA methods. We initially outlined the need and motivations for developing AMAP and its potential applications, and finally, elaborated the design and implementation of AMAP. As of now, AMAP is a proof-of-concept tailored to joint experimentation/workflow design for use cases from pilot projects in SFB 950 (with a focus on word spotting and writing style analysis). A first larger-scale use case analysis along with user studies from a variety of domains is in the making to solicit user feedback and improve the interface design.
Acknowledgement
We gratefully acknowledge support by Sonderforschungsbereich (SFB) 950 - Manuscript Cultures in Asia, Africa and Europe - (Faculty of Humanities of Universität Hamburg) through Deutsche Forschungsgemeinschaft (DFG).

Bibliography

Hassner, T., Rehbein, M., Stokes, P. A., and Wolf, L. (2015). Computation and Palaeography: Potentials and Limits. Kodikologie und Paläographie im Digitalen Zeitalter 3: Codicology and Palaeography in the Digital Age 3, 1.

Kasturi, R., O’Gorman, L., and Govindaraju, V. (2002). Document Image Analysis: A Primer. Sadhana, 27(1), 3-22.

Mahony, S., and Bodard, G. (2016). A Virtual Research Environment for the Study of Documents and Manuscripts. In Digital Research in the Study of Classical Antiquity (pp. 107-124). Routledge.

Narayanan, N. H., and Hübscher, R. (1998). Visual Language Theory: Towards a Human-Computer Interaction Perspective. In Visual language theory (pp. 87-128). Springer, New York, NY.

Rajan, V., and Stiehl, H. S. (2018a). Bringing Paleography to the Table: Developing an Interactive Manuscript Exploration System for Large Multi-Touch Devices. In DAS 2018 Short Papers Booklet, Document Analysis Systems 2018. Vienna, Austria.

Rajan, V., and Stiehl, H. S. (2018b). From Eye-to-Eye to Hand-in-Hand: Collaborative Solution Building in Interdisciplinary Manuscript Research. In Burghardt, M. and Müller-Birn, C. (Hrsg.), INF-DH-2018. Bonn: Gesellschaft für Informatik e.V..

Rajan, V and Stiehl, H.S (2019). Turning Black into White through Visual Programming: Peeking into the Black Box of Computational Manuscript Analysis. In Manuscript Cultures (mc No 12).

Stokes, P. A., Brookes, S., Buomprisco, G., Marques de Matos, D., and Watson, M. (2014). The DigiPal Framework for Script and Image. In DH2014 Book of Abstracts, EPFL-UNIL, Lausanne, Switzerland, 539-41.

Trower, J., and Gray, J. (2015). Blockly Language Creation and Applications: Visual Programming for Media Computation and Bluetooth Robotics Control. In Proceedings of the 46th ACM Technical Symposium on Computer Science Education (pp. 5-5). ACM.

Wolber, D. (2011). App Inventor and Real-World Motivation. In Proceedings of the 42nd ACM technical symposium on Computer science education (pp. 601-606). ACM.

Würsch, M., Ingold, R., and Liwicki, M. (2016). SDK Reinvented: Document Image Analysis Methods as Restful Web Services. In Document Analysis Systems (DAS), 2016 12th IAPR Workshop on (pp. 90-95). IEEE.

Würsch, M., Seuret M., Imstepf, L., Fischer, A., and Ingold, R. (2019). Creating Workflows with a Human in the Loop for Document Image Analysis. In Manuscript Cultures (mc No 12).