RAT 2.0

Historical maps are progressively digitized and added to the inventory of digital libraries. Beside their value as historical objects, such maps are an important source of information for researchers in various scientific disciplines. This ranges from the actual history of cartography and general history to the geographic and social sciences. However, for most of these digital libraries, the available metadata include only limited information about the content of the maps, for example author, title, size, and/or creation date.

Whereas given information extraction methods are designed for modern maps and mostly limited to certain types that share similar graphical features, there exist a limited number of tools that rely on a manual recording to visualize certain properties such as distortions as well as support a content-based querying. Examples concern the development of places over time, toponym changes over time, and the identification of the position of places (historical map vs. modern map). This also applies to place markers and text labels, which contain inherent information and so the annotation and geo-referencing of place markers is a crucial task, which can be supported with computer based tools (Budig and Dijk, 2015, Höhn et al., 2013, Shaw and Bajcsy, 2011, Simon et al., 2011).

As already presented in previous contributions (Höhn and Schommer, 2016, Höhn et al., 2013), the Referencing and Annotation Tool RAT supports an identification of place markers in digitized historical maps. RAT facilitates a geo-referencing by suggesting the most likely modern places based on an estimated mapping. The suggestions can be constrained by additional filters, for example by applying a phonetic search (with the Kölner Phonetik) to places, which sound similar to names given on the map. This allows an identification of modern places, whose historic name has changed over time but where its name still is close. RAT performs a template matching algorithm based on the normalized cross-correlation for the identification of place markers. If there are colored place markers in a map, a color segmentation methodology can be used to detect these markers. With respect to the geo-referencing, RAT uses the implemented phonetic search and an estimation of the positions of the place markers.

Figure 1. Architecture of the Convolutional Neural Network, which shows the operations used for processing the image

In addition to the original template-based place marker recognition algorithm, we integrated a place marker recognition algorithm based on convolutional neural networks (CNNs; RAT 2.0).

For these algorithms, the user is asked to manually annotate a small subset of the map. Regarding the template-based variant, the user has then to select a template for each type of place marker. From the templates, which are manually chosen by the user, the system creates automatically variations; based on the performance - measured with the annotated small subset of the map -, the best performing template variants are then chosen. In the template-matching algorithm, however, the normalized cross-correlation is used as a similarity measure because of its robustness against changes in brightness and contrast.

Regarding the detection trough a convolutional neural network, the manually annotated (small) subset of the map is used and split in a training and validation part. This is used to train the network, which

has - at this stage - a very basic structure. It consists only of convolutional and pooling layers as presented in Figure 1.

Both the template matching approach and the convolutional neural network approach share similar performances. Our tests have shown - for the template matching approach - a detection precision of 98.2% and a recall rate (discovered place markers divided by all existing place markers on the map) of 87.7%. The convolutional neural network approach reaches only a precision of 94.4%, but gives a recall rate of 96.2%. So, there are more place markers found; but the result contains also some more wrong matches in between. Therefore, it depends on the use case, which result is “better”, but for the manual post-correction it seems easier to check the CNN results for those additional wrong matches then finding the missed matches from the template-based approach.

The reason behind the use of the convolutional neural network approach has been an algorithmic limitation of the template matching approach. So far, RAT 2.0 uses only a fundamental convolutional system (at present, there are no additional techniques used, like, for example, data augmentation or pre-training).

As a future point, we work on training the convolutional neural network on multiple maps in order to find a classification model that learns the characteristics of place markers and that detects these on unseen maps.

Shaw, T. and Bajcsy, P. (2011). “Automation of Digital Historical Map Analyses.” Proceedings of the IS&T/SPIE Electronic Imaging, Vol. 7869.

Simon, R., Haslhofer, B., Robitza, W. and Momeni, E.

(2011). “Semantically Augmented Annotations in Digitized Map Collections.” Proceedings of the 11th Annual International ACM/IEEE Joint Conference on Digital Libraries, pp. 199-202.

Bibliography

Budig, B. and van Dijk, T. C. (2015). “Active Learning for Classifying Template Matches in Historical Maps.” International Conference on Discovery Science, pp. 33-47.

Höhn, W., Schmidt, H.-G. and Schöneberg, H. (2013). “Semiautomatic Recognition and Georeferencing of

Places in Early Maps.” Proceedings of the 13th ACM/IEEE-

CSJoint Conference on Digital Libraries, pp. 335-38.

Höhn, W. and Schommer, C. (2016). “Annotating and Georeferencing of Digitized Early Maps.” Digital Humanities 2016, pp. 807-808

Höhn, W. and Schommer, C. (2016). “RAT: A Referencing and Annotation Tool for Digitized Early Maps.” Digital Humanities BeneLux 2016